A Few-Shot Wireless Gesture Recognition Method Based on Meta-Actions

Abstract

The present invention relates to a small-sample wireless gesture recognition method based on meta-actions, and belongs to the field of human-computer interaction technology. The method includes two stages: offline training and online recognition. In the offline training stage, meta-action gesture data is collected by millimeter-wave radar, and after pre-processing to eliminate noise interference, the Doppler spectrum features of the first gesture and the curvature features of the first gesture trajectory are extracted; the second gesture features are synthesized based on the consistency of physical laws, and the two features are fused using a weighted fusion method to train the perception network. In the online recognition stage, the trained perception network is used to realize online gesture recognition. The present invention uses meta-action feature combination technology to generate a diverse set of new gesture features with only a small number of samples; combined with the dual-feature fusion strategy of Doppler spectrum and trajectory curvature, it effectively distinguishes gestures with similar motion trajectories, solving the problem that existing wireless gesture recognition technology relies on a large amount of training data and has low differentiation between similar gestures.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a small-sample wireless gesture recognition method based on meta-actions. Background Art

[0002] Gesture recognition technology, a key development in the field of human-computer interaction, is valued for its ability to deliver a natural and intuitive user experience. Currently, mainstream technical solutions fall into three categories: sensor-based contact recognition, computer vision-based optical recognition, and contactless recognition based on wireless signals. Each approach faces different technical bottlenecks in practical application.

[0003] Sensor-based contact recognition solutions typically require users to wear specialized equipment equipped with integrated sensors such as accelerometers and gyroscopes to analyze gestures by collecting three-dimensional motion data. While this approach offers high measurement accuracy, the physical contact nature reduces user comfort and increases equipment maintenance costs. Computer vision-based optical recognition solutions use cameras to capture hand images and analyze gesture features using image processing algorithms. While their non-contact nature improves the user experience, changes in ambient lighting conditions can significantly affect image quality, leading to insufficient recognition stability.

[0004] Wireless signal recognition technology, developed in recent years, analyzes the modulation characteristics of radio frequency signals caused by gestures, achieving recognition through non-contact and strong environmental adaptability. However, this technology requires the collection of a large number of training samples for each gesture category. This data collection process is affected by factors such as signal interference and device layout, resulting in high system deployment costs. Furthermore, existing algorithms struggle to effectively distinguish between gestures with similar motion trajectories, limiting the technology's reliable application in real-world scenarios. Summary of the Invention

[0005] In light of this, the present invention provides a small-sample wireless gesture recognition method based on meta-actions. Meta-action gesture features are combined based on the characteristics of gesture motion trajectories to generate new gesture Doppler spectrum features and trajectory curvature features. This overcomes the challenge of requiring different types of gesture data for wireless gesture recognition while ensuring consistency in physical laws. By obtaining the Doppler spectrum features and trajectory curvature features of meta-actions, the difficulty in distinguishing gestures with similar Doppler spectrum features is effectively addressed. A weighted fusion of gesture Doppler spectrum features and gesture trajectory curvature features is then used for network training, enabling efficient wireless gesture recognition with a small sample size.

[0006] To this end, the present invention provides the following technical solutions:

[0007] The present invention provides a small sample wireless gesture recognition method based on meta-actions, comprising: an offline training phase and an online recognition phase; in the offline training phase, millimeter wave radar equipment is used to collect meta-action gesture data to form a gesture action set, and each meta-action gesture data in the gesture action set is preprocessed to eliminate the interference of meta-action gesture signal noise; for the preprocessed meta-action gesture signal, a first gesture Doppler spectrum feature and a first gesture trajectory curvature feature are extracted; based on the first gesture Doppler spectrum feature and the first gesture trajectory curvature feature, a second gesture Doppler spectrum feature and a second gesture trajectory curvature feature are synthesized to obtain The gesture trajectory curvature feature is obtained by merging the Doppler spectrum feature of the second gesture and the curvature feature of the second gesture trajectory; the fusion feature of each gesture in the gesture action set is fused into the perception network for training; in the online recognition stage, gesture data is collected to construct gesture samples to be recognized, noise interference of the gesture signal to be recognized is removed, the distance-time image of the gesture sample to be recognized is obtained, the Doppler spectrum feature of the gesture sample to be recognized and the curvature feature of the gesture trajectory to be recognized are extracted, the two features of the gesture are effectively fused by the weighted fusion method, and the results are input into the trained perception network to realize wireless gesture recognition.

[0008] Furthermore, the gesture data of each element in the gesture action set is preprocessed, including: obtaining multiple channel radar raw data and processing the radar data frame by frame; performing time domain windowing on each receiving channel signal in each frame, and eliminating the DC component and static interference in the signal by de-averaging.

[0009] Furthermore, the Doppler spectrum feature of the first gesture is extracted, including: performing a distance dimension fast Fourier transform on the pre-processed meta-action gesture signal to obtain a distance-time image of the meta-action gesture; performing a fast Fourier transform on the distance-time image along the slow time dimension to obtain a Doppler spectrum of the meta-action gesture; performing a dimensionality reduction process on the Doppler spectrum of the meta-action gesture ... The two-dimensional matrix is converted into a one-dimensional vector while retaining the Doppler spectrum waveform signal of the gesture; the obtained waveform signal is traversed to locate the first point where the sign changes from negative to positive and ensure Meet one condition: sign change point There is a sign change point from positive to negative before to determine the integrity of the first lower envelope of the curve; reverse the one-dimensional vector to find the sign change point ; restore the data to its original order and Before and The gesture Doppler data after the point is set to zero, and only the Doppler signal between the two points is retained, thereby extracting the first gesture Doppler spectrum feature.

[0010] Furthermore, by performing dimensionality reduction on the Doppler spectrogram of the meta-gesture, The two-dimensional matrix is converted into a one-dimensional vector while retaining the Doppler spectrum waveform signal of the gesture, including: Line and back The sum of the rows and columns is obtained and , By comparison and The energy value is selectively retained by the numerical size to obtain a one-dimensional vector; by performing median filtering and smoothing on the one-dimensional vector, a waveform signal reflecting the motion characteristics of the gesture is obtained.

[0011] Furthermore, the curvature feature of the first gesture trajectory is extracted, including: performing a distance-dimensional fast Fourier transform on the meta-action gesture signal to obtain a distance-time image of the meta-action gesture; analyzing the distance information of the gesture relative to the radar at a certain moment in the distance-time image to obtain the distance values corresponding to the gesture at different time points, forming a position sequence : ;in, Indicates the meta-action gesture at a time point The distance value relative to the radar at the time; using the position sequence , calculate the curvature of the meta-action gesture trajectory;

[0012] Based on the obtained sign change point and sign change point At the position of the time axis, the curvature features of the meta-action gesture are extracted, and the curvature data of the meta-action gesture trajectory between the two sign-changing points are retained to obtain the curvature features of the meta-action gesture trajectory.

[0013] Furthermore, the curvature is calculated using the following formula using the discrete points of the gesture trajectory: : ;in, and Represent the first-order derivative and second-order derivative of the trajectory, namely the velocity and acceleration; a length of The curvature sequence vector : ;in, Indicates the meta-action gesture at a time point The curvature value.

[0014] Further, synthesizing the first gesture Doppler spectrum feature and the first gesture trajectory curvature feature to obtain a second gesture Doppler spectrum feature and a second gesture trajectory curvature feature, including: performing time normalization processing on the first gesture Doppler spectrum feature and the first gesture trajectory curvature feature;

[0015] Generating a second gesture Doppler spectrum feature, including: accelerating the normalized first gesture Doppler spectrum feature to simulate the speed variation characteristics of different gestures in actual motion; synthesizing and splicing the Doppler spectra of the meta-action gesture along a time axis according to the composition sequence and motion characteristics of the target gesture; after the Doppler spectra are spliced, performing time normalization processing on the generated second gesture Doppler spectrum so that its time length is consistent with that of the meta-action gesture, thereby obtaining the second gesture Doppler spectrum feature;

[0016] Generating a second gesture trajectory curvature feature includes: analyzing the motion trajectory of the meta-action gesture in space and extracting its key features; accelerating the first gesture trajectory curvature feature by compressing the time axis of the gesture trajectory curvature graph; combining multiple meta-action gesture trajectory curvature graphs according to time sequence and spatial relationship to form a second gesture trajectory curvature graph, and normalizing the graph so that its time length is consistent with the meta-action gesture, thereby obtaining the second gesture trajectory curvature feature.

[0017] Furthermore, the input of the perception network is a two-dimensional feature matrix whose label is the number of gesture categories. The difference between the model prediction and the true label is quantified by calculating the loss value, where the loss function is defined as: ; in represents the total number of categories of labeled samples, represents the one-hot encoded label of the sample, Represents the predicted output probability of the perception network; the gesture feature matrix passes through the convolution layer Conv, activation layer ReLU and full connection layer Fc in the perception network to train the perception network model.

[0018] Advantages and positive effects of the present invention:

[0019] In this invention, new gesture data is generated by collecting several sets of meta-gestures and combining them to expand the gesture set. By analyzing the temporal sequence and spatial relationships (such as connection points and movement directions) between the meta-gestures, these meta-gesture features are combined according to specific rules to form new complex gesture features, thereby expanding the diversity of the gesture set. Each gesture is represented by Doppler spectrum features and trajectory curvature features. Network training and learning are performed by weighted fusion of these two features for each gesture, achieving efficient wireless gesture recognition using a small number of samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 This is a flowchart of the working process in an embodiment of the present invention;

[0022] Figure 2 It is a non-gesture signal generated by the hand-raising and hand-releasing action in the embodiment of the present invention;

[0023] Figure 3 This is the gesture recognition network structure in the embodiment of the present invention;

[0024] Figure 4 This is an expanded combination of gestures in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] This paper proposes a small-sample wireless gesture recognition method based on meta-actions. Meta-actions are simple, basic gesture units with universal characteristics and are also the basic components of complex gestures. Based on the characteristics of gesture writing trajectories, it is found that some complex gestures can be synthesized by combining and expanding meta-actions. Specifically, the principle of gesture synthesis is based on the spatiotemporal relationship of meta-actions: first, a complex gesture is decomposed into multiple meta-action sequences, each representing a basic motion unit of the gesture; second, by analyzing the motion characteristics and spatial relationships between the meta-actions, these meta-actions are combined according to specific rules to form a complete new gesture feature. For example, the motion characteristics and writing trajectory of gesture "h" can be obtained by splicing gestures "1" and "n". Specifically, gesture "1" represents a vertical downward linear motion, while gesture "n" represents a continuous curved motion. The two are combined in space through a connecting point to form the complete motion characteristics and trajectory characteristics of gesture "h".

[0028] This invention comprehensively considers the scalability of Doppler spectrum features and trajectory curvature features of gesture samples derived from wireless signals, combines several sets of meta-action gesture samples to obtain new gesture data, and expands the gesture action set while ensuring consistency with physical laws. The key to this invention is that, first, by eliminating the influence of static interference and non-gesture signals generated by raising and releasing the hand, the Doppler spectrum features of the first gesture and the curvature features of the first gesture trajectory are extracted; second, based on the principle of synthesizing new gestures from meta-actions, while ensuring consistency with physical laws, the meta-action gesture features are combined to obtain a new category of Doppler spectrum features and gesture trajectory curvature features of the second gesture; finally, by weightedly fusing the second gesture Doppler spectrum features and the second gesture trajectory curvature features, a network model is trained to achieve efficient wireless gesture recognition using a small number of samples.

[0029] like Figure 1 As shown in FIG, a method for wireless gesture recognition based on a few samples of meta-actions in an embodiment of the present invention has an overall process divided into two parts: offline training and online recognition.

[0030] In the offline training stage, first, meta-action gesture samples are collected and noise interference is removed to generate a range-time map (RTM), and then the Doppler spectrum features of the first gesture and the curvature features of the first gesture trajectory are extracted; secondly, the second gesture feature samples are synthesized based on the first gesture features to expand the gesture action set; finally, the second gesture Doppler spectrum features and the second gesture trajectory curvature features are fused to train the network model.

[0031] In the online recognition stage, gesture data is collected to construct gesture samples to be recognized, noise interference of the gesture signals to be recognized is removed, the RTM images of the gesture samples to be recognized are obtained, and two features of the gesture samples to be recognized are extracted. After feature fusion, the features are input into the trained network model to complete gesture recognition.

[0032] The system configuration for implementing the few-sample wireless gesture recognition method in this embodiment is as follows: the system operates on a Texas Instruments IWR6843 FMCW radar transceiver; the transceiver operates at 60 GHz with a bandwidth set to 3.06 GHz, providing a range resolution of 4.9 cm; the transceiver uses three transmit and four receive antennas, and using TDM-MIMO mode, it can form a 12-element virtual array, providing an angular resolution of 29°.

[0033] (1) Offline training of synthesizing new gestures based on meta-action gestures, specifically including the following steps:

[0034] S11, using a millimeter wave radar device to collect meta-motion gesture data, and eliminating interference from meta-motion gesture signal noise after pre-processing;

[0035] In the specific implementation, first, the gesture recognition system is initialized based on preset radar parameters (including the number of sampling points, antenna configuration, range resolution, and Doppler resolution). Second, the radar data is processed frame by frame by acquiring raw data from multiple channels of radar. Finally, each receiving channel signal within each frame is windowed in the time domain (such as a Hamming window), and the DC component and static interference in the signal are eliminated by de-averaging.

[0036] S12, extracting the Doppler spectrum features of the first gesture of the meta-action and the curvature features of the first gesture trajectory from the preprocessed meta-action gesture signal;

[0037] The extraction of the Doppler spectrum features of the first gesture includes: first, performing a range-dimensional fast Fourier transform (Range-FFT) on the preprocessed meta-action gesture signal to obtain the RTM image of the meta-action gesture; second, performing a fast Fourier transform (Doppler-FFT) on the RTM image along the slow time dimension to obtain the Doppler spectrum of the meta-action gesture, which reflects the dynamic characteristics of the gesture by capturing the frequency change characteristics of the gesture.

[0038] Although the Doppler spectrum based on meta-motion gestures eliminates the influence of static interference, the influence of non-gesture signals such as raising and releasing hands when making gestures in the air still exists. Since the Doppler spectrum can intuitively reflect the change of target speed over time, when the experimenter performs the hand-raising operation, the speed of the hand will experience a process of first accelerating and then decelerating until the speed drops to zero, and the hand gradually approaches the radar; then, the experimenter completes the gesture in the air; after the gesture is completed, when the experimenter puts his hand down, the speed of the hand will also experience a process of first accelerating and then decelerating until the speed drops to zero, and the hand gradually moves away from the radar. Figure 2 As shown in the figure, the hand-raising action corresponds to the first lower envelope waveform in the Doppler spectrum, while the hand-releasing action corresponds to the last upper envelope waveform. To remove the non-gesture interference generated during the hand-raising and releasing processes, it is necessary to find the end point of the first lower envelope waveform and the starting point of the last upper envelope waveform in the gesture Doppler spectrum. By retaining the signal between these two points, the Doppler spectrum features of the meta-action gesture are extracted.

[0039] By reducing the dimensionality of the Doppler spectrum of the meta-gesture, The two-dimensional matrix is converted into a one-dimensional vector while retaining the Doppler spectrum waveform information of the gesture. First, the front of the two-dimensional matrix Line and back The sum of the rows and columns is obtained and ( ), that is, summing up the Doppler energy of each frame; secondly, by comparing and The energy value is selectively retained based on the numerical value of , and a one-dimensional vector is obtained. Finally, a waveform signal reflecting the motion characteristics of the gesture is obtained by performing median filtering and smoothing on the one-dimensional vector.

[0040] Traverse the obtained waveform signal, first locate the first sign change point from negative to positive and ensure Meet one condition: sign change point There is a sign change point from positive to negative before, to determine the integrity of the first lower envelope of the curve; secondly, the one-dimensional vector is reversed and the sign change point is found using the same method as above ; Finally, restore the data to its original order and Before and The gesture Doppler data after the point is set to zero, and only the Doppler signal between the two points is retained, thereby extracting the first gesture Doppler spectrum feature.

[0041] Among them, extracting the curvature feature of the first gesture trajectory includes: performing distance-dimensional fast Fourier transform on the meta-action gesture signal to obtain the RTM image of the meta-action gesture, which can intuitively reflect the energy distribution of the gesture changing with time in the distance dimension. By analyzing the distance information of the gesture relative to the radar at a certain moment in the RTM image, the distance values corresponding to the gesture at different time points are obtained to form a position sequence :

[0042] ;in, Indicates the meta-action gesture at a time point The distance value relative to the radar at different time points is used to calculate the curvature of the meta-action gesture trajectory using the position sequence of these gestures at different time points.

[0043] The curvature calculation method is based on the change between consecutive points. Specifically, the curvature is calculated using the discrete points of the gesture trajectory using the following formula:

[0044] ;in, and They represent the first-order derivative and second-order derivative of the trajectory, namely velocity and acceleration, respectively.

[0045] The curvature is calculated to obtain a length of The curvature sequence vector : ;in, Indicates the meta-action gesture at a time point The curvature value.

[0046] Based on the obtained sign change point and sign change point At the position on the time axis, the curvature feature of the meta-action gesture is extracted, and the curvature data of the meta-action gesture trajectory between the two sign-change points is retained to obtain the first gesture trajectory curvature feature.

[0047] S13, synthesizing the first gesture Doppler spectrum feature and the first gesture trajectory curvature feature to obtain a second gesture Doppler spectrum feature and a second gesture trajectory curvature feature;

[0048] S111, performing time normalization processing on the first gesture feature to synthesize the second gesture;

[0049] S112, generating a second hand gesture Doppler spectrum feature;

[0050] Analysis of gesture motion characteristics reveals that some gestures exhibit opposite motion relative to the radar trajectory, meaning their Doppler spectra are symmetrical along the frequency axis. For example, by frequency-flipping the Doppler spectrum of the acquired meta-gesture "n," a mirror-symmetric Doppler spectrum is obtained, generating the new Doppler spectrum of gesture "u." Some complex gestures can be composed of simple meta-gesture features. For example, gesture "m" can be composed of two sets of gesture "n," and gesture "h" can be composed of gesture "1" and gesture "n." Based on these findings, the specific method for generating the second gesture Doppler spectrum features is as follows: First, the normalized meta-gesture Doppler spectrum features are accelerated to simulate the velocity variations of different gestures in real motion. Then, the Doppler spectra of the meta-gestures are synthesized and spliced along the time axis based on the composition order and motion characteristics of the target gesture. For example, by concatenating the two sets of Doppler spectra for gesture "n" along their motion trajectories, a new Doppler spectra for gesture "m" can be generated. Similarly, by concatenating the Doppler spectra for gestures "1" and "n" along their motions, a new Doppler spectra for gesture "h" can be generated. After the Doppler spectra are concatenated, the generated new Doppler spectra are time-normalized to align their duration with the original gesture, yielding the Doppler spectra features for the second gesture.

[0051] S113 . Combining the primitive gesture trajectory curvature features according to the writing characteristics of the gesture in space to generate a second gesture trajectory curvature feature.

[0052] Specifically, first, the motion trajectory of the meta-action gesture in space is analyzed to extract its key features; second, the curvature features of the meta-action gesture trajectory are accelerated by compressing the time axis of the meta-action gesture trajectory curvature graph; finally, multiple meta-action gesture trajectory curvature graphs are combined according to the time sequence and spatial relationship to form a second gesture trajectory curvature graph, and it is normalized so that its time length is consistent with the meta-action gesture, thereby obtaining the curvature features of the second gesture trajectory.

[0053] S14, using a weighted fusion method to fuse the generated second gesture Doppler spectrum features and the second gesture trajectory curvature features;

[0054] A gesture action set is constructed by combining meta-action gestures with synthesized new gestures, where each gesture contains two key features: Doppler spectrum features and gesture trajectory curvature features. The Doppler spectrum features are extracted through the Doppler effect of the radar signal, reflecting the velocity distribution characteristics of the gesture target, and can effectively characterize the dynamic motion pattern of the gesture; the gesture trajectory curvature features characterize the spatial motion characteristics of the gesture by calculating the curvature change of the gesture motion path, and can effectively distinguish gestures with similar Doppler features but different spatial trajectories. In order to make full use of the complementarity of these two features, a weighted fusion method is adopted in the embodiment of the present invention to fuse the gesture Doppler spectrum features and trajectory curvature features. By assigning a corresponding weight coefficient to each feature and adjusting the weight ratio in combination with the motion characteristics of the gesture, weighted fusion of features is achieved.

[0055] S15, sending the fusion features of each gesture in the gesture action set into the perception network for training;

[0056] The input of the network is a two-dimensional feature matrix whose label is the number of gesture categories. The difference between the model prediction and the true label is quantified by calculating the loss value, where the loss function is defined as: ; in represents the total number of categories of labeled samples, represents the one-hot encoded label of the sample, Represents the predicted output probability of the perception network. Figure 3 As shown in the figure, the gesture feature matrix passes through the convolution layer Conv, the activation layer ReLU and the fully connected layer Fc to train the perception network model.

[0057] (2) Using the trained perception network for online gesture recognition, specifically including the following steps:

[0058] S21, constructing gesture samples to be recognized by actually collecting gesture data;

[0059] S22, removing noise interference from the gesture signal to be recognized, obtaining an RTM image of the gesture sample to be recognized, and extracting Doppler spectrum features and trajectory curvature features of the gesture sample to be recognized;

[0060] S23. Use the weighted fusion method to effectively fuse the two features of the gesture and input them into the perception network to realize wireless gesture recognition.

[0061] For ease of understanding, the above-mentioned small-sample wireless gesture recognition method based on meta-actions is described below with a specific example.

[0062] A method for wireless gesture recognition based on a small number of samples of a meta-action comprises the following steps:

[0063] S31. Meta-action gesture signal preprocessing:

[0064] Based on the handwriting trajectory characteristics of numbers and letters, four meta-gestures, "1," "c," "3," and "n," were selected for synthesizing new gestures. Four seconds of meta-gesture data were collected using radar equipment, with 64 sampling points and a range resolution of 4.9 cm. This example processes the collected radar data frame by frame, retaining 60 frames of data and using a deaveraging method to eliminate DC components and static interference from the signal.

[0065] S32. Extract the Doppler spectrum features of the first hand gesture:

[0066] The pre-processed meta-action gesture data is subjected to a distance-dimensional fast Fourier transform to obtain the RTM image of the meta-action gesture, and then a fast Fourier transform is performed along the slow time dimension to obtain the Doppler spectrum of the meta-action gesture. The Doppler spectrum of the meta-action gesture signal is subjected to dimensionality reduction processing, and the two-dimensional Doppler matrix is reduced to a one-dimensional vector while retaining the key waveform information. In this embodiment, 1 second corresponds to 15 frames, 1 frame corresponds to 96 chirps, and 4 seconds of signal is retained, that is, 60 frames of data are collected, so the gesture Doppler matrix size is 96×60. By taking the column sum of the first 48 rows of the two-dimensional matrix, we can get , and the sum of the columns in the last 48 rows is , extract effective features by comparing the front and back halves, that is, comparing and Size, if , it indicates that the main energy is concentrated in the positive frequency band, retaining The value of and take the opposite number, if , it indicates that the main energy is concentrated in the negative frequency band, retaining The value of is obtained, and a one-dimensional vector is obtained. After smoothing the obtained one-dimensional vector, its curve characteristics are displayed through an image. Traverse the data of the one-dimensional vector and locate the first sign change point from negative to positive. At the same time, ensure that there is a sign change point from positive to negative before the sign change point to confirm the integrity of the first lower envelope in the curve. Mark this sign change point as the starting point T of the gesture signal. Subsequently, the curve image of the one-dimensional vector is reversed, and the same method is used to locate the sign change point and mark the end point F of the gesture signal. Finally, the data is reversed to restore to the original order, and the precise marking of the starting point T and the end point F of the gesture signal is completed, and the first gesture Doppler spectrum feature between the two points is extracted.

[0067] S33. Extract curvature features of the first gesture trajectory:

[0068] The RTM image obtained by performing a distance-dimensional fast Fourier transform on the meta-motion gesture data is saved. By performing a detailed analysis on the RTM image of the meta-motion gesture, the key features in the image that can reflect the trajectory of the gesture are extracted. Based on these features, the curvature information of the meta-motion gesture is further calculated. Specifically, by analyzing the distance information of the gesture relative to the radar at each moment in the RTM image, the distance values corresponding to the gesture at different time points are obtained. In this embodiment, a total of 60 frames of distance information are extracted from the meta-motion gesture to construct a complete position sequence: ;in, Indicates the frame The distance value of the meta-motion gesture relative to the radar is calculated based on the position sequence of the meta-motion gesture at different time points. The calculation of curvature relies on the analysis of the change relationship between consecutive points. Specifically, the curvature is calculated using the discrete point data of the gesture trajectory using the following formula: ;in, and Represent the velocity and acceleration of the meta-action gesture trajectory position respectively, and then obtain a curvature sequence vector with a length of 60, each element of which is the curvature value at that time point: ;in, Indicates the frame The curvature value of the gesture trajectory of the Zhongyuan action, It represents the curvature vector of the meta-action gesture trajectory. Using the starting point of the meta-action gesture signal and end point At the position of the time axis, the influence of non-gesture signals generated by hand-raising and hand-releasing actions is eliminated, and the curvature features of the meta-action gesture trajectory are extracted.

[0069] S34. Combine the first gesture feature to obtain a second gesture feature:

[0070] The extracted first gesture features are time-normalized and unified to the same time scale (4 seconds) to ensure the temporal consistency of the data. According to the motion characteristics of the gestures, the Doppler spectrum features of the meta-action gestures are accelerated to simulate the speed change characteristics of different gestures in actual motion. The Doppler spectra of the meta-action gestures are combined in sequence to generate new gesture data, such as Figure 4As shown, by flipping the Doppler spectrum of meta-action gesture "n," a new gesture "u" is obtained. By flipping the Doppler spectrum of meta-action gesture "1," a new gesture "1 (up)" is obtained. Gesture "1 (up)" is used to assist in synthesizing new gestures. Gesture "h" is obtained by accelerating and combining the Doppler spectra of meta-action gestures "1" and "n." Gesture "d" is obtained by accelerating and combining the Doppler spectra of meta-action gestures "c" and "1 (up)." Gesture "b" is obtained by accelerating and combining the Doppler spectra of meta-action gestures "1 (up)" and "3." Gesture "m" is obtained by accelerating and combining the Doppler spectra of meta-action gestures "n" and "n." Similarly, by analyzing the spatial motion trajectories of meta-action gestures and combining the curvature features of their trajectories, a second gesture trajectory curvature feature is obtained. In this experiment, only four meta-action gesture data were collected to generate nine gesture data types (including the four collected meta-action gestures and the five synthesized new gestures), effectively expanding the gesture collection.

[0071] S35. Fusion of two gesture features: Based on the second gesture Doppler spectrum and the second gesture trajectory curvature, perform weighted fusion. The gesture Doppler spectrum feature is a two-dimensional matrix data, and the gesture trajectory curvature feature is a one-dimensional vector:

[0072] 1) Doppler spectrum feature matrix ,in is the number of time frames (i.e. 60 frames), is the number of chirps in each frame (i.e., 96), which is the number of frequency units obtained after fast Fourier transform;

[0073] 2) Gesture trajectory curvature feature vector ,in is the number of time frames (i.e. 60 frames).

[0074] First, the gesture trajectory curvature feature vector is expanded to have the same dimension as the Doppler spectrum feature matrix, and then a two-dimensional matrix is obtained. , the expanded curvature matrix and Doppler spectrum matrix Perform weighted fusion: ;in, is the fused feature matrix, and are the weight coefficients of Doppler feature and curvature feature respectively, is 0.7, is 0.3, where the dot product represents the multiplication of the matrix elements one by one.

[0075] S36. Training network model:

[0076] The weighted fused gesture feature matrix is used as the input of the perception network. There are 9 gestures in the gesture dataset (4 meta-action gestures and 5 synthesized new gestures). The labels are the number of gesture categories. The difference between the model prediction and the true label is quantified by calculating the loss value, where the loss function is defined as: ; in represents the total number of categories of labeled samples, represents the one-hot encoded label of the sample, Represents the predicted output probability of the perception network. The gesture data passes through the convolution layer Conv, the activation layer ReLU, and the fully connected layer Fc, and finally outputs the training result.

[0077] S37, Online Gesture Recognition:

[0078] By actually collecting nine gestures, namely "1", "c", "n", "3", "B", "d", "h", "m" and "u", a gesture dataset to be recognized is constructed. The noise interference of the gesture signals to be recognized is removed, the RTM images of the gesture samples to be recognized are obtained, the Doppler spectrum features and trajectory curvature features of the gesture samples to be recognized are extracted, the two features are fused, and the features are input into the trained perception network for gesture recognition.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for wireless gesture recognition based on a few samples of meta-actions, characterized by: include: Offline training phase and online recognition phase; During the offline training phase, millimeter-wave radar equipment is used to collect meta-gesture data to form a gesture set, and each meta-gesture data in the gesture set is preprocessed to eliminate interference from meta-gesture signal noise; Extracting the first gesture Doppler spectrum features and the first gesture trajectory curvature features from the preprocessed meta-action gesture signal; A second gesture Doppler spectrum feature and a second gesture trajectory curvature feature are synthesized based on the first gesture Doppler spectrum feature and the first gesture trajectory curvature feature; The weighted fusion method is used to fuse the Doppler spectrum features of the second gesture and the curvature features of the second gesture trajectory; The fusion features of each gesture in the gesture action set are sent to the perception network for training; In the online recognition stage, gesture data is collected to construct gesture samples to be recognized. Noise interference in the gesture signals is removed, and the range-time image of the gesture samples to be recognized is obtained. The Doppler spectrum features and trajectory curvature features of the gesture samples to be recognized are extracted. The two gesture features are effectively fused using a weighted fusion method and input into the trained perception network to achieve wireless gesture recognition. The extraction of the first hand gesture Doppler spectrum features includes: Based on the preprocessed meta-motion gesture signal, a distance-dimensional fast Fourier transform is performed on it to obtain a distance-time image of the meta-motion gesture; the distance-time image is fast Fourier transformed along the slow time dimension to obtain a Doppler spectrum of the meta-motion gesture; By reducing the dimensionality of the Doppler spectrum of the meta-gesture, The two-dimensional matrix is converted into a one-dimensional vector while retaining the Doppler spectrum waveform signal of the gesture; Traverse the obtained waveform signal and locate the first sign change point from negative to positive and ensure Meet one condition: sign change point There is a sign change point from positive to negative before to determine the integrity of the first lower envelope of the curve; reverse the one-dimensional vector to find the sign change point ; restore the data to its original order and Before and The gesture Doppler data after the point is set to zero, and only the Doppler signal between the two points is retained, thereby extracting the first gesture Doppler spectrum feature.

2. The method for wireless gesture recognition based on a small number of samples based on meta-actions according to claim 1, characterized in that: Preprocessing the gesture data of each element in the gesture action set includes: Obtain multiple channel radar raw data and process the radar data frame by frame; The signal of each receiving channel in each frame is windowed in the time domain, and the DC component and static interference in the signal are eliminated by de-averaging.

3. The method for wireless gesture recognition based on a small number of samples based on meta-actions according to claim 1, characterized in that: By reducing the dimensionality of the Doppler spectrum of the meta-gesture, The two-dimensional matrix is converted into a one-dimensional vector while retaining the Doppler spectrum waveform signal of the gesture, including: For the two-dimensional matrix Line and back The sum of the rows and columns is obtained and , ; By comparison and The numerical value of is used to selectively retain the energy value and obtain a one-dimensional vector; By performing median filtering and smoothing on the one-dimensional vector, a waveform signal reflecting the motion characteristics of the gesture is obtained.

4. The method for wireless gesture recognition based on a small number of samples based on meta-actions according to claim 1, characterized in that: Extracting the curvature features of the first gesture trajectory, including: By performing distance-dimensional fast Fourier transform on the meta-motion gesture signal, the distance-time image of the meta-motion gesture is obtained; By analyzing the distance information of the gesture relative to the radar at a certain moment in the distance-time image, the distance values corresponding to the gesture at different time points are obtained to form a position sequence : ;in, Indicates the meta-action gesture at a time point The distance value relative to the radar at time ; Using the position sequence , calculate the curvature of the meta-action gesture trajectory; Based on the obtained sign change point and sign change point At the position of the time axis, the curvature features of the meta-action gesture are extracted, and the curvature data of the meta-action gesture trajectory between the two sign-changing points are retained to obtain the curvature features of the meta-action gesture trajectory.

5. The method for wireless gesture recognition based on a small number of samples based on meta-actions according to claim 4, characterized in that: Using the discrete points of the gesture trajectory, the curvature is calculated using the following formula : ;in, and They represent the first and second derivatives of the trajectory, namely velocity and acceleration; The curvature is calculated to obtain a length of The curvature sequence vector : ;in, Indicates the meta-action gesture at a time point The curvature value.

6. The method for wireless gesture recognition based on a small number of samples based on meta-actions according to claim 1, characterized in that: The second gesture Doppler spectrum feature and the second gesture trajectory curvature feature are synthesized based on the first gesture Doppler spectrum feature and the first gesture trajectory curvature feature, including: Performing time normalization processing on the Doppler spectrum features of the first gesture and the curvature features of the first gesture trajectory; Generating a second gesture Doppler spectrum feature, including: accelerating the normalized first gesture Doppler spectrum feature to simulate the speed variation characteristics of different gestures in actual motion; synthesizing and splicing the Doppler spectra of the meta-action gesture along a time axis according to the composition sequence and motion characteristics of the target gesture; after the Doppler spectra are spliced, performing time normalization processing on the generated second gesture Doppler spectrum so that its time length is consistent with that of the meta-action gesture, thereby obtaining the second gesture Doppler spectrum feature; Generating a second gesture trajectory curvature feature includes: analyzing the motion trajectory of the meta-action gesture in space and extracting its key features; accelerating the first gesture trajectory curvature feature by compressing the time axis of the gesture trajectory curvature graph; combining multiple meta-action gesture trajectory curvature graphs according to time sequence and spatial relationship to form a second gesture trajectory curvature graph, and normalizing the graph so that its time length is consistent with the meta-action gesture, thereby obtaining the second gesture trajectory curvature feature.

7. The method for wireless gesture recognition based on a small number of samples based on meta-actions according to any one of claims 1 to 6, characterized in that: The input of the perception network is a two-dimensional feature matrix whose label is the number of gesture categories. The difference between the model prediction and the true label is quantified by calculating the loss value, where the loss function Defined as: ; in, represents the total number of categories of labeled samples, represents the one-hot encoded label of the sample, Represents the predicted output probability of the perception network; the gesture feature matrix passes through the convolution layer, activation layer and full connection layer in the perception network respectively to train the perception network model.

Images