A millimeter wave radar gesture control method and related device

Abstract

The application discloses a millimeter wave radar gesture control method and related devices, and relates to the technical field of data analysis. The method comprises: performing clutter suppression processing on each frame of gesture signal data; performing effective gesture signal extraction and signal reconstruction on each frame of gesture signal data after clutter suppression processing based on modal component screening, to obtain a plurality of frames of target gesture signal data; performing distance-angle graph analysis and enhanced radar spectrum graph analysis on the plurality of frames of target gesture signal data, and performing feature extraction by using a multiple signal classification algorithm, to obtain target feature data; determining target weights of each frame of target gesture signal data, and performing gesture recognition by using a multi-task learning gesture recognition model in combination with the target feature data; and matching operation instructions based on a gesture recognition result to perform device control. The application can improve the reliability of gesture recognition and realize more accurate gesture control.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to a millimeter-wave radar gesture control method and related device. Background Technology

[0002] In recent years, with the development of intelligent technology, human-computer interaction technology has been gradually applied to daily life. Gesture recognition, as one of the common means of interaction, allows users to interact with machines in a more natural way through the movement of their palms or fingers, thereby achieving convenient and quick gesture control. Noise reduction and effective signal extraction of radar gesture signals are crucial steps in gesture recognition. Current methods mainly employ wavelet analysis, but this approach has poor noise reduction performance and easily leads to the reduction of some effective signals, resulting in unreliable gesture signal data and affecting the accuracy of gesture recognition. Furthermore, current methods typically rely solely on single range Doppler image features for gesture recognition, but the feature data used is not comprehensive enough to capture sufficient detailed features, leading to insufficient accuracy in gesture recognition and affecting the reliability of gesture control. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a millimeter-wave radar gesture control method and related device, which can improve the reliability of gesture recognition and achieve higher precision gesture control.

[0004] To address the aforementioned technical problems, this invention provides a millimeter-wave radar gesture control method, the method comprising:

[0005] Acquire several frames of gesture signal data from millimeter-wave radar, and perform clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame.

[0006] Based on modal component screening, effective hand gesture signal extraction and signal reconstruction are performed on the clutter-suppressed hand gesture signal data of each frame to obtain several frames of target hand gesture signal data.

[0007] Range angle map analysis and enhanced radar spectrum analysis are performed on several frames of target gesture signal data to obtain target range angle map and enhanced radar spectrum. Based on the target range angle map and enhanced radar spectrum, feature extraction is performed using a multi-signal classification algorithm to obtain target feature data.

[0008] The target weights of the target gesture signal data in each frame are determined, and gesture recognition is performed using a multi-task learning gesture recognition model based on the target weights and target feature data to obtain gesture recognition results.

[0009] The corresponding operation command is matched based on the gesture recognition result, and the device is controlled based on the operation command.

[0010] Optionally, the step of performing clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame includes:

[0011] The phase of each frame of gesture signal data is obtained, and the corresponding mean pulse is determined based on the phase of each frame of gesture signal data;

[0012] The filtering and smoothing operator for each frame of gesture signal data is determined, and clutter suppression processing is performed on each frame of gesture signal data based on the mean pulse and the filtering and smoothing operator to obtain clutter-suppressed gesture signal data for each frame.

[0013] Optionally, the step of effectively extracting and reconstructing the gesture signal data after clutter suppression processing of each frame based on modal component filtering to obtain several frames of target gesture signal data includes:

[0014] The gesture signal data after clutter suppression processing in each frame is decomposed to obtain several corresponding modal components;

[0015] The energy value of each modal component is determined, and effective gesture signal is extracted from the clutter-suppressed gesture signal data of each frame based on the energy value of each modal component to obtain effective gesture signal data;

[0016] Noise signal data is determined based on the effective gesture signal data, and the noise signal data is denoised based on the noise suppression network to obtain denoised signal data. The noise suppression network adopts a time-frequency loss function and a multi-head self-attention mechanism.

[0017] Based on the denoised signal data and the effective gesture signal data, signal reconstruction is performed to obtain several frames of target gesture signal data.

[0018] Optionally, the decomposition of the gesture signal data after clutter suppression processing in each frame to obtain several corresponding modal components includes:

[0019] Determine the range of decomposition parameters, and use the objective function to determine the target decomposition parameters based on the range of decomposition parameters;

[0020] The target envelope of the clutter-suppressed gesture signal data for each frame is determined, and the clutter-suppressed gesture signal data for each frame is decomposed based on the target envelope and target decomposition parameters to obtain several corresponding modal components.

[0021] Optionally, the step of performing range angle map analysis and enhanced radar spectrum analysis on several frames of target gesture signal data to obtain target range angle map and enhanced radar spectrum, and extracting target feature data based on the target range angle map and enhanced radar spectrum using a multi-signal classification algorithm, includes:

[0022] Determine the distance dimension and Doppler dimension of the target gesture signal data in each frame, and generate an initial distance angle map based on the distance dimension and Doppler dimension;

[0023] The initial distance-angle map is subjected to translation transformation and data augmentation processing to obtain the data-augmented initial distance-angle map;

[0024] A continuous range angle frame sequence is determined based on the initial range angle map after data augmentation, and a target range angle map is generated based on the continuous range angle frame sequence;

[0025] A micro-Doppler spectrum is generated based on several frames of target gesture signal data, and histogram equalization and homomorphic filtering are performed on the micro-Doppler spectrum to obtain an enhanced radar spectrum.

[0026] Based on a multi-signal classification algorithm, an angle-time map is determined using several frames of target gesture signal data. Based on the target distance-angle map, enhanced radar spectrum map, and angle-time map, feature extraction is performed to obtain target feature data.

[0027] Optionally, determining the target weights of the target gesture signal data in each frame, and performing gesture recognition using a multi-task learning gesture recognition model based on the target weights and target feature data to obtain gesture recognition results, includes:

[0028] The target weight of each frame of target gesture signal data is determined based on time sorting.

[0029] The target feature data is input into the multi-task learning gesture recognition model to determine the gesture type classification probability corresponding to the target gesture signal data in each frame;

[0030] Gesture recognition is performed based on the gesture type classification probability and the target weight to obtain the gesture recognition result.

[0031] Optionally, the step of matching the corresponding operation command based on the gesture recognition result and controlling the device based on the operation command includes:

[0032] Based on the gesture recognition results, the corresponding operation instructions are matched in the instruction database;

[0033] The operation instruction is sent to the target device, the target device parses the operation instruction to obtain parsed instruction information, and the target device executes the parsed instruction information.

[0034] In addition, the present invention provides a millimeter-wave radar gesture control device, the device comprising:

[0035] Signal clutter suppression module: used to acquire several frames of gesture signal data from millimeter-wave radar, and to perform clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame.

[0036] Signal processing module: used to extract and reconstruct effective gesture signals from the clutter-suppressed gesture signal data of each frame based on modal component filtering, and obtain several frames of target gesture signal data;

[0037] Feature extraction module: used to perform range angle map analysis and enhanced radar spectrum analysis on several frames of target gesture signal data to obtain target range angle map and enhanced radar spectrum, and to perform feature extraction based on the target range angle map and enhanced radar spectrum using a multi-signal classification algorithm to obtain target feature data;

[0038] Gesture recognition module: used to determine the target weight of the target gesture signal data in each frame, and to perform gesture recognition using a multi-task learning gesture recognition model based on the target weight and target feature data, so as to obtain the gesture recognition result;

[0039] Device control module: used to match the corresponding operation command based on the gesture recognition result, and to control the device based on the operation command.

[0040] In addition, the present invention provides an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the above-described millimeter-wave radar gesture control method.

[0041] In addition, the present invention provides a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the millimeter-wave radar gesture control method described above.

[0042] In this embodiment of the invention, clutter suppression processing is performed on each frame of gesture signal data. Based on modal component filtering, effective gesture signal extraction and signal reconstruction are performed on the clutter-suppressed gesture signal data of each frame to obtain several frames of target gesture signal data, making the obtained target gesture signal data more reliable and avoiding affecting the accuracy of subsequent feature extraction. Range angle map analysis and enhanced radar spectrum analysis are performed on several frames of target gesture signal data to obtain target range angle maps and enhanced radar spectra. Feature extraction is then performed using a multi-signal classification algorithm, which can obtain more comprehensive and accurate feature data, while capturing sufficient detailed features, overcoming the problem of insufficient feature information leading to low gesture recognition accuracy in the prior art. The target weight of each frame of target gesture signal data is determined. Based on the target weight and target feature data, a multi-task learning gesture recognition model is used for gesture recognition. Based on the gesture recognition results, corresponding operation commands are matched for device control, which can improve the reliability of gesture recognition and achieve higher precision gesture control. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the millimeter-wave radar gesture control method in an embodiment of the present invention.

[0045] Figure 2 This is a flowchart illustrating a millimeter-wave radar gesture control method according to another embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of the structural composition of the millimeter-wave radar gesture control device in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1

[0050] Please see Figure 1 , Figure 1 This is a flowchart illustrating the millimeter-wave radar gesture control method according to an embodiment of the present invention. The method includes:

[0051] S11: Acquire several frames of gesture signal data from the millimeter-wave radar, and perform clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame.

[0052] In the specific implementation of this invention, several frames of gesture signal data from millimeter-wave radar are acquired, the phase of each frame of gesture signal data is acquired, and the corresponding mean pulse is determined based on the phase of each frame of gesture signal data. The filtering and smoothing operator for each frame of gesture signal data is determined, and clutter suppression processing is performed on each frame of gesture signal data based on the mean pulse and the filtering and smoothing operator to obtain clutter-suppressed gesture signal data for each frame. Clutter suppression processing can effectively eliminate interference signals and improve the reliability of gesture signal data.

[0053] S12: Based on modal component screening, effective hand gesture signal extraction and signal reconstruction are performed on the clutter suppression processed hand gesture signal data of each frame to obtain several frames of target hand gesture signal data.

[0054] In the specific implementation of this invention, the energy value of each modal component is determined. Based on the energy value of each modal component, a clustering algorithm is used to extract effective gesture signals from the clutter-suppressed gesture signal data of each frame to obtain effective gesture signal data. Based on the effective gesture signal data, noise signal data is determined, and noise signal data is denoised using a noise suppression network to obtain denoised signal data. Based on the denoised signal data and the effective gesture signal data, signal reconstruction is performed to obtain several frames of target gesture signal data, which can effectively extract the required target gesture signal data and provide sufficiently accurate data support for subsequent feature extraction.

[0055] S13: Perform range angle map analysis and enhanced radar spectrum analysis on several frames of target gesture signal data to obtain target range angle map and enhanced radar spectrum, and extract features based on the target range angle map and enhanced radar spectrum using a multi-signal classification algorithm to obtain target feature data;

[0056] In the specific implementation of this invention, the distance dimension and Doppler dimension of each frame of target gesture signal data are determined, and an initial distance angle map is generated based on the distance dimension and Doppler dimension; the initial distance angle map is subjected to translation transformation and data augmentation processing to obtain an initial distance angle map after data augmentation processing; a continuous distance angle frame sequence is determined based on the initial distance angle map after data augmentation processing, and a target distance angle map is generated based on the continuous distance angle frame sequence; a micro-Doppler spectrum is generated based on several frames of target gesture signal data, and histogram equalization and homomorphic filtering are performed on the micro-Doppler spectrum to obtain an enhanced radar spectrum; an angle time map is determined using several frames of target gesture signal data based on a multiple signal classification algorithm, and feature extraction is performed based on the target distance angle map, the enhanced radar spectrum, and the angle time map to obtain target feature data, making the extracted feature data more comprehensive and accurate, and effectively improving the reliability of gesture recognition.

[0057] S14: Determine the target weight of the target gesture signal data in each frame, and perform gesture recognition using a multi-task learning gesture recognition model based on the target weight and target feature data to obtain the gesture recognition result;

[0058] In the specific implementation of this invention, the target weight of each frame of target gesture signal data is determined based on time sorting; the target feature data is input into the multi-task learning gesture recognition model to determine the gesture type classification probability corresponding to each frame of target gesture signal data; gesture recognition is performed based on the gesture type classification probability combined with the target weight, which effectively improves the accuracy of gesture recognition.

[0059] S15: Match the corresponding operation command based on the gesture recognition result, and control the device based on the operation command.

[0060] In the specific implementation of this invention, the corresponding operation command is matched in the command database based on the gesture recognition result; the operation command is sent to the target device, the target device parses the operation command to obtain the parsed command information, and the target device executes the parsed command information, so that the target device can accurately execute the gesture command made by the user, thereby improving the user's gesture control operation experience.

[0061] In this embodiment of the invention, clutter suppression processing is performed on each frame of gesture signal data. Based on modal component filtering, effective gesture signal extraction and signal reconstruction are performed on the clutter-suppressed gesture signal data of each frame to obtain several frames of target gesture signal data, making the obtained target gesture signal data more reliable and avoiding affecting the accuracy of subsequent feature extraction. Range angle map analysis and enhanced radar spectrum analysis are performed on several frames of target gesture signal data to obtain target range angle maps and enhanced radar spectra. Feature extraction is then performed using a multi-signal classification algorithm, which can obtain more comprehensive and accurate feature data, while capturing sufficient detailed features, overcoming the problem of insufficient feature information leading to low gesture recognition accuracy in the prior art. The target weight of each frame of target gesture signal data is determined. Based on the target weight and target feature data, a multi-task learning gesture recognition model is used for gesture recognition. Based on the gesture recognition results, corresponding operation commands are matched for device control, which can improve the reliability of gesture recognition and achieve higher precision gesture control.

[0062] Example 2

[0063] Please see Figure 2 , Figure 2 This is a flowchart illustrating a millimeter-wave radar gesture control method according to another embodiment of the present invention, the method comprising:

[0064] S201: Acquire several frames of gesture signal data from the millimeter-wave radar, and perform clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame.

[0065] In the specific implementation of this invention, the step of performing clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame includes: acquiring the phase of each frame of gesture signal data and determining the corresponding mean pulse based on the phase of each frame of gesture signal data; determining the filtering and smoothing operator for each frame of gesture signal data, and performing clutter suppression processing on each frame of gesture signal data based on the mean pulse and the filtering and smoothing operator to obtain clutter-suppressed gesture signal data for each frame.

[0066] Specifically, millimeter-wave radar emits radar signals to the surrounding area. When a user makes a gesture, the radar signal encounters the gesture and generates an echo signal. Collecting several frames of echo signal data corresponding to the gesture constitutes the gesture signal data. The phase of each frame of gesture signal data is obtained. By combining the frequency of the radar wave with the phase formula, the phase of the gesture signal data can be obtained. Based on the phase of each frame of gesture signal data, the corresponding mean pulse is determined. A mean cancellation algorithm can be used to calculate the mean pulse by averaging the phases of each frame of gesture signal data.

[0067] A filtering and smoothing operator is determined for each frame of gesture signal data. The filtering and smoothing operator is used to eliminate noise or outliers in the signal. Based on the mean pulse and the filtering and smoothing operator, clutter suppression processing is performed on each frame of gesture signal data. The gesture signal data is then canceled according to the mean pulse. The cancellation processing can effectively eliminate interference caused by objects or low-frequency objects. By filtering and smoothing the canceled gesture signal data of each frame through the filtering and smoothing operator, clutter suppression filtering processing can be achieved, and clutter-suppressed gesture signal data of each frame can be obtained.

[0068] S202: Decompose the gesture signal data after clutter suppression processing of each frame to obtain several corresponding modal components;

[0069] In the specific implementation of this invention, the step of decomposing the clutter-suppressed gesture signal data of each frame to obtain several corresponding modal components includes: determining the decomposition parameter range, and determining the target decomposition parameters based on the decomposition parameter range using an objective function; determining the target envelope of the clutter-suppressed gesture signal data of each frame, and decomposing the clutter-suppressed gesture signal data of each frame based on the target envelope and the target decomposition parameters to obtain several corresponding modal components.

[0070] Specifically, the decomposition parameter range is determined, including the number of modal components. Based on the decomposition parameter range, the target decomposition parameters are determined using an objective function, with the minimum energy loss coefficient as the objective function. An optimization algorithm is used to determine the target decomposition parameters using the decomposition parameter range and the minimum energy loss function. The optimization algorithm can be the locust optimization algorithm, which finds the optimal decomposition parameters as the target decomposition parameters in an iterative loop.

[0071] The target envelope of each frame of clutter-suppressed gesture signal data is determined. This can be obtained through polynomial fitting interpolation or cubic spline interpolation. Based on the target envelope and target decomposition parameters, the clutter-suppressed gesture signal data of each frame is decomposed. The frequency points corresponding to the maxima within a preset range of the gesture signal data are determined. The midpoints of adjacent frequency points are used as boundary points. The target envelope is divided into intervals based on the boundary points, and the wavelet function and scaling function of each interval are determined. Based on the wavelet function, scaling function, and target decomposition parameters, a neural network model is used to decompose the clutter-suppressed gesture signal data of each frame to obtain several corresponding modal components. This process can preserve the detailed features of the signal data while avoiding interference from random noise in the decomposition.

[0072] S203: Determine the energy value of each modal component, and extract the effective gesture signal from the clutter-suppressed gesture signal data of each frame based on the energy value of each modal component to obtain the effective gesture signal data;

[0073] In the specific implementation of this invention, the energy density and average period of each modal component are calculated, and the energy value of each modal component is determined based on the product of the energy density and the average period. Based on the energy values ​​of each modal component, valid gesture signals are extracted from the clutter-suppressed gesture signal data of each frame. Starting from the energy value of the first modal component, the energy value of the first modal component is multiplied by a preset multiple to obtain the product result. This product result is compared with the energy value of the next modal component. If the energy value of the next modal component is greater than the product result, this modal component is used as the boundary component. The modal components after the boundary component are considered valid gesture signals, thus obtaining valid gesture signal data.

[0074] S204: Based on the effective gesture signal data, noise signal data is determined, and the noise signal data is denoised based on the noise suppression network to obtain denoised signal data. The noise suppression network adopts a time-frequency loss function and a multi-head self-attention mechanism.

[0075] In the specific implementation of this invention, modal components other than the effective gesture signal data are treated as noise signal data. The noise signal data is input into a noise suppression network for denoising processing to obtain denoised signal data. The noise suppression network adopts a time-frequency loss function and a multi-head self-attention mechanism. The multi-head attention mechanism can enhance the performance of the noise suppression network and capture information across different distances. The time-frequency loss function can better learn the physical meaning of the signal in the frequency domain.

[0076] S205: Based on the denoised signal data and effective gesture signal data, perform signal reconstruction to obtain several frames of target gesture signal data;

[0077] In the specific implementation of this invention, the signal is reconstructed using the denoised signal data and effective gesture signal data through the corresponding inverse transform algorithm to obtain several frames of target gesture signal data, which can improve the signal-to-noise ratio and ensure the stability and reliability of radar signal data.

[0078] S206: Perform range angle map analysis and enhanced radar spectrum analysis on several frames of target gesture signal data to obtain target range angle map and enhanced radar spectrum, and extract features based on the target range angle map and enhanced radar spectrum using a multi-signal classification algorithm to obtain target feature data;

[0079] In the specific implementation of this invention, the step of performing range-angle map analysis and enhanced radar spectrum analysis on several frames of target gesture signal data to obtain target range-angle maps and enhanced radar spectra, and extracting features based on the target range-angle maps and enhanced radar spectra using a multi-signal classification algorithm to obtain target feature data, includes: determining the range dimension and Doppler dimension of each frame of target gesture signal data, and generating an initial range-angle map based on the range dimension and Doppler dimension; performing translation transformation and data enhancement processing on the initial range-angle map to obtain an initial range-angle map after data enhancement processing; determining a continuous range-angle frame sequence based on the initial range-angle map after data enhancement processing, and generating a target range-angle map based on the continuous range-angle frame sequence; generating a micro-Doppler spectrum based on several frames of target gesture signal data, and performing histogram equalization and homomorphic filtering on the micro-Doppler spectrum to obtain an enhanced radar spectrum; determining an angle-time map based on several frames of target gesture signal data using a multi-signal classification algorithm, and extracting features based on the target range-angle map, enhanced radar spectrum, and angle-time map to obtain target feature data.

[0080] Specifically, the distance dimension and Doppler dimension of the target gesture signal data in each frame are determined. Based on the target gesture signal data in each frame, the corresponding intermediate frequency (IF) signal is determined. The IF signal includes the distance, speed, and angle information of the gesture. The IF signal is converted to the frequency domain using a Fast Fourier Transform (FFT) to obtain the distance dimension and Doppler dimension. The distance dimension expresses the distance information of the gesture, and the Doppler dimension expresses the speed information of the gesture. An initial distance-angle map is generated based on the distance dimension and Doppler dimension. A first three-dimensional matrix is ​​generated by combining the distance dimension and Doppler dimension with the antenna dimension of the millimeter-wave radar. A FFT is performed on the first three-dimensional matrix along the antenna dimension to obtain a second three-dimensional matrix. The angle information of the gesture is added to the second three-dimensional matrix. The second three-dimensional matrix is ​​accumulated along the Doppler dimension, and noise is eliminated from the second three-dimensional matrix to obtain the initial distance-angle map.

[0081] The initial distance angle map is subjected to translation transformation and data augmentation processing. The initial distance angle map is translated to obtain a first distance angle sequence at different positions. The first distance angle sequence is frame interpolated and downsampled to obtain a second distance angle sequence at different speeds. The second distance angle sequence is inverted according to symmetry to obtain a third distance angle sequence that is symmetrical to the second distance angle sequence. The second distance angle sequence and the third distance angle sequence are combined to realize the data augmentation of the distance angle map, that is, to obtain the initial distance angle map after data augmentation processing.

[0082] Based on the initial distance angle map after data augmentation, a continuous distance angle frame sequence is determined. The start time and end time of the gesture in the initial distance angle map after data augmentation are determined by a neural network model. The continuous distance angle frame sequence is determined based on the start time and end time of the gesture. The target distance angle map is generated based on the continuous distance angle frame sequence. In other words, the final target distance angle map is composed of the continuous distance angle frame sequence.

[0083] A micro-Doppler spectrum is generated based on several frames of target gesture signal data. One-dimensional and two-dimensional fast Fourier transforms are performed on the target gesture signal data in the time dimension to obtain the micro-Doppler spectrum. Histogram equalization and homomorphic filtering are then applied to the micro-Doppler spectrum. Grayscale spectral processing is then performed to obtain a grayscale micro-Doppler spectrum. Homomorphic filtering is then applied to the grayscale micro-Doppler spectrum to achieve brightness compression and contrast enhancement. Histogram equalization is then applied to the homomorphically filtered grayscale micro-Doppler spectrum to expand the dynamic range of grayscale pixel values, increase grayscale differences, and homogenize the distribution of pixel values, making the details of the grayscale micro-Doppler spectrum clearer and richer, thereby achieving spectral detail enhancement and obtaining an enhanced radar spectrum.

[0084] An angle-time map is determined using several frames of target gesture signal data based on a multiple signal classification algorithm. Multiple signal classification is a type of spatial spectrum estimation algorithm. Its idea is to use the covariance matrix of the received data for eigenvalue decomposition, separating the signal subspace and noise subspace. The orthogonality between the signal direction vector and the noise subspace is used to construct a spatial scanning spectrum, and a global search for spectral peaks is performed to achieve signal parameter estimation. The angle of the gesture target is estimated using the multiple signal classification algorithm. An angle-time map is generated by combining the estimated angles of the gesture target in chronological order. Feature extraction is then performed based on the target distance-angle map, enhanced radar spectrum, and angle-time map. A corresponding neural network feature extractor can be used to extract features from the target distance-angle map, enhanced radar spectrum, and angle-time map to obtain gesture feature data, i.e., target feature data.

[0085] S207: Determine the target weight of the target gesture signal data in each frame, and perform gesture recognition using a multi-task learning gesture recognition model based on the target weight and target feature data to obtain the gesture recognition result;

[0086] In the specific implementation of this invention, determining the target weight of each frame of target gesture signal data and performing gesture recognition based on the target weight and target feature data using a multi-task learning gesture recognition model to obtain gesture recognition results includes: determining the target weight of each frame of target gesture signal data based on time sorting; inputting target feature data into the multi-task learning gesture recognition model to determine the gesture type classification probability corresponding to each frame of target gesture signal data; and performing gesture recognition based on the gesture type classification probability combined with the target weight to obtain gesture recognition results.

[0087] Specifically, the target weight of each frame of target gesture signal data is determined based on time sorting. The newer the radar gesture signal, the more it reflects the current gesture, and therefore the greater the target weight of the gesture signal. Thus, the newer the target gesture signal data in time sorting, the greater the target weight.

[0088] The target feature data is input into the multi-task learning gesture recognition model to determine the gesture type classification probability corresponding to the target gesture signal data in each frame. The multi-task learning gesture recognition model is a converged model obtained by inputting the sample dataset into the deep neural network model for training. The multi-task learning gesture recognition model includes a convolutional neural network layer, a long short-term memory network layer, and a fully connected layer.

[0089] Gesture recognition is performed based on the gesture type classification probability and the target weight. The target probability of the corresponding gesture type is calculated according to the gesture type classification probability and target weight of each frame of target gesture signal data. The gesture type with the highest target probability is taken as the final recognition result, that is, the gesture recognition result is obtained. By comprehensively considering the weight and classification probability, the accuracy of gesture recognition can be improved.

[0090] S208: Match the corresponding operation command based on the gesture recognition result, and control the device based on the operation command.

[0091] In a specific implementation of the present invention, the step of matching the corresponding operation instruction based on the gesture recognition result and controlling the device based on the operation instruction includes: matching the corresponding operation instruction in the instruction database based on the gesture recognition result; sending the operation instruction to the target device; the target device parsing the operation instruction to obtain parsed instruction information; and the target device executing the parsed instruction information.

[0092] Specifically, based on the gesture recognition results, corresponding operation commands are matched in the command database. Different operation commands are matched according to different gesture types. The command database contains operations corresponding to different gesture types; for example, if the gesture type is "move right," the operation command is "switch page to the right." The operation command is sent to the target device, such as a smart screen. The target device parses the operation command. The target device's central processing unit parses the operation command to obtain the execution command information corresponding to the operation command, i.e., obtains the parsed command information, such as "page switching." The target device executes the parsed command information to achieve high-precision gesture control.

[0093] In this embodiment of the invention, clutter suppression processing is performed on each frame of gesture signal data. Based on modal component filtering, effective gesture signal extraction and signal reconstruction are performed on the clutter-suppressed gesture signal data of each frame to obtain several frames of target gesture signal data, making the obtained target gesture signal data more reliable and avoiding affecting the accuracy of subsequent feature extraction. Range angle map analysis and enhanced radar spectrum analysis are performed on several frames of target gesture signal data to obtain target range angle maps and enhanced radar spectra. Feature extraction is then performed using a multi-signal classification algorithm, which can obtain more comprehensive and accurate feature data, while capturing sufficient detailed features, overcoming the problem of insufficient feature information leading to low gesture recognition accuracy in the prior art. The target weight of each frame of target gesture signal data is determined. Based on the target weight and target feature data, a multi-task learning gesture recognition model is used for gesture recognition. Based on the gesture recognition results, corresponding operation commands are matched for device control, which can improve the reliability of gesture recognition and achieve higher precision gesture control.

[0094] Example 3

[0095] Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of a millimeter-wave radar gesture control device according to an embodiment of the present invention. The device includes:

[0096] Signal clutter suppression module 31: used to acquire several frames of gesture signal data from millimeter-wave radar, and to perform clutter suppression processing on each frame of gesture signal data to obtain clutter-suppressed gesture signal data for each frame.

[0097] Signal processing module 32: used to extract effective gesture signals and reconstruct signals from the gesture signal data after clutter suppression processing of each frame based on modal component screening, so as to obtain several frames of target gesture signal data;

[0098] Feature extraction module 33: used to perform range angle map analysis and enhanced radar spectrum analysis on several frames of target gesture signal data to obtain target range angle map and enhanced radar spectrum, and to perform feature extraction based on the target range angle map and enhanced radar spectrum using a multi-signal classification algorithm to obtain target feature data;

[0099] Gesture recognition module 34: used to determine the target weight of the target gesture signal data in each frame, and to perform gesture recognition using a multi-task learning gesture recognition model based on the target weight and target feature data, so as to obtain the gesture recognition result;

[0100] Device control module 35: used to match the corresponding operation command based on the gesture recognition result, and to control the device based on the operation command.

[0101] In the specific implementation of this invention, the specific implementation of the device item can be referred to the implementation of the method item above, and will not be repeated here.

[0102] In this embodiment of the invention, clutter suppression processing is performed on each frame of gesture signal data. Based on modal component filtering, effective gesture signal extraction and signal reconstruction are performed on the clutter-suppressed gesture signal data of each frame to obtain several frames of target gesture signal data, making the obtained target gesture signal data more reliable and avoiding affecting the accuracy of subsequent feature extraction. Range angle map analysis and enhanced radar spectrum analysis are performed on several frames of target gesture signal data to obtain target range angle maps and enhanced radar spectra. Feature extraction is then performed using a multi-signal classification algorithm, which can obtain more comprehensive and accurate feature data, while capturing sufficient detailed features, overcoming the problem of insufficient feature information leading to low gesture recognition accuracy in the prior art. The target weight of each frame of target gesture signal data is determined. Based on the target weight and target feature data, a multi-task learning gesture recognition model is used for gesture recognition. Based on the gesture recognition results, corresponding operation commands are matched for device control, which can improve the reliability of gesture recognition and achieve higher precision gesture control.

[0103] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the millimeter-wave radar gesture control method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0104] Example 4

[0105] Please see Figure 4 , Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0106] This invention also provides an electronic device, such as... Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. Those skilled in the art will understand that... Figure 4The illustrated electronic device does not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 41 can be used to store computer program 42 and various functional modules. Processor 43 runs the computer program 42 stored in memory 41, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. Processor 43 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or a processor 43, or any conventional processor, etc. The processors and memories disclosed in this invention include, but are not limited to, these types of processors and memories. The processors and memories disclosed in this invention are merely examples and not intended to be limiting.

[0107] As one embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to perform the millimeter-wave radar gesture control method in any of the above embodiments. For specific implementation processes, please refer to the above embodiments, which will not be repeated here.

[0108] In this embodiment of the invention, clutter suppression processing is performed on each frame of gesture signal data. Based on modal component filtering, effective gesture signal extraction and signal reconstruction are performed on the clutter-suppressed gesture signal data of each frame to obtain several frames of target gesture signal data, making the obtained target gesture signal data more reliable and avoiding affecting the accuracy of subsequent feature extraction. Range angle map analysis and enhanced radar spectrum analysis are performed on several frames of target gesture signal data to obtain target range angle maps and enhanced radar spectra. Feature extraction is then performed using a multi-signal classification algorithm, which can obtain more comprehensive and accurate feature data, while capturing sufficient detailed features, overcoming the problem of insufficient feature information leading to low gesture recognition accuracy in the prior art. The target weight of each frame of target gesture signal data is determined. Based on the target weight and target feature data, a multi-task learning gesture recognition model is used for gesture recognition. Based on the gesture recognition results, corresponding operation commands are matched for device control, which can improve the reliability of gesture recognition and achieve higher precision gesture control.

[0109] Furthermore, the above provides a detailed description of a millimeter-wave radar gesture control method and related devices provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A millimeter wave radar gesture control method, characterized by, The method comprises: Obtaining several frames of gesture signal data of the millimeter wave radar, and performing clutter suppression processing on each frame of gesture signal data to obtain each frame of gesture signal data after clutter suppression processing; Based on the modal component screening, the effective gesture signal extraction and signal reconstruction are performed on each frame of gesture signal data after clutter suppression processing to obtain several frames of target gesture signal data; The target distance angle graph and the enhanced radar spectrum graph are obtained by performing distance angle graph analysis and enhanced radar spectrum graph analysis on the several frames of target gesture signal data, and the feature extraction is performed based on the target distance angle graph and the enhanced radar spectrum graph by using the multiple signal classification algorithm to obtain the target feature data; The target weight of each frame of target gesture signal data is determined, and gesture recognition is performed based on the target weight and the target feature data by using the multi-task learning gesture recognition model to obtain the gesture recognition result; Based on the gesture recognition result, the corresponding operation instruction is matched, and the device control is performed based on the operation instruction; Wherein, the effective gesture signal extraction and signal reconstruction are performed on each frame of gesture signal data after clutter suppression processing based on the modal component screening to obtain several frames of target gesture signal data, including: decomposing each frame of gesture signal data after clutter suppression processing to obtain corresponding several modal components; the energy value of each modal component is determined, and the effective gesture signal extraction is performed on each frame of gesture signal data after clutter suppression processing based on the energy value of each modal component to obtain effective gesture signal data; the noise signal data is determined based on the effective gesture signal data, and the noise signal data is denoised by using the noise suppression network to obtain denoised signal data, wherein the noise suppression network adopts time-frequency loss function and multi-head self-attention mechanism; the signal reconstruction is performed based on the denoised signal data and the effective gesture signal data to obtain several frames of target gesture signal data; The target distance angle graph and the enhanced radar spectrum graph are obtained by performing distance angle graph analysis and enhanced radar spectrum graph analysis on the several frames of target gesture signal data, and the feature extraction is performed based on the target distance angle graph and the enhanced radar spectrum graph by using the multiple signal classification algorithm to obtain the target feature data, including: the distance dimension and the Doppler dimension of each frame of target gesture signal data are determined, and the initial distance angle graph is generated based on the distance dimension and the Doppler dimension; the initial distance angle graph is processed by translation transformation and data enhancement to obtain the initial distance angle graph after data enhancement processing; the continuous distance angle frame sequence is determined based on the initial distance angle graph after data enhancement processing, and the target distance angle graph is generated based on the continuous distance angle frame sequence; the micro-Doppler spectrum is generated based on the several frames of target gesture signal data, and the histogram equalization and homomorphic filtering are performed on the micro-Doppler spectrum to obtain the enhanced radar spectrum graph; the angle-time graph is determined based on the several frames of target gesture signal data by using the multiple signal classification algorithm, and the feature extraction is performed based on the target distance angle graph, the enhanced radar spectrum graph and the angle-time graph to obtain the target feature data.

2. The mmWave radar gesture control method of claim 1, wherein, The clutter suppression processing is performed on each frame of gesture signal data to obtain each frame of gesture signal data after clutter suppression processing, including: acquire phases of the gesture signal data of each frame, and determine mean pulses corresponding to the gesture signal data of each frame based on the phases of the gesture signal data of each frame; determine filter smoothing operators of the gesture signal data of each frame, and perform clutter suppression processing on the gesture signal data of each frame based on the mean pulses and the filter smoothing operators to obtain gesture signal data of each frame after clutter suppression processing. 3.The millimeter wave radar gesture control method of claim 1, wherein, The gesture signal data of each frame after clutter suppression processing is decomposed to obtain corresponding modal components, including: determining a decomposition parameter range, and determining a target decomposition parameter using a target function based on the decomposition parameter range; determining a target envelope of the gesture signal data of each frame after clutter suppression processing, and decomposing the gesture signal data of each frame after clutter suppression processing based on the target envelope and the target decomposition parameter to obtain corresponding modal components. 4.The millimeter wave radar gesture control method of claim 1, wherein, The target weight of each frame of target gesture signal data is determined, and gesture recognition is performed using a multi-task learning gesture recognition model based on the target weight and the target feature data to obtain a gesture recognition result, including: determining the target weight of each frame of target gesture signal data based on time sorting; inputting the target feature data into the multi-task learning gesture recognition model to determine the gesture type classification probability corresponding to each frame of target gesture signal data; performing gesture recognition based on the gesture type classification probability combined with the target weight to obtain a gesture recognition result. 5.The millimeter wave radar gesture control method of claim 1, wherein, The corresponding operation instruction is matched based on the gesture recognition result, and the device is controlled based on the operation instruction, including: matching the corresponding operation instruction in the instruction database based on the gesture recognition result; sending the operation instruction to the target device, and the target device analyzes the operation instruction to obtain analysis instruction information, and the target device executes the analysis instruction information.

6. A millimeter wave radar gesture control device, characterized by, The device includes: a signal clutter suppression module for acquiring a plurality of frames of gesture signal data of a millimeter wave radar, and performing clutter suppression processing on the gesture signal data of each frame to obtain gesture signal data of each frame after clutter suppression processing; a signal processing module for performing effective gesture signal extraction and signal reconstruction on the gesture signal data of each frame after clutter suppression processing based on modal component screening to obtain a plurality of frames of target gesture signal data; a feature extraction module for performing distance-angle map analysis and enhanced radar spectrum map analysis on the plurality of frames of target gesture signal data to obtain a target distance-angle map and an enhanced radar spectrum map, and performing feature extraction using a multiple signal classification algorithm based on the target distance-angle map and the enhanced radar spectrum map to obtain target feature data; a gesture recognition module for determining a target weight of each frame of target gesture signal data, and performing gesture recognition using a multi-task learning gesture recognition model based on the target weight and the target feature data to obtain a gesture recognition result; a device control module for matching a corresponding operation instruction based on the gesture recognition result, and controlling a device based on the operation instruction. The effective gesture signal extraction and signal reconstruction are performed on the gesture signal data after each frame of clutter suppression processing based on the modal component, and a plurality of frames of target gesture signal data are obtained, including: decomposing the gesture signal data after each frame of clutter suppression processing to obtain a plurality of corresponding modal components; determining the energy values of the modal components, and performing effective gesture signal extraction on the gesture signal data after each frame of clutter suppression processing based on the energy values of the modal components to obtain effective gesture signal data; determining noise signal data based on the effective gesture signal data, and performing denoising processing on the noise signal data based on a noise suppression network to obtain denoised signal data, the noise suppression network using a time-frequency loss function and a multi-head self-attention mechanism; and performing signal reconstruction based on the denoised signal data and the effective gesture signal data to obtain a plurality of frames of target gesture signal data. The distance-angle graph analysis and enhanced radar spectrum graph analysis are performed on the plurality of frames of target gesture signal data to obtain a target distance-angle graph and an enhanced radar spectrum graph, and feature extraction is performed on the target distance-angle graph and the enhanced radar spectrum graph based on a multiple signal classification algorithm to obtain target feature data, including: determining the distance dimension and the Doppler dimension of each frame of target gesture signal data, and generating an initial distance-angle graph based on the distance dimension and the Doppler dimension; performing translation transformation and data enhancement processing on the initial distance-angle graph to obtain the initial distance-angle graph after data enhancement processing; determining a continuous distance-angle frame sequence based on the initial distance-angle graph after data enhancement processing, and generating a target distance-angle graph based on the continuous distance-angle frame sequence; generating a micro-Doppler spectrum based on the plurality of frames of target gesture signal data, and performing histogram equalization and homomorphic filtering on the micro-Doppler spectrum to obtain an enhanced radar spectrum graph; determining an angle-time graph based on the plurality of frames of target gesture signal data using the multiple signal classification algorithm, and performing feature extraction based on the target distance-angle graph, the enhanced radar spectrum graph, and the angle-time graph to obtain target feature data. 7.An electronic device comprising a processor and a memory, wherein The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the millimeter wave radar gesture control method in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions run on the electronic device, so that the electronic device executes the millimeter wave radar gesture control method in any one of claims 1 to 5.

Images