Human body movement instruction recognition method based on high-frequency sound waves

Abstract

The invention relates to man-machine interaction, in particular to a human body limb movement instruction recognition method based on high-frequency sound waves, and the method comprises the steps: transmitting a high-frequency sound wave signal through an array sensor, and receiving an echo signal reflected by the surface of a human body; determining the three-dimensional position of each part of the human limb according to the time delay from transmitting the high-frequency sound wave signal to receiving the echo signal; determining the movement speed of each part of the human body based on the Doppler effect; in combination with the three-dimensional position and the movement speed of each part of the human body limb, accurate position information and speed information of each part of the human body limb are obtained; extracting frequency spectrum features in direct relation with the movement mode of the human limbs in the echo signals; according to the accurate position information and speed information of each part of the human body limb, a human body limb movement track is constructed, and key nodes and movement modes are extracted from the movement track according to the frequency spectrum characteristics of the echo signals; according to the technical scheme provided by the invention, the defect that the human body limb action instruction recognition precision is low can be overcome.

Description

Technical Field

[0001] The present invention relates to human-computer interaction, and in particular to a method for recognizing human body movement instructions based on high-frequency sound waves. Background Art

[0002] With the continuous development of science and technology, the human-computer interaction mode is undergoing profound changes. Traditional human-computer interaction modes such as buttons and touch screens can no longer meet people's usage needs, while new human-computer interaction modes based on vision, sound, etc. have gradually become a research hotspot. Among them, the human body movement command recognition method based on high-frequency sound waves has attracted much attention due to its advantages such as non-contact, non-invasive, and fast response speed.

[0003] At present, there have been many studies on human body movement command recognition methods based on high-frequency sound waves at home and abroad. Most of these methods use ultrasonic sensors as signal transmitters and receivers to identify the user's body movements by capturing and analyzing the reflected echo signals. However, these methods still have deficiencies in signal processing, feature extraction, and model training, resulting in the need to improve recognition accuracy and real-time performance.

[0004] With the continuous development of artificial intelligence technologies such as machine learning and deep learning, the human body movement command recognition method based on high-frequency sound waves has also ushered in new development opportunities. By introducing advanced models and algorithms, the recognition accuracy and robustness can be further improved, while reducing the requirements for hardware equipment and improving the user interaction experience. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method for recognizing human limb movement commands based on high-frequency sound waves, which can effectively overcome the defect of low recognition accuracy of human limb movement commands in the prior art.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A method for recognizing human body movement instructions based on high-frequency sound waves comprises the following steps:

[0010] S1, using an array sensor to transmit a high-frequency sound wave signal and receive an echo signal reflected from the human body surface;

[0011] S2, determining the three-dimensional position of each part of the human body according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal;

[0012] S3, determine the movement speed of various parts of human limbs based on the Doppler effect;

[0013] S4, combining the three-dimensional position and movement speed of each part of the human body limbs to obtain accurate position information and speed information of each part of the human body limbs;

[0014] S5, extracting the frequency spectrum features directly related to the movement pattern of human limbs in the echo signal;

[0015] S6, constructing the motion trajectory of the human limbs according to the accurate position information and speed information of each part of the human limbs, and extracting key nodes and motion patterns from the motion trajectory according to the spectrum characteristics of the echo signal;

[0016] S7, inputting the accurate position information and speed information of each part of the human limbs and the frequency spectrum characteristics of the echo signal into the pre-trained human limb motion recognition model to obtain the human limb motion recognition result;

[0017] S8. Determine corresponding human body movement instructions according to the human body movement recognition results, control the response of the device through the human body movement instructions, and improve the user interaction experience.

[0018] Preferably, S1 uses an array sensor to transmit a high-frequency sound wave signal and receives an echo signal reflected from the surface of the human body, including:

[0019] Control the emission angle, frequency and intensity of high-frequency sound wave signals to ensure coverage of the target area of ​​the human body;

[0020] Use MEMS microphone array to transmit high-frequency sound wave signals and receive echo signals reflected from the human body surface;

[0021] The high-frequency sound wave signal is a sound wave signal above 20KHz obtained by processing using the FWCM method. The sound wave signal can propagate around the human body and interact with human limbs, and can also penetrate the air well and be reflected by the human body surface.

[0022] Preferably, in S2, determining the three-dimensional position of each part of the human body limb according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal includes:

[0023] Based on the time delay from transmitting the high-frequency sound wave signal to receiving the echo signal, the change in the distance at which each sensor receives the echo signal is calculated to determine the three-dimensional position of each part of the human body.

[0024] Preferably, determining the movement speed of each part of a human body limb based on the Doppler effect in S3 includes:

[0025] Based on the Doppler effect, the movement speed of each part of the human body is determined by calculating the frequency shift of the echo signal received by each sensor;

[0026] Among them, the Doppler effect is that due to the movement of human limbs, the frequency of the echo signal will shift. If the human limbs move toward the array sensor, the frequency of the echo signal will increase. If the human limbs move away from the array sensor, the frequency of the echo signal will decrease.

[0027] Preferably, extracting the frequency spectrum features directly related to the motion pattern of human limbs in the echo signal in S5 includes:

[0028] The echo signal is converted into the frequency domain using Fourier transform, the changes in the spectrum are analyzed, and the spectrum features that are directly related to the movement patterns of human limbs are extracted;

[0029] Among them, the frequency spectrum features include amplitude spectrum and phase spectrum.

[0030] Preferably, in S7, the accurate position information and speed information of each part of the human limb and the frequency spectrum characteristics of the echo signal are input into the pre-trained human limb motion recognition model to obtain the human limb motion recognition result, including:

[0031] S71, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0032] S72, setting a loss function and an optimizer for a human body movement recognition model;

[0033] S73, inputting the training set into the human body movement recognition model for model training;

[0034] S74, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0035] S75, if the loss value is less than the preset threshold, the model training is completed, and the current human limb movement recognition model is the trained human limb movement recognition model, otherwise return to S73, and continue to use the training set to perform model training;

[0036] S76. Input the validation set into the trained human body movement recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0037] S77, inputting the test set into the tuned human body movement recognition model to evaluate the model performance;

[0038] Among them, the human body movement recognition model is built based on convolutional neural network CNN or recurrent neural network RNN ​​or long short-term memory network LSTM;

[0039] Convolutional neural network (CNN) is suitable for processing spatial features and can automatically extract local features of signals;

[0040] Recurrent neural networks (RNN) or long short-term memory (LSTM) networks are suitable for processing time series data. They can learn the continuous movement characteristics of human limbs and adapt to the dynamic changes of human limb movements.

[0041] Preferably, in S71, the historical data set is divided into a training set, a validation set and a test set according to a preset ratio, including:

[0042] S711, using an array sensor to transmit a high-frequency sound wave signal, and receiving an echo signal reflected from the human body surface under various human body movements;

[0043] S712, performing a preprocessing operation on the collected echo signal, and performing data enhancement on the preprocessed echo signal;

[0044] S713, determining the three-dimensional position of each part of the human body under various human body movements according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal;

[0045] S714, determining the movement speed of various parts of human limbs under various human limb movements based on the Doppler effect;

[0046] S715, combining the three-dimensional positions and movement speeds of various parts of the human body under various human body movements, obtaining accurate position information and speed information of various parts of the human body under various human body movements;

[0047] S716, extracting frequency spectrum features directly related to the motion patterns of human limbs in echo signals under various human limb movements;

[0048] S717, setting action labels according to various human body movements, accurate position information and speed information of various parts of human body parts under various human body movements, and frequency spectrum characteristics of echo signals under various human body movements, to form a historical data set;

[0049] S718, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0050] Among them, the action labels include raising the left hand, extending the right hand forward, and squatting.

[0051] Preferably, in S712, the collected echo signals are preprocessed, and data enhancement is performed on the preprocessed echo signals, including:

[0052] Perform preprocessing operations including denoising, normalization, and signal truncation on the collected echo signals;

[0053] The preprocessed echo signal is perturbed in time and frequency to enhance the robustness of the human limb movement recognition model.

[0054] Preferably, before inputting the accurate position information and speed information of each part of the human limb and the frequency spectrum characteristics of the echo signal into the pre-trained human limb action recognition model to obtain the human limb action recognition result in S7, the method further includes:

[0055] Integrate the pre-trained human body movement recognition model into applications that need to recognize human body movement instructions;

[0056] Among them, applications that require recognition of human body movement commands include smart homes, virtual reality, and robot control.

[0057] (III) Beneficial effects

[0058] Compared with the prior art, the present invention provides a method for recognizing human limb motion commands based on high-frequency sound waves. The pre-trained human limb motion recognition model can effectively cope with the rapid changes of gestures and noise interference in different environments based on the position information and speed information of each part of the human limb and the spectral characteristics of the echo signal during the real-time recognition of human limb motion commands. At the same time, combined with multi-sensor data and deep learning methods, it can accurately track the dynamic changes of human limb movements and realize accurate recognition of human limb motion commands. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] A method for recognizing human body movement commands based on high-frequency sound waves, such as Figure 1 As shown, S1, using an array sensor to transmit a high-frequency sound wave signal and receiving an echo signal reflected from the human body surface, specifically including:

[0063] Control the emission angle, frequency and intensity of high-frequency sound wave signals to ensure coverage of the target area of ​​the human body;

[0064] Use MEMS microphone array to transmit high-frequency sound wave signals and receive echo signals reflected from the human body surface;

[0065] The high-frequency sound wave signal is a sound wave signal above 20KHz obtained by processing using the FWCM method. The sound wave signal can propagate around the human body and interact with human limbs, and can also penetrate the air well and be reflected by the human body surface.

[0066] S2. Determine the three-dimensional position of each part of the human body according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal, specifically including:

[0067] Based on the time delay from transmitting the high-frequency sound wave signal to receiving the echo signal, the change in the distance at which each sensor receives the echo signal is calculated to determine the three-dimensional position of each part of the human body.

[0068] S3. Determine the movement speed of various parts of the human body based on the Doppler effect, specifically including:

[0069] Based on the Doppler effect, the movement speed of each part of the human body is determined by calculating the frequency shift of the echo signal received by each sensor;

[0070] Among them, the Doppler effect is that due to the movement of human limbs, the frequency of the echo signal will shift. If the human limbs move toward the array sensor, the frequency of the echo signal will increase. If the human limbs move away from the array sensor, the frequency of the echo signal will decrease.

[0071] S4. Combine the three-dimensional position and movement speed of each part of the human body to obtain accurate position information and speed information of each part of the human body.

[0072] S5. Extracting the frequency spectrum features in the echo signal that are directly related to the motion pattern of human limbs, specifically including:

[0073] The echo signal is converted into the frequency domain using Fourier transform, the changes in the spectrum are analyzed, and the spectrum features that are directly related to the movement patterns of human limbs are extracted;

[0074] Among them, the frequency spectrum features include amplitude spectrum and phase spectrum.

[0075] S6. Construct the motion trajectory of the human limbs according to the precise position information and speed information of each part of the human limbs, and extract key nodes and motion patterns from the motion trajectory according to the spectrum characteristics of the echo signal.

[0076] S7. Input the precise position information and speed information of each part of the human limbs and the frequency spectrum characteristics of the echo signal into the pre-trained human limb motion recognition model to obtain the human limb motion recognition result.

[0077] 1) In S7, the accurate position information and speed information of each part of the human limbs and the frequency spectrum characteristics of the echo signal are input into the pre-trained human limb action recognition model to obtain the human limb action recognition result, including:

[0078] S71, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0079] S72, setting a loss function and an optimizer for a human body movement recognition model;

[0080] S73, inputting the training set into the human body movement recognition model for model training;

[0081] S74, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0082] S75, if the loss value is less than the preset threshold, the model training is completed, and the current human limb movement recognition model is the trained human limb movement recognition model, otherwise return to S73, and continue to use the training set to perform model training;

[0083] S76. Input the validation set into the trained human body movement recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0084] S77, inputting the test set into the tuned human body movement recognition model to evaluate the model performance;

[0085] Among them, the human body movement recognition model is built based on convolutional neural network CNN or recurrent neural network RNN ​​or long short-term memory network LSTM;

[0086] Convolutional neural network (CNN) is suitable for processing spatial features and can automatically extract local features of signals;

[0087] Recurrent neural networks (RNN) or long short-term memory (LSTM) networks are suitable for processing time series data. They can learn the continuous movement characteristics of human limbs and adapt to the dynamic changes of human limb movements.

[0088] Specifically, in S71, the historical data set is divided into a training set, a validation set, and a test set according to a preset ratio, including:

[0089] S711, using an array sensor to transmit a high-frequency sound wave signal, and receiving an echo signal reflected from the human body surface under various human body movements;

[0090] S712, performing a preprocessing operation on the collected echo signal, and performing data enhancement on the preprocessed echo signal;

[0091] S713, determining the three-dimensional position of each part of the human body under various human body movements according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal;

[0092] S714, determining the movement speed of various parts of human limbs under various human limb movements based on the Doppler effect;

[0093] S715, combining the three-dimensional positions and movement speeds of various parts of the human body under various human body movements, obtaining accurate position information and speed information of various parts of the human body under various human body movements;

[0094] S716, extracting frequency spectrum features directly related to the motion patterns of human limbs in echo signals under various human limb movements;

[0095] S717, setting action labels according to various human body movements, accurate position information and speed information of various parts of human body parts under various human body movements, and frequency spectrum characteristics of echo signals under various human body movements, to form a historical data set;

[0096] S718, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0097] Among them, the action labels include raising the left hand, extending the right hand forward, and squatting.

[0098] Specifically, in S712, the collected echo signals are preprocessed, and data enhancement is performed on the preprocessed echo signals, including:

[0099] Perform preprocessing operations including denoising, normalization, and signal truncation on the collected echo signals;

[0100] The preprocessed echo signal is perturbed in time and frequency to enhance the robustness of the human limb movement recognition model.

[0101] 2) In S7, the accurate position information and speed information of each part of the human limb and the frequency spectrum characteristics of the echo signal are input into the pre-trained human limb action recognition model to obtain the human limb action recognition result, and the following is also included:

[0102] Integrate the pre-trained human body movement recognition model into applications that need to recognize human body movement instructions;

[0103] Among them, applications that require recognition of human body movement commands include smart homes, virtual reality, and robot control.

[0104] S8. Determine corresponding human body movement instructions according to the human body movement recognition results, control the response of the device through the human body movement instructions, and improve the user interaction experience.

[0105] In the technical solution of this application, the pre-trained human limb motion recognition model can effectively cope with the rapid changes of gestures and noise interference in different environments in the process of real-time recognition of human limb motion instructions based on the position information and speed information of various parts of the human limbs and the spectrum characteristics of the echo signal. At the same time, combined with multi-sensor data and deep learning methods, it can accurately track the dynamic changes of human limb movements and realize accurate recognition of human limb motion instructions. In addition, by further optimizing the algorithm, it can reduce calculation time and delay, improve real-time performance, and ensure rapid response in dynamic environments.

[0106] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing human body movement instructions based on high-frequency sound waves, characterized in that: The following steps are involved: S1, using an array sensor to transmit a high-frequency sound wave signal and receive an echo signal reflected from the human body surface; S2, determining the three-dimensional position of each part of the human body according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal; S3, determine the movement speed of various parts of human limbs based on the Doppler effect; S4, combining the three-dimensional position and movement speed of each part of the human body limbs to obtain accurate position information and speed information of each part of the human body limbs; S5, extracting the frequency spectrum features directly related to the movement pattern of human limbs in the echo signal; S6, constructing the motion trajectory of the human limbs according to the accurate position information and speed information of each part of the human limbs, and extracting key nodes and motion patterns from the motion trajectory according to the spectrum characteristics of the echo signal; S7, inputting the accurate position information and speed information of each part of the human limbs and the frequency spectrum characteristics of the echo signal into the pre-trained human limb motion recognition model to obtain the human limb motion recognition result; S8. Determine corresponding human body movement instructions according to the human body movement recognition results, control the response of the device through the human body movement instructions, and improve the user interaction experience.

2. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 1, characterized in that: S1 uses an array sensor to transmit high-frequency sound wave signals and receive echo signals reflected from the human body surface, including: Control the emission angle, frequency and intensity of high-frequency sound wave signals to ensure coverage of the target area of ​​the human body; Use MEMS microphone array to transmit high-frequency sound wave signals and receive echo signals reflected from the human body surface; The high-frequency sound wave signal is a sound wave signal above 20KHz obtained by processing using the FWCM method. The sound wave signal can propagate around the human body and interact with human limbs, and can also penetrate the air well and be reflected by the human body surface.

3. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 2, characterized in that: In S2, the three-dimensional position of each part of the human body is determined according to the time delay between the emission of the high-frequency sound wave signal and the reception of the echo signal, including: Based on the time delay from transmitting the high-frequency sound wave signal to receiving the echo signal, the change in the distance at which each sensor receives the echo signal is calculated to determine the three-dimensional position of each part of the human body.

4. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 3, characterized in that: In S3, the movement speed of each part of the human body is determined based on the Doppler effect, including: Based on the Doppler effect, the movement speed of each part of the human body is determined by calculating the frequency shift of the echo signal received by each sensor; Among them, the Doppler effect is that due to the movement of human limbs, the frequency of the echo signal will shift. If the human limbs move toward the array sensor, the frequency of the echo signal will increase. If the human limbs move away from the array sensor, the frequency of the echo signal will decrease.

5. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 4, characterized in that: S5 extracts the spectral features in the echo signal that are directly related to the motion pattern of the human limbs, including: The echo signal is converted into the frequency domain using Fourier transform, the changes in the spectrum are analyzed, and the spectrum features that are directly related to the movement patterns of human limbs are extracted; Among them, the frequency spectrum features include amplitude spectrum and phase spectrum.

6. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 5, characterized in that: In S7, the precise position information and speed information of each part of the human limbs and the frequency spectrum characteristics of the echo signal are input into the pre-trained human limb motion recognition model to obtain the human limb motion recognition result, including: S71, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio; S72, setting a loss function and an optimizer for a human body movement recognition model; S73, inputting the training set into the human body movement recognition model for model training; S74, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information; S75, if the loss value is less than the preset threshold, the model training is completed, and the current human limb movement recognition model is the trained human limb movement recognition model, otherwise return to S73, and continue to use the training set to perform model training; S76. Input the validation set into the trained human body movement recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model; S77, inputting the test set into the tuned human body movement recognition model to evaluate the model performance; Among them, the human body movement recognition model is built based on convolutional neural network CNN or recurrent neural network RNN ​​or long short-term memory network LSTM; Convolutional neural network (CNN) is suitable for processing spatial features and can automatically extract local features of signals; Recurrent neural networks (RNN) or long short-term memory (LSTM) networks are suitable for processing time series data. They can learn the continuous motion characteristics of human limbs and adapt to the dynamic changes of human limb movements.

7. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 6, characterized in that: In S71, the historical data set is divided into a training set, a validation set, and a test set according to a preset ratio, including: S711, using an array sensor to transmit a high-frequency sound wave signal, and receiving an echo signal reflected from the human body surface under various human body movements; S712, performing a preprocessing operation on the collected echo signal, and performing data enhancement on the preprocessed echo signal; S713, determining the three-dimensional position of each part of the human body under various human body movements according to the time delay between transmitting the high-frequency sound wave signal and receiving the echo signal; S714, determining the movement speed of various parts of human limbs under various human limb movements based on the Doppler effect; S715, combining the three-dimensional positions and movement speeds of various parts of the human body under various human body movements, obtaining accurate position information and speed information of various parts of the human body under various human body movements; S716, extracting frequency spectrum features directly related to the motion patterns of human limbs in echo signals under various human limb movements; S717, setting action labels according to the precise position information and speed information of various parts of the human limbs under various human limb actions, and the frequency spectrum characteristics of the echo signals under various human limb actions, to form a historical data set; S718, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio; Among them, the action labels include raising the left hand, extending the right hand forward, and squatting.

8. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 7, characterized in that: In S712, the collected echo signals are preprocessed, and data enhancement is performed on the preprocessed echo signals, including: Perform preprocessing operations including denoising, normalization, and signal truncation on the collected echo signals; The preprocessed echo signal is perturbed in time and frequency to enhance the robustness of the human limb movement recognition model.

9. The method for recognizing human body movement instructions based on high-frequency sound waves according to claim 6, characterized in that: In S7, the accurate position information and speed information of each part of the human limbs and the frequency spectrum characteristics of the echo signal are input into the pre-trained human limb action recognition model. Before obtaining the human limb action recognition result, the following is also included: Integrate the pre-trained human body movement recognition model into applications that need to recognize human body movement instructions; Among them, applications that require recognition of human body movement commands include smart homes, virtual reality, and robot control.

Images