Voice command recognition method based on millimeter wave sensing

Abstract

The invention discloses a voice command recognition method based on millimeter wave sensing, and the method comprises the steps: firstly building a millimeter wave radar data collection platform, and collecting echo data of different voice commands; then, the collected echo data are preprocessed; performing short-time Fourier transform on the preprocessed echo data on each distance unit, extracting Doppler information of the target, and then performing incoherent superposition on Doppler spectrums of all the distance units to generate a micro-Doppler spectrogram; according to the method, an improved deep convolutional adversarial network is utilized to perform data amplification, finally, a vocal cord vibration recognition network BAC-Net is constructed, and an optimal model is obtained by training different voice command words, so that different voice commands are recognized. According to the invention, the limitation of the prior art in the aspects of processing independent voice commands and detecting weak sounds (such as patient voices) is solved, and the difficulty of model design and expansion caused by limited data volume is overcome at the same time.

Description

Technical Field

[0001] The present invention belongs to the field of wireless sensing technology, and in particular relates to a voice command recognition method based on millimeter wave sensing. Background Art

[0002] With the rapid development of Internet technology, the number of smart devices continues to increase, and voice assistants have been widely used in smart driving, smart home, smart medical and other fields. However, existing voice assistants mainly rely on microphones to collect voice signals, which works well in an ideal environment without noise, but multiple sound sources, noise and far-field interference in complex acoustic environments can easily cause signal distortion, making it difficult for traditional speech recognition technology to meet actual needs. In addition, the microphone method cannot be applied in scenarios such as through-wall perception. In order to improve the performance of speech recognition, researchers began to explore the possibility of non-acoustic sensors and proposed two methods: contact and non-contact. The contact method generates voice signals by capturing the vibration of the vocal cords and surrounding tissues through wearable devices, but long-term contact with the skin may cause inconvenience. Therefore, research has gradually turned to non-contact human voice perception. Non-contact perception was first based on computer vision, and its principle is similar to the way human language communicates. However, this method has personal privacy issues and is limited by ambient light conditions, making it difficult to apply in scenarios where users wear masks.

[0003] In recent years, millimeter-wave radar has been widely used in various human-computer interaction scenarios due to its advantages such as low cost, small size, high resolution and privacy protection. Millimeter-wave radar transmits radio signals through antennas and receives radio signals reflected back after phase modulation due to glottal vibration. The echo signal collected is highly similar to the voice signal recorded by the microphone. Its operating frequency range is 76-81GHz, corresponding to a wavelength of about 4mm. It has a finer-grained sensing capability. The technology uses phase information to distinguish distance changes with an accuracy of millimeters. It can penetrate specific materials such as clothes, masks, and plastics, and is not affected by harsh environmental conditions such as ambient light, dust, and rain and fog. In addition, users do not need to wear additional electronic devices and are non-invasive. Zhang et al. proposed AmbiEar, the first millimeter-wave (mmWave) speech recognition method suitable for non-line-of-sight scenarios. Experiments show that AmbiEar's word recognition accuracy in non-natural language scenarios reaches 87.21%, and the recognition error is reduced by 35.1%. Fan et al. proposed a novel millimeter-wave-based speech recognition solution to solve the problems of multi-source interference and environmental noise. This method uses a single-channel millimeter wave signal to accurately extract the multimodal features of lip movement and vocal cord vibration. Experimental results in a real environment show that the average speech recognition accuracy of this method reached 92.8%, especially in the recognition of vowels and consonants. Wu et al. conducted a comprehensive technical review of the most advanced wireless voice sensing technology, indicating that voiceprint recognition based on wireless sensing has good development prospects. Existing methods mainly focus on voice security, identity authentication and voice recovery, and have limited application in speech recognition. Summary of the invention

[0004] The purpose of the present invention is to provide a voice command recognition method based on millimeter wave sensing, which solves the limitations of the prior art in processing independent voice commands and detecting weak sounds (such as patient voices), while overcoming the difficulties brought about by limited data volume to model design and expansion.

[0005] The technical solution adopted by the present invention is a voice command recognition method based on millimeter wave sensing, which is characterized by being implemented in the following steps: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 2: preprocessing the echo data collected in step 1; Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0006] The present invention is also characterized in that: Step 1 is implemented according to the following steps: Step 1.1, build a millimeter wave radar data acquisition platform, including hardware architecture and software configuration. The hardware needs to install the AWR1642BOOST evaluation module development board and DCA1000EVM data acquisition card produced by Texas Instruments (TI), including the main board. The main board is equipped with two transmitting antennas and four receiving antennas. The main board is installed with TI's official mmWave Studio02.01.01.00, and the millimeter wave radar related parameters are set, as shown in the following table:

[0007] Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: To avoid errors that may be caused by signal segmentation, the present invention collects echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 (5*10) word groups in each category. The multi-channel fusion method is used to integrate the signals of the 4 receiving antennas in the millimeter-wave radar. Therefore, there are 200 (50*4) word groups in each category to enhance the vocal cord micro-vibration perception ability.

[0008] Step 2 is implemented as follows: Step 2.1: The echo data of different voice command words collected in step 1 are expressed in the form of a three-dimensional matrix with the dimensions:

[0009] in, is the sampling point index in the fast time dimension, is the Chirp index on the slow time dimension, is the index of the receiving channel, The data type of the matrix is ​​complex, that is, each data point can be represented as , contains the amplitude and phase information of the signal, Represents the number of sampling points in the fast time dimension. Each fast time sampling point corresponds to a signal sample captured by the analog-to-digital converter ADC in one sampling cycle. The fast time is usually related to the distance information of the target. Indicates the number of linear frequency modulation chirps in the slow time dimension. A chirp is a single frequency modulation signal sent by the radar to obtain the speed information of the target. Indicates the number of receiving channels; Step 2.2: Perform fast Fourier transform on the echo data obtained in step 2.1 in the fast time dimension to extract the distance information of the target and transform the original ADC data from the time domain to the distance domain. The specific calculation formula is:

[0010] is the distance unit index and the sampling point in the fast time dimension One-to-one correspondence, finally generating a distance-time matrix ; Step 2.3: The distance-time matrix of step 2.2 The phasor mean algorithm is used to filter out static clutter and all Chirp Take the phase mean, calculate the static clutter component, and subtract it from the original matrix:

[0011] To filter out the distance-time matrix after static clutter, so as to enhance the display of dynamic targets and obtain clearer dynamic target signals; Step 2.4: Step 2.3 Remove the DC component, and the calculation formula of the DC component is:

[0012] from Subtract the calculated DC component from the output to eliminate background interference:

[0013] Finally, the matrix with the DC component removed is obtained .

[0014] Step 3 is implemented as follows: Step 3.1: The echo data matrix obtained in step 2.4 , for each distance unit Perform short-time Fourier transform on the target to extract the Doppler information of the target:

[0015] Indicates distance unit ,frequency , receiving channel Micro-Doppler spectrum on ; Step 3.2: Based on the Doppler spectrum of each distance unit obtained in step 3.1, the Doppler spectra of all distance units are incoherently superimposed to generate a micro-Doppler spectrum:

[0016] Indicates frequency and receiving channel Micro-Doppler spectrum on is the square of the amplitude of the micro-Doppler spectrum and is used for incoherent accumulation to generate the final micro-Doppler spectrum.

[0017] Step 4 is implemented according to the following steps: Step 4.1: Build an improved deep convolutional adversarial network; Step 4.2: Input the micro-Doppler spectra on all receiving antennas generated in step 3.2 into the improved DCGAN in step 4.1 to obtain the amplified micro-Doppler map; Step 4.3: Merge the enhanced data obtained in step 4.2 with the original data, and divide the data set according to the ratio of training set: test set: validation set = 7:1.5:1.5.

[0018] The specific structure of the improved deep convolutional adversarial network in step 4.1 is as follows: The improved deep convolution adversarial network DCGAN is used to amplify the data to generate enhanced data equal to the number of original micro-Doppler images of each category. The loss function of the original DCGAN is the Sigmoid cross entropy loss function, which judges whether the input sample is correctly classified and does not punish the discrimination of the generated samples, which leads to the problem of gradient disappearance during GAN training. The loss function of the improved DCGAN is the least squares function, and the corresponding discriminator loss function is:

[0019] The loss function of the generator is:

[0020] represents the expected value or mean, Represents the discriminator for the real sample The output probability of Represents the discriminator to generate samples The output probability of represents samples sampled from the true data distribution, represents the noise vector sampled from the noise distribution, the first term Indicates that the discriminator hopes to The output is as close to 1 as possible, and the second Indicates that the discriminator hopes to generate samples The output of is as close to 0 as possible. The network still provides learning errors when the discriminator makes an error, and penalizes the samples that are correctly judged according to their distance to the decision boundary, pulling the generated samples far from the decision boundary to the decision boundary. This method generates more gradients when the generator is updated, which can effectively alleviate the gradient vanishing problem. At the same time, by pulling the generated samples far from the decision boundary to the decision boundary, the generated samples can be closer to the distribution of real samples, effectively improving the quality of the generated pictures and generating more realistic sample data.

[0021] The specific model structure of the improved deep convolutional adversarial network in step 4.1 is: The generator first receives 100-dimensional random Gaussian distribution noise and fully connects it to the dimension of 8×8×512. Then it performs 4 alternating upsampling and convolution operations. The upsampling uses a transposed convolution kernel size of 4x4 and a step size of 2. The number of convolution kernels is 512, 256, 128, and 64 respectively. Finally, the generated micro-Doppler image is output through a convolution layer with a convolution kernel size of 3x3 and a step size of 1. The discriminator receives the generated samples and the original sample data and performs 4 convolution operations. The number of kernels is 64, 128, 256 and 512 respectively. Each time, a convolution kernel with a size of 4x4, a stride of 2 and a padding of 1 is used to extract image features. Then, a convolution layer with a convolution kernel size of 4x4 and a stride of 1 is used to compress the feature map into a 1x1x1 output. The Sigmoid activation function is used to output the true or false judgment result of the image. After continuous alternating iterative optimization between the generator and the discriminator, the two reach Nash equilibrium, that is, a generated image that meets the requirements is obtained.

[0022] Step 5 is implemented according to the following steps: Step 5.1, the vocal cord vibration recognition network BAC-Net is based on the two-dimensional convolutional neural network 2D-CNN, and uses the Bayesian optimization algorithm to dynamically adjust the key hyperparameters of the model, so as to efficiently find the best hyperparameter combination suitable for the specific RGB image classification task. On this basis, the convolution block attention module CBAM is added after each layer of convolution operation, and the optimization algorithm is used to dynamically determine whether the CBAM module is introduced after each layer of convolution, and optimize the channel and spatial weight calculation details inside the CBAM. This module combines the channel attention mechanism and the spatial attention mechanism, and can dynamically adjust the weight of the feature map to guide the network to pay more attention to the key features, thereby improving the classification performance and generalization ability of the model; among them, the key hyperparameters include the convolution kernel size, the number of channels per layer, the optimizer selection, the dropout rate, and the learning rate; in addition, the pruning mechanism is introduced. In each round of experiments, the hyperparameter combination with a classification accuracy lower than 10% is directly eliminated to reduce the interference of invalid combinations on the overall optimization; Step 5.2: Send the training set in the data set constructed in step 4 to the BAC-Net constructed in step 5.1 for training. The network integrates Bayesian optimization and convolutional block attention modules, focuses on key frequency bands and micro-Doppler features, improves the efficiency of hyperparameter tuning and recognition performance, and obtains the optimal model of the validation set by training different voice command words. Finally, the test set is used for detection to realize the recognition and classification of different voice commands, including "hello", "goodbye", "yes", "no", "eat", "drink water", "toilet", "thank you", "sleep" and "help".

[0023] The beneficial effect of the present invention is that the voice command recognition method based on millimeter wave sensing, referred to as mmMedVoice, is intended to solve the limitations of existing millimeter wave radar-based voice recognition technology in processing independent voice commands and detecting weak sounds (such as patient voices), while overcoming the difficulties caused by limited data volume to model design and expansion. The method uses a 77GHz multi-input multi-output frequency-modulated continuous wave millimeter wave radar to sense the vibration of the human vocal cords in a non-contact manner, and studies the correlation between speech and the micro-motion characteristics of vocal cord vibration. By sensing the phase change of the vocal cord vibration signal, the frequency-time spectrum is accurately extracted from the millimeter wave single-channel signal, and the echo signal extraction of the tiny vibration of the human vocal cord is realized. This method not only effectively reduces costs and avoids synchronization problems between multiple sensors, but also achieves efficient monitoring while protecting the privacy of the subjects. The method includes: building a millimeter-wave radar data acquisition platform to collect echo data; preprocessing the echo data, generating a distance-time matrix through fast Fourier transform, using the phasor mean algorithm to filter out static clutter and remove the DC component; extracting the Doppler information of each distance unit through short-time Fourier transform, and incoherently superposing the Doppler spectra of all distance units to generate a micro-Doppler spectrum; using an improved deep convolutional adversarial network to achieve micro-Doppler data amplification, construct and divide the data set; proposing a vocal cord vibration recognition network BAC-Net, integrating Bayesian optimization and convolutional block attention modules, focusing on key frequency bands and micro-Doppler features, and improving the efficiency of hyperparameter tuning and recognition performance. Comprehensive experiments show that in indoor environments, the accuracy rate is as high as 94%, exceeding the current most advanced methods and six classic deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a framework diagram of the voice command recognition method based on millimeter wave sensing of the present invention; Figure 2 is a micro-Doppler image corresponding to the command word "drink water" provided in an embodiment of the present invention; Figure 3 is a micro-Doppler image corresponding to the command word "eat" provided by an embodiment of the present invention; FIG4 (a) is a real sample of the command word "hello" provided by an embodiment of the present invention; Figure 4 (b) is the generated sample of Figure 4 (a) after being enhanced by the improved DCGAN; Figure 5 is a neural network model diagram after optimization provided by an embodiment of the present invention; Figure 6 It is the confusion matrix of the BAC-Net model detection result provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] The invention relates to a method for voice command recognition based on millimeter wave sensing. Figure 1 , follow the steps below: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 1 is implemented according to the following steps: Step 1.1, build a millimeter wave radar data acquisition platform, including hardware architecture and software configuration. The hardware needs to install the AWR1642BOOST evaluation module development board and DCA1000EVM data acquisition card produced by Texas Instruments (TI), including the main board. The main board is equipped with two transmitting antennas and four receiving antennas. The main board is installed with TI's official mmWave Studio02.01.01.00, and the millimeter wave radar related parameters are set, as shown in the following table:

[0027] Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: To avoid errors that may be caused by signal segmentation, the present invention collects echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 (5*10) word groups in each category. The multi-channel fusion method is used to integrate the signals of the 4 receiving antennas in the millimeter-wave radar. Therefore, there are 200 (50*4) word groups in each category to enhance the vocal cord micro-vibration perception ability.

[0028] Step 2, preprocessing the echo data collected in step 1; Step 2 is implemented as follows: Step 2.1: The echo data of different voice command words are collected through step 1, and the data is in the form of a three-dimensional matrix In the form of:

[0029] in, is the sampling point index in the fast time dimension, is the Chirp index on the slow time dimension, is the index of the receiving channel, The data type of the matrix is ​​complex, that is, each data point can be represented as , contains the amplitude and phase information of the signal, Represents the number of sampling points in the fast time dimension. Each fast time sampling point corresponds to a signal sample captured by the analog-to-digital converter ADC in one sampling cycle. The fast time is usually related to the distance information of the target. Indicates the number of linear frequency modulation chirps in the slow time dimension. A chirp is a single frequency modulation signal sent by the radar to obtain the speed information of the target. Indicates the number of receiving channels; Step 2.2: Perform fast Fourier transform on the echo data obtained in step 2.1 in the fast time dimension to extract the distance information of the target and transform the original ADC data from the time domain to the distance domain. The specific calculation formula is:

[0030] is the distance unit index and the sampling point in the fast time dimension One-to-one correspondence, finally generating a distance-time matrix ; Step 2.3: The distance-time matrix of step 2.2 The phasor mean algorithm is used to filter out static clutter and all Chirp Take the phase mean, calculate the static clutter component, and subtract it from the original matrix:

[0031] To filter out the distance-time matrix after static clutter, so as to enhance the display of dynamic targets and obtain clearer dynamic target signals; Step 2.4: Step 2.3 Remove the DC component, and the calculation formula of the DC component is:

[0032] from Subtract the calculated DC component from the output to eliminate background interference:

[0033] Finally, the matrix with the DC component removed is obtained .

[0034] Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 3 is implemented as follows: Step 3.1: The echo data matrix obtained in step 2.4 , for each distance unit Perform short-time Fourier transform on the target to extract the Doppler information of the target:

[0035] Indicates distance unit ,frequency , receiving channel Micro-Doppler spectrum on ; Step 3.2: Based on the Doppler spectrum of each distance unit obtained in step 3.1, the Doppler spectra of all distance units are incoherently superimposed to generate a micro-Doppler spectrum:

[0036] Indicates frequency and receiving channel Micro-Doppler spectrum on is the square of the amplitude of the micro-Doppler spectrum and is used for incoherent accumulation to generate the final micro-Doppler spectrum.

[0037] Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 4 is implemented according to the following steps: Step 4.1: Build an improved deep convolutional adversarial network; The specific structure of the improved deep convolutional adversarial network in step 4.1 is as follows: The improved deep convolution adversarial network DCGAN is used to amplify the data to generate enhanced data equal to the number of original micro-Doppler images of each category. The loss function of the original DCGAN is the Sigmoid cross entropy loss function, which judges whether the input sample is correctly classified and does not punish the discrimination of the generated samples, which leads to the problem of gradient disappearance during GAN training. The loss function of the improved DCGAN is the least squares function, and the corresponding discriminator loss function is:

[0038] The loss function of the generator is:

[0039] represents the expected value or mean, Represents the discriminator for the real sample The output probability of Represents the discriminator to generate samples The output probability of represents samples sampled from the true data distribution, represents the noise vector sampled from the noise distribution, the first term Indicates that the discriminator hopes to The output is as close to 1 as possible, and the second Indicates that the discriminator hopes to generate samples The output of is as close to 0 as possible. The network still provides learning errors when the discriminator makes an error, and penalizes the samples that are correctly judged according to their distance to the decision boundary, pulling the generated samples far from the decision boundary to the decision boundary. This method generates more gradients when the generator is updated, which can effectively alleviate the gradient vanishing problem. At the same time, by pulling the generated samples far from the decision boundary to the decision boundary, the generated samples can be closer to the distribution of real samples, effectively improving the quality of the generated pictures and generating more realistic sample data.

[0040] The specific model structure of the improved deep convolutional adversarial network in step 4.1 is: The generator first receives 100-dimensional random Gaussian distribution noise and fully connects it to the dimension of 8×8×512. Then it performs 4 alternating upsampling and convolution operations. The upsampling uses a transposed convolution kernel size of 4x4 and a step size of 2. The number of convolution kernels is 512, 256, 128, and 64 respectively. Finally, the generated micro-Doppler image is output through a convolution layer with a convolution kernel size of 3x3 and a step size of 1. The discriminator receives the generated samples and the original sample data and performs 4 convolution operations. The number of kernels is 64, 128, 256 and 512 respectively. Each time, a convolution kernel with a size of 4x4, a stride of 2 and a padding of 1 is used to extract image features. Then, a convolution layer with a convolution kernel size of 4x4 and a stride of 1 is used to compress the feature map into a 1x1x1 output. The Sigmoid activation function is used to output the true or false judgment result of the image. After continuous alternating iterative optimization between the generator and the discriminator, the two reach Nash equilibrium, that is, a generated image that meets the requirements is obtained.

[0041] Step 4.2: Input the micro-Doppler spectra on all receiving antennas generated in step 3.2 into the improved DCGAN in step 4.1 to obtain the amplified micro-Doppler map; Step 4.3: Merge the enhanced data obtained in step 4.2 with the original data, and divide the data set according to the ratio of training set: test set: validation set = 7:1.5:1.5.

[0042] Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0043] Step 5 is implemented according to the following steps: Step 5.1, the vocal cord vibration recognition network BAC-Net is based on the two-dimensional convolutional neural network 2D-CNN, and uses the Bayesian optimization algorithm to dynamically adjust the key hyperparameters of the model, so as to efficiently find the best hyperparameter combination suitable for the specific RGB image classification task. On this basis, the convolution block attention module CBAM is added after each layer of convolution operation, and the optimization algorithm is used to dynamically determine whether the CBAM module is introduced after each layer of convolution, and optimize the channel and spatial weight calculation details inside the CBAM. This module combines the channel attention mechanism and the spatial attention mechanism, and can dynamically adjust the weight of the feature map to guide the network to pay more attention to the key features, thereby improving the classification performance and generalization ability of the model; among them, the key hyperparameters include the convolution kernel size, the number of channels per layer, the optimizer selection, the dropout rate, and the learning rate; in addition, the pruning mechanism is introduced. In each round of experiments, the hyperparameter combination with a classification accuracy lower than 10% is directly eliminated to reduce the interference of invalid combinations on the overall optimization; Step 5.2: Send the training set in the data set constructed in step 4 to the BAC-Net constructed in step 5.1 for training. The network integrates Bayesian optimization and convolutional block attention modules, focusing on key frequency bands and micro-Doppler features to improve the efficiency of hyperparameter tuning and recognition performance. The network obtains the optimal model of the validation set by training different voice command words, and finally uses the test set for detection to achieve recognition and classification of different voice commands.

[0044] Example 1 The invention relates to a method for voice command recognition based on millimeter wave sensing. Figure 1 , follow the steps below: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 2, preprocessing the echo data collected in step 1; Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0045] Example 2 The invention relates to a method for voice command recognition based on millimeter wave sensing. Figure 1 , follow the steps below: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 1 is implemented according to the following steps: Step 1.1, build a millimeter wave radar data acquisition platform, including hardware architecture and software configuration. The hardware needs to install the AWR1642BOOST evaluation module development board and DCA1000EVM data acquisition card produced by Texas Instruments (TI), including the main board. The main board is equipped with two transmitting antennas and four receiving antennas. The main board is installed with TI's official mmWave Studio02.01.01.00, and the millimeter wave radar related parameters are set, as shown in the following table:

[0046] Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: To avoid errors that may be caused by signal segmentation, the present invention collects echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 (5*10) word groups in each category. The multi-channel fusion method is used to integrate the signals of the 4 receiving antennas in the millimeter-wave radar. Therefore, there are 200 (50*4) word groups in each category to enhance the vocal cord micro-vibration perception ability.

[0047] Step 2, preprocessing the echo data collected in step 1; Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 3 is implemented as follows: Step 3.1: The echo data matrix obtained in step 2.4 , for each distance unit Perform short-time Fourier transform on the target to extract the Doppler information of the target:

[0048] Indicates distance unit ,frequency , receiving channel Micro-Doppler spectrum on ; Step 3.2: Based on the Doppler spectrum of each distance unit obtained in step 3.1, the Doppler spectra of all distance units are incoherently superimposed to generate a micro-Doppler spectrum:

[0049] Indicates frequency and receiving channel Micro-Doppler spectrum on is the square of the amplitude of the micro-Doppler spectrum and is used for incoherent accumulation to generate the final micro-Doppler spectrum.

[0050] Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0051] Example 3 The invention relates to a method for voice command recognition based on millimeter wave sensing. Figure 1 , follow the steps below: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 1 is implemented according to the following steps: Step 1.1, build a millimeter wave radar data acquisition platform, including hardware architecture and software configuration. The hardware needs to install the AWR1642BOOST evaluation module development board and DCA1000EVM data acquisition card produced by Texas Instruments (TI), including the main board. The main board is equipped with two transmitting antennas and four receiving antennas. The main board is installed with TI's official mmWave Studio02.01.01.00, and the millimeter wave radar related parameters are set, as shown in the following table:

[0052] Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: To avoid errors that may be caused by signal segmentation, the present invention collects echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 (5*10) word groups in each category. The multi-channel fusion method is used to integrate the signals of the 4 receiving antennas in the millimeter-wave radar. Therefore, there are 200 (50*4) word groups in each category to enhance the vocal cord micro-vibration perception ability.

[0053] Step 2, preprocessing the echo data collected in step 1; Step 2 is implemented as follows: Step 2.1: The echo data of different voice command words are collected through step 1, and the data is in the form of a three-dimensional matrix In the form of:

[0054] in, is the sampling point index in the fast time dimension, is the Chirp index on the slow time dimension, is the index of the receiving channel, The data type of the matrix is ​​complex, that is, each data point can be represented as , contains the amplitude and phase information of the signal, Represents the number of sampling points in the fast time dimension. Each fast time sampling point corresponds to a signal sample captured by the analog-to-digital converter ADC in one sampling cycle. The fast time is usually related to the distance information of the target. Indicates the number of linear frequency modulation chirps in the slow time dimension. A chirp is a single frequency modulation signal sent by the radar to obtain the speed information of the target. Indicates the number of receiving channels; Step 2.2: Perform fast Fourier transform on the echo data obtained in step 2.1 in the fast time dimension to extract the distance information of the target and transform the original ADC data from the time domain to the distance domain. The specific calculation formula is:

[0055] is the distance unit index and the sampling point in the fast time dimension One-to-one correspondence, finally generating a distance-time matrix ; Step 2.3: The distance-time matrix of step 2.2 The phasor mean algorithm is used to filter out static clutter and all Chirp Take the phase mean, calculate the static clutter component, and subtract it from the original matrix:

[0056] To filter out the distance-time matrix after static clutter, so as to enhance the display of dynamic targets and obtain clearer dynamic target signals; Step 2.4: Step 2.3 Remove the DC component, and the calculation formula of the DC component is:

[0057] from Subtract the calculated DC component from the output to eliminate background interference:

[0058] Finally, the matrix with the DC component removed is obtained .

[0059] Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 3 is implemented as follows: Step 3.1: The echo data matrix obtained in step 2.4 , for each distance unit Perform short-time Fourier transform on the target to extract the Doppler information of the target:

[0060] Indicates distance unit ,frequency , receiving channel Micro-Doppler spectrum on ; Step 3.2: Based on the Doppler spectrum of each distance unit obtained in step 3.1, the Doppler spectra of all distance units are incoherently superimposed to generate a micro-Doppler spectrum:

[0061] Indicates frequency and receiving channel Micro-Doppler spectrum on is the square of the amplitude of the micro-Doppler spectrum and is used for incoherent accumulation to generate the final micro-Doppler spectrum.

[0062] Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0063] Example 4 The invention relates to a method for voice command recognition based on millimeter wave sensing. Figure 1 , follow the steps below: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 1 is implemented according to the following steps: Step 1.1, build a millimeter wave radar data acquisition platform, including hardware architecture and software configuration. The hardware needs to install the AWR1642BOOST evaluation module development board and DCA1000EVM data acquisition card produced by Texas Instruments (TI), including the main board. The main board is equipped with two transmitting antennas and four receiving antennas. The main board is installed with TI's official mmWave Studio02.01.01.00, and the millimeter wave radar related parameters are set, as shown in the following table:

[0064] Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: To avoid errors that may be caused by signal segmentation, the present invention collects echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 (5*10) word groups in each category. The multi-channel fusion method is used to integrate the signals of the 4 receiving antennas in the millimeter-wave radar. Therefore, there are 200 (50*4) word groups in each category to enhance the vocal cord micro-vibration perception ability.

[0065] Step 2, preprocessing the echo data collected in step 1; Step 2 is implemented as follows: Step 2.1: The echo data of different voice command words are collected through step 1, and the data is in the form of a three-dimensional matrix In the form of:

[0066] in, is the sampling point index in the fast time dimension, is the Chirp index on the slow time dimension, is the index of the receiving channel, The data type of the matrix is ​​complex, that is, each data point can be represented as , contains the amplitude and phase information of the signal, Represents the number of sampling points in the fast time dimension. Each fast time sampling point corresponds to a signal sample captured by the analog-to-digital converter ADC in one sampling cycle. The fast time is usually related to the distance information of the target. Indicates the number of linear frequency modulation chirps in the slow time dimension. A chirp is a single frequency modulation signal sent by the radar to obtain the speed information of the target. Indicates the number of receiving channels; Step 2.2: Perform fast Fourier transform on the echo data obtained in step 2.1 in the fast time dimension to extract the distance information of the target and transform the original ADC data from the time domain to the distance domain. The specific calculation formula is:

[0067] is the distance unit index and the sampling point in the fast time dimension One-to-one correspondence, finally generating a distance-time matrix ; Step 2.3: The distance-time matrix of step 2.2 The phasor mean algorithm is used to filter out static clutter and all Chirp Take the phase mean, calculate the static clutter component, and subtract it from the original matrix:

[0068] To filter out the distance-time matrix after static clutter, so as to enhance the display of dynamic targets and obtain clearer dynamic target signals; Step 2.4: Step 2.3 Remove the DC component, and the calculation formula of the DC component is:

[0069] from Subtract the calculated DC component from the output to eliminate background interference:

[0070] Finally, the matrix with the DC component removed is obtained .

[0071] Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 3 is implemented as follows: Step 3.1: The echo data matrix obtained in step 2.4 , for each distance unit Perform short-time Fourier transform on the target to extract the Doppler information of the target:

[0072] Indicates distance unit ,frequency , receiving channel Micro-Doppler spectrum on ; Step 3.2: Based on the Doppler spectrum of each distance unit obtained in step 3.1, the Doppler spectra of all distance units are incoherently superimposed to generate a micro-Doppler spectrum:

[0073] Indicates frequency and receiving channel Micro-Doppler spectrum on is the square of the amplitude of the micro-Doppler spectrum and is used for incoherent accumulation to generate the final micro-Doppler spectrum.

[0074] Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 4 is implemented according to the following steps: Step 4.1: Build an improved deep convolutional adversarial network; The specific structure of the improved deep convolutional adversarial network in step 4.1 is as follows: The improved deep convolution adversarial network DCGAN is used to amplify the data to generate enhanced data equal to the number of original micro-Doppler images of each category. The loss function of the original DCGAN is the Sigmoid cross entropy loss function, which judges whether the input sample is correctly classified and does not punish the discrimination of the generated samples, which leads to the problem of gradient disappearance during GAN training. The loss function of the improved DCGAN is the least squares function, and the corresponding discriminator loss function is:

[0075] The loss function of the generator is:

[0076] represents the expected value or mean, Represents the discriminator for the real sample The output probability of Represents the discriminator to generate samples The output probability of represents samples sampled from the true data distribution, represents the noise vector sampled from the noise distribution, the first term Indicates that the discriminator hopes to The output is as close to 1 as possible, and the second Indicates that the discriminator hopes to generate samples The output of is as close to 0 as possible. The network still provides learning errors when the discriminator makes an error, and penalizes the samples that are correctly judged according to their distance to the decision boundary, pulling the generated samples far from the decision boundary to the decision boundary. This method generates more gradients when the generator is updated, which can effectively alleviate the gradient vanishing problem. At the same time, by pulling the generated samples far from the decision boundary to the decision boundary, the generated samples can be closer to the distribution of real samples, effectively improving the quality of the generated pictures and generating more realistic sample data.

[0077] Step 4.2: Input the micro-Doppler spectra on all receiving antennas generated in step 3.2 into the improved DCGAN in step 4.1 to obtain the amplified micro-Doppler map; Step 4.3: Merge the enhanced data obtained in step 4.2 with the original data, and divide the data set according to the ratio of training set: test set: validation set = 7:1.5:1.5.

[0078] Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0079] Example 5 The invention relates to a method for voice command recognition based on millimeter wave sensing. Figure 1 , follow the steps below: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 1 is implemented according to the following steps: Step 1.1, build a millimeter wave radar data acquisition platform, including hardware architecture and software configuration. The hardware needs to install the AWR1642BOOST evaluation module development board and DCA1000EVM data acquisition card produced by Texas Instruments (TI), including the main board. The main board is equipped with two transmitting antennas and four receiving antennas. The main board is installed with TI's official mmWave Studio02.01.01.00, and the millimeter wave radar related parameters are set, as shown in the following table:

[0080] Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: To avoid errors that may be caused by signal segmentation, the present invention collects echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 (5*10) word groups in each category. The multi-channel fusion method is used to integrate the signals of the 4 receiving antennas in the millimeter-wave radar. Therefore, there are 200 (50*4) word groups in each category to enhance the vocal cord micro-vibration perception ability.

[0081] Step 2, preprocessing the echo data collected in step 1; Step 2 is implemented as follows: Step 2.1: The echo data of different voice command words are collected through step 1, and the data is in the form of a three-dimensional matrix In the form of:

[0082] in, is the sampling point index in the fast time dimension, is the Chirp index on the slow time dimension, is the index of the receiving channel, The data type of the matrix is ​​complex, that is, each data point can be represented as , contains the amplitude and phase information of the signal, Represents the number of sampling points in the fast time dimension. Each fast time sampling point corresponds to a signal sample captured by the analog-to-digital converter ADC in one sampling cycle. The fast time is usually related to the distance information of the target. Indicates the number of linear frequency modulation chirps in the slow time dimension. A chirp is a single frequency modulation signal sent by the radar to obtain the speed information of the target. Indicates the number of receiving channels; Step 2.2: Perform fast Fourier transform on the echo data obtained in step 2.1 in the fast time dimension to extract the distance information of the target and transform the original ADC data from the time domain to the distance domain. The specific calculation formula is:

[0083] is the distance unit index and the sampling point in the fast time dimension One-to-one correspondence, finally generating a distance-time matrix ; Step 2.3: The distance-time matrix of step 2.2 The phasor mean algorithm is used to filter out static clutter and all Chirp Take the phase mean, calculate the static clutter component, and subtract it from the original matrix:

[0084] To filter out the distance-time matrix after static clutter, so as to enhance the display of dynamic targets and obtain clearer dynamic target signals; Step 2.4: Step 2.3 Remove the DC component, and the calculation formula of the DC component is:

[0085] from Subtract the calculated DC component from the output to eliminate background interference:

[0086] Finally, the matrix with the DC component removed is obtained .

[0087] Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 4: Generate micro-Doppler spectra on all receiving antennas in step 3, use the improved deep convolutional adversarial network to perform data amplification, construct a data set and divide it into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 5. Construct a vocal cord vibration recognition network BAC-Net, integrate Bayesian optimization and convolutional block attention modules, focus on key frequency bands and micro-Doppler features, improve hyperparameter tuning efficiency and recognition performance, and use the training set obtained in step 4 as the input of the vocal cord vibration recognition network BAC-Net. The network obtains the optimal model by training different voice command words, thereby recognizing different voice commands.

[0088] Step 5 is implemented according to the following steps: Step 5.1, the vocal cord vibration recognition network BAC-Net is based on the two-dimensional convolutional neural network 2D-CNN, and uses the Bayesian optimization algorithm to dynamically adjust the key hyperparameters of the model. The convolution block attention module CBAM is added after each layer of convolution operation, and the optimization algorithm is used to dynamically determine whether the CBAM module is introduced after each layer of convolution, and optimize the channel and spatial weight calculation details inside the CBAM. This module combines the channel attention mechanism and the spatial attention mechanism to dynamically adjust the weight of the feature map to guide the network to pay more attention to key features, thereby improving the classification performance and generalization ability of the model; among them, the key hyperparameters include the convolution kernel size, the number of channels per layer, the optimizer selection, the Dropout rate, and the learning rate; in addition, the pruning mechanism is introduced. In each round of experiments, the hyperparameter combination with a classification accuracy lower than 10% is directly eliminated to reduce the interference of invalid combinations on the overall optimization; Step 5.2: Send the training set in the data set constructed in step 4 to the BAC-Net constructed in step 5.1 for training. The network integrates Bayesian optimization and convolutional block attention modules, focuses on key frequency bands and micro-Doppler features, improves the efficiency of hyperparameter tuning and recognition performance, and obtains the optimal model of the validation set by training different voice command words. Finally, the test set is used for detection to realize the recognition and classification of different voice commands, including "hello", "goodbye", "yes", "no", "eat", "drink water", "toilet", "thank you", "sleep" and "help". Figure 2 is the micro-Doppler map corresponding to the command word “drink water”; Figure 3 is the micro-Doppler map corresponding to the command word "eat"; Figure 4 (a) is the real sample of the command word "hello"; Figure 4 (b) is the generated sample of Figure 4 (a) after being enhanced by the improved DCGAN.

[0089] Example 6 Figure 5 It is a neural network model diagram after optimization provided by an embodiment of the present invention, wherein the optimizer uses RootMean Square Propagation, the learning rate is 0.00012, the dropout of the first and second fully connected layers are both 0.4, the epoch is 150, the batch_size is 16, corresponding to 4 convolution blocks with a convolution kernel size of 3*3, and the number of channels in each layer are 64, 32, 45, and 46 respectively.

[0090] Figure 6 It is the confusion matrix of the detection result of the BAC-Net model provided by the embodiment of the present invention, in which the numbers 0-9 represent the 10 voice command words of the classification. It can be seen that the values ​​on the main diagonal are basically above 0.90, indicating that the model has a high recognition accuracy in most categories and can effectively distinguish various categories, especially categories 0, 4, 6, 7, 8, etc., with a recognition rate of 95% and above. The recognition rate of category 1 is 91%, and 3% of the samples are misclassified as category 2, indicating that the model has some confusion between category 1 and category 2. The recognition rate of category 3 is 90%, and 6% of the samples are misclassified as category 5; at the same time, the recognition rate of category 5 is 87%, and 3% of the samples are misclassified as category 3. This shows that there is difficulty in distinguishing between category 3 and category 5, which may be due to the similarity of the two categories. The recognition rates of category 7 and category 9 are 97% and 93%, respectively. 4% of the samples of category 9 are misclassified as category 8. Overall, the model performs well in recognizing most categories, but there is confusion between some similar categories (such as category 3 and category 5).

[0091] Table 1 is a comparison of the accuracy of the method of the present invention and the traditional deep learning method.

[0092] Table 1 Comparison of the accuracy of the method of the present invention and the traditional deep learning method

[0093] Table 2 is a comparison of the accuracy of the method of the present invention and the traditional millimeter wave radar based speech method.

[0094]

Claims

1. A voice command recognition method based on millimeter wave sensing, characterized in that: Follow the steps below to implement it: Step 1: Build a millimeter-wave radar data acquisition platform to collect echo data of different voice commands; Step 2, preprocessing the echo data collected in step 1; Step 3: Perform short-time Fourier transform on the preprocessed echo data at each range unit to extract the Doppler information of the target, and then perform incoherent superposition on the Doppler spectra of all range units to generate a micro-Doppler spectrum; Step 4: Use the improved deep convolutional adversarial network to perform data augmentation and construct training sets, validation sets, and test sets; Step 5: Construct the vocal cord vibration recognition network BAC-Net, and obtain the optimal model by training different voice command words to recognize different voice commands.

2. The method for voice command recognition based on millimeter wave sensing according to claim 1, characterized in that: The step 1 is specifically implemented according to the following steps: Step 1.1: Build a millimeter-wave radar data acquisition platform, including a mainboard equipped with two transmitting antennas and four receiving antennas. Install TI's mmWave Studio 02.01.01.00 on the mainboard and set millimeter-wave radar related parameters, as shown in the following table: Step 1.2, design and arrange the experimental scene, the radar is placed on a tripod with a radial distance of 0.5-0.8m and 10-30cm in front of the target; Step 1.3: Collect echo signals of different voice command words in indoor scenes by recording voice commands one by one. Each subject needs to say the required 10 command words in turn. During the recording of each command word, the subject repeats the same command word 5 times according to the gesture guidance of the collector. Data collection is automatically performed at a fixed 5-second interval. There are an average of 50 groups of words in each category. The multi-channel fusion method is used to integrate the signals of the four receiving antennas in the millimeter-wave radar. Therefore, there are 200 groups of words in each category to enhance the perception ability of vocal cord micro-vibration.

3. The method for voice command recognition based on millimeter wave sensing according to claim 2, characterized in that: The step 2 is specifically implemented according to the following steps: Step 2.1: The echo data of different voice command words are collected through step 1, and the data is in the form of a three-dimensional matrix In the form of: in, is the sampling point index in the fast time dimension, is the Chirp index on the slow time dimension, is the index of the receiving channel, The data type of the matrix is ​​complex, that is, each data point can be represented as , contains the amplitude and phase information of the signal, represents the number of sampling points in the fast time dimension, Indicates the number of linear frequency modulation chirps in the slow time dimension. A chirp is a single frequency modulation signal sent by the radar to obtain the speed information of the target. Indicates the number of receiving channels; Step 2.2: Perform fast Fourier transform on the echo data obtained in step 2.1 in the fast time dimension to extract the distance information of the target and transform the original ADC data from the time domain to the distance domain. The specific calculation formula is: is the distance unit index and the sampling point in the fast time dimension One-to-one correspondence, finally generating a distance-time matrix ; Step 2.3: The distance-time matrix of step 2.2 The phasor mean algorithm is used to filter out static clutter and all Chirp Take the phase mean, calculate the static clutter component, and subtract it from the original matrix: To filter out the distance-time matrix after static clutter and obtain a clearer dynamic target signal; Step 2.4: Step 2.3 Remove the DC component, and the calculation formula of the DC component is: from Subtract the calculated DC component from the output to eliminate background interference: Finally, the matrix with the DC component removed is obtained .

4. The method for voice command recognition based on millimeter wave sensing according to claim 3, characterized in that: The step 3 is specifically implemented according to the following steps: Step 3.1: The echo data matrix obtained in step 2.4 , for each distance unit Perform short-time Fourier transform on the target to extract the Doppler information of the target: Indicates distance unit ,frequency , receiving channel Micro-Doppler spectrum on ; Step 3.2: Based on the Doppler spectrum of each distance unit obtained in step 3.1, the Doppler spectra of all distance units are incoherently superimposed to generate a micro-Doppler spectrum: Indicates frequency and receiving channel Micro-Doppler spectrum on is the square of the amplitude of the micro-Doppler spectrum and is used for incoherent accumulation to generate the final micro-Doppler spectrum.

5. The method for voice command recognition based on millimeter wave sensing according to claim 4, characterized in that: The step 4 is specifically implemented according to the following steps: Step 4.1: Build an improved deep convolutional adversarial network; Step 4.2: Input the micro-Doppler spectra on all receiving antennas generated in step 3.2 into the improved DCGAN in step 4.1 to obtain the amplified micro-Doppler map; Step 4.3: Merge the enhanced data obtained in step 4.2 with the original data, and divide the data set according to the ratio of training set: test set: validation set = 7:1.5:1.

5.

6. The method for voice command recognition based on millimeter wave sensing according to claim 5, characterized in that: The specific structure of the improved deep convolutional adversarial network in step 4.1 is as follows: The improved deep convolution adversarial network DCGAN is used to amplify the data to generate enhanced data equal to the number of original micro-Doppler images of each category. The loss function of the original DCGAN is the Sigmoid cross entropy loss function, which judges whether the input sample is correctly classified and does not punish the discrimination of the generated samples, which leads to the problem of gradient disappearance during GAN training. The loss function of the improved DCGAN is the least squares function, and the corresponding discriminator loss function is: The loss function of the generator is: represents the expected value or mean, Represents the discriminator for the real sample The output probability of Represents the discriminator to generate samples The output probability of represents samples sampled from the true data distribution, represents the noise vector sampled from the noise distribution, the first term Indicates that the discriminator hopes to The output is as close to 1 as possible, and the second Indicates that the discriminator hopes to generate samples The output is as close to 0 as possible.

7. The method for voice command recognition based on millimeter wave sensing according to claim 6, characterized in that: The specific model structure of the improved deep convolutional adversarial network in step 4.1 is: The generator first receives 100-dimensional random Gaussian distribution noise and fully connects it to the dimension of 8×8×512. Then it performs 4 alternating upsampling and convolution operations. The upsampling uses a transposed convolution kernel size of 4x4 and a step size of 2. The number of convolution kernels is 512, 256, 128, and 64 respectively. Finally, the generated micro-Doppler image is output through a convolution layer with a convolution kernel size of 3x3 and a step size of 1. The discriminator receives the generated samples and the original sample data and performs 4 convolution operations. The number of kernels is 64, 128, 256 and 512 respectively. Each time, a convolution kernel with a size of 4x4, a stride of 2 and a padding of 1 is used to extract image features. Then, a convolution layer with a convolution kernel size of 4x4 and a stride of 1 is used to compress the feature map into a 1x1x1 output. The Sigmoid activation function is used to output the true or false judgment result of the image. After continuous alternating iterative optimization between the generator and the discriminator, the two reach Nash equilibrium, that is, a generated image that meets the requirements is obtained.

8. The method for voice command recognition based on millimeter wave sensing according to claim 7, characterized in that: The step 5 is specifically implemented according to the following steps: Step 5.1, the vocal cord vibration recognition network BAC-Net is based on the two-dimensional convolutional neural network 2D-CNN, and uses the Bayesian optimization algorithm to dynamically adjust the key hyperparameters of the model. The convolution block attention module CBAM is added after each layer of convolution operation, and the optimization algorithm is used to dynamically determine whether the CBAM module is introduced after each layer of convolution, and optimize the channel and spatial weight calculation details inside the CBAM. Among them, the key hyperparameters include the convolution kernel size, the number of channels per layer, the optimizer selection, the Dropout rate, and the learning rate. In addition, the pruning mechanism is introduced. In each round of experiments, the hyperparameter combination with a classification accuracy lower than 10% is directly eliminated to reduce the interference of invalid combinations on the overall optimization. Step 5.2: Send the training set in the data set constructed in step 4 to the BAC-Net constructed in step 5.1 for training. The network obtains the optimal model of the verification set by training different voice command words, and finally uses the test set for detection to achieve recognition and classification of different voice commands.

Images