Human body feature recognition method based on millimeter wave radar

Abstract

The embodiment of the invention relates to the technical field of human body feature recognition, and discloses a human body feature recognition method based on a millimeter wave radar, and the method comprises the steps: building a millimeter wave radar hardware collection system, and connecting the millimeter wave radar hardware collection system with an upper computer; performing parameter configuration on the millimeter-wave radar hardware acquisition system based on configuration software in the upper computer, and starting the millimeter-wave radar hardware acquisition system to perform acquisition; the method comprises the following steps: filtering a target signal acquired by a millimeter wave radar hardware acquisition system by using a filter, and performing short-time Fourier transform on the filtered target signal to obtain a spectrogram of the target signal; converting the spectrogram of the target signal into an array matrix form, performing singular value decomposition, obtaining a principal component based on the singular value decomposition, and obtaining a feature vector of the target signal; and inputting the feature vector of the target signal into a QDA classifier to obtain a human body feature recognition result. The method is free from the influence of environmental factors, and the stability and accuracy of human body feature recognition are greatly improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of human feature recognition, and in particular to a human feature recognition method based on millimeter wave radar. Background Art

[0002] Gestures and lip readings express human thoughts and play an important role in the study of behavioral characteristics. Their applications can be traced back to multiple fields, including human-computer interaction, biomedical engineering, and security detection. With the rise of emerging technologies such as virtual reality and augmented reality, the demand for human-computer interaction is getting higher and higher. Gesture recognition and lip reading recognition can provide a natural, intuitive, and non-physical contact interaction method to help humans interact with computers, robots, and other devices more conveniently. For patients with limb disabilities caused by certain diseases, gesture recognition can help them more conveniently control electric wheelchairs, intelligent prostheses, and other equipment, improving their quality of life, while lip reading recognition can help some hearing-impaired patients communicate better with people around them. Millimeter-wave radar has the advantages of strong penetration and being unaffected by light. It is also widely used in the field of security detection. Through lip reading recognition and gesture recognition, it can be determined whether passengers are lying or holding dangerous items. Based on the above application needs, research on lip reading recognition and gesture recognition based on millimeter-wave radar has become a hot topic in recent years.

[0003] Accurately detecting and extracting different human features is the key to human feature recognition technology, and optical information acquisition and processing is a commonly used technology for human feature recognition. However, affected by the environment, this technology has great instability, and its reliability will quickly decrease in extreme environments. At the same time, the steps for processing optical images are relatively cumbersome, requiring high computer computing power, and have great limitations in practical applications. At present, some research teams are also using infrared and acoustic technologies to perform the same operations, but these technologies have certain limitations, such as low resolution and susceptibility to sound and temperature.

[0004] With the continuous improvement of millimeter-wave radar integrated chip technology, the stability and application scope of millimeter-wave radar have been continuously expanded. The millimeter-wave radar integrated chip is equipped with a transmitting antenna and a receiving antenna, which has great advantages due to its lightness and high resolution. Compared with infrared, optical and acoustic technologies, millimeter-wave radar is less affected by environmental factors such as light, weather, temperature and sound, and is adaptable to a wider range of working environments. In addition, the anti-stealth capability of millimeter-wave radar is also better than other technologies. Therefore, under complex environmental factors, millimeter-wave radar is obviously more applicable. However, no research team has yet proposed a stable and effective method for human feature recognition based on millimeter-wave radar. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method for human feature recognition based on millimeter-wave radar, which collects target signals through a millimeter-wave radar hardware acquisition system and performs recognition based on singular value decomposition technology and QDA technology. The entire process is free from the influence of environmental factors, thereby greatly improving the stability and accuracy of human feature recognition.

[0006] In order to solve the above technical problems, an embodiment of the present application proposes a human feature recognition method based on millimeter-wave radar, and the method includes the following steps: building a millimeter-wave radar hardware acquisition system, and connecting it to a host computer via a USB cable and a network cable; configuring the parameters of the millimeter-wave radar hardware acquisition system based on the configuration software installed in the host computer, and after completing the parameter configuration, starting the millimeter-wave radar hardware acquisition system to collect data; using a filter to filter the target signal collected by the millimeter-wave radar hardware acquisition system, and performing a short-time Fourier transform on the filtered target signal to obtain a spectrum diagram of the target signal; converting the spectrum diagram of the target signal into the form of an array RI matrix and performing a singular value decomposition, and obtaining a feature vector of the target signal based on the principal components obtained after the singular value decomposition; inputting the feature vector of the target signal into a pre-trained QDA classifier to obtain a human feature recognition result of the target signal output by the QDA classifier.

[0007] The embodiment of the present application also proposes a human feature recognition system based on millimeter wave radar, the system comprising: a hardware construction module, a parameter configuration module, a signal processing module, a singular value decomposition module and a QDA classification module; the hardware construction module is used to build a millimeter wave radar hardware acquisition system, and connect the millimeter wave radar hardware acquisition system to a host computer through a USB cable and a network cable; the parameter configuration module is used to configure the parameters of the millimeter wave radar hardware acquisition system based on the configuration software installed in the host computer, and after completing the parameter configuration, the millimeter wave radar hardware acquisition system is started to collect data; The signal processing module is used to filter the target signal collected by the millimeter-wave radar hardware acquisition system using a filter, and perform short-time Fourier transform on the filtered target signal to obtain a frequency spectrum of the target signal; the singular value decomposition module is used to convert the frequency spectrum of the target signal into the form of an array RI matrix and perform singular value decomposition, and obtain the characteristic vector of the target signal based on the principal components obtained after the singular value decomposition; the QDA classification module is used to input the characteristic vector of the target signal into a pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier.

[0008] An embodiment of the present application also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned millimeter-wave radar-based human feature recognition method.

[0009] An embodiment of the present application further proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the above-mentioned method for human feature recognition based on millimeter-wave radar can be implemented.

[0010] The embodiment of the present application provides a human feature recognition method based on millimeter wave radar, builds a millimeter wave radar hardware acquisition system, and uses the millimeter wave radar hardware acquisition system to collect data and obtain a target signal. The millimeter wave radar hardware acquisition system is not affected by environmental factors such as light, weather, temperature and sound, which effectively improves the stability and reliability of human feature recognition. At the same time, the millimeter wave radar hardware acquisition system does not rely on visual images and has advantages in protecting privacy. After obtaining the target signal, the target signal is filtered using a filter, and a short-time Fourier transform is performed to obtain a spectrum diagram of the target signal. The target signal is often mixed with high-frequency noise and clutter signals. At the same time, a considerable part of the target signal is low-frequency signals of no interest, such as the echo of a stationary object. Using a filter to filter out this useless information can reduce the complexity of subsequent processing and improve the accuracy of human feature recognition. The frequency spectrum of the target signal is converted into the form of an array RI matrix and subjected to singular value decomposition. Based on the principal components obtained after the singular value decomposition, the eigenvector of the target signal is obtained. Finally, the eigenvector of the target signal is input into the pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier. The singular value decomposition technology and QDA technology are used for human feature recognition. The singular value decomposition can compress the feature dimension, and the QDA discriminator has the characteristics of high flexibility and strong discrimination ability, which further greatly improves the stability and accuracy of human feature recognition.

[0011] In some optional embodiments, the millimeter-wave radar hardware acquisition system is built and connected to the host computer through a USB cable and a network cable, including: based on the AWR2243BOOST evaluation board and the DCA1000EVM signal acquisition board, the millimeter-wave radar hardware acquisition system is built, and the DCA1000EVM signal acquisition board is connected to the host computer through a USB cable and a network cable; the millimeter-wave radar hardware acquisition system is parameterized based on the configuration software installed in the host computer, and after the parameter configuration is completed, the millimeter-wave radar hardware acquisition system is started for data acquisition, including: based on the mmWavestud installed in the host computer io software configures the parameters of the millimeter-wave radar hardware acquisition system and establishes serial port communication between the millimeter-wave radar hardware acquisition system and the host computer. After completing the parameter configuration, the millimeter-wave radar hardware acquisition system is started for data acquisition; the AWR2243BOOST evaluation board transmits a high-frequency signal to the target under test and receives the echo signal, and then passes the original signal to the DCA1000EVM signal acquisition board after obtaining it; the DCA1000EVM signal acquisition board converts the original signal into a digital signal and transmits it to the host computer through a high-speed serial interface, USB cable and network cable. The digital signal is the target signal collected by the millimeter-wave radar hardware acquisition system.

[0012] In some optional embodiments, the use of a filter to filter the target signal collected by the millimeter-wave radar hardware acquisition system includes: first, using an MTI filter to coarsely filter the target signal collected by the millimeter-wave radar hardware acquisition system to filter out low-frequency signals of no interest, and obtain a coarsely filtered target signal; then using a Butterworth fourth-order filter to finely filter the coarsely filtered target signal to filter out high-frequency noise and clutter signals, and obtain a finely filtered target signal; performing a short-time Fourier transform on the filtered target signal to obtain a spectrum diagram of the target signal, including: setting a short-time Fourier transform window length, overlap percentage, and FFT filling factor, and based on the set short-time Fourier transform window length, overlap percentage, and FFT filling factor, performing a short-time Fourier transform on the finely filtered target signal to obtain a spectrum diagram of the target signal.

[0013] In some optional embodiments, before training the QDA classifier, training samples are collected through the millimeter wave radar hardware acquisition system, and the training samples include lip reading training samples and gesture training samples; the lip reading training sample collection technicians are designed to read out three lip reading signals of the letters "A", "I", and "M", and each lip reading signal is repeatedly collected S times to obtain 3S lip reading training samples; the gesture training sample collection technicians are designed to make three gesture signals of "extending the thumb", "extending the thumb and index finger", and "extending the thumb, index finger and little finger", and each gesture signal is repeatedly collected S times to obtain 3S gesture training samples; the real labels are marked for each lip reading training sample and each gesture training sample, and a training sample set and a test sample set are generated.

[0014] In some optional embodiments, the QDA classifier is trained, including: loading all training samples in the training sample set, loading all training samples in the training sample set, adjusting the size of each training sample to 100px×100px, and storing them in the array RI matrix; performing singular value decomposition on the array RI matrix, and calculating the cumulative contribution of each singular value obtained by the singular value decomposition to the total variance, and determining the number k of principal components retained based on the cumulative contribution rate and a preset contribution rate threshold; selecting the first k principal components to generate a feature vector of each training sample, using the feature vector of each training sample to calculate the feature mean of each category, and then subtracting the feature mean of the corresponding category from the feature vector of each type of training sample to obtain the feature bias q of each type of training sample. a ; Feature bias q based on various training samples a and the feature bias q of various training samples a Transpose Calculate the covariance matrix ∑ for each category k , and calculate Σ k -1 ; Feature bias q based on test samples b , the feature bias q of the test sample b Transpose And Σ for each category k -1 Calculate the prediction value of the test sample for each category, compare the prediction value of the test sample for each category, and determine the category corresponding to the minimum prediction value as the human feature recognition result of the test sample; based on the human feature recognition result of the test sample and the true label marked on the test sample, iteratively train the QDA classifier until convergence to obtain a trained QDA classifier.

[0015] In some optional embodiments, based on the feature bias qb of the test sample, the transposition of the feature bias qb of the test sample And Σ for each category k-1 Calculate the predicted value of the test sample for each category using the following formula:

[0016]

[0017] Among them, P k Represents the calculated prediction value of the test sample for the kth category.

[0018] In some optional embodiments, the preset contribution rate threshold is 95%, that is, the number k of retained principal components is determined so that the cumulative contribution rate reaches 95%. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0020] Figure 1 is a flow chart of a method for human feature recognition based on millimeter wave radar provided by an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a data collection scenario provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of the composition of an FMCW system provided in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of 2D-FFT processing of FMCW provided in one embodiment of the present application;

[0024] Figure 5 is a schematic diagram of the composition structure of a human feature recognition system based on millimeter wave radar provided by another embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0026] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0027] In order to solve the problem that traditional infrared, optical, and acoustic human feature recognition technologies are easily affected by environmental factors and result in poor recognition effects, an embodiment of the present application proposes a human feature recognition method based on millimeter wave radar, which is applied to an electronic device, wherein the electronic device may be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the human feature recognition method based on millimeter wave radar proposed in this embodiment are described below. The following content is only the implementation details provided for ease of understanding and is not necessary for the implementation of this solution.

[0028] The specific process of the human feature recognition method based on millimeter wave radar proposed in this embodiment is as follows: Figure 1 As shown, including:

[0029] Step 101, build a millimeter wave radar hardware acquisition system and connect it to the host computer via a USB cable and a network cable.

[0030] In the specific implementation, considering that the millimeter-wave radar is not affected by environmental factors such as light, weather, temperature and sound, the server builds a millimeter-wave radar hardware acquisition system for data collection. After the construction is completed, the millimeter-wave radar hardware acquisition system is connected to the host computer (ie, the server) via a USB cable and a network cable.

[0031] In one example, the server built a millimeter-wave radar hardware acquisition system based on the AWR2243BOOST evaluation board and the DCA1000EVM signal acquisition board, and connected the DCA1000EVM signal acquisition board to the host computer via a USB cable and a network cable.

[0032] Step 102, based on the configuration software installed in the host computer, the millimeter wave radar hardware acquisition system is configured with parameters. After the parameter configuration is completed, the millimeter wave radar hardware acquisition system is started to collect data.

[0033] In the specific implementation, after the millimeter-wave radar hardware acquisition system is connected to the host computer, the server needs to configure the parameters of the millimeter-wave radar hardware acquisition system based on the configuration software installed in the host computer. After completing the parameter configuration, the millimeter-wave radar hardware acquisition system is started to collect data. The data collection scenario can be as follows Figure 2 shown.

[0034] In one example, the server configures parameters of the millimeter-wave radar hardware acquisition system based on the mmWave studio software installed in the host computer, establishes serial port communication between the millimeter-wave radar hardware acquisition system and the host computer, and after completing the parameter configuration, starts the millimeter-wave radar hardware acquisition system for data acquisition.

[0035] The parameters that need to be configured on the server are shown in Table 1.

[0036] Table 1: Parameter configuration table

[0037] Parameters Parameter configuration Parameters Parameter configuration Tx channel Tx0 Data rate 600Mbps Rx channel Rx0 / Rx1 / Rx2 / RX3 Lane format Format 0 adc bits 16 Lane config Lane1 Lane2 adc format Complex1x profile id 0 ADC IQ SWAP I First start frequency 77GHz ADC mode Regular ADC Frequency slope 61.653MHz / μs data path LVDS Idel time 7μs PACKET 0 ADC_ONLY Tx start time 1μs PACKET 1 Suppress Packet adc start time 6μs Lane clock DDR Clock ADC samples 256

[0038] After all the parameters in Table 1 are configured, you can click the DCA1000ARM button in the mmWave studio software to start the plug-in tool of the DCA1000EVM signal acquisition board to ensure that the device is in a ready-to-work state. Next, click the Trigger Frame button to control the acquisition and transmission of radar signals. After clicking this button, the mmWave studio software will send a trigger signal to the millimeter-wave radar hardware acquisition system to trigger a data acquisition. The AWR2243BOOST evaluation board transmits a high-frequency signal to the target under test and receives the echo signal. After obtaining the original signal, it is passed to the DCA1000EVM signal acquisition board. The DCA1000EVM signal acquisition board is responsible for converting the original signal into a digital signal and transmitting the digital signal to the host computer through a high-speed serial interface, USB cable and network cable. The digital signal is the target signal collected by the millimeter-wave radar hardware acquisition system. The collected target signal is saved in the pre-set path in the format of a binary file.

[0039] In one example, the FMCW system may be composed as follows: Figure 3 As shown, the 2D-FFT processing of FMCW can be as follows Figure 4 shown.

[0040] Step 103: Filter the target signal collected by the millimeter wave radar hardware acquisition system using a filter, and perform short-time Fourier transform on the filtered target signal to obtain a frequency spectrum of the target signal.

[0041] In the specific implementation, the server uses a filter to filter the target signal collected by the millimeter-wave radar hardware acquisition system, retaining only the part of interest in the target information, and then performs a short-time Fourier transform on the filtered target signal to obtain the spectrum diagram of the target signal.

[0042] In one example, the Moving Target Indication (MTI) technology in radar signal processing can distinguish between stationary and moving objects and improve the detection accuracy of moving targets. MTI uses a bandpass filter and utilizes the motion characteristics of the target to suppress the stationary object signal through the bandpass filter and retain the moving object signal. This technology is widely used in radar systems to detect moving targets such as aircraft and vehicles. In short, MTI can replace the Butterworth fourth-order filter due to its similar filtering characteristics to achieve effective filtering in signal processing and communication.

[0043] When filtering, the server first uses the MTI filter to coarsely filter the target signal collected by the millimeter wave radar hardware acquisition system to filter out low-frequency signals of no interest. The MTI filter is like a sieve that can filter out low-frequency signals that are not of interest to human feature recognition, that is, the echo of stationary objects. The target signal after coarse filtering is the high-frequency signal of interest to human feature recognition, that is, the echo of moving objects.

[0044] The server then uses a Butterworth fourth-order filter to finely filter the target signal after coarse filtering, filtering out high-frequency noise and clutter signals to obtain a finely filtered target signal. Even after being processed by the MTI filter, there may still be some high-frequency noise and clutter in the signal. At this time, the server uses a Butterworth fourth-order filter to further remove these noises. The Butterworth fourth-order filter is like a finer sieve that can filter out those tiny impurities more carefully.

[0045] After obtaining the finely filtered target signal, the server needs to set the short-time Fourier transform window length, overlap percentage and FFT filling factor, and perform short-time Fourier transform on the finely filtered target signal based on the set short-time Fourier transform window length, overlap percentage and FFT filling factor to obtain the spectrum of the target signal. The window length determines the accuracy of the observed signal, the overlap percentage determines the smoothness of the image, and the FFT filling factor determines how clearly the frequency details of the signal can be seen.

[0046] Step 104, converting the frequency spectrum of the target signal into the form of an array RI matrix and performing singular value decomposition, and obtaining the eigenvector of the target signal based on the principal components obtained after the singular value decomposition.

[0047] In the specific implementation, after obtaining the spectrum diagram of the target signal, the server needs to convert the spectrum diagram of the target signal into the form of an array RI matrix and perform singular value decomposition (SVD), and obtain the eigenvector of the target signal based on the principal components obtained after the singular value decomposition.

[0048] SVD is a common method for data dimensionality reduction and an important matrix decomposition technique in linear algebra. It can convert high-dimensional data into low-dimensional data and retain the main features of the original data as much as possible, thereby reducing the amount of calculation and improving calculation efficiency. SVD decomposes any m×n matrix A into the product of three specific matrices in the following form:

[0049] A=U∑V T

[0050] Among them, U is an m×m orthogonal matrix, that is, U T U=I,U T represents the transpose of U, whose columns are called left singular vectors, ∑ is an m×n diagonal matrix, the non-negative real numbers on its diagonal are called singular values, and are arranged in descending order. The singular values ​​represent the degree of "stretching" or "compression" of the original matrix in the direction of the corresponding singular vectors, V is an n×n orthogonal matrix, VV T =I, V T represents the transpose of V, and its columns are called right singular vectors.

[0051] The steps of singular value decomposition are usually as follows:

[0052] Calculate the covariance matrix. If the original matrix A is not centralized, you first need to calculate its covariance matrix. For matrix A, the covariance matrix C can be calculated by the following formula:

[0053]

[0054] Where m is the number of rows in matrix A.

[0055] Eigenvalue decomposition: perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues ​​and eigenvectors, C = V∑V T , V is a matrix of eigenvectors, ∑ is a diagonal matrix whose diagonal elements are eigenvalues.

[0056] Calculate the singular values ​​and singular vectors. The singular values ​​are the square roots of the eigenvalues ​​of the covariance matrix and are arranged in descending order. The elements on the diagonal of the singular value matrix ∑ are the singular values ​​of the original matrix A, and the corresponding column vectors of the eigenvector matrix V are the right singular vectors of the original matrix A.

[0057] Calculate the left singular vector. The left singular vector can be calculated by the following formula:

[0058] U=AV∑- 1

[0059] Among them, ∑- 1 is the inverse of the singular value matrix, containing only the inverses of the nonzero singular values.

[0060] Normalize the singular vectors to ensure that the column vectors of the left singular vectors U and the column vectors of the right singular vectors V are both unit vectors, that is, their lengths are 1.

[0061] Selecting principal components: In the context of dimensionality reduction, the first k largest singular values ​​and their corresponding left and right singular vectors can be selected to approximate the original matrix A.

[0062] In general, SVD can reduce high-dimensional data to low dimensions through linear transformation while retaining the main features. It is often used in fields such as data compression and visualization.

[0063] Step 105 , inputting the feature vector of the target signal into a pre-trained QDA classifier, and obtaining a human feature recognition result of the target signal output by the QDA classifier.

[0064] In a specific implementation, after obtaining the feature vector of the target signal, the server can input the feature vector of the target signal into a pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier.

[0065] Quadratic Discriminant Analysis (QDA) is a commonly used machine learning algorithm, a multi-class classification technique, and a generalization of Linear Discriminant Analysis (LDA). QDA is used to classify data points that belong to different categories, which may have different distributions in the feature space.

[0066] QDA assumes that the data for each class follows a multivariate normal distribution, but unlike LDA, QDA allows each class to have a different covariance matrix. QDA estimates a unique covariance matrix for each class, which allows it to better adapt to the diversity of data.

[0067] For each class k, QDA constructs a quadratic discriminant function about the data point x, which has the following form:

[0068]

[0069] Among them, μ k is the mean vector of class k, Σ k is the covariance matrix of class k, |Σ k | is the determinant of the covariance matrix.

[0070] Given a new observation x, QDA calculates the discriminant function value for each category and assigns it to the category with the largest discriminant function value.

[0071] During the training phase, QDA needs to estimate the mean vector and covariance matrix for each class, which is usually done through maximum likelihood estimation or Bayesian estimation.

[0072] Since the covariance matrix needs to be estimated and inverted for each category, QDA has high computational complexity. At the same time, QDA is sensitive to outliers and noise.

[0073] In practical applications, QDA may require regularization techniques to avoid overfitting, especially when the amount of data is small or the feature dimension is high. QDA is suitable for situations where features have different distributions in different categories, especially when these distributions are not linearly separable.

[0074] When the covariance matrices of all classes are the same, QDA degenerates into LDA. LDA is computationally simpler, but has stricter assumptions about the data. In practical applications, the performance of the QDA model needs to be evaluated, which is usually done through techniques such as cross-validation. The quadratic discriminator is at the core of the classification decision in QDA. The quadratic discriminator provides a flexible way to deal with the diversity of data in different categories. However, it also needs to be used with caution, especially when the amount of data is not enough to accurately estimate the covariance matrix of each class.

[0075] In one example, before training the QDA classifier, the server can collect training samples through the millimeter wave radar hardware acquisition system, and the training samples include lip reading training samples and gesture training samples. The server designs the lip reading training sample acquisition technician to read the three lip reading signals of the letters "A", "I", and "M". Each lip reading signal is repeatedly collected S times to obtain 3S lip reading training samples. The gesture training sample acquisition technician is designed to make three gesture signals of "extending the thumb", "extending the thumb and index finger", and "extending the thumb, index finger, and pinky finger". Each gesture signal needs to be repeatedly collected S times to obtain 3S gesture training samples. Then, the real label is annotated for each lip reading training sample and each gesture training sample to generate a training sample set and a test sample set.

[0076] In one example, when the server trains the QDA classifier, it first loads all the training samples in the training sample set, adjusts the size of each training sample to 100px×100px, and stores them in the array RI matrix. Then, the array RI matrix is ​​singularly decomposed, and the cumulative contribution of each singular value obtained by the singular value decomposition to the total variance is calculated. Based on the cumulative contribution rate and the preset contribution rate threshold, the number of principal components k to be retained is determined. The server selects the first k principal components to generate the feature vector of each training sample, uses the feature vector of each training sample to calculate the feature mean of each category, and then subtracts the feature mean of the corresponding category from the feature vector of each type of training sample to obtain the feature bias q of each type of training sample. a , and then based on the feature bias q of various training samples a and the feature bias q of various training samples a Transpose Calculate the covariance matrix ∑ for each category k , and calculate ∑ k -1 Afterwards, the server biases q based on the features of the test sampleb , the feature bias q of the test sample b Transpose And the ∑ k -1 Calculate the prediction value of the test sample for each category, compare the prediction value of the test sample for each category, determine the category corresponding to the minimum prediction value as the human feature recognition result of the test sample, and based on the human feature recognition result of the test sample and the true label marked on the test sample, iteratively train the QDA classifier until convergence to obtain a trained QDA classifier.

[0077] In one example, the server biases q based on the features of the test sample b , the feature bias q of the test sample b Transpose And Σ for each category k -1 Calculating the predicted value of the test sample for each category can be achieved through the following formula:

[0078]

[0079] Among them, P k Represents the calculated prediction value of the test sample for the kth category.

[0080] In one example, the preset contribution rate threshold is 95%, that is, the number k of retained principal components is determined so that the cumulative contribution rate reaches 95%.

[0081] In this embodiment, a millimeter wave radar hardware acquisition system is built to collect data and obtain target signals. The millimeter wave radar hardware acquisition system is not affected by environmental factors such as light, weather, temperature and sound, which effectively improves the stability and reliability of human feature recognition. At the same time, the millimeter wave radar hardware acquisition system does not rely on visual images and has advantages in protecting privacy. After obtaining the target signal, the target signal is filtered by a filter, and a short-time Fourier transform is performed to obtain a spectrum diagram of the target signal. The target signal is often mixed with high-frequency noise and clutter. At the same time, a considerable part of the target signal is low-frequency signals of no interest, such as the echo of a stationary object. Using a filter to filter out this useless information can reduce the complexity of subsequent processing and improve the accuracy of human feature recognition. The frequency spectrum of the target signal is converted into the form of an array RI matrix and subjected to singular value decomposition. Based on the principal components obtained after the singular value decomposition, the eigenvector of the target signal is obtained. Finally, the eigenvector of the target signal is input into the pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier. The singular value decomposition technology and QDA technology are used for human feature recognition. The singular value decomposition can compress the feature dimension, and the QDA discriminator has the characteristics of high flexibility and strong discrimination ability, which further greatly improves the stability and accuracy of human feature recognition.

[0082] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0083] Another embodiment of the present application proposes a human feature recognition system based on millimeter wave radar. The implementation details of the human feature recognition system based on millimeter wave radar proposed in this embodiment are specifically described below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0084] The structural diagram of the human feature recognition system based on millimeter wave radar proposed in this embodiment can be shown as follows: Figure 5 As shown, it includes: a hardware construction module 201, a parameter configuration module 203, a signal processing module 204, a singular value decomposition module 205 and a QDA classification module 206.

[0085] The hardware building module 201 is used to build a millimeter wave radar hardware acquisition system 202, and connect the millimeter wave radar hardware acquisition system 202 to a host computer via a USB cable and a network cable.

[0086] The parameter configuration module 203 is used to perform parameter configuration on the millimeter wave radar hardware acquisition system 202 based on the configuration software installed in the host computer, and after completing the parameter configuration, start the millimeter wave radar hardware acquisition system 202 to collect data.

[0087] The signal processing module 204 is used to filter the target signal collected by the millimeter wave radar hardware collection system 202 using a filter, and perform short-time Fourier transform on the filtered target signal to obtain a frequency spectrum of the target signal.

[0088] The singular value decomposition module 205 is used to convert the frequency spectrum of the target signal into the form of an array RI matrix and perform singular value decomposition, and obtain the eigenvector of the target signal based on the principal components obtained after the singular value decomposition.

[0089] The QDA classification module 206 is used to input the feature vector of the target signal into a pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier.

[0090] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0091] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.

[0092] In another embodiment, in order to evaluate the superiority of the human feature recognition method based on millimeter wave radar proposed in this application, technicians conducted a series of simulation experiments.

[0093] Analysis of gesture recognition results: In gesture recognition, there are three types of data collected, with twenty groups of data in each type. That is, for the sixty groups of gesture signals collected at a distance of 20 cm from the radar, the classifiers based on SVD and QDA are trained in sequence with the proportion of the training set being 50%, 60%, 70%, and 80%, and the test set is used to test its classification accuracy.

[0094] Table 2 shows the classification accuracy of the classifiers trained based on SVD and QDA on the training and test sets with different proportions of gesture recognition data.

[0095] Table 2: Classification accuracy of gesture recognition with different training set ratios

[0096] The proportion of training set 50％ 60％ 70％ 80％ Classification Accuracy 86％ 88.8％ 91.6％ 94.4％

[0097] According to the calculation method of singular value decomposition, we can calculate that when the training set proportion of gesture recognition data is 50%, 60%, 70%, and 80% respectively, we can draw the conclusion that the total amount of gesture recognition data remains unchanged, and as the proportion of training set increases, the accuracy of gesture recognition classification will gradually improve.

[0098] Analysis of lip reading recognition results: In lip reading recognition, there are three types of data collected, with twenty groups of data in each type. That is, for the sixty groups of lip reading signals collected at a distance of 20 cm from the radar, the classifiers based on SVD and QDA are trained in sequence with the proportion of training sets being 50%, 60%, 70%, and 80%, and the test set is used to test its classification accuracy.

[0099] Table 3 shows the classification accuracy of the classifiers trained based on SVD and QDA on the training and test sets of lip reading recognition data with different proportions.

[0100] Table 3: Classification accuracy of lip reading recognition with different proportions of training sets

[0101] The proportion of training set 50％ 60％ 70％ 80％ Classification Accuracy 63.3％ 75％ 83.3％ 83.3％

[0102] According to the calculation method of singular value decomposition, we can calculate that when the training set proportion of lip reading recognition data is 50%, 60%, 70%, and 80% respectively, we can draw the conclusion: the total amount of lip reading recognition data remains unchanged, and as the proportion of training set increases, the accuracy of lip reading recognition classification will gradually improve.

[0103] The application of millimeter-wave radar technology in lip reading and gesture recognition has great potential and future development prospects. With the increase in the number of people with hearing impairments, millimeter-wave radar technology can become a safe and reliable means of identification, providing more convenience for people's daily life and social interaction. In lip reading recognition, millimeter-wave radar can accurately identify the words spoken by the person being tested by capturing lip movements and changes in shape. For people with hearing impairments, this can greatly improve their communication efficiency and quality of life. In gesture recognition, millimeter-wave radar can accurately identify the meaning of gestures by capturing hand movements and posture changes, thereby achieving contactless interaction and control.

[0104] In the future, millimeter-wave radar technology will continue to develop and optimize, and its application in lip reading and gesture recognition will be more widely promoted and applied. On the one hand, millimeter-wave radar technology can improve the accuracy and stability of recognition by optimizing algorithms and adjusting parameters, while combining a large number of samples to train the model to adapt to the recognition of targets under different conditions. On the other hand, millimeter-wave radar technology can also achieve a more intelligent and automated interaction mode through integration with other sensors and smart devices.

[0105] In short, the application of millimeter-wave radar technology in lip reading recognition and gesture recognition has broad prospects and huge development potential, which will bring more convenience and innovation to people's lives and social activities. Therefore, it is of great significance for us to reproduce millimeter-wave radar for human behavior recognition and optimize various recognition algorithms.

[0106] Another embodiment of the present application provides an electronic device, the structure of which is as follows: Figure 6 As shown, it includes: at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can execute the human feature recognition method based on millimeter wave radar described in the above-mentioned method embodiments.

[0107] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and will not be further described in this article. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0108] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0109] Another embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the human feature recognition method based on millimeter-wave radar of the above-mentioned method embodiments can be implemented.

[0110] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0111] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A human feature recognition method based on millimeter wave radar, characterized in that: include: Build a millimeter-wave radar hardware acquisition system and connect it to the host computer via USB cable and network cable; The millimeter-wave radar hardware acquisition system is configured with parameters based on the configuration software installed in the host computer. After the parameter configuration is completed, the millimeter-wave radar hardware acquisition system is started to collect data; The target signal collected by the millimeter wave radar hardware acquisition system is filtered by a filter, and the filtered target signal is short-time Fourier transformed to obtain a spectrum diagram of the target signal; The frequency spectrum of the target signal is converted into the form of an array RI matrix and subjected to singular value decomposition. Based on the principal components obtained after the singular value decomposition, the eigenvector of the target signal is obtained; The feature vector of the target signal is input into the pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier.

2. The method for human feature recognition based on millimeter wave radar according to claim 1, characterized in that: Build a millimeter-wave radar hardware acquisition system and connect it to the host computer via USB cable and network cable, including: Based on the AWR2243BOOST evaluation board and the DCA1000EVM signal acquisition board, a millimeter-wave radar hardware acquisition system was built, and the DCA1000EVM signal acquisition board was connected to the host computer via a USB cable and a network cable; The millimeter-wave radar hardware acquisition system is configured with parameters based on the configuration software installed in the host computer. After completing the parameter configuration, the millimeter-wave radar hardware acquisition system is started for data acquisition, including: Based on the mmWave studio software installed in the host computer, the millimeter-wave radar hardware acquisition system is configured with parameters, and serial port communication between the millimeter-wave radar hardware acquisition system and the host computer is established. After completing the parameter configuration, the millimeter-wave radar hardware acquisition system is started for data acquisition. The AWR2243BOOST evaluation board transmits high-frequency signals to the target under test and receives echo signals, and then passes the original signals to the DCA1000EVM signal acquisition board; The DCA1000EVM signal acquisition board converts the original signal into a digital signal and transmits it to the host computer through a high-speed serial interface, USB cable and network cable. The digital signal is the target signal collected by the millimeter-wave radar hardware acquisition system.

3. The method for human feature recognition based on millimeter wave radar according to claim 1, characterized in that: The target signal collected by the millimeter wave radar hardware acquisition system is filtered using a filter, including: First, the target signal collected by the millimeter-wave radar hardware acquisition system is coarsely filtered using an MTI filter to filter out low-frequency signals of no interest and obtain the target signal after coarse filtering; Then, a Butterworth fourth-order filter is used to finely filter the target signal after coarse filtering to remove high-frequency noise and clutter signals, thereby obtaining a finely filtered target signal; Perform short-time Fourier transform on the filtered target signal to obtain the spectrum of the target signal, including: The window length, overlap percentage and FFT filling factor of the short-time Fourier transform are set. Based on the set window length, overlap percentage and FFT filling factor of the short-time Fourier transform, the finely filtered target signal is short-time Fourier transformed to obtain the spectrum diagram of the target signal.

4. The method for human feature recognition based on millimeter wave radar according to claim 1, characterized in that: Before training the QDA classifier, the millimeter wave radar hardware acquisition system is used to collect training samples, including lip reading training samples and gesture training samples; Design lip reading training sample collection technicians read three lip reading signals of letters "A", "I", and "M", and each lip reading signal is repeatedly collected S times to obtain 3S lip reading training samples; Design gesture training samples to collect three gesture signals of technicians making "extending thumb", "extending thumb and index finger", and "extending thumb, index finger and pinky finger". Each gesture signal is collected repeatedly S times to obtain 3S gesture training samples; The true labels are marked for each lip reading training sample and each gesture training sample to generate a training sample set and a test sample set.

5. The method for human feature recognition based on millimeter wave radar according to claim 4, characterized in that: Training the QDA classifier includes: Load all training samples in the training sample set, adjust the size of each training sample to 100px×100px, and store them in the array RI matrix; Perform singular value decomposition on the array RI matrix, and calculate the cumulative contribution of each singular value obtained by the singular value decomposition to the total variance, and determine the number k of principal components to be retained based on the cumulative contribution rate and the preset contribution rate threshold; Select the first k principal components to generate the feature vector of each training sample, use the feature vector of each training sample to calculate the feature mean of each category, and then subtract the feature mean of the corresponding category from the feature vector of each type of training sample to obtain the feature bias q of each type of training sample a ; Feature bias q based on various training samples a and the feature bias q of various training samples a Transpose Calculate the covariance matrix Σ for each category k , and calculate Σ k -1 ; Feature bias q based on test samples b , the feature bias q of the test sample b Transpose And Σ for each category k -1 Calculate the prediction value of the test sample for each category, compare the prediction value of the test sample for each category, and determine the category corresponding to the minimum prediction value as the human feature recognition result of the test sample; Based on the human feature recognition results of the test samples and the true labels marked on the test samples, the QDA classifier is iteratively trained until convergence to obtain a trained QDA classifier.

6. The method for human feature recognition based on millimeter wave radar according to claim 5, characterized in that: Feature bias q based on test samples b , the feature bias q of the test sample b Transpose And Σ for each category k -1 Calculate the predicted value of the test sample for each category using the following formula: Among them, P k Represents the calculated prediction value of the test sample for the kth category.

7. The method for human feature recognition based on millimeter wave radar according to claim 5, characterized in that: The preset contribution rate threshold is 95%, that is, the number k of principal components retained is determined so that the cumulative contribution rate reaches 95%.

8. A human feature recognition system based on millimeter wave radar, characterized in that: include: Hardware building module, parameter configuration module, signal processing module, singular value decomposition module and QDA classification module; The hardware building module is used to build the millimeter-wave radar hardware acquisition system and connect the millimeter-wave radar hardware acquisition system to the host computer through a USB cable and a network cable; The parameter configuration module is used to configure the parameters of the millimeter-wave radar hardware acquisition system based on the configuration software installed in the host computer. After completing the parameter configuration, the millimeter-wave radar hardware acquisition system is started to collect data; The signal processing module is used to filter the target signal collected by the millimeter wave radar hardware acquisition system using a filter, and perform short-time Fourier transform on the filtered target signal to obtain a spectrum diagram of the target signal; The singular value decomposition module is used to convert the frequency spectrum of the target signal into the form of an array RI matrix and perform singular value decomposition, and obtain the eigenvector of the target signal based on the principal components obtained after the singular value decomposition; The QDA classification module is used to input the feature vector of the target signal into a pre-trained QDA classifier to obtain the human feature recognition result of the target signal output by the QDA classifier.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the human feature recognition method based on millimeter wave radar as described in any one of claims 1 to claim 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement the human feature recognition method based on millimeter wave radar as described in any one of claims 1 to 7.

Images