Human body perception method and system based on millimeter wave radar

Abstract

The present application discloses a human body perception method and system based on millimeter wave radar, which relates to the field of millimeter wave radar perception technology. It transmits radar signals multiple times through millimeter wave radar, and parses the received radar echoes to obtain a sequence of echo signals, and uses artificial intelligence-based signal processing and analysis technology to extract the time-frequency features and semantic association of the echo signals, so as to automatically obtain the perception result of whether a person exists based on the contextual semantic association features of the time-frequency features of the multiple echo signals in the time series. The human body perception method of the present application can obtain more fine-grained echo signal time series data by transmitting radar signals multiple times, and uses artificial intelligence to perform real-time analysis of the received echo signals to identify and analyze complex signal patterns, so as to understand the meaning and characteristics of the echo signals more carefully, which helps to more accurately capture the changes in the human body state, thereby improving the accuracy of human body perception.

Description

Technical Field

[0001] The present application relates to the field of millimeter wave radar sensing technology, and more specifically, to a human body sensing method and system based on millimeter wave radar. Background Art

[0002] Human perception is a technology that uses various sensors and algorithms to monitor and identify the presence, position, movement and behavior of the human body. This technology has applications in many fields, including but not limited to medical health monitoring, smart home, security monitoring and human-computer interaction.

[0003] At present, the mainstream sensor technologies used to detect the distribution of people around mainly include infrared sensors, microwave sensors and electric field sensors. Infrared sensors are favored for their wide application and high sensitivity to detecting moving and stationary human bodies. However, there are various interference factors in the detection environment, such as heat sources and hot and cold wind currents. Infrared sensors may generate false alarms, resulting in a high false alarm rate. Microwave sensors use the Doppler effect to detect moving objects and can accurately identify moving human bodies, but they may have difficulties in distinguishing certain objects, which may lead to false detection or missed detection. Electric field sensors sense the human body by detecting tiny changes in capacitance in the surrounding electric field, but because the distribution of the human body's electric field has a large inconsistency, this affects the stability and detection range of the electric field sensor, making it limited in reliability. Therefore, Chinese patent CN110045370A discloses a human body perception method based on millimeter wave radar, which uses a radar module to transmit a radar signal and receive an echo signal, and extracts motion information and physiological information by performing two discrete Fourier transforms on the echo signal, respectively, and combines these information to determine whether there are people in the current environment and the status of the people.

[0004] However, the above-mentioned prior art extracts motion information and physiological information respectively through two discrete Fourier transforms. This double processing increases the complexity and computational cost of the algorithm, and may cause delays, especially in real-time applications. In addition, using only discrete Fourier transform to extract features may limit the accurate recognition of complex human behaviors and fail to fully capture the diversity of human activities.

[0005] Therefore, an optimized millimeter-wave radar-based human perception method is desired. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a human perception method and system based on millimeter wave radar, which transmits radar signals multiple times through millimeter wave radar, and parses the received radar echoes to obtain a sequence of echo signals, and uses artificial intelligence-based signal processing and analysis technology to extract the time-frequency features and semantic association of the echo signals, so as to automatically obtain the perception result of whether a person exists based on the contextual semantic association features of the time-frequency features of the multiple echo signals in the time series. The human perception method of the present application can obtain more fine-grained echo signal time series data by transmitting radar signals multiple times, and uses artificial intelligence to perform real-time analysis of the received echo signals to identify and analyze complex signal patterns, so as to understand the meaning and characteristics of the echo signals more carefully, which helps to more accurately capture the changes in the human body state, thereby improving the accuracy of human perception.

[0007] According to one aspect of the present application, a human body perception method based on millimeter wave radar is provided, which includes:

[0008] The radar signal is transmitted multiple times by the millimeter wave radar, and the received radar echo is analyzed to obtain a sequence of echo signals;

[0009] Performing time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of the echo signals;

[0010] Extracting echo signal time-frequency features from each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain a sequence of echo signal time-frequency associated semantic feature vectors;

[0011] Inputting the sequence of the echo signal time-frequency associated semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency associated semantic feature vectors;

[0012] Performing temporal context association coding on the sequence of the optimized echo signal time-frequency association semantic feature vectors to obtain an echo signal temporal context semantic association feature vector as an echo signal temporal context semantic association feature;

[0013] Based on the time series context semantic association feature of the echo signal, a perception result is obtained, and the perception result is used to indicate whether a person exists.

[0014] In the above-mentioned human body perception method based on millimeter wave radar, each echo signal in the sequence of echo signals is time-frequency converted to obtain a sequence of two-dimensional time-frequency images of echo signals, including: performing wavelet transform on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of echo signals.

[0015] In the above-mentioned human body perception method based on millimeter wave radar, each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images is subjected to echo signal time-frequency feature extraction to obtain a sequence of echo signal time-frequency associated semantic feature vectors, including: each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images is passed through an echo signal time-frequency feature extractor based on a void convolutional neural network model to obtain a sequence of echo signal time-frequency associated semantic feature vectors.

[0016] In the above-mentioned human perception method based on millimeter wave radar, the sequence of the echo signal time-frequency association semantic feature vectors is input into the feature enhancement network to obtain the sequence of optimized echo signal time-frequency association semantic feature vectors, including: calculating the mean vector of the sequence of the echo signal time-frequency association semantic feature vectors to obtain the echo signal time-frequency association semantic sequence distribution center vector; calculating the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector and the echo signal time-frequency association semantic sequence distribution center vector to obtain the sequence of echo signal time-frequency association semantic implicit correlation coefficients; calculating the reciprocal of each echo signal time-frequency association semantic implicit correlation coefficient in the sequence of the echo signal time-frequency association semantic implicit correlation coefficient to obtain the sequence of echo signal time-frequency association semantic anti-correlation coefficients; using the Softmax function The sequence of the echo signal time-frequency association semantic anti-correlation coefficients is normalized to obtain a sequence of the echo signal time-frequency association semantic anti-correlation energy coefficients; each echo signal time-frequency association semantic anti-correlation energy coefficient in the sequence of the echo signal time-frequency association semantic anti-correlation energy coefficients is used as a weight to weight each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vectors to obtain a sequence of inversely suppressed echo signal time-frequency association semantic feature vectors; the positional difference of each corresponding echo signal time-frequency association semantic feature vector and the inversely suppressed echo signal time-frequency association semantic feature vector between the sequence of the echo signal time-frequency association semantic feature vectors and the sequence of inversely suppressed echo signal time-frequency association semantic feature vectors is calculated to obtain a sequence of optimized echo signal time-frequency association semantic feature vectors.

[0017] In the above-mentioned human perception method based on millimeter wave radar, the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector and the center vector of the echo signal time-frequency association semantic sequence distribution is calculated to obtain a sequence of echo signal time-frequency association semantic implicit correlation coefficients, including: calculating the position point multiplication between the echo signal time-frequency association semantic feature vector and the center vector of the echo signal time-frequency association semantic sequence distribution to obtain the echo signal time-frequency association semantic class support feature vector; calculating the sum of the eigenvalues ​​of each position in the echo signal time-frequency association semantic class support feature vector to obtain the echo signal time-frequency association semantic class support feature vector. Semantic class support sum value; calculate the square sum of the eigenvalues ​​at each position in the echo signal time-frequency association semantic feature vector and the square sum of the eigenvalues ​​at each position in the echo signal time-frequency association semantic sequence distribution center vector to obtain the echo signal time-frequency association semantic square value and the echo signal time-frequency association semantic sequence distribution center square value; calculate the square root of the sum of the echo signal time-frequency association semantic square value and the echo signal time-frequency association semantic sequence distribution center square value to obtain the echo signal time-frequency association value; divide the echo signal time-frequency association semantic class support sum value by the echo signal time-frequency association value to obtain the echo signal time-frequency association semantic implicit association coefficient.

[0018] In the above-mentioned human body perception method based on millimeter wave radar, the sequence of the time-frequency association semantic anti-correlation coefficients of the echo signal is normalized using the Softmax function to obtain a sequence of the time-frequency association semantic anti-correlation energy coefficients of the echo signal, including: using each echo signal time-frequency association semantic anti-correlation coefficient of the sequence of the time-frequency association semantic anti-correlation coefficients of the echo signal as an exponent of a natural constant to calculate an exponential function value based on the natural constant according to the position to obtain a sequence of the time-frequency association semantic anti-correlation exponential coefficients of the echo signal; calculating the time-frequency association semantic anti-correlation exponential coefficients of the echo signal. The sum of the time-frequency association semantic anti-correlation index coefficients of each echo signal in the sequence is obtained to obtain the total sum of the time-frequency association semantic anti-correlation index coefficients of the echo signal; the sum of the total sum of the time-frequency association semantic anti-correlation index coefficients of the echo signal and a constant one is calculated to obtain the time-frequency association semantic anti-correlation modulation coefficient of the echo signal; each echo signal time-frequency association semantic anti-correlation index coefficient in the sequence of the echo signal time-frequency association semantic anti-correlation index coefficients is divided by the echo signal time-frequency association semantic anti-correlation modulation coefficient to obtain a sequence of the echo signal time-frequency association semantic anti-correlation energy coefficients.

[0019] In the above-mentioned human perception method based on millimeter wave radar, the sequence of the optimized echo signal time-frequency association semantic feature vectors is subjected to temporal context association encoding to obtain the echo signal temporal context semantic association feature vector, including: inputting the sequence of the optimized echo signal time-frequency association semantic feature vectors into the echo signal temporal context association encoder based on the Bi-LSTM model to obtain the echo signal temporal context semantic association feature vector.

[0020] In the above-mentioned human body perception method based on millimeter wave radar, a perception result is obtained based on the temporal context semantic association feature of the echo signal, including: inputting the temporal context semantic association feature vector of the echo signal into a human body sensor based on a classifier to obtain the perception result.

[0021] According to another aspect of the present application, a human body perception system based on millimeter wave radar is provided, comprising:

[0022] A radar signal transmission and analysis module is used to transmit radar signals multiple times through a millimeter wave radar, and analyze the received radar echoes to obtain a sequence of echo signals;

[0023] An echo signal time-frequency conversion module, used for performing time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of the echo signals;

[0024] A signal time-frequency associated semantic feature extraction module, used for extracting echo signal time-frequency features from each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain a sequence of echo signal time-frequency associated semantic feature vectors;

[0025] A signal time-frequency association semantic feature optimization module, used for inputting the sequence of echo signal time-frequency association semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency association semantic feature vectors;

[0026] A signal time sequence context semantic association module, used for performing time sequence context association encoding on the sequence of the optimized echo signal time-frequency association semantic feature vector to obtain an echo signal time sequence context semantic association feature vector as an echo signal time sequence context semantic association feature;

[0027] The perception result generation module is used to obtain a perception result based on the semantic association feature of the time sequence context of the echo signal, and the perception result is used to indicate whether a person exists.

[0028] In the above-mentioned millimeter-wave radar-based human perception system, the signal time-frequency association semantic feature extraction module is used to: respectively pass each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images through an echo signal time-frequency feature extractor based on a void convolutional neural network model to obtain a sequence of the echo signal time-frequency association semantic feature vectors.

[0029] The present application has at least the following technical effects: Compared with the prior art, the present application provides a human perception method and system based on millimeter wave radar, which transmits radar signals multiple times through millimeter wave radar, and parses the received radar echoes to obtain a sequence of echo signals, and uses artificial intelligence-based signal processing and analysis technology to extract the time-frequency features and semantic association of the echo signals, so as to automatically obtain the perception result of whether a person exists based on the contextual semantic association features of the time-frequency features of the multiple echo signals in the time series. The human perception method of the present application can obtain more fine-grained echo signal time series data by transmitting radar signals multiple times, and uses artificial intelligence to perform real-time analysis of the received echo signals to identify and analyze complex signal patterns, thereby more carefully understanding the meaning and characteristics of the echo signals, helping to more accurately capture changes in human state, and thus improve the accuracy of human perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0031] Figure 1 The figure is a flow chart of a human body perception method based on millimeter wave radar according to an embodiment of the present application.

[0032] Figure 2 Schematic diagram of the architecture of a human body perception method based on millimeter wave radar according to an embodiment of the present application.

[0033] Figure 3 The present invention is a flowchart of inputting the sequence of time-frequency associated semantic feature vectors of the echo signal into a feature enhancement network to obtain a sequence of optimized time-frequency associated semantic feature vectors of the echo signal in the millimeter wave radar-based human perception method according to an embodiment of the present application.

[0034] Figure 4A flowchart for calculating the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of echo signal time-frequency association semantic feature vectors and the center vector of the echo signal time-frequency association semantic sequence distribution in the millimeter wave radar-based human perception method according to an embodiment of the present application to obtain a sequence of echo signal time-frequency association semantic implicit correlation coefficients.

[0035] Figure 5 The present invention is a flowchart of using a Softmax function to normalize the sequence of time-frequency correlation semantic anti-correlation coefficients of the echo signal in a human perception method based on a millimeter-wave radar according to an embodiment of the present application to obtain a sequence of time-frequency correlation semantic anti-correlation energy coefficients of the echo signal.

[0036] Figure 6 4 is a block diagram of a human perception system based on millimeter wave radar according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0038] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0039] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0040] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0041] Human perception is a technology that uses various sensors and algorithms to monitor and identify the presence, position, movement and behavior of the human body. This technology has applications in many fields, including but not limited to medical health monitoring, smart home, security monitoring and human-computer interaction.

[0042] The technical concept of the present application is to transmit radar signals multiple times through millimeter wave radar, and parse the received radar echoes to obtain a sequence of echo signals, and use artificial intelligence-based signal processing and analysis technology to extract the time-frequency features and semantic association of the echo signals, so as to automatically obtain the perception result of whether a person exists based on the contextual semantic association features of the time-frequency features of the multiple echo signals in the time series. The human body perception method of the present application can obtain more fine-grained echo signal time series data by transmitting radar signals multiple times, and use artificial intelligence to perform real-time analysis of the received echo signals to identify and analyze complex signal patterns, so as to understand the meaning and characteristics of the echo signals more carefully, which helps to more accurately capture changes in human body status, thereby improving the accuracy of human body perception.

[0043] Specifically, Figure 1 The figure is a flow chart of a human body perception method based on millimeter wave radar according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of a human body perception method based on millimeter wave radar according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the human perception method based on millimeter wave radar includes: S110, transmitting radar signals multiple times by millimeter wave radar, and parsing the received radar echoes to obtain a sequence of echo signals; S120, performing time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of echo signals; S130, performing echo signal time-frequency feature extraction on each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain a sequence of echo signal time-frequency associated semantic feature vectors; S140, inputting the sequence of echo signal time-frequency associated semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency associated semantic feature vectors; S150, performing temporal context association encoding on the sequence of optimized echo signal time-frequency associated semantic feature vectors to obtain an echo signal temporal context semantic association feature vector as an echo signal temporal context semantic association feature; S160, obtaining a perception result based on the echo signal temporal context semantic association feature, and the perception result is used to indicate whether a person exists.

[0044] In step S110, the millimeter wave radar transmits radar signals multiple times, and the received radar echoes are parsed to obtain a sequence of echo signals. It should be understood that the millimeter wave radar is a radar technology that works in the millimeter wave band (30~300 GHz, wavelength of 1~10 mm). Due to the short wavelength of the millimeter wave, the millimeter wave radar has the characteristics of high resolution and high precision, which enables it to accurately detect and locate the human body, and even recognize small movements and posture changes. Based on this, in the technical solution of the present application, the millimeter wave radar transmits radar signals multiple times, and the received radar echoes are parsed to obtain a sequence of echo signals, and the signal analysis and processing are performed on them, so as to accurately perceive the situation of the personnel.

[0045] In step S120, each echo signal in the sequence of the echo signals is subjected to time-frequency conversion to obtain a sequence of two-dimensional time-frequency images of the echo signals. Specifically, in an embodiment of the present application, each echo signal in the sequence of the echo signals is subjected to time-frequency conversion to obtain a sequence of two-dimensional time-frequency images of the echo signals, including: each echo signal in the sequence of the echo signals is subjected to wavelet transformation to obtain a sequence of two-dimensional time-frequency images of the echo signals. Accordingly, considering that each echo signal in the sequence of the echo signals contains frequency characteristic information of different time periods of the signal, these characteristic information play an important role in the subsequent judgment of whether a person exists. Therefore, in the technical solution of the present application, each echo signal in the sequence of the echo signals is subjected to wavelet transformation to obtain a sequence of two-dimensional time-frequency images of the echo signals. It should be understood that wavelet transformation can simultaneously capture the localized features of the time domain and frequency domain of the signal, which helps to accurately locate the instantaneous features of the signal in the time-frequency domain of the echo signal, thereby helping to identify information such as the presence, activity and posture of the human body.

[0046] In step S130, each echo signal two-dimensional time-frequency image in the sequence of the echo signal two-dimensional time-frequency images is subjected to echo signal time-frequency feature extraction to obtain a sequence of echo signal time-frequency associated semantic feature vectors. Specifically, in an embodiment of the present application, each echo signal two-dimensional time-frequency image in the sequence of the echo signal two-dimensional time-frequency images is subjected to echo signal time-frequency feature extraction to obtain a sequence of echo signal time-frequency associated semantic feature vectors, including: each echo signal two-dimensional time-frequency image in the sequence of the echo signal two-dimensional time-frequency images is subjected to an echo signal time-frequency feature extractor based on a hole convolutional neural network model to obtain a sequence of echo signal time-frequency associated semantic feature vectors. Considering that different local features in each echo signal two-dimensional time-frequency image in the sequence of the echo signal two-dimensional time-frequency images have different time-frequency feature information, the hole convolution can expand the receptive field of the convolutional neural network and effectively capture the time-frequency features of different positions, which helps to improve the context association of feature extraction. Therefore, in the technical solution of the present application, each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images is respectively passed through an echo signal time-frequency feature extractor based on a void convolutional neural network model to respectively capture and extract the correlation between the local time-frequency characteristics in each echo signal two-dimensional time-frequency image, thereby obtaining a sequence of echo signal time-frequency correlation semantic feature vectors.

[0047] In step S140, the sequence of the time-frequency association semantic feature vectors of the echo signal is input into the feature enhancement network to obtain a sequence of optimized time-frequency association semantic feature vectors of the echo signal. Accordingly, considering that each time-frequency association semantic feature vector of the echo signal in the sequence of the time-frequency association semantic feature vectors of the echo signal expresses the time-frequency association semantic features of the echo signal in a local time period, which contains the key loop time-frequency association semantic features of the echo signal that are closely related to the judgment of the personnel, but there are also some features, which are not significant enough in terms of the correlation between them and the overall echo signal. Therefore, in order to extract the most influential and valuable features from the sequence of the time-frequency association semantic feature vectors of the echo signal, while reducing the interference of those irrelevant or minor features, and then optimizing the performance and role of these key features in the judgment of the personnel, in the technical solution of the present application, the input feature enhancement network is used to obtain a sequence of optimized time-frequency association semantic feature vectors of the echo signal. Specifically, first, the average value of the time-frequency association semantic feature vectors of the echo signal is calculated to understand the basic mode of the entire time-frequency association semantic data set of the echo signal and obtain the distribution center vector of the time-frequency association semantic sequence of the echo signal. Next, the correlation between each echo signal time-frequency association semantic feature vector and the average feature vector is calculated to identify features that are highly correlated with the overall trend. Then, the inverse of each correlation is calculated to obtain the anti-correlation coefficient, which helps to increase the weight of features with high correlation and reduce the weight of features with low correlation in subsequent processing. Next, each anti-correlation coefficient is normalized using the Softmax function to ensure that the sum of the weights of all features is 1, which helps to balance the importance of different features in the model, and the normalized weights are applied to each echo signal time-frequency association semantic feature vector to obtain a sequence of weighted reverse suppression feature vectors. Finally, the difference between the echo signal time-frequency association semantic feature vector and the weighted feature vector is calculated to reversely screen out features that contribute more to the model performance, while reducing the impact of noise, so as to obtain a more representative sequence of optimized echo signal time-frequency association semantic feature vectors.

[0048] Figure 3 The present invention is a flowchart of inputting the sequence of the time-frequency associated semantic feature vectors of the echo signal into the feature enhancement network to obtain the sequence of the time-frequency associated semantic feature vectors of the echo signal in the human body perception method based on the millimeter wave radar according to the embodiment of the present application. Specifically, in the embodiment of the present application, Figure 3As shown, the sequence of the echo signal time-frequency association semantic feature vectors is input into the feature enhancement network to obtain the sequence of optimized echo signal time-frequency association semantic feature vectors, including: S210, calculating the mean vector of the sequence of the echo signal time-frequency association semantic feature vectors to obtain the echo signal time-frequency association semantic sequence distribution center vector; S220, calculating the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vectors and the echo signal time-frequency association semantic sequence distribution center vector to obtain the sequence of echo signal time-frequency association semantic implicit correlation coefficients; S230, calculating the reciprocal of each echo signal time-frequency association semantic implicit correlation coefficient in the sequence of the echo signal time-frequency association semantic implicit correlation coefficients to obtain the sequence of echo signal time-frequency association semantic anti-correlation coefficients; S240, using the Softmax function to The sequence of echo signal time-frequency association semantic anti-correlation coefficients is normalized to obtain a sequence of echo signal time-frequency association semantic anti-correlation energy coefficients; S250, using each echo signal time-frequency association semantic anti-correlation energy coefficient in the sequence of echo signal time-frequency association semantic anti-correlation energy coefficients as a weight, weighting each echo signal time-frequency association semantic feature vector in the sequence of echo signal time-frequency association semantic feature vectors to obtain a sequence of inversely suppressed echo signal time-frequency association semantic feature vectors; S260, calculating the positional difference of each corresponding echo signal time-frequency association semantic feature vector and the inversely suppressed echo signal time-frequency association semantic feature vector between the sequence of echo signal time-frequency association semantic feature vectors and the sequence of inversely suppressed echo signal time-frequency association semantic feature vectors to obtain the sequence of optimized echo signal time-frequency association semantic feature vectors.

[0049] Figure 4 The present invention is a flowchart of calculating the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector and the echo signal time-frequency association semantic sequence distribution center vector in the human body perception method based on the millimeter wave radar according to the embodiment of the present application to obtain the sequence of echo signal time-frequency association semantic implicit correlation coefficients. More specifically, in the embodiment of the present application, if Figure 4As shown, the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector and the center vector of the echo signal time-frequency association semantic sequence distribution is calculated to obtain a sequence of echo signal time-frequency association semantic implicit correlation coefficients, including: S310, calculating the position point multiplication between the echo signal time-frequency association semantic feature vector and the center vector of the echo signal time-frequency association semantic sequence distribution to obtain the echo signal time-frequency association semantic class support feature vector; S320, calculating the sum of the feature values ​​at each position in the echo signal time-frequency association semantic class support feature vector to obtain the echo signal time-frequency association semantic class support sum value; S3 30, calculate the sum of the squares of the eigenvalues ​​at each position in the echo signal time-frequency association semantic feature vector and the sum of the squares of the eigenvalues ​​at each position in the echo signal time-frequency association semantic sequence distribution center vector to obtain the echo signal time-frequency association semantic square value and the echo signal time-frequency association semantic sequence distribution center square value; S340, calculate the square root of the sum of the echo signal time-frequency association semantic square value and the echo signal time-frequency association semantic sequence distribution center square value to obtain the echo signal time-frequency association value; S350, divide the echo signal time-frequency association semantic class support sum value by the echo signal time-frequency association value to obtain the echo signal time-frequency association semantic implicit association coefficient.

[0050] Figure 5 The present invention is a flowchart of using a Softmax function to normalize the sequence of time-frequency correlation semantic anti-correlation coefficients of the echo signal in the human body perception method based on millimeter wave radar according to an embodiment of the present application to obtain a sequence of time-frequency correlation semantic anti-correlation energy coefficients of the echo signal. More specifically, in the embodiment of the present application, Figure 5As shown, the sequence of the echo signal time-frequency association semantic anti-correlation coefficients is normalized using the Softmax function to obtain a sequence of the echo signal time-frequency association semantic anti-correlation energy coefficients, including: S410, using each echo signal time-frequency association semantic anti-correlation coefficient of the sequence of the echo signal time-frequency association semantic anti-correlation coefficients as an exponent of a natural constant to calculate an exponential function value based on the natural constant according to the position to obtain a sequence of the echo signal time-frequency association semantic anti-correlation exponential coefficients; S420, calculating each echo signal in the sequence of the echo signal time-frequency association semantic anti-correlation exponential coefficients. S430, calculate the sum of the total sum of the time-frequency association semantic anti-correlation index coefficients of the echo signal and the constant one to obtain the time-frequency association semantic anti-correlation modulation coefficient of the echo signal; S440, divide each echo signal time-frequency association semantic anti-correlation index coefficient in the sequence of the echo signal time-frequency association semantic anti-correlation index coefficients by the echo signal time-frequency association semantic anti-correlation modulation coefficient to obtain the sequence of the echo signal time-frequency association semantic anti-correlation energy coefficients.

[0051] In an embodiment of the present application, specifically, the sequence of the time-frequency associated semantic feature vectors of the echo signal is input into a feature enhancement network to obtain a sequence of optimized time-frequency associated semantic feature vectors of the echo signal, which is used to: input the sequence of the time-frequency associated semantic feature vectors of the echo signal into a feature enhancement network, and process it with the following optimization formula to obtain the sequence of optimized time-frequency associated semantic feature vectors of the echo signal; wherein the optimization formula is:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] in, is a sequence of time-frequency associated semantic feature vectors of the echo signal, is the number of feature vectors in the sequence of the echo signal time-frequency associated semantic feature vectors, is the first in the sequence of the echo signal time-frequency associated semantic feature vector echo signal time-frequency correlation semantic feature vector, is the center vector of the time-frequency associated semantic sequence distribution of the echo signal, It is the point multiplication by position. is the number of eigenvalues ​​in each of the echo signal time-frequency associated semantic feature vectors, It is The echo signal time-frequency associated semantic feature vector The eigenvalues ​​at the positions, is the center vector of the time-frequency correlation semantic sequence distribution of the echo signal The eigenvalues ​​at the positions, is the first in the sequence of the time-frequency correlation semantic implicit correlation coefficients of the echo signal The time-frequency correlation semantic implicit correlation coefficient of the echo signal, is the first in the sequence of the time-frequency correlation semantic anti-correlation coefficients of the echo signal The time-frequency correlation semantic anti-correlation coefficient of the echo signal, is the first in the sequence of the time-frequency correlation semantic anti-correlation energy coefficients of the echo signal The time-frequency correlation semantic anti-correlation energy coefficient of the echo signal, is a sequence of time-frequency associated semantic feature vectors of the reverse suppressed echo signal, is the first in the sequence of the time-frequency associated semantic feature vector of the reverse suppressed echo signal The time-frequency correlation semantic feature vector of the reverse suppressed echo signal, is the sequence of the optimized echo signal time-frequency associated semantic feature vectors, is the first in the sequence of the optimized echo signal time-frequency associated semantic feature vector An optimized echo signal time-frequency correlation semantic feature vector.

[0059] In step S150, the sequence of the optimized echo signal time-frequency association semantic feature vectors is subjected to temporal context association encoding to obtain the echo signal temporal context semantic association feature vector as the echo signal temporal context semantic association feature. Specifically, in an embodiment of the present application, the sequence of the optimized echo signal time-frequency association semantic feature vectors is subjected to temporal context association encoding to obtain the echo signal temporal context semantic association feature vector, including: inputting the sequence of the optimized echo signal time-frequency association semantic feature vectors into the echo signal temporal context association encoder based on the Bi-LSTM model to obtain the echo signal temporal context semantic association feature vector. It should be understood that, considering that there is an association between the contexts of the echo signal with different time spans between the sequence of the optimized echo signal time-frequency association semantic feature vectors, the Bi-LSTM model can effectively capture the long-term dependencies between different time scales in the time series data, which is very effective for processing time series data. Based on this, in the technical solution of the present application, the sequence of the optimized echo signal time-frequency association semantic feature vectors is input into the echo signal temporal context association encoder based on the Bi-LSTM model to mine and extract the time-frequency semantic associations between the contexts existing between each optimized echo signal time-frequency association semantic feature, thereby obtaining the echo signal temporal context semantic association feature vector.

[0060] In step S160, based on the semantic association features of the time sequence context of the echo signal, a perception result is obtained, and the perception result is used to indicate whether a person exists. Specifically, in an embodiment of the present application, based on the semantic association features of the time sequence context of the echo signal, a perception result is obtained, including: inputting the semantic association feature vector of the time sequence context of the echo signal into a human sensor based on a classifier to obtain the perception result. That is, the semantic association features of the time sequence context of the echo signal obtained by context association encoding using the sequence of the optimized time-frequency association semantic feature vector of the echo signal are classified and processed, so as to automatically obtain the perception result of whether a person exists. By transmitting radar signals multiple times, more fine-grained echo signal time sequence data can be obtained, and artificial intelligence is used to perform real-time analysis of the received echo signal to identify and analyze complex signal patterns, so as to more accurately understand the meaning and characteristics of the echo signal, which helps to more accurately capture changes in human body status, thereby improving the accuracy of human perception.

[0061] It should be understood that in the technical solution of the present application, each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vectors respectively represents the local time-frequency image semantic features of the echo signal two-dimensional time-frequency image determined by wavelet transform of each echo signal in the sequence of the echo signal. Although the feature expression significance of each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector can be effectively improved after being processed by the feature enhancement network, this will also cause when the sequence of the optimized echo signal time-frequency association semantic feature vector is input into the echo signal temporal context association encoder based on the Bi-LSTM model, the echo signal temporal context association encoder based on the Bi-LSTM model will be overly attracted by the significant information in the feature sequence and ignore other useful information, such as feature complementary information, etc., thereby causing the echo signal temporal context semantic association feature vector to have a more significant local distribution difference, which makes it have a harmonious offset relative to the predetermined category label of the classifier, affecting the classification iteration effect.

[0062] Based on this, in a preferred embodiment of the present application, before the echo signal temporal context semantic association feature vector is input into the human sensor based on the classifier, the echo signal temporal context semantic association feature vector is subjected to feature distribution modulation, and the process includes:

[0063] Determine the mean and variance of a feature set composed of feature values ​​of the echo signal temporal context semantic association feature vector, and divide the mean by the variance to obtain an echo signal temporal context semantic association probability weight value:

[0064]

[0065] in, and represent the mean and variance of the feature set composed of the feature values ​​of the semantic association feature vector of the temporal context of the echo signal, respectively, Indicates the semantic association probability weight value of the echo signal temporal context;

[0066] The echo signal temporal context semantic association feature vector is multiplied by the inverse of the maximum eigenvalue of the echo signal temporal context semantic association feature vector to obtain the echo signal temporal context semantic association probability constraint vector:

[0067]

[0068] in, represents the inverse of the maximum eigenvalue of the semantic association feature vector of the echo signal temporal context, represents the semantic association feature vector of the echo signal temporal context, It means point multiplication by position. Represents the semantic association probability constraint vector of the echo signal temporal context;

[0069] The echo signal temporal context semantic association probability constraint vector is added with the echo signal temporal context semantic association probability weight value, and the logarithm with base 2 is calculated to obtain the echo signal temporal context semantic association information interaction vector:

[0070]

[0071] in, represents the semantic association probability constraint vector of the echo signal temporal context, represents the semantic association probability weight value of the echo signal temporal context, Indicates adding by position point. Represents the interactive vector of semantic association information of the echo signal temporal context;

[0072] After subtracting the echo signal temporal context semantic association probability constraint vector from the echo signal temporal context semantic association probability weight value, the inverse of each eigenvalue is calculated to obtain the echo signal temporal context semantic association sequence constraint vector:

[0073]

[0074] in, represents the semantic association probability constraint vector of the echo signal temporal context, represents the semantic association probability weight value of the echo signal temporal context, It means to subtract by position point. Represents the semantic association sequence constraint vector of the echo signal temporal context;

[0075] Finally, the echo signal temporal context semantic association information interaction vector is added to the echo signal temporal context semantic association sequence constraint vector to obtain an optimized echo signal temporal context semantic association feature vector.

[0076] The above optimization process simulates the mesoscale interaction structure under probability constraints between the eigenvalue scale and the eigenvector scale of the echo signal temporal context semantic association feature vector through the probability statistics characteristics of the echo signal temporal context semantic association feature vector, so as to construct a bidirectional hidden variable motif based on the mesoscale short sequence relative to the probability statistics value to perform mesoscale bidirectional migration, and perform posterior recovery based on the sequence constraints on the interactive information, thereby improving the convergence effect of the echo signal temporal context semantic association feature vector in the probability density domain, thereby improving the feature set of the echo signal temporal context semantic association feature vector in the predetermined category harmony in the associated target classification domain. In this way, the speed of classification training and the accuracy of classification results of the echo signal temporal context semantic association feature vector when it is classified by the human sensor based on the classifier are improved. In this way, more fine-grained echo signal temporal data can be obtained by transmitting radar signals multiple times, and artificial intelligence is used to perform real-time analysis of the received echo signal to identify and analyze complex signal patterns, so as to understand the meaning and characteristics of the echo signal more carefully, which helps to capture the changes in the human body state more accurately, thereby improving the accuracy of human body perception.

[0077] In summary, a human perception method based on millimeter-wave radar according to an embodiment of the present application is explained, which transmits radar signals multiple times through millimeter-wave radar, and parses the received radar echoes to obtain a sequence of echo signals, and uses artificial intelligence-based signal processing and analysis technology to extract time-frequency features and semantic association of the echo signals, so as to automatically obtain the perception result of whether a person exists based on the contextual semantic association features of the time-frequency features of the multiple echo signals in the time series. The human perception method of the present application can obtain more fine-grained echo signal time series data by transmitting radar signals multiple times, and uses artificial intelligence to perform real-time analysis of the received echo signals to identify and analyze complex signal patterns, thereby more carefully understanding the meaning and characteristics of the echo signals, which helps to more accurately capture changes in human body status, thereby improving the accuracy of human perception.

[0078] Figure 6 FIG. 1 is a block diagram of a human body perception system based on a millimeter wave radar according to an embodiment of the present application. Figure 6As shown, according to an embodiment of the present application, a human perception system 100 based on a millimeter wave radar includes: a radar signal transmission and analysis module 110, which is used to transmit radar signals multiple times through a millimeter wave radar, and analyze the received radar echoes to obtain a sequence of echo signals; an echo signal time-frequency conversion module 120, which is used to perform time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of echo signals; a signal time-frequency associated semantic feature extraction module 130, which is used to perform echo signal time-frequency feature extraction on each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain echo signal time-frequency associated semantic features. A signal time-frequency association semantic feature optimization module 140 is used to input the sequence of echo signal time-frequency association semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency association semantic feature vectors; a signal timing context semantic association module 150 is used to perform timing context association encoding on the sequence of optimized echo signal time-frequency association semantic feature vectors to obtain an echo signal timing context semantic association feature vector as an echo signal timing context semantic association feature; a perception result generation module 160 is used to obtain a perception result based on the echo signal timing context semantic association feature, and the perception result is used to indicate whether a person exists.

[0079] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned millimeter wave radar-based human perception system have been referred to above. Figures 1 to 5 The invention has been introduced in detail in the description of the millimeter wave radar-based human perception method, and therefore, its repeated description will be omitted.

[0080] As described above, the millimeter wave radar-based human perception system 100 according to the embodiment of the present disclosure can be implemented in various wireless terminals, such as a server with a millimeter wave radar-based human perception algorithm. In one possible implementation, the millimeter wave radar-based human perception system 100 according to the embodiment of the present disclosure can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the millimeter wave radar-based human perception system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the millimeter wave radar-based human perception system 100 can also be one of the many hardware modules of the wireless terminal.

[0081] Alternatively, in another example, the millimeter-wave radar-based human perception system 100 and the wireless terminal may also be separate devices, and the millimeter-wave radar-based human perception system 100 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0082] The above is merely an example of the principles of the present disclosure, and various modifications may be made by those skilled in the art without departing from the scope of the present disclosure. The above embodiments are presented for the purpose of illustration rather than limitation. The present disclosure may also take many forms other than those explicitly described herein.

Claims

1. A human body perception method based on millimeter wave radar, characterized in that: include: The radar signal is transmitted multiple times by the millimeter wave radar, and the received radar echo is analyzed to obtain a sequence of echo signals; Performing time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of the echo signals; Extracting echo signal time-frequency features from each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain a sequence of echo signal time-frequency associated semantic feature vectors; Inputting the sequence of the echo signal time-frequency associated semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency associated semantic feature vectors; Performing temporal context association coding on the sequence of the optimized echo signal time-frequency association semantic feature vectors to obtain an echo signal temporal context semantic association feature vector as an echo signal temporal context semantic association feature; Based on the time series context semantic association feature of the echo signal, a perception result is obtained, and the perception result is used to indicate whether a person exists.

2. The human body perception method based on millimeter wave radar according to claim 1 is characterized in that: Performing time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of echo signals includes: performing wavelet transformation on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of echo signals.

3. The human body perception method based on millimeter wave radar according to claim 2 is characterized in that: The method comprises: extracting echo signal time-frequency features from each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain a sequence of echo signal time-frequency associated semantic feature vectors, including: passing each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images through an echo signal time-frequency feature extractor based on a hole convolutional neural network model to obtain a sequence of echo signal time-frequency associated semantic feature vectors.

4. The human body perception method based on millimeter wave radar according to claim 3 is characterized in that: Inputting the sequence of the echo signal time-frequency associated semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency associated semantic feature vectors, including: Calculating the mean vector of the sequence of the echo signal time-frequency associated semantic feature vectors to obtain the echo signal time-frequency associated semantic sequence distribution center vector; Calculating the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector and the echo signal time-frequency association semantic sequence distribution center vector to obtain a sequence of echo signal time-frequency association semantic implicit association coefficients; Calculating the reciprocal of each echo signal time-frequency association semantic implicit correlation coefficient in the sequence of echo signal time-frequency association semantic implicit correlation coefficients to obtain a sequence of echo signal time-frequency association semantic anti-correlation coefficients; Using a Softmax function to normalize the sequence of the echo signal time-frequency association semantic anti-correlation coefficients to obtain a sequence of the echo signal time-frequency association semantic anti-correlation energy coefficients; Taking each echo signal time-frequency association semantic anti-correlation energy coefficient in the sequence of echo signal time-frequency association semantic anti-correlation energy coefficients as a weight, weighting each echo signal time-frequency association semantic feature vector in the sequence of echo signal time-frequency association semantic feature vectors to obtain a sequence of reversely suppressed echo signal time-frequency association semantic feature vectors; The positional difference between each corresponding time-frequency association semantic feature vector of the echo signal and the sequence of time-frequency association semantic feature vectors of the reverse suppressed echo signal is calculated to obtain the sequence of optimized echo signal time-frequency association semantic feature vectors.

5. The human body perception method based on millimeter wave radar according to claim 4 is characterized in that: Calculating the correlation coefficient between each echo signal time-frequency association semantic feature vector in the sequence of the echo signal time-frequency association semantic feature vector and the echo signal time-frequency association semantic sequence distribution center vector to obtain a sequence of echo signal time-frequency association semantic implicit association coefficients, including: Calculate the point-by-point multiplication between the echo signal time-frequency association semantic feature vector and the echo signal time-frequency association semantic sequence distribution center vector to obtain the echo signal time-frequency association semantic class support feature vector; Calculating the sum of the feature values ​​of each position in the echo signal time-frequency association semantic class support feature vector to obtain the echo signal time-frequency association semantic class support sum value; Calculate the square sum of the eigenvalues ​​at each position in the echo signal time-frequency association semantic feature vector and the square sum of the eigenvalues ​​at each position in the echo signal time-frequency association semantic sequence distribution center vector to obtain the echo signal time-frequency association semantic square value and the echo signal time-frequency association semantic sequence distribution center square value; Calculating the square root of the sum of the square value of the echo signal time-frequency association semantics and the square value of the center of the echo signal time-frequency association semantic sequence distribution to obtain the echo signal time-frequency association value; The echo signal time-frequency association semantic class support sum value is divided by the echo signal time-frequency association value to obtain the echo signal time-frequency association semantic implicit association coefficient.

6. The human body perception method based on millimeter wave radar according to claim 5 is characterized in that: The sequence of the time-frequency correlation semantic anti-correlation coefficients of the echo signal is normalized using a Softmax function to obtain a sequence of the time-frequency correlation semantic anti-correlation energy coefficients of the echo signal, including: Using each echo signal time-frequency association semantic anti-correlation coefficient of the sequence of echo signal time-frequency association semantic anti-correlation coefficients as an exponent of a natural constant to calculate an exponential function value based on the natural constant according to position to obtain a sequence of echo signal time-frequency association semantic anti-correlation exponential coefficients; Calculating the sum of the time-frequency association semantic anti-correlation index coefficients of each echo signal in the sequence of the time-frequency association semantic anti-correlation index coefficients of the echo signal to obtain the total sum of the time-frequency association semantic anti-correlation index coefficients of the echo signal; Calculate the sum of the total sum of the time-frequency correlation semantic anti-correlation index coefficients of the echo signal and a constant of one to obtain the time-frequency correlation semantic anti-correlation modulation coefficient of the echo signal; Each echo signal time-frequency association semantic anti-correlation index coefficient in the sequence of echo signal time-frequency association semantic anti-correlation index coefficients is divided by the echo signal time-frequency association semantic anti-correlation modulation coefficient to obtain a sequence of echo signal time-frequency association semantic anti-correlation energy coefficients.

7. The human body perception method based on millimeter wave radar according to claim 6, characterized in that: The sequence of the optimized echo signal time-frequency association semantic feature vectors is subjected to temporal context association encoding to obtain the echo signal temporal context semantic association feature vector, comprising: inputting the sequence of the optimized echo signal time-frequency association semantic feature vectors into an echo signal temporal context association encoder based on a Bi-LSTM model to obtain the echo signal temporal context semantic association feature vector.

8. The human body perception method based on millimeter wave radar according to claim 7, characterized in that: Based on the semantic association feature of the time series context of the echo signal, a perception result is obtained, including: inputting the semantic association feature vector of the time series context of the echo signal into a human body sensor based on a classifier to obtain the perception result.

9. A human body perception system based on millimeter wave radar, characterized in that: include: A radar signal transmission and analysis module is used to transmit radar signals multiple times through a millimeter wave radar, and analyze the received radar echoes to obtain a sequence of echo signals; An echo signal time-frequency conversion module, used for performing time-frequency conversion on each echo signal in the sequence of echo signals to obtain a sequence of two-dimensional time-frequency images of the echo signals; A signal time-frequency associated semantic feature extraction module, used for extracting echo signal time-frequency features from each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images to obtain a sequence of echo signal time-frequency associated semantic feature vectors; A signal time-frequency association semantic feature optimization module, used for inputting the sequence of echo signal time-frequency association semantic feature vectors into a feature enhancement network to obtain a sequence of optimized echo signal time-frequency association semantic feature vectors; A signal time sequence context semantic association module, used for performing time sequence context association encoding on the sequence of the optimized echo signal time-frequency association semantic feature vector to obtain an echo signal time sequence context semantic association feature vector as an echo signal time sequence context semantic association feature; The perception result generation module is used to obtain a perception result based on the semantic association feature of the echo signal time sequence context, and the perception result is used to indicate whether a person exists.

10. The millimeter wave radar-based human perception system according to claim 9, characterized in that: The signal time-frequency associated semantic feature extraction module is used to: pass each echo signal two-dimensional time-frequency image in the sequence of echo signal two-dimensional time-frequency images through an echo signal time-frequency feature extractor based on a hole convolutional neural network model to obtain a sequence of echo signal time-frequency associated semantic feature vectors.

Images