Millimeter-wave Radar Smart Home Sensing Method and System

Abstract

The present application discloses a millimeter-wave radar smart home sensing method and system, which relates to the field of intelligent sensing technology. After filtering the millimeter-wave detection echo signal and performing range-Doppler domain conversion, signal processing and analysis algorithms based on artificial intelligence and deep learning are introduced at the backend to analyze the converted range-Doppler signal, so as to capture and depict the semantic features of the low-frequency part of the range-Doppler signal, thereby sensing and detecting the heart rate frequency of static personnel. It can utilize the frequency band advantages of millimeter-wave radar, and combine artificial intelligence and deep learning technologies to more accurately sense and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, better adapting to changes in complex home environments, improving the flexibility and generalization ability of smart home sensing, being able to more effectively respond to different application scenarios, and improving the intelligent level of the smart home sensing system.

Description

Technical Field

[0001] The present application relates to the field of intelligent sensing technology, and more specifically, to a millimeter wave radar smart home sensing method and system. Background Art

[0002] With the aging population and the increasing demand for a healthier lifestyle, intelligent sensing technology in the home is becoming increasingly important. Sensing the heart rate of static individuals (i.e., those at rest or relatively still) is a crucial capability, as it can be applied to various areas, including health management and emergency response. For example, for the elderly or those with chronic illnesses, continuous heart rate monitoring can help detect abnormalities promptly and enable necessary action.

[0003] Millimeter-wave radar, operating in the millimeter-wave frequency band (typically between 30 GHz and 300 GHz), has the ability to penetrate non-metallic materials and is insensitive to changes in light and temperature, enabling stable operation in a variety of environments. Furthermore, millimeter-wave radar can detect subtle movements, such as the slight rise and fall of the chest caused by breathing or heartbeat, making it ideal for monitoring the vital signs of static personnel.

[0004] Chinese patent CN117741647A discloses a millimeter-wave radar smart home perception method and device. It extracts the target's phase signal within a specific time period and performs linear regression and filtering on the phase signal. It then uses empirical mode decomposition (EMD) and fast Fourier transform (FFT) to obtain the spectrum of each signal component, selecting the component between [0.2, 0.6] Hz to estimate the respiratory rate of a static person.

[0005] Among the aforementioned millimeter-wave radar smart home perception methods, empirical mode decomposition (EMD) and Fast Fourier Transform (FFT) are classic signal processing methods, primarily used to analyze the time-frequency characteristics of signals. However, when faced with the multiple interference factors found in complex home environments, these methods may not be able to effectively distinguish and extract useful frequency component features of respiratory signals. This increases the difficulty of extracting pure respiratory signals, thereby reducing the perception capabilities of smart homes. Furthermore, traditional signal processing methods require manual feature design, and once established, the EMD and FFT-based models are difficult to adjust and optimize. This, to a certain extent, limits the flexibility and adaptability of smart home perception. Feature selection and parameter settings need to be adjusted for different human bodies and environmental conditions.

[0006] Therefore, an optimized millimeter-wave radar smart home perception solution is needed to solve the above technical problems. Summary of the Invention

[0007] To solve the above technical problems, the present application is proposed. An embodiment of the present application provides a millimeter-wave radar smart home sensing method and system. After filtering and range-Doppler domain conversion of the millimeter-wave detection echo signal, signal processing and analysis algorithms based on artificial intelligence and deep learning are introduced at the backend to analyze the converted range-Doppler signal, so as to capture and depict the semantic features of the low-frequency part of the range-Doppler signal, thereby sensing and detecting the heart rate frequency of static personnel. It can utilize the frequency band advantages of the millimeter-wave radar and combine artificial intelligence and deep learning technologies to more accurately sense and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention and being able to better adapt to changes in complex home environments, improving the flexibility and generalization ability of smart home sensing, being able to more effectively respond to different application scenarios, and improving the intelligent level of the smart home sensing system.

[0008] According to one aspect of the present application, a millimeter-wave radar smart home sensing method is provided, which includes:

[0009] Collecting millimeter-wave detection echo signals through an oscilloscope;

[0010] After performing high-pass filtering processing on the millimeter-wave detection echo signal, converting the obtained filtered millimeter-wave detection echo signal from the time domain to the range-Doppler domain to obtain a range-Doppler signal;

[0011] Passing the range-Doppler signal through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal;

[0012] Inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal;

[0013] Performing heart rate frequency sensing based on the semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal to determine a sensing result, and the sensing result is used to represent the heart rate frequency decoding value.

[0014] According to another aspect of the present application, a millimeter-wave radar smart home sensing system is provided, which includes:

[0015] A signal acquisition module for collecting millimeter-wave detection echo signals through an oscilloscope;

[0016] A signal preprocessing module for performing high-pass filtering processing on the millimeter-wave detection echo signal and then converting the obtained filtered millimeter-wave detection echo signal from the time domain to the range-Doppler domain to obtain a range-Doppler signal;

[0017] A signal feature filtering module, configured to pass the range-Doppler signal through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal;

[0018] A feature selection, optimization and aggregation module, configured to input the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection, optimization and aggregation network based on feature distribution characteristics to obtain a semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal;

[0019] A perception result determination module, configured to perform heart rate frequency perception based on the semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal to determine a perception result, where the perception result is used to represent a heart rate frequency decoding value.

[0020] The present application has at least the following technical effects:

[0021] Compared with the prior art, a millimeter-wave radar smart home perception method and system provided by the present application, after filtering the millimeter-wave detection echo signal and performing range-Doppler domain conversion, introduces signal processing and analysis algorithms based on artificial intelligence and deep learning at the backend to analyze the converted range-Doppler signal, so as to capture and depict the semantic features of the low-frequency part of the range-Doppler signal, thereby perceiving and detecting the heart rate frequency of static personnel. It can utilize the frequency band advantage of the millimeter-wave radar and combine artificial intelligence and deep learning technologies to more accurately perceive and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, better adapting to the changes in complex home environments, improving the flexibility and generalization ability of smart home perception, being able to more effectively respond to different application scenarios, and improving the intelligent level of the smart home perception system. Description of the Drawings

[0022] By describing the embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features and advantages of the present application will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 It is a flowchart of a millimeter-wave radar smart home perception method according to an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of data flow of a millimeter-wave radar smart home perception method according to an embodiment of the present application;

[0025] Figure 3 It is a block diagram of a millimeter-wave radar smart home perception system according to an embodiment of the present application. Detailed implementation manners

[0026] Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0027] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0029] In the present application, flowcharts are used to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations before or below are not necessarily executed precisely in order. On the contrary, as needed, various steps can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0030] Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0031] It should be noted that in the present application, all actions of obtaining signals, information, or data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining the authorization given by the owner of the corresponding device.

[0032] In the technical solution of the present application, a millimeter-wave radar smart home sensing method is proposed. Figure 1 is a flowchart of the millimeter-wave radar smart home sensing method according to the embodiments of the present application. Figure 2 is a schematic diagram of data flow of the millimeter-wave radar smart home sensing method according to the embodiments of the present application. As Figure 1 and Figure 2As shown, the millimeter-wave radar smart home sensing method according to an embodiment of the present application includes the steps of: S1, collecting millimeter-wave detection echo signals through an oscilloscope; S2, after performing high-pass filtering on the millimeter-wave detection echo signals, converting the obtained filtered millimeter-wave detection echo signals from the time domain to the range-Doppler domain to obtain range-Doppler signals; S3, passing the range-Doppler signals through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signals; S4, inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signals into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signals; S5, performing heart rate frequency sensing based on the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signals to determine a sensing result, and the sensing result is used to represent a heart rate frequency decoding value.

[0033] In particular, S1 and S2 collect millimeter-wave detection echo signals through an oscilloscope; and after high-pass filtering the millimeter-wave detection echo signals, the obtained filtered millimeter-wave detection echo signals are converted from the time domain to the range-Doppler domain to obtain the range-Doppler signal. In a specific example of the present application, the millimeter-wave detection echo signals are first high-pass filtered to obtain filtered millimeter-wave detection echo signals. The echo signals received by the millimeter-wave radar may contain DC components and other low-frequency noise, which are generally unnecessary because they do not carry information about micro-movements such as breathing or heartbeat. High-pass filtering can effectively remove these components, allowing subsequent processing to focus more on the signal of interest. Therefore, the millimeter-wave detection echo signals are further high-pass filtered to obtain filtered millimeter-wave detection echo signals. In other words, the chest rise and fall caused by physiological phenomena such as breathing and heartbeat is very weak, and is also reflected in the radar signal as a signal with a lower frequency but smaller amplitude. High-pass filtering eliminates signals below a certain frequency, thereby highlighting these useful signals and making subsequent signal processing and analysis easier and more accurate. Furthermore, since the high-pass filtered signal is clearer, it helps accurately analyze and locate the target's position and motion. Therefore, the filtered millimeter-wave detection echo signal is further subjected to a range-Doppler transform to convert it from the time domain to the range-Doppler domain to obtain a range-Doppler signal, thereby facilitating more effective analysis and extraction of useful information from the signal. The range transform converts the signal into distance-related data, enabling identification of targets at different distances. This is crucial for millimeter-wave radar detection echo signal analysis, as it allows the radar system to distinguish between targets at different locations. In a smart home environment, this means determining the distance of a person or object relative to the radar device. The Doppler transform enables the radar system to detect the radial velocity of the target. In a smart home environment, this velocity information can be used to monitor subtle human movements, such as the rise and fall of the chest during breathing or micro-movements caused by a heartbeat. In this way, the radar can sense the vital signs of a static individual. Therefore, the signal obtained after the range-Doppler transform (i.e., the range-Doppler signal) contains comprehensive information about the target's range and velocity, which is very beneficial for further signal processing. For example, when analyzing respiratory rate or heart rate, this information can help filter out unnecessary interference signals and focus on the frequency components related to breathing or heart rate.

[0034] Specifically, in step S3, the range-Doppler signal is passed through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal. Since the range-Doppler signal contains the range and velocity information of the target. However, it is not easy to directly obtain useful physiological information (such as heart rate) from these signals. Specifically, considering that in millimeter-wave radar sensing, activities such as the breathing and heartbeat of static personnel will leave weak low-frequency components in the signal. Therefore, in order to effectively extract the semantic features of the low-frequency part related to the heart rate frequency detection task of the target person from the range-Doppler signal, in the technical solution of this application, the range-Doppler signal is further input into a signal feature filter based on a dilated convolutional neural network model to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal. It is worth mentioning that the signal feature filter based on the dilated convolutional neural network model can effectively capture these semantic features of the low-frequency part in the range-Doppler signal without losing resolution by increasing the receptive field. These semantic feature vectors of the low-frequency part can reflect important information related to human activities, such as the rhythm change of the heartbeat, which helps to perceive and detect the heart rate frequency of the target person.

[0035] Specifically, in step S4, the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is input into a selection optimization aggregation network based on the feature distribution characteristics to obtain a semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal. Among them, each semantic feature vector of the low-frequency part of the range-Doppler signal in the set contains different semantic features of the low-frequency part of the range and velocity information of the target in the millimeter-wave detection echo signal. These semantic features may contain noise or outliers, and these components will interfere with subsequent analysis. Therefore, in order to be able to screen out the central semantics of the low-frequency part, that is, the key semantic information related to the heart rate frequency perception task, in the technical solution of this application, the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is further input into a selection optimization aggregation network based on the feature distribution characteristics to obtain a semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal. Among them, the selection optimization aggregation network based on the feature distribution characteristics can perform feature selection based on the analysis of the energy distribution spectrum, aiming to optimize the expression of the set of semantic feature vectors of the low-frequency part of the range-Doppler signal by quantifying the energy distribution of each semantic feature of the signal low-frequency part, so as to select feature vectors within a certain range related to the semantic center and the heart rate frequency perception task, thereby excluding the influence of noise and outliers.

[0036] In an embodiment of the present application, inputting a set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal includes: First, calculating a feature energy distribution spectrum vector of each semantic feature vector of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal; By calculating the feature energy distribution spectrum vector, these semantic feature vectors of the low-frequency part of the range-Doppler signal are mapped to the energy distribution spectrum space, which helps to capture the distribution of each semantic feature of the low-frequency part of the range-Doppler signal at different frequency or energy levels. At the same time, the energy distribution spectrum can reveal the periodic patterns or trends in the feature vector, which may not be obvious in the original feature space. Next, calculating the position-wise mean vector of the sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal as the semantic energy spectrum distribution center vector of the low-frequency part of the range-Doppler signal; Evaluating the proximity of other feature vectors to the overall distribution through the semantic energy spectrum distribution center vector of the low-frequency part of the range-Doppler signal, so as to select low-frequency semantic features with a higher degree of relevance to the semantic center, that is, the heart rate perception task, and optimize the feature expression ability. Furthermore, calculating the energy distribution spectrum span factor between the semantic energy spectrum distribution center vector of the low-frequency part of the range-Doppler signal and each semantic feature energy distribution spectrum vector in the sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signal; Among them, based on the difference (energy distribution spectrum span factor) and the degree of dispersion between each semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal and the energy spectrum distribution center vector, selection can be made to retain those feature vectors that are most representative in terms of energy distribution. Subsequently, based on the comparison between each semantic energy distribution spectrum span factor in the sequence of semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signal and a preset threshold, determining the set of filtered semantic feature vectors of the low-frequency part of the range-Doppler signal; Finally, cascading each filtered semantic feature vector in the set of filtered semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal.That is, based on the comparison between the semantic energy distribution spectrum span factor of the low-frequency part of each range-Doppler signal and a preset threshold, the semantic feature vectors of the low-frequency part of the range-Doppler signals that meet the conditions are selected to form a sequence. In this way, the screening of the feature vectors is realized by selecting the semantic features of the low-frequency part whose distance from the center vector of the energy spectrum distribution is within a certain range, which helps to remove noise and outliers, retains the most representative and informative features, can more accurately reflect the true state and vital sign information of the target object, and helps to more accurately perceive and detect the heart rate of the target object subsequently.

[0037] Among them, calculating the eigen - energy distribution spectrum vectors of each of the semantic feature vectors of the low - frequency part of the range - Doppler signal to obtain a sequence of semantic feature energy distribution spectrum vectors of the low - frequency part of the range - Doppler signal includes: calculating the eigen - distribution energy collaborative representation vectors between each of the semantic feature vectors of the low - frequency part of the range - Doppler signal and each of the other semantic feature vectors of the low - frequency part of the range - Doppler signal in the set of semantic feature vectors of the low - frequency part of the range - Doppler signal to obtain a sequence of multiple semantic feature distribution energy collaborative representation vectors of the low - frequency part of the range - Doppler signal; calculating the eigen - distribution energy collaborative factor vectors of each sequence of the semantic feature distribution energy collaborative representation vectors in the sequence of multiple semantic feature distribution energy collaborative representation vectors of the low - frequency part of the range - Doppler signal to obtain the sequence of semantic feature energy distribution spectrum vectors of the low - frequency part of the range - Doppler signal. Specifically, the process of calculating the eigen - distribution energy collaborative representation vectors between each of the semantic feature vectors of the low - frequency part of the range - Doppler signal and each of the other semantic feature vectors of the low - frequency part of the range - Doppler signal in the set of semantic feature vectors of the low - frequency part of the range - Doppler signal to obtain a sequence of multiple semantic feature distribution energy collaborative representation vectors of the low - frequency part of the range - Doppler signal includes: extracting a predetermined semantic feature vector of the low - frequency part of the range - Doppler signal from the sequence of eigen - energy distribution spectrum vectors of the semantic feature vectors of the low - frequency part of the range - Doppler signal; performing element - by - element maximum extraction on the predetermined semantic feature vector of the low - frequency part of the range - Doppler signal and each of the other semantic feature vectors of the low - frequency part of the range - Doppler signal to obtain a sequence of semantic feature distribution energy collaborative representation vectors corresponding to the predetermined semantic feature vector of the low - frequency part of the range - Doppler signal; and the process of calculating the eigen - distribution energy collaborative factor vectors of each sequence of the semantic feature distribution energy collaborative representation vectors in the sequence of multiple semantic feature distribution energy collaborative representation vectors of the low - frequency part of the range - Doppler signal to obtain the sequence of semantic feature energy distribution spectrum vectors of the low - frequency part of the range - Doppler signal includes: respectively extracting the maximum value of each semantic feature distribution energy collaborative representation vector in the sequence of semantic feature distribution energy collaborative representation vectors of the low - frequency part of the range - Doppler signal to obtain a sequence of maximum values of semantic features of the low - frequency part of the range - Doppler signal; respectively calculating the mean and variance of each semantic feature distribution energy collaborative representation vector to obtain a sequence of means of semantic features of the low - frequency part of the range - Doppler signal and a sequence of variances of semantic features of the low - frequency part of the range - Doppler signal;Sum the sequence of the variances of the semantic features of the low-frequency part of the range-Doppler signal and a preset hyperparameter at each position to obtain a sequence of the first energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal; calculate the square of the difference between the maximum value of the semantic features of the low-frequency part of the range-Doppler signal and the mean value of the semantic features of the low-frequency part of the range-Doppler signal in each group of the sequence of the maximum values of the semantic features of the low-frequency part of the range-Doppler signal and the sequence of the mean values of the semantic features of the low-frequency part of the range-Doppler signal to obtain a sequence of the semantic difference values of the low-frequency part of the range-Doppler signal; multiply the sequence of the variances of the semantic features of the low-frequency part of the range-Doppler signal by a constant two at each position to obtain a sequence of the variances of the modulated semantic features of the low-frequency part of the range-Doppler signal; after adding the semantic difference values of the low-frequency part of the range-Doppler signal and the variances of the modulated semantic features of the low-frequency part of the range-Doppler signal in each group of the sequence of the semantic difference values of the low-frequency part of the range-Doppler signal and the sequence of the variances of the modulated semantic features of the low-frequency part of the range-Doppler signal, divide the sum value by the preset hyperparameter to obtain a sequence of the second energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal; divide the sequence of the first energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal by the sequence of the second energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal at each position to obtain the semantic feature energy distribution spectral vector of the low-frequency part of the range-Doppler signal composed of multiple feature distribution energy cooperation factors.;

[0038] More specifically, the process of calculating the energy distribution spectrum span factors between each distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector in the sequence of the distance-Doppler signal low-frequency part semantic energy spectrum distribution center vector and the distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector to obtain the sequence of distance-Doppler signal low-frequency part semantic energy distribution spectrum span factors includes: calculating the position-wise difference between the distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector and the distance-Doppler signal low-frequency part semantic energy spectrum distribution center vector to obtain the distance-Doppler signal low-frequency part semantic difference feature vector; calculating the covariance matrix between the distance-Doppler signal low-frequency part semantic energy spectrum distribution center vector and the distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector, and calculating the reciprocal of the covariance matrix to obtain the distance-Doppler signal low-frequency part semantic inverse covariance matrix; calculating the product of the transposed vector of the distance-Doppler signal low-frequency part semantic difference feature vector and the distance-Doppler signal low-frequency part semantic inverse covariance matrix and the distance-Doppler signal low-frequency part semantic difference feature vector to obtain the distance-Doppler signal low-frequency part semantic distribution value; calculating the square root of the distance-Doppler signal low-frequency part semantic distribution value to obtain the distance-Doppler signal low-frequency part semantic energy distribution spectrum span factor.

[0039] In summary, in the above embodiments, inputting the set of distance-Doppler signal low-frequency part semantic feature vectors into the selection optimization aggregation network based on feature distribution characteristics to obtain the distance-Doppler signal low-frequency part semantic aggregation purification representation vector includes: inputting the set of distance-Doppler signal low-frequency part semantic feature vectors into the selection optimization aggregation network based on feature distribution characteristics and processing them according to the following feature selection optimization aggregation formula to obtain the distance-Doppler signal low-frequency part semantic aggregation purification representation vector; wherein, the feature selection optimization aggregation formula is:

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] Among them, and respectively represent the -th and -th semantic feature vectors of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal, represents taking the maximum value between two vectors element by element, is the distance-Doppler signal low-frequency part semantic feature distribution energy collaborative representation vector between the -th and -th semantic feature vectors of the low-frequency part of the range-Doppler signal, is the maximum value of the extracted vector, and are respectively 's mean and variance, is a preset hyperparameter, is the -th feature distribution energy collaborative factor in the -th distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector in the sequence of distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, is the -th feature distribution energy collaborative factor in the -th distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector in the sequence of distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, represents the sequence of distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, is the -th distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector in the sequence of distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, is the number of vectors in the sequence of distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, is the distance-Doppler signal low-frequency part semantic energy spectrum distribution center vector, is the transpose of the vector, is the -th covariance matrix between the distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector and the distance-Doppler signal low-frequency part semantic energy spectrum distribution center vector, is the -th distance-Doppler signal low-frequency part semantic energy distribution spectrum span factor corresponding to the distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vector, is a preset threshold, is the a semantic feature vector of the low-frequency part of the filtered range-Doppler signal respectively represents the th semantic feature vector of the low-frequency part of the filtered range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the filtered range-Doppler signal is the set of semantic feature vectors of the low-frequency part of the filtered range-Doppler signal represents vector concatenation is the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal

[0049] Specifically, for S5, heart rate frequency perception is performed based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine a perception result, and the perception result is used to represent a heart rate frequency decoding value. In a specific example of the present application, the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal is input into a decoder-based heart rate frequency perception module to obtain the perception result, and the perception result is used to represent a heart rate frequency decoding value. That is, the semantic purification and aggregation features of the low-frequency part of the range-Doppler signal are used for decoding regression to perceive and detect the heart rate frequency of a static person. In this way, the frequency band advantage of the millimeter-wave radar can be utilized, combined with artificial intelligence and deep learning technologies to more accurately perceive and detect the heart rate frequency of a static person, thereby reducing the need for human intervention and being able to better adapt to changes in a complex home environment, improving the flexibility and generalization ability of smart home perception. In this way, different application scenarios can be more effectively dealt with, and the intelligent level of the smart home perception system can be improved

[0050] Among them, each semantic feature vector of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal respectively represents the low-frequency part image semantic feature of the range-Doppler signal determined based on a filtering factor. Thus, when the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is input into a selection optimization aggregation network based on feature distribution characteristics, considering that the full-frequency domain aggregation offset caused by the significant difference in feature energy of each frequency band in the range-Doppler signal will lead to the lack of feature instance determination of the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal, thereby affecting the accuracy of the perception result obtained by inputting it into the decoder-based heart rate frequency perception module that has completed training

[0051] Preferably, inputting the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal into a decoder-based heart rate frequency perception module to obtain a perception result includes

[0052] Calculate the distance between each pair of eigenvalues of the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, such as the L2 distance, and take the square root of the distance to obtain the semantic aggregation purification distance representation matrix of the low-frequency part of the range-Doppler signal, that is:

[0053]

[0054] Wherein, represents the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, and represent the and eigenvalues of the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, represents the distance between the and eigenvalues of the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, represents the eigenvalue at the position in the semantic aggregation purification distance representation matrix of the low-frequency part of the range-Doppler signal;

[0055] Obtain the semantic aggregation purification autocorrelation matrix of the low-frequency part of the range-Doppler signal of the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal as a row vector, that is , wherein, represents the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, represents the transpose of the vector, represents matrix multiplication, represents the semantic aggregation purification autocorrelation matrix of the low-frequency part of the range-Doppler signal;

[0056] Perform matrix multiplication on the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal and the semantic aggregation purification distance representation matrix of the low-frequency part of the range-Doppler signal to obtain the semantic aggregation purification first-level mapping vector of the low-frequency part of the range-Doppler signal, that is , wherein, represents the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, represents the semantic aggregation purification distance representation matrix of the low-frequency part of the range-Doppler signal, represents matrix multiplication, represents the semantic aggregation purification first-level mapping vector of the low-frequency part of the range-Doppler signal;

[0057] Multiply the semantic aggregation and purification first-level mapping vector of the low-frequency part of the range-Doppler signal by the matrix product of the semantic aggregation and purification distance representation matrix of the low-frequency part of the range-Doppler signal and the semantic aggregation and purification self-correlation matrix of the low-frequency part of the range-Doppler signal to obtain the semantic aggregation and purification multi-level mapping vector of the low-frequency part of the range-Doppler signal , where represents the semantic aggregation and purification first-level mapping vector of the low-frequency part of the range-Doppler signal, represents the semantic aggregation and purification distance representation matrix of the low-frequency part of the range-Doppler signal, represents matrix multiplication, represents the semantic aggregation and purification self-correlation matrix of the low-frequency part of the range-Doppler signal, represents the semantic aggregation and purification multi-level mapping vector of the low-frequency part of the range-Doppler signal;

[0058] Dot the semantic aggregation and purification multi-level mapping vector of the low-frequency part of the range-Doppler signal with the semantic aggregation and purification associated eigenvector composed of the eigenvalues of the semantic aggregation and purification self-correlation matrix of the low-frequency part of the range-Doppler signal to obtain an optimized semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal, where interpolation or zero-padding is performed when the eigenvalues are insufficient;

[0059] Input the optimized semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal into the heartbeat frequency perception module based on the decoder to obtain the perception result.

[0060] Therefore, through the linear target mapping representation of the similarity distance representation matrix based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal, the complete similarity instantiation of the self-correlation of the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal is performed based on the quadratic target mapping representation of the multi-level distribution hierarchy, and the associated fusion kernel bias is used to compensate for the negative influence factor of the association mismatch, so as to improve the eigenvalue instance decision degree of the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal under similarity constraints, that is, the significance degree of the eigenvalue as an instance for the cyclic regression decision, and improve the accuracy of the perception result obtained by the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal through the heartbeat frequency perception module based on the decoder, so as to be able to more accurately perceive and detect the heart rate frequency of static personnel, improve the flexibility and generalization ability of smart home perception, and make the smart home perception system have a higher level of intelligence.

[0061] In summary, the millimeter-wave radar smart home sensing method according to the embodiments of the present application is elucidated. After filtering the millimeter-wave detection echo signal and performing range-Doppler domain conversion, a signal processing and analysis algorithm based on artificial intelligence and deep learning is introduced at the backend to analyze the converted range-Doppler signal, thereby capturing and characterizing the semantic features of the low-frequency part of the range-Doppler signal, so as to sense and detect the heart rate frequency of static personnel. It can utilize the frequency band advantage of the millimeter-wave radar and combine artificial intelligence and deep learning technologies to more accurately sense and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, better adapting to the changes in complex home environments, improving the flexibility and generalization ability of smart home sensing, and being able to more effectively respond to different application scenarios, improving the intelligence level of the smart home sensing system.

[0062] Furthermore, a millimeter-wave radar smart home sensing system is also provided.

[0063] Figure 3 FIG. is a block diagram of the millimeter-wave radar smart home sensing system according to the embodiments of the present application. As Figure 3 shown, the millimeter-wave radar smart home sensing system 300 according to the embodiments of the present application includes: a signal acquisition module 310 for acquiring millimeter-wave detection echo signals through an oscilloscope; a signal preprocessing module 320 for performing high-pass filtering on the millimeter-wave detection echo signals and then converting the obtained filtered millimeter-wave detection echo signals from the time domain to the range-Doppler domain to obtain range-Doppler signals; a signal feature filtering module 330 for passing the range-Doppler signals through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signals; a feature selection optimization aggregation module 340 for inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signals into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signals; a sensing result determination module 350 for performing heart rate frequency sensing based on the semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signals to determine a sensing result, where the sensing result is used to represent the heart rate frequency decoding value.

[0064] As described above, the millimeter-wave radar smart home sensing system 300 according to the embodiments of the present application can be implemented in various wireless terminals, such as a server with a millimeter-wave radar smart home sensing algorithm. In a possible implementation, the millimeter-wave radar smart home sensing system 300 according to the embodiments of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the millimeter-wave radar smart home sensing system 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the millimeter-wave radar smart home sensing system 300 can also be one of the many hardware modules of the wireless terminal.

[0065] Alternatively, in another example, the millimeter-wave radar smart home sensing system 300 and the wireless terminal can also be separate devices, and the millimeter-wave radar smart home sensing system 300 can be connected to the wireless terminal through a wired and / or wireless network and transmit interaction information in accordance with a predefined data format.

[0066] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A millimeter-wave radar intelligent home sensing method, characterized in that Including: Collecting millimeter-wave detection echo signals through an oscilloscope; After performing high-pass filtering on the millimeter-wave detection echo signals, converting the obtained filtered millimeter-wave detection echo signals from the time domain to the range-Doppler domain to obtain range-Doppler signals; Passing the range-Doppler signals through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signals, which includes: Inputting the range-Doppler signals into a signal feature filter based on a dilated convolutional neural network model to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signals; Inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signals into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signals; Performing heart rate frequency perception based on the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signals to determine a perception result, and the perception result is used to represent the heart rate frequency decoding value.

2. The millimeter-wave radar smart home sensing method according to claim 1, wherein After performing high-pass filtering on the millimeter-wave detection echo signals, converting the obtained filtered millimeter-wave detection echo signals from the time domain to the range-Doppler domain to obtain range-Doppler signals, including: Performing high-pass filtering on the millimeter-wave detection echo signals to obtain filtered millimeter-wave detection echo signals; Performing range-Doppler transformation on the filtered millimeter-wave detection echo signals to convert the filtered millimeter-wave detection echo signals from the time domain to the range-Doppler domain to obtain the range-Doppler signals.

3. The millimeter-wave radar smart home sensing method according to claim 2, characterized in that, Inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signals into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signals, including: Calculating the feature energy distribution spectrum vectors of each semantic feature vector of the low-frequency part of the range-Doppler signals in the set of semantic feature vectors of the low-frequency part of the range-Doppler signals to obtain a sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signals; Calculating the position-wise mean vector of the sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signals as the semantic energy spectrum distribution center vector of the low-frequency part of the range-Doppler signals; Calculating the energy distribution spectrum span factors between the semantic energy spectrum distribution center vector of the low-frequency part of the range-Doppler signals and each semantic feature energy distribution spectrum vector in the sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signals to obtain a sequence of semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signals; Based on the comparison between each semantic energy distribution spectrum span factor in the sequence of semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signals and a preset threshold, determining a set of filtered semantic feature vectors of the low-frequency part of the range-Doppler signals; Cascade each of the filtered semantic feature vectors of the low-frequency part of the range-Doppler signal in the set of the filtered semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain the aggregated and purified representation vector of the low-frequency part of the range-Doppler signal.

4. The millimeter-wave radar intelligent home sensing method according to claim 3, wherein Calculate the eigen-energy distribution spectrum vector of each semantic feature vector of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of eigen-energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal, including: Calculate the eigen-distribution energy collaborative representation vector between each of the semantic feature vectors of the low-frequency part of the range-Doppler signal and each of the other semantic feature vectors of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of multiple eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal; Calculate the eigen-distribution energy collaborative factor vector of each sequence of eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal in the sequence of multiple eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal to obtain the sequence of eigen-energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal.

5. The millimeter-wave radar smart home sensing method according to claim 4, characterized in that Calculate the eigen-distribution energy collaborative representation vector between each of the semantic feature vectors of the low-frequency part of the range-Doppler signal and each of the other semantic feature vectors of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of multiple eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal, including: Extract a predetermined semantic feature vector of the low-frequency part of the range-Doppler signal from the sequence of eigen-energy distribution spectrum vectors of the semantic feature vectors of the low-frequency part of the range-Doppler signal; Perform element-wise maximum extraction on the predetermined semantic feature vector of the low-frequency part of the range-Doppler signal and each of the other semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal corresponding to the predetermined semantic feature vector of the low-frequency part of the range-Doppler signal.

6. The millimeter-wave radar smart home sensing method according to claim 5, characterized in that, Calculate the eigen-distribution energy collaborative factor vector of each sequence of eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal in the sequence of multiple eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal to obtain the sequence of eigen-energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal, including: Extract the maximum value of each eigen-distribution energy collaborative representation vector of the low-frequency part of the range-Doppler signal in the sequence of eigen-distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of maximum values of the semantic features of the low-frequency part of the range-Doppler signal; Calculate the mean and variance of the semantic feature distribution energy collaborative representation vectors of the low-frequency part of each of the distance-Doppler signals respectively to obtain a sequence of the semantic feature means of the low-frequency part of the distance-Doppler signals and a sequence of the semantic feature variances of the low-frequency part of the distance-Doppler signals; Add the sequence of the semantic feature variances of the low-frequency part of the distance-Doppler signals and a preset hyperparameter in a position-wise manner to obtain a sequence of the first energy distribution coefficients of the semantic features of the low-frequency part of the distance-Doppler signals; Calculate the square of the difference between the maximum value of the semantic features of the low-frequency part of the distance-Doppler signals and the mean value of the semantic features of the low-frequency part of the distance-Doppler signals in each group in the sequence of the maximum values of the semantic features of the low-frequency part of the distance-Doppler signals and the sequence of the mean values of the semantic features of the low-frequency part of the distance-Doppler signals to obtain a sequence of the semantic difference values of the low-frequency part of the distance-Doppler signals; Multiply the sequence of the semantic feature variances of the low-frequency part of the distance-Doppler signals by a constant two in a position-wise manner to obtain a sequence of the modulated semantic feature variances of the low-frequency part of the distance-Doppler signals; After adding the semantic difference values of the low-frequency part of the distance-Doppler signals and the modulated semantic feature variances of the low-frequency part of the distance-Doppler signals in each group in the sequence of the semantic difference values of the low-frequency part of the distance-Doppler signals and the sequence of the modulated semantic feature variances of the low-frequency part of the distance-Doppler signals, divide the sum value by the preset hyperparameter to obtain a sequence of the second energy distribution coefficients of the semantic features of the low-frequency part of the distance-Doppler signals; Divide the sequence of the first energy distribution coefficients of the semantic features of the low-frequency part of the distance-Doppler signals and the sequence of the second energy distribution coefficients of the semantic features of the low-frequency part of the distance-Doppler signals in a position-wise manner to obtain the semantic feature energy distribution spectrum vector of the low-frequency part of the distance-Doppler signals composed of multiple feature distribution energy collaborative factors; 7. The millimeter-wave radar smart home sensing method according to claim 6, wherein Calculate the energy distribution spectrum span factors between the semantic energy spectrum distribution center vector of the low-frequency part of the distance-Doppler signals and each semantic feature energy distribution spectrum vector in the sequence of the semantic feature energy distribution spectrum vectors of the low-frequency part of the distance-Doppler signals to obtain a sequence of the semantic energy distribution spectrum span factors of the low-frequency part of the distance-Doppler signals, including: Calculate the position-wise difference between the semantic feature energy distribution spectrum vector of the low-frequency part of the distance-Doppler signals and the semantic energy spectrum distribution center vector of the low-frequency part of the distance-Doppler signals to obtain the semantic differential feature vector of the low-frequency part of the distance-Doppler signals; Calculate the covariance matrix between the semantic energy spectrum distribution center vector of the low-frequency part of the distance-Doppler signals and the semantic feature energy distribution spectrum vector of the low-frequency part of the distance-Doppler signals, and calculate the reciprocal of the covariance matrix to obtain the semantic inverse covariance matrix of the low-frequency part of the distance-Doppler signals; Calculate the transpose vector of the semantic difference feature vector of the low-frequency part of the range-Doppler signal and the product of the semantic inverse covariance matrix of the low-frequency part of the range-Doppler signal and the semantic difference feature vector of the low-frequency part of the range-Doppler signal to obtain the semantic distribution value of the low-frequency part of the range-Doppler signal; Calculate the square root of the semantic distribution value of the low-frequency part of the range-Doppler signal to obtain the semantic energy distribution spectrum span factor of the low-frequency part of the range-Doppler signal.

8. The millimeter-wave radar smart home sensing method according to claim 7, wherein Perform heart rate frequency perception based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine the perception result, where the perception result is used to represent the heart rate frequency decoding value, including: input the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal into the heart rate frequency perception module based on the decoder to obtain the perception result, and the perception result is used to represent the heart rate frequency decoding value.

9. The millimeter-wave radar smart home perception system is characterized in that, Comprising: A signal acquisition module, configured to acquire a millimeter-wave detection echo signal through an oscilloscope; A signal preprocessing module, configured to perform high-pass filtering on the millimeter-wave detection echo signal, and then convert the obtained filtered millimeter-wave detection echo signal from the time domain to the range-Doppler domain to obtain a range-Doppler signal; A signal feature filtering module, configured to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal by passing the range-Doppler signal through a signal feature filter, which includes: Input the range-Doppler signal into the signal feature filter based on the dilated convolutional neural network model to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal; A feature selection, optimization and aggregation module, configured to input the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into the selection, optimization and aggregation network based on the feature distribution characteristics to obtain the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal; A perception result determination module, configured to perform heart rate frequency perception based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine the perception result, and the perception result is used to represent the heart rate frequency decoding value.

Images