Millimeter wave radar smart home sensing method and system

Abstract

The invention discloses a millimeter wave radar smart home sensing method and a millimeter wave radar smart home sensing system, and relates to the technical field of smart sensing. A signal processing and analysis algorithm based on artificial intelligence and deep learning is introduced at the rear end to analyze the converted distance-Doppler signal, so that the semantic features of the low-frequency part of the distance-Doppler signal are captured and described, and the heart rate frequency of a static person is perceived and detected; according to the method, the frequency band advantage of the millimeter wave radar can be utilized, artificial intelligence and the deep learning technology are combined to more accurately sense and detect the heart rate frequency of a static person, and therefore the requirement for human intervention is reduced, the method can better adapt to changes of the complex family environment, the flexibility and generalization ability of intelligent home sensing are improved, and the intelligent home experience is improved. Different application scenes can be more effectively coped with, and the intelligent level of the intelligent home sensing system is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent sensing technology, and more specifically, to a millimeter wave radar smart home sensing method and system. Background Art

[0002] As the aging population intensifies and people pursue a healthier quality of life, smart sensing technology in the home environment is becoming more and more important. Among them, sensing the heart rate of static people (i.e. people who are at rest or relatively still) is an important function because it can be applied to health management, emergency response and other aspects. For example, for the elderly or people with chronic diseases, continuous monitoring of heart rate can help detect abnormalities in time and take necessary measures.

[0003] Millimeter-wave radar operates in the millimeter-wave frequency band (usually between 30GHz and 300GHz), has the ability to penetrate non-metallic materials, and is insensitive to light and temperature changes, which enables it to work stably in various environments. In addition, millimeter-wave radar can detect subtle movements, such as slight chest movements caused by breathing or heartbeats, which makes it very suitable for monitoring the vital signs of static personnel.

[0004] Chinese patent CN117741647A discloses a millimeter-wave radar smart home perception method and device, which extracts the phase signal of the target within a specific time length and performs linear regression and filtering on the phase signal, and then uses the empirical mode decomposition (EMD) plus fast Fourier transform (FFT) method to obtain the spectrum of each signal component, and selects the component between [0.2, 0.6] Hz to estimate the breathing frequency of static personnel.

[0005] In the above-mentioned millimeter-wave radar smart home perception method, empirical mode decomposition and FFT are classic signal processing methods, which are mainly used to analyze the time-frequency characteristics of the signal. However, when faced with multiple interference factors in a complex home environment, these methods may not be able to effectively distinguish and extract useful frequency component features of the respiratory signal, which will increase the difficulty of extracting pure respiratory signals, thereby reducing the perception ability of the smart home. In addition, traditional signal processing methods require manual feature design, and once the method based on empirical mode decomposition and FFT is determined, its model is not easy to adjust and optimize, which to some extent limits the flexibility and adaptability of smart home perception. For different human bodies and different environmental conditions, feature selection and parameter settings need to be adjusted.

[0006] Therefore, an optimized millimeter-wave radar smart home perception solution is needed to solve the above technical problems. Summary of the invention

[0007] In order to solve the above technical problems, this application is proposed. The embodiment of the present application provides a millimeter wave radar smart home perception method and system, which filters the millimeter wave detection echo signal and converts it into the range-Doppler domain, and then introduces a signal processing and analysis algorithm based on artificial intelligence and deep learning at the back end to analyze the converted range-Doppler signal, so as to capture and characterize the semantic features of the low-frequency part of the range-Doppler signal, so as to perceive and detect the heart rate frequency of static personnel, and can utilize the frequency band advantage of millimeter wave radar, and combine artificial intelligence and deep learning technology to more accurately perceive and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, and can better adapt to changes in complex home environments, improve the flexibility and generalization ability of smart home perception, and can more effectively respond to different application scenarios, and improve the intelligence level of smart home perception systems.

[0008] According to one aspect of the present application, a millimeter wave radar smart home perception method is provided, which includes:

[0009] Collect millimeter wave detection echo signals through an oscilloscope;

[0010] After high-pass filtering the millimeter-wave detection echo signal, the obtained filtered millimeter-wave detection echo signal is converted from the time domain to the range-Doppler domain to obtain a range-Doppler signal;

[0011] Passing the range-Doppler signal through a signal feature filter to obtain a set of semantic feature vectors of a low-frequency portion of the range-Doppler signal;

[0012] Inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signal;

[0013] Heart rate frequency perception is performed based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine a perception result, and the perception result is used to represent the heart rate frequency decoding value.

[0014] According to another aspect of the present application, a millimeter wave radar smart home perception system is provided, which includes:

[0015] A signal acquisition module, used for acquiring millimeter wave detection echo signals through an oscilloscope;

[0016] A signal preprocessing module, used for performing high-pass filtering on the millimeter-wave detection echo signal, and then converting the obtained filtered millimeter-wave detection echo signal from the time domain to the range-Doppler domain to obtain a range-Doppler signal;

[0017] A signal feature filtering module, used for passing the range-Doppler signal through a signal feature filter to obtain a set of semantic feature vectors of a low-frequency part of the range-Doppler signal;

[0018] A feature selection optimization aggregation module is used to input the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signal;

[0019] The sensing result determination module is used to sense the heart rate frequency based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine the sensing result, and the sensing result is used to represent the heart rate frequency decoding value.

[0020] This application has at least the following technical effects:

[0021] Compared with the prior art, the present application provides a millimeter-wave radar smart home perception method and system, which filters the millimeter-wave detection echo signal and converts it into the range-Doppler domain, and then introduces a signal processing and analysis algorithm based on artificial intelligence and deep learning at the back end to analyze the converted range-Doppler signal, so as to capture and characterize the semantic features of the low-frequency part of the range-Doppler signal, thereby perceiving and detecting the heart rate frequency of static personnel. It can utilize the frequency band advantage of the millimeter-wave radar, and combine artificial intelligence and deep learning technology to more accurately perceive and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, and can better adapt to changes in complex home environments, improve the flexibility and generalization ability of smart home perception, and can more effectively respond to different application scenarios, and improve the intelligence level of the smart home perception system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 This is a flow chart of a millimeter wave radar smart home sensing method according to an embodiment of the present application;

[0024] Figure 2 A schematic diagram of data flow of a millimeter wave radar smart home sensing method according to an embodiment of the present application;

[0025] Figure 3 The figure is a block diagram of a millimeter wave radar smart home sensing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0029] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] It should be noted that in this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization of the owner of the corresponding device.

[0032] In the technical solution of the present application, a millimeter wave radar smart home perception method is proposed. Figure 1 The figure is a flow chart of the millimeter wave radar smart home perception method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the millimeter wave radar smart home sensing method according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to the millimeter wave radar smart home perception method of the embodiment of the present application, the steps include: S1, collecting millimeter wave detection echo signals through an oscilloscope; S2, after high-pass filtering the millimeter wave detection echo signals, converting the filtered millimeter wave detection echo signals from the time domain to the range-Doppler domain to obtain range-Doppler signals; S3, passing the range-Doppler signals through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signals; S4, inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signals into a selective optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signals; S5, performing heartbeat frequency perception based on the semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signals to determine a perception result, and the perception result is used to represent a heart rate frequency decoding value.

[0033] In particular, the S1 and S2 collect millimeter wave detection echo signals through an oscilloscope; and after high-pass filtering the millimeter wave detection echo signals, the obtained filtered millimeter wave detection echo signals are converted from the time domain to the range-Doppler domain to obtain the range-Doppler signal. In a specific example of the present application, first, the millimeter wave detection echo signals are high-pass filtered to obtain filtered millimeter wave detection echo signals. Among them, the echo signals received by the millimeter wave radar may contain DC components and other low-frequency noises, which are usually unnecessary because they do not carry information about micro-movements such as breathing or heartbeats. High-pass filtering can effectively remove these components, so that subsequent processing can be more focused on the signals of interest. Therefore, the millimeter wave detection echo signals are further high-pass filtered to obtain filtered millimeter wave detection echo signals. In other words, the chest fluctuations caused by physiological phenomena such as breathing and heartbeats are very weak, which are also reflected in the radar signal as signals with lower frequencies but smaller amplitudes. By high-pass filtering, signals below a certain frequency can be eliminated, thereby highlighting these useful signals, making subsequent signal processing and analysis easier and more accurate. Furthermore, since the signal after high-pass filtering is clearer, it is helpful to accurately analyze and locate the position and motion state of the target. Therefore, the filtered millimeter-wave detection echo signal is further subjected to a range-Doppler transform to convert the filtered millimeter-wave detection echo signal from the time domain to the range-Doppler domain to obtain a range-Doppler signal, thereby facilitating more effective analysis and extraction of useful information in the signal. Among them, by distance transformation, the signal can be converted into distance-related data, so that targets at different distances can be identified. This is very important for the analysis of the detection echo signal of the millimeter-wave radar, because it allows the radar system to distinguish target objects at different positions. In a smart home environment, this means that the distance of a person or object relative to the radar device can be determined. The Doppler transform enables the radar system to detect the radial velocity of the target. In a smart home environment, this velocity information can be used to monitor the tiny movements of the human body, such as the rise and fall of the chest during breathing or the micro-movement caused by the heartbeat. In this way, the radar can sense the vital signs information of static personnel. Therefore, the signal obtained after the range-Doppler transform (i.e., the range-Doppler signal) contains comprehensive information about the target distance and speed, which is very beneficial for further signal processing. For example, when analyzing the breathing rate or heart rate, this information can help filter out unnecessary interference signals and focus on the frequency components related to breathing or heart rate.

[0034] In particular, in S3, the distance-Doppler signal is passed through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the distance-Doppler signal. Since the distance-Doppler signal contains the distance and speed information of the target. However, it is not easy to obtain useful physiological information (such as heart rate) directly from these signals. In particular, considering that in millimeter-wave radar perception, activities such as breathing and heartbeat of static personnel will leave weak low-frequency components in the signal. Therefore, in order to effectively extract the low-frequency semantic features related to the target person's heart rate frequency detection task from the distance-Doppler signal, in the technical solution of the present application, the distance-Doppler signal is further input into a signal feature filter based on a hole convolutional neural network model to obtain a set of semantic feature vectors of the low-frequency part of the distance-Doppler signal. It is worth mentioning that the signal feature filter based on the hole convolutional neural network model can effectively capture these low-frequency semantic features in the distance-Doppler signal without losing resolution by increasing the receptive field. These low-frequency semantic feature vectors can reflect important information related to human activities, such as changes in heartbeat rhythm, which helps to perceive and detect the heart rate frequency of the target person.

[0035] In particular, the S4 inputs the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signal. Among them, each semantic feature vector of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal respectively contains different semantic features of the low-frequency part of the distance and speed information of the target in the millimeter-wave detection echo signal, and these semantic features may contain noise or outliers, which will interfere with the subsequent analysis. Therefore, in order to be able to screen out the central semantics of the low-frequency part, that is, the key semantic information related to the heart rate perception task, in the technical solution of the present application, the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is further input into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signal. Among them, the selection optimization aggregation network based on feature distribution characteristics can perform feature selection based on the analysis of the energy distribution spectrum, aiming to optimize the expression of the set of semantic feature vectors of the low-frequency part of the range-Doppler signal by quantifying the energy distribution of the semantic features of the low-frequency part of each signal, so as to select feature vectors within a certain range of distances related to the semantic center and heart rate perception tasks, thereby eliminating the influence of noise and outliers.

[0036] In an embodiment of the present application, the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is input into a selective optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal, including: first, calculating the characteristic energy distribution spectrum vector of each semantic feature vector of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal; mapping these semantic feature vectors of the low-frequency part of the range-Doppler signal to the energy distribution spectrum space by calculating the characteristic energy distribution spectrum vector, thereby helping to capture the distribution of each semantic feature of the low-frequency part of the range-Doppler signal at different frequencies or energy levels, and at the same time, the energy distribution spectrum can reveal periodic patterns or trends in the feature vectors, which may not be obvious in the original feature space. Next, the position mean vector of the sequence of the energy distribution spectrum vector of the semantic feature of the low frequency part of the distance-Doppler signal is calculated as the center vector of the energy distribution spectrum of the semantic feature of the low frequency part of the distance-Doppler signal; the proximity of other feature vectors to the overall distribution is evaluated by the center vector of the energy distribution spectrum of the semantic feature of the low frequency part of the distance-Doppler signal, so as to select low-frequency semantic features with higher relevance to the semantic center, that is, the heartbeat frequency perception task, and optimize the feature expression ability. Furthermore, the energy distribution spectrum span factor between the center vector of the energy distribution spectrum of the semantic feature of the low frequency part of the distance-Doppler signal and each energy distribution spectrum vector of the semantic feature of the low frequency part of the distance-Doppler signal in the sequence of the energy distribution spectrum vector of the semantic feature of the low frequency part of the distance-Doppler signal is calculated to obtain a sequence of the energy distribution spectrum span factor of the semantic feature of the low frequency part of the distance-Doppler signal; wherein, the selection is based on the difference (energy distribution spectrum span factor) between each energy distribution spectrum vector of the semantic feature of the low frequency part of the distance-Doppler signal and the center vector of the energy spectrum distribution and the degree of dispersion, so that the most representative feature vectors in terms of energy distribution can be retained. Subsequently, based on the comparison between each distance-Doppler signal low-frequency part semantic energy distribution spectrum span factor in the sequence of distance-Doppler signal low-frequency part semantic energy distribution spectrum span factors and a preset threshold, the set of filtered semantic feature vectors of the distance-Doppler signal low-frequency part is determined; finally, each filtered semantic feature vector of the distance-Doppler signal low-frequency part in the set of filtered semantic feature vectors of the distance-Doppler signal low-frequency part is cascaded to obtain the distance-Doppler signal low-frequency part semantic aggregation purified representation vector.That is, based on the comparison between the span factor of the semantic energy distribution spectrum of the low-frequency part of each range-Doppler signal and the preset threshold, the semantic feature vectors of the low-frequency part of the range-Doppler signal that meet the conditions are selected to form a sequence. In this way, the feature vector is screened by selecting the semantic features of the low-frequency part whose distance from the center vector of the energy spectrum distribution is within a certain range. This helps to remove noise and outliers, retain the most representative and informative features, and can more accurately reflect the real state and vital signs information of the target object, which helps to more accurately perceive and detect the heart rate of the target object in the future.

[0037] Among them, calculating the characteristic energy distribution spectrum vector of each semantic feature vector of the low-frequency part of the distance-Doppler signal in the set of semantic feature vectors of the low-frequency part of the distance-Doppler signal to obtain a sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the distance-Doppler signal, includes: respectively calculating the characteristic distribution energy synergy representation vector between each semantic feature vector of the low-frequency part of the distance-Doppler signal and other semantic feature vectors of the low-frequency part of the distance-Doppler signal in the set of semantic feature vectors of the low-frequency part of the distance-Doppler signal to obtain a sequence of multiple semantic feature distribution energy synergy representation vectors of the low-frequency part of the distance-Doppler signal; calculating the characteristic distribution energy synergy factor vector of the sequence of each semantic feature distribution energy synergy representation vector of the low-frequency part of the distance-Doppler signal in the sequence of multiple semantic feature distribution energy synergy representation vectors of the low-frequency part of the distance-Doppler signal to obtain a sequence of semantic feature energy distribution spectrum vectors of the low-frequency part of the distance-Doppler signal. Specifically, the process of respectively calculating the feature distribution energy collaborative representation vector between each of the semantic feature vectors of the low-frequency part of the distance-Doppler signal and other semantic feature vectors of the low-frequency part of the distance-Doppler signal in the set of the semantic feature vectors of the low-frequency part of the distance-Doppler signal to obtain a sequence of multiple semantic feature distribution energy collaborative representation vectors of the low-frequency part of the distance-Doppler signal includes: extracting a predetermined semantic feature vector of the low-frequency part of the distance-Doppler signal from the sequence of feature energy distribution spectrum vectors of the semantic feature vectors of the low-frequency part of the distance-Doppler signal; performing element-by-element maximum value extraction on the predetermined semantic feature vector of the low-frequency part of the distance-Doppler signal and other semantic feature vectors of the low-frequency part of the distance-Doppler signal to obtain the semantic feature distribution energy collaborative representation vector of the low-frequency part of the distance-Doppler signal corresponding to the predetermined semantic feature vector of the low-frequency part of the distance-Doppler signal A sequence of vectors; and a process of calculating the feature distribution energy coordination factor vector of each sequence of the multiple distance-Doppler signal low-frequency part semantic feature distribution energy coordination representation vectors to obtain the sequence of the distance-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, comprising: extracting the maximum value of each distance-Doppler signal low-frequency part semantic feature distribution energy coordination representation vector in the sequence of the distance-Doppler signal low-frequency part semantic feature distribution energy coordination representation vector respectively to obtain a sequence of maximum values ​​of the distance-Doppler signal low-frequency part semantic features; calculating the mean and variance of each distance-Doppler signal low-frequency part semantic feature distribution energy coordination representation vector respectively to obtain a sequence of distance-Doppler signal low-frequency part semantic feature means and a sequence of distance-Doppler signal low-frequency part semantic feature variances;The sequence of the semantic feature variance of the low-frequency part of the distance-Doppler signal and the preset hyperparameter are added by position to obtain the sequence of the first energy distribution coefficient of the semantic feature of the low-frequency part of the distance-Doppler signal; the square of the difference between the maximum value of the semantic feature of the low-frequency part of the distance-Doppler signal and the mean value of the semantic feature of the low-frequency part of the distance-Doppler signal in each group corresponding to the sequence of the maximum value of the semantic feature of the low-frequency part of the distance-Doppler signal and the sequence of the mean value of the semantic feature of the low-frequency part of the distance-Doppler signal is calculated to obtain the sequence of the semantic difference value of the low-frequency part of the distance-Doppler signal; the sequence of the semantic feature variance of the low-frequency part of the distance-Doppler signal is multiplied by constant 2 by position to obtain the sequence of the modulated semantic feature variance of the low-frequency part of the distance-Doppler signal; After adding each corresponding set of the semantic difference values ​​of the low-frequency part of the distance-Doppler signal and the variance of the semantic feature of the low-frequency part of the modulated distance-Doppler signal in the sequence of the semantic difference values ​​of the low-frequency part of the distance-Doppler signal and the variance of the semantic feature of the low-frequency part of the modulated distance-Doppler signal, the sum is added to the preset hyperparameter to obtain the sequence of the second energy distribution coefficients of the semantic feature of the low-frequency part of the distance-Doppler signal; the sequence of the first energy distribution coefficients of the semantic feature of the low-frequency part of the distance-Doppler signal and the sequence of the second energy distribution coefficients of the semantic feature of the low-frequency part of the distance-Doppler signal are divided by position to obtain the energy distribution spectrum vector of the semantic feature of the low-frequency part of the distance-Doppler signal composed of multiple feature distribution energy synergy factors. ;

[0038] More specifically, the process of calculating the energy distribution spectrum span factor between the central vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal and each semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal in the sequence of the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal to obtain the sequence of semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signal includes: calculating the position difference between the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal and the central vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal to obtain the semantic differential feature vector of the low-frequency part of the range-Doppler signal; calculating the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal The covariance matrix between the center vector of the semantic energy spectrum distribution and the semantic feature energy distribution spectrum vector of the low-frequency part of the distance-Doppler signal is obtained, and the inverse of the covariance matrix is ​​calculated to obtain the semantic inverse covariance matrix of the low-frequency part of the distance-Doppler signal; the transposed vector of the semantic differential feature vector of the low-frequency part of the distance-Doppler signal and the product of the semantic inverse covariance matrix of the low-frequency part of the distance-Doppler signal and the semantic differential feature vector of the low-frequency part of the distance-Doppler signal are calculated to obtain the semantic distribution value of the low-frequency part of the distance-Doppler signal; the square root of the semantic distribution value of the low-frequency part of the distance-Doppler signal is calculated to obtain the spectral span factor of the semantic energy distribution of the low-frequency part of the distance-Doppler signal.

[0039] In summary, in the above embodiment, the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is input into the selection optimization aggregation network based on the feature distribution characteristics to obtain the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, including: inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into the selection optimization aggregation network based on the feature distribution characteristics to process with the following feature selection optimization aggregation formula to obtain the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal; wherein the feature selection optimization aggregation formula is:

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] in, and Respectively represent the first and The semantic feature vector of the low-frequency part of the range-Doppler signal, It means taking the maximum value between two vectors element by element. For the and The distance-Doppler signal low-frequency part semantic feature vectors are the distance-Doppler signal low-frequency part semantic feature distribution energy collaborative representation vectors, To extract the maximum value of a vector, and They are The mean and variance of To preset hyperparameters, is the first in the sequence of the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal The energy distribution spectrum vector of the semantic feature of the low-frequency part of the range-Doppler signal The characteristic distribution energy synergy factor, For the The energy distribution spectrum vector of the semantic feature of the low-frequency part of the range-Doppler signal The characteristic distribution energy synergy factor, A sequence representing the energy distribution spectrum vector of the semantic feature of the low-frequency part of the range-Doppler signal, is the first in the sequence of the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal The energy distribution spectrum vector of the semantic feature of the low-frequency part of the range-Doppler signal, is the number of vectors in the sequence of the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal, is the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal, is the transpose of a vector, It is The covariance matrix between the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal and the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal, For the The span factor of the semantic energy distribution spectrum of the low-frequency part of the range-Doppler signal corresponding to the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal, is the preset threshold, is the first in the set of semantic feature vectors of the low-frequency part of the filtered range-Doppler signal. The semantic feature vector of the low-frequency part of the filtered range-Doppler signal, respectively represent the first The semantic feature vector of the low-frequency part of the filtered range-Doppler signal, is the set of semantic feature vectors of the low-frequency part of the filtered range-Doppler signal, represents vector concatenation, It is a semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal.

[0049] In particular, the S5 performs heart rate perception based on the semantically aggregated purified representation vector of the low-frequency part of the distance-Doppler signal to determine the perception result, and the perception result is used to represent the heart rate frequency decoding value. In a specific example of the present application, the semantically aggregated purified representation vector of the low-frequency part of the distance-Doppler signal is input into a decoder-based heart rate frequency perception module to obtain the perception result, and the perception result is used to represent the heart rate frequency decoding value. That is, the semantically purified aggregation features of the low-frequency part of the distance-Doppler signal are used to perform decoding regression, so as to perceive and detect the heart rate frequency of static personnel. In this way, the frequency band advantage of millimeter-wave radar can be utilized, and artificial intelligence and deep learning technology can be combined to more accurately perceive and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, and can better adapt to changes in complex home environments, improve the flexibility and generalization ability of smart home perception, and in this way, it can more effectively cope with different application scenarios and improve the intelligence level of smart home perception systems.

[0050] Among them, each semantic feature vector of the low-frequency part of the range-Doppler signal in the set of semantic feature vectors of the low-frequency part of the range-Doppler signal respectively represents the semantic feature of the low-frequency part image of the range-Doppler signal determined based on the filtering factor. Therefore, when the set of semantic feature vectors of the low-frequency part of the range-Doppler signal is input into the selection optimization aggregation network based on the feature distribution characteristics, the full-frequency domain aggregation offset caused by the significant difference in feature energy of each frequency band in the range-Doppler signal will lead to the lack of feature instance determination of the semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, thereby affecting the accuracy of the perception result obtained by the decoder-based heartbeat frequency perception module that has been input and trained.

[0051] Preferably, inputting the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal into a decoder-based heartbeat frequency perception module to obtain a perception result includes:

[0052] The distance between each pair of eigenvalues ​​of the semantic aggregation and purification representation vector of the low-frequency part of the distance-Doppler signal is calculated, for example, the L2 distance, and the square root of the distance is taken to obtain the distance representation matrix of the semantic aggregation and purification of the low-frequency part of the distance-Doppler signal, that is:

[0053]

[0054] in, The semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal is represented. and The first representation vector of the semantic aggregation and purification of the low-frequency part of the range-Doppler signal is represented by and Eigenvalues, The first representation vector of the semantic aggregation and purification of the low-frequency part of the range-Doppler signal is represented by and The distance between eigenvalues, Representation of the low-frequency part of the distance-Doppler signal semantic aggregation purification distance representation matrix The eigenvalue of the position;

[0055] The distance-Doppler signal low-frequency part semantic aggregation purification autocorrelation matrix of the distance-Doppler signal low-frequency part semantic aggregation purification representation vector as a row vector is obtained, that is, ,in, The semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal is represented. represents the transpose of a vector, represents matrix multiplication, Represents the semantically aggregated purified autocorrelation matrix of the low-frequency part of the range-Doppler signal;

[0056] The semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal is matrix-multiplied with the semantic aggregation and purification distance representation matrix of the low-frequency part of the range-Doppler signal to obtain the semantic aggregation and purification first-level mapping vector of the low-frequency part of the range-Doppler signal, that is, ,in, The semantically aggregated and purified representation vector of the low-frequency part of the range-Doppler signal is represented. Represents the distance-Doppler signal low-frequency part semantic aggregation purification distance representation matrix, represents matrix multiplication, Represents the semantic aggregation and purification of the low-frequency part of the range-Doppler signal first-level mapping vector;

[0057] The distance-Doppler signal low-frequency part semantic aggregation purification first-level mapping vector is matrix-multiplied with the distance-Doppler signal low-frequency part semantic aggregation purification distance representation matrix and the distance-Doppler signal low-frequency part semantic aggregation purification self-correlation matrix to obtain the distance-Doppler signal low-frequency part semantic aggregation purification multi-level mapping vector ,in, Represents the semantic aggregation and purification of the low-frequency part of the range-Doppler signal. Represents the distance-Doppler signal low-frequency part semantic aggregation purification distance representation matrix, represents matrix multiplication, Represents the semantic aggregation and purification of the low-frequency part of the range-Doppler signal autocorrelation matrix, Represents the semantic aggregation and purification of the low-frequency part of the range-Doppler signal by a multi-level mapping vector;

[0058] The multi-level mapping vector of the semantic aggregation and purification of the low-frequency part of the distance-Doppler signal is interpolated with the associated eigenvector of the semantic aggregation and purification of the low-frequency part of the distance-Doppler signal composed of the eigenvalues ​​of the semantic aggregation and purification of the low-frequency part of the distance-Doppler signal auto-association matrix to obtain an optimized semantic aggregation and purification representation vector of the low-frequency part of the distance-Doppler signal, wherein interpolation or zero padding is performed when the eigenvalue is insufficient;

[0059] The semantically aggregated and purified representation vector of the optimized low-frequency part of the range-Doppler signal is input into a decoder-based heartbeat frequency perception module to obtain a perception result.

[0060] Therefore, through the linear target mapping representation of the similarity distance representation matrix based on the semantic aggregation and purification representation vector of the low-frequency part of the distance-Doppler signal, the self-correlation of the complete similarity instantiation of the semantic aggregation and purification representation vector of the low-frequency part of the distance-Doppler signal is instantiated to perform a secondary target mapping representation based on a multi-level distribution hierarchy, and the association mismatch negative influence factor is compensated by the association fusion kernel bias to improve the eigenvalue instance determination degree of the semantic aggregation and purification representation vector of the low-frequency part of the distance-Doppler signal under similarity constraints, that is, the significance degree of the eigenvalue as an instance for the circular regression judgment, and improve the accuracy of the perception result obtained by the decoder-based heart rate perception module of the semantic aggregation and purification representation vector of the low-frequency part of the distance-Doppler signal. It can more accurately perceive and detect the heart rate frequency of static people, improve the flexibility and generalization ability of smart home perception, and make the smart home perception system have a higher level of intelligence.

[0061] In summary, the millimeter-wave radar smart home perception method according to the embodiment of the present application is explained. After filtering the millimeter-wave detection echo signal and converting it into the range-Doppler domain, a signal processing and analysis algorithm based on artificial intelligence and deep learning is introduced at the back end to analyze the converted range-Doppler signal, so as to capture and characterize the semantic features of the low-frequency part of the range-Doppler signal, thereby perceiving and detecting the heart rate frequency of static personnel. It can utilize the frequency band advantage of the millimeter-wave radar, and combine artificial intelligence and deep learning technology to more accurately perceive and detect the heart rate frequency of static personnel, thereby reducing the need for human intervention, and can better adapt to changes in complex home environments, improve the flexibility and generalization ability of smart home perception, and can more effectively respond to different application scenarios, and improve the intelligence level of smart home perception systems.

[0062] Furthermore, a millimeter-wave radar smart home sensing system is also provided.

[0063] Figure 3 FIG. 1 is a block diagram of a millimeter wave radar smart home sensing system according to an embodiment of the present application. Figure 3 As shown, according to the millimeter wave radar smart home perception system 300 of the embodiment of the present application, it includes: a signal acquisition module 310, which is used to collect millimeter wave detection echo signals through an oscilloscope; a signal preprocessing module 320, which is used to perform high-pass filtering on the millimeter wave detection echo signals, and then convert the filtered millimeter wave detection echo signals from the time domain to the range-Doppler domain to obtain the range-Doppler signals; a signal feature filtering module 330, which is used to pass the range-Doppler signals through the signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signals; a feature selection optimization aggregation module 340, which is used to input the set of semantic feature vectors of the low-frequency part of the range-Doppler signals into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signals; a perception result determination module 350, which is used to perform heart rate frequency perception based on the semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signals to determine the perception result, and the perception result is used to represent the heart rate frequency decoding value.

[0064] As described above, the millimeter wave radar smart home perception system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a millimeter wave radar smart home perception algorithm. In one possible implementation, the millimeter wave radar smart home perception system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the millimeter wave radar smart home perception system 300 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the millimeter wave radar smart home perception system 300 can also be one of the many hardware modules of the wireless terminal.

[0065] Alternatively, in another example, the millimeter wave radar smart home sensing system 300 and the wireless terminal may also be separate devices, and the millimeter wave radar smart home sensing system 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0066] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A millimeter wave radar smart home sensing method, characterized in that: include: Collect millimeter wave detection echo signals through an oscilloscope; After high-pass filtering the millimeter-wave detection echo signal, the obtained filtered millimeter-wave detection echo signal is converted from the time domain to the range-Doppler domain to obtain a range-Doppler signal; Passing the range-Doppler signal through a signal feature filter to obtain a set of semantic feature vectors of a low-frequency portion of the range-Doppler signal; Inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signal; Heart rate frequency perception is performed based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine a perception result, and the perception result is used to represent the heart rate frequency decoding value.

2. The millimeter wave radar smart home sensing method according to claim 1, characterized in that: After high-pass filtering the millimeter wave detection echo signal, the filtered millimeter wave detection echo signal is converted from the time domain to the range-Doppler domain to obtain a range-Doppler signal, including: Performing high-pass filtering on the millimeter-wave detection echo signal to obtain a filtered millimeter-wave detection echo signal; The filtered millimeter-wave detection echo signal is subjected to a range-Doppler transform to convert the filtered millimeter-wave detection echo signal from a time domain to a range-Doppler domain to obtain the range-Doppler signal.

3. The millimeter wave radar smart home sensing method according to claim 2 is characterized in that: The range-Doppler signal is passed through a signal feature filter to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal, including: inputting the range-Doppler signal into a signal feature filter based on a hole convolutional neural network model to obtain a set of semantic feature vectors of the low-frequency part of the range-Doppler signal.

4. The millimeter wave radar smart home sensing method according to claim 3 is characterized in that: Inputting the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purification representation vector of the low-frequency part of the range-Doppler signal, including: Calculating the characteristic energy distribution spectrum vector of each range-Doppler signal low-frequency part semantic feature vector in the set of range-Doppler signal low-frequency part semantic feature vectors to obtain a sequence of range-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors; Calculating the position mean vector of the sequence of the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal as the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal; Calculate the energy distribution spectrum span factor between the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal and each of the semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal in the sequence of the semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of the semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signal; Determine a set of filtered semantic feature vectors of the low-frequency part of the range-Doppler signal based on a comparison between each semantic energy distribution spectrum span factor of the low-frequency part of the range-Doppler signal in the sequence of the semantic energy distribution spectrum span factor of the low-frequency part of the range-Doppler signal and a preset threshold; The filtered semantic feature vectors of the low-frequency part of the range-Doppler signal in the set of the filtered semantic feature vectors of the low-frequency part of the range-Doppler signal are cascaded to obtain the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal.

5. The millimeter wave radar smart home sensing method according to claim 4 is characterized in that: Calculating the characteristic energy distribution spectrum vector of each range-Doppler signal low-frequency part semantic feature vector in the set of the range-Doppler signal low-frequency part semantic feature vectors to obtain a sequence of range-Doppler signal low-frequency part semantic feature energy distribution spectrum vectors, including: Respectively calculating the feature distribution energy collaborative representation vector between each of the semantic feature vectors of the low-frequency part of the range-Doppler signal and other semantic feature vectors of the low-frequency part of the range-Doppler signal in the set of the semantic feature vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of multiple semantic feature distribution energy collaborative representation vectors of the low-frequency part of the range-Doppler signal; The characteristic distribution energy coordination factor vector of each sequence of the semantic feature distribution energy coordination representation vector of the low-frequency part of the range-Doppler signal in the sequence of the multiple semantic feature distribution energy coordination representation vectors of the low-frequency part of the range-Doppler signal is calculated to obtain the sequence of the semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal.

6. The millimeter wave radar smart home sensing method according to claim 5, characterized in that: The method comprises: calculating the feature distribution energy synergistic representation vector between each of the semantic feature vectors of the low frequency part of the range-Doppler signal and other semantic feature vectors of the low frequency part of the range-Doppler signal in the set of the semantic feature vectors of the low frequency part of the range-Doppler signal respectively to obtain a sequence of multiple semantic feature distribution energy synergistic representation vectors of the low frequency part of the range-Doppler signal, including: Extracting a predetermined range-Doppler signal low-frequency part semantic feature vector from a sequence of characteristic energy distribution spectrum vectors of the range-Doppler signal low-frequency part semantic feature vector; The predetermined semantic feature vector of the low-frequency part of the range-Doppler signal and each other semantic feature vector of the low-frequency part of the range-Doppler signal are respectively subjected to element-by-element maximum value extraction to obtain a sequence of collaborative representation vectors of the semantic feature distribution energy of the low-frequency part of the range-Doppler signal corresponding to the predetermined semantic feature vector of the low-frequency part of the range-Doppler signal.

7. The millimeter wave radar smart home sensing method according to claim 6, characterized in that: Calculating the feature distribution energy synergy factor vector of each sequence of the semantic feature distribution energy synergy representation vector of the low-frequency part of the range-Doppler signal in the sequence of the multiple semantic feature distribution energy synergy representation vectors of the low-frequency part of the range-Doppler signal to obtain the sequence of the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal, including: Respectively extracting the maximum value of each range-Doppler signal low-frequency part semantic feature distribution energy collaborative representation vector in the sequence of the range-Doppler signal low-frequency part semantic feature distribution energy collaborative representation vector to obtain a sequence of maximum values ​​of the range-Doppler signal low-frequency part semantic features; Calculating the mean and variance of the energy cooperative representation vector of the semantic feature distribution of the low-frequency part of each range-Doppler signal respectively to obtain a sequence of the mean of the semantic feature of the low-frequency part of the range-Doppler signal and a sequence of the variance of the semantic feature of the low-frequency part of the range-Doppler signal; Adding the sequence of variances of the semantic features of the low-frequency part of the range-Doppler signal and the preset hyperparameters by position to obtain a sequence of first energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal; Calculating the square of the difference between the maximum value of the semantic feature of the low frequency part of the range-Doppler signal and the mean value of the semantic feature of the low frequency part of the range-Doppler signal in each group corresponding to the sequence of the maximum value of the semantic feature of the low frequency part of the range-Doppler signal and the sequence of the mean value of the semantic feature of the low frequency part of the range-Doppler signal to obtain a sequence of semantic difference values ​​of the low frequency part of the range-Doppler signal; Multiplying the sequence of semantic feature variances of the low-frequency part of the range-Doppler signal by constant 2 by position to obtain a sequence of semantic feature variances of the low-frequency part of the modulated range-Doppler signal; After adding each corresponding group of the semantic difference value of the low-frequency part of the distance-Doppler signal and the variance of the semantic feature of the low-frequency part of the modulated distance-Doppler signal in the sequence of the semantic difference value of the low-frequency part of the distance-Doppler signal and the sequence of the variance of the semantic feature of the low-frequency part of the modulated distance-Doppler signal, the sum is added to the preset hyperparameter to obtain a sequence of the second energy distribution coefficient of the semantic feature of the low-frequency part of the distance-Doppler signal; The sequence of the first energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal is divided by the sequence of the second energy distribution coefficients of the semantic features of the low-frequency part of the range-Doppler signal by position to obtain an energy distribution spectrum vector of the semantic features of the low-frequency part of the range-Doppler signal composed of multiple feature distribution energy synergy factors.

8. The millimeter wave radar smart home sensing method according to claim 7, characterized in that: Calculating the energy distribution spectrum span factor between the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal and each of the semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal in the sequence of the semantic feature energy distribution spectrum vectors of the low-frequency part of the range-Doppler signal to obtain a sequence of the semantic energy distribution spectrum span factors of the low-frequency part of the range-Doppler signal, including: Calculating the position difference between the semantic feature energy distribution spectrum vector of the low-frequency part of the range-Doppler signal and the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal to obtain the semantic differential feature vector of the low-frequency part of the range-Doppler signal; Calculate the covariance matrix between the center vector of the semantic energy spectrum distribution of the low-frequency part of the range-Doppler signal and the spectrum vector of the semantic feature energy distribution of the low-frequency part of the range-Doppler signal, and calculate the inverse of the covariance matrix to obtain the semantic inverse covariance matrix of the low-frequency part of the range-Doppler signal; Calculating the product of the transposed vector of the semantic differential feature vector of the low frequency part of the range-Doppler signal and the semantic inverse covariance matrix of the low frequency part of the range-Doppler signal and the semantic differential feature vector of the low frequency part of the range-Doppler signal to obtain the semantic distribution value of the low frequency part of the range-Doppler signal; The square root of the semantic distribution value of the low-frequency part of the range-Doppler signal is calculated to obtain the spectral span factor of the semantic energy distribution of the low-frequency part of the range-Doppler signal.

9. The millimeter wave radar smart home sensing method according to claim 8, characterized in that: Heart rate frequency perception is performed based on the semantically aggregated purified representation vector of the low-frequency part of the distance-Doppler signal to determine a perception result, and the perception result is used to represent the heart rate frequency decoding value, including: inputting the semantically aggregated purified representation vector of the low-frequency part of the distance-Doppler signal into a decoder-based heart rate frequency perception module to obtain the perception result, and the perception result is used to represent the heart rate frequency decoding value.

10. Millimeter wave radar smart home perception system, characterized in that: include: A signal acquisition module, used for acquiring millimeter wave detection echo signals through an oscilloscope; A signal preprocessing module, used for performing high-pass filtering on the millimeter-wave detection echo signal, and then converting the obtained filtered millimeter-wave detection echo signal from the time domain to the range-Doppler domain to obtain a range-Doppler signal; A signal feature filtering module, used for passing the range-Doppler signal through a signal feature filter to obtain a set of semantic feature vectors of a low-frequency part of the range-Doppler signal; A feature selection optimization aggregation module is used to input the set of semantic feature vectors of the low-frequency part of the range-Doppler signal into a selection optimization aggregation network based on feature distribution characteristics to obtain a semantic aggregation purified representation vector of the low-frequency part of the range-Doppler signal; The sensing result determination module is used to sense the heart rate frequency based on the semantic aggregation and purification representation vector of the low-frequency part of the range-Doppler signal to determine the sensing result, and the sensing result is used to represent the heart rate frequency decoding value.

Images