Millimeter-wave radar sleep behavior recognition method based on adaptive synchronous compression transform

Abstract

The present invention proposes a method for identifying sleep behavior using a millimeter-wave radar based on adaptive synchronous compression transform. The present invention relates to the field of sleep monitoring technology. The millimeter-wave radar obtains intermediate frequency signals of sleep behavior and uses a sliding window method to suppress background clutter. The intermediate frequency signal after background interference is removed is subjected to a fast Fourier transform to generate a range image, and a unit average constant false alarm rate detection algorithm is used to achieve target detection and range unit positioning. At the range unit where the target is located, the range image is subjected to an adaptive synchronous compression transform along the slow time axis, and the window length of the short-time Fourier transform is adaptively adjusted according to the main frequency change trend to improve the time-frequency energy concentration and compensate for the signal attenuation and frequency shift caused by distance and angle changes. Finally, a cascaded CNN-LSTM network model is constructed. CNN extracts spatial features, and LSTM captures temporal dependencies to achieve accurate identification of sleep behavior under different distance and angle conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of sleep monitoring, and in particular to a millimeter-wave radar sleep behavior recognition method based on adaptive synchronous compression transform. Background Art

[0002] With the growing severity of sleep disorders, sleep monitoring technology has garnered widespread attention. Existing sleep monitoring methods primarily fall into two categories: contact and non-contact. While polysomnography and wearable devices offer high measurement accuracy, they are inconvenient to wear and may disrupt natural sleep. Non-contact methods, such as millimeter-wave radar, utilize the micro-Doppler effect to enable remote, non-intrusive monitoring and hold broad application prospects.

[0003] In practical applications, millimeter-wave radar monitoring still faces challenges. For example, background clutter interferes with the extraction of micro-motion signals, and the non-stationary nature of the signal makes it difficult for traditional short-time Fourier transforms (STFTs) to achieve both time and frequency resolution. Furthermore, Doppler signatures are significantly affected by changes in target angle and distance, especially at different azimuths. Signal attenuation can lead to a decrease in behavior recognition accuracy. Summary of the Invention

[0004] This paper proposes a millimeter-wave radar sleep behavior recognition method based on adaptive synchronous compression transform. This method adaptively adjusts the STFT window length and dynamically optimizes the time-frequency resolution based on the dominant frequency, effectively suppressing the dispersion of non-stationary signals and enhancing Doppler feature expression. A cascaded CNN-LSTM network fully utilizes spatial features and temporal information to achieve high-precision sleep behavior recognition, improving the system's stability and adaptability.

[0005] The technical solution proposed by the present invention is:

[0006] The method for recognizing sleeping behavior by millimeter-wave radar based on adaptive synchronous compression transform includes the following steps:

[0007] Step 1: Initialize the millimeter-wave radar system, set radar operating parameters, and collect intermediate frequency signals of different sleep behaviors to provide data for subsequent behavior recognition;

[0008] Step 2: Use the sliding window averaging method to suppress dynamic background clutter on the intermediate frequency signal. The mean value within the window is calculated as the background estimate, and a differential operation is performed to remove static clutter, thereby improving the clutter suppression rate while ensuring the integrity of the micro-Doppler characteristics.

[0009] Step 3: The filtered signal is then subjected to a fast Fourier transform along the fast time axis to obtain a range profile. A global detection threshold is set based on the spectral mean of the range profile, and the presence of a target is determined using a cell-averaged constant false alarm (CA-CFAR) algorithm. After a target is detected, the range cell in which the target is located is determined by searching for the maximum value of the range profile.

[0010] Step 4: At the target range unit, perform adaptive synchronous compression transform (ASST) on the range image along the slow time axis to obtain a time-frequency representation with enhanced Doppler characteristics.

[0011] Step 5: Repeat the adaptive synchronous compression transform (ASST) on all radar data frames, extract the instantaneous main frequency of each frame and splice them in chronological order to generate the enhanced Doppler time-frequency map (EDTM), thereby fully characterizing the change of the Doppler characteristics of the target motion over time.

[0012] Step 6: Input EDTM into the cascaded CNN-LSTM network and train the model to recognize sleep behavior;

[0013] Step 7: Use the trained model to identify five sleep behaviors in the validation set, including sitting up, lying down, turning over, waving hands, and falling out of bed.

[0014] Furthermore, in step 1, the initialization of the millimeter wave radar includes setting the transmission frequency, bandwidth, sampling rate and number of transceiver antennas; and collecting intermediate frequency signals of five sleeping behaviors including sitting up, lying down, turning over, waving hands and falling out of bed.

[0015] Furthermore, in step 2, the local background average within the previous fixed window is calculated for each chirp of the collected intermediate frequency signal using a sliding window method. This is then subtracted from the current chirp, effectively suppressing static interference. After background suppression, the filtered signal is subjected to a one-dimensional Fourier transform along the fast time axis, converting the time domain signal to the frequency domain to generate a range profile.

[0016] In step 3, the present invention regards the entire spectrum of the range image of the current data frame as a reference unit without a protection unit, averages the spectrum within the reference unit, obtains a global background noise energy estimate, multiplies the global background noise energy estimate by a scaling factor α to obtain a global detection threshold, detects the entire range image using the detection threshold, and determines whether the range image contains a target based on whether any range unit is greater than the detection threshold. If the target is contained, the range unit r where the target is located can be obtained by searching for the maximum value position of the range image.

[0017] In step 4, at the target range unit, adaptive synchronous compression transform (ASST) is performed on the range image along the slow time axis to obtain a time-frequency representation with enhanced Doppler characteristics. The specific process is as follows:

[0018] (4.1) From the range image generated by the radar, select the target distance unit r and extract its slow time dimension signal sequence x r (t), where t represents the slow time axis. Then for the signal x r (t) Perform Fourier transform, calculate frequency distribution, calculate frequency distribution x r (f):

[0019]

[0020] According to X r (f) is the peak value of the amplitude spectrum, and determines the main frequency f of the current signal c and its changing trends.

[0021] (4.2) Dynamically adjust the length of the STFT window function based on the frequency range of the signal's main frequency. c Higher than the preset frequency domain threshold f th When the short window function w is used short (t), the window length is N short , by shortening the time window to improve the time resolution, ensure the capture of the transient characteristics of high-frequency components; if the signal main frequency f c Below the preset frequency domain threshold f th When using the long window function w long (t), window length N long By extending the time window, the frequency resolution is improved to ensure the spectrum refinement of the low-frequency components. r (t) Perform STFT calculation:

[0022]

[0023] Output adaptive time-frequency distribution S r (t,ω), where is the window function ω(t) and ω is the angular frequency.

[0024] (4.3) Calculate the STFT result S r The phase derivative of (t,ω) gives the instantaneous frequency estimate

[0025]

[0026] where arg(S r (t,ω)) is the phase value of the STFT calculation result. According to the estimated instantaneous frequency, the energy of STFT|S r (t,ω)| 2 Reassigned to the estimated instantaneous frequency position to form a synchronous compressed time-frequency representation T r (t,ω), the redistribution process can be expressed as:

[0027]

[0028] Where δ(·) represents the Dirac function. Through the energy redistribution of SST, a compressed time-frequency representation is obtained, which has a higher time-frequency resolution.

[0029] In step 5, based on the processing of the single frame radar signal in step 3 and step 4, the above processing flow is sequentially executed for all data frames collected by the radar. Specifically, first, for each frame signal Calculate its corresponding adaptive synchronous compression transform time-frequency representation The frames obtained The frames are arranged and spliced ​​in chronological order to form a continuous feature representation frame by frame and smoothed at the boundaries between adjacent frames to obtain the Doppler feature enhanced time-frequency map EDTM. The time is used as the horizontal axis and the Doppler frequency shift is used as the vertical axis. The numerical value represents the signal strength, which characterizes the change of the Doppler characteristics of the target movement over time.

[0030] In step 6, the Doppler feature-enhanced time-frequency map (EDTM) is fed into a cascaded CNN-LSTM network for training. In this network structure, the convolutional neural network (CNN) module extracts spatial features from the EDTM, while the long short-term memory (LSTM) module captures temporal correlation information in the feature sequence.

[0031] During training, labeled sleep behavior data is used as a supervisory signal. The cross-entropy loss function is used to calculate the error between the model's predicted categories and the true labels. The network parameters are then updated based on this error through backpropagation. The Adam algorithm is used as the optimizer, adaptively adjusting the learning rate to accelerate convergence and minimize the loss function. After training is complete, the optimal network weights are saved for subsequent inference.

[0032] In step 7, the trained cascaded CNN-LSTM network is used to perform a recognition test on the validation set. The specific process involves first loading the saved model weights, then inputting the Doppler time-frequency maps (EDTMs) from the validation set into the network, and outputting the corresponding category prediction results. Finally, the recognition accuracy is calculated and the confusion matrix is ​​plotted to comprehensively evaluate the model's recognition performance.

[0033] The present invention uses adaptive synchronous compression transform to process radar intermediate frequency signals to generate Doppler time maps, and adopts a cascaded CNN-LSTM neural network to extract spatial and temporal features of sleep behavior to improve recognition accuracy.

[0034] The beneficial effects of the present invention are:

[0035] 1. The present invention adopts adaptive synchronous compression transform (ASST) to effectively compensate for the weakening of Doppler signals caused by changes in target azimuth or distance, improve the time-frequency energy focusing, thereby enhancing the distinguishability of the signal and reducing the impact of signal attenuation on behavior recognition.

[0036] 2. This paper adopts a cascaded CNN-LSTM network, uses 2D-CNN to extract the spatial information of the Doppler time map, and uses LSTM to capture the time series characteristics of sleep behavior, accurately distinguishing different sleep behaviors and significantly improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the overall flow chart of the sleep behavior recognition method of the present invention;

[0038] Figure 2 is a flow chart of radar data processing of the present invention;

[0039] Figure 3 The five sleep behaviors defined in the embodiment of the present invention are based on the Doppler time map of the short-time Fourier transform, including (a) lying down, (b) sitting up, (c) turning over, (d) waving hands, and (e) falling out of bed;

[0040] Figure 4 The five sleep behaviors defined in the embodiment of the present invention are Doppler feature-enhanced time-frequency images based on adaptive synchronous compression transform, including (a) lying down, (b) sitting up, (c) turning over, (d) waving hands, and (e) falling out of bed.

[0041] Figure 5 It is the network structure diagram of the present invention, where (a) is the overall cascade CNN-LSTM network structure diagram, (b) is the 2D-CNN network structure, (c) is the feature flattening network structure, and (d) is the LSTM network structure. DETAILED DESCRIPTION

[0042] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings to more clearly illustrate the advantages and features of the present invention and to make it easier for those skilled in the art to understand the essence of the present invention. The description of the specific embodiments is intended to further clarify the scope of protection of the present invention and provide a basis for the definition of the claims.

[0043] refer to Figures 1 to 5 The millimeter wave radar sleep behavior recognition method based on adaptive synchronous compression transform includes the following steps:

[0044] Step 1: Initialize the millimeter-wave radar system and set radar operating parameters, including transmission frequency, bandwidth, sampling rate, etc., to ensure that the system can meet the requirements of capturing different sleep behavior characteristics;

[0045] In the acquisition environment, the radar equipment was installed on the ceiling to ensure coverage of the test area. The subjects were placed in the test area directly below the radar to collect data on five sleep behaviors: sitting up, lying down, turning over, waving hands, and falling out of bed. The radar transmits a frequency-modulated continuous wave signal, receives the echo signal after it is reflected by the target, and obtains the intermediate frequency signal through mixing processing, which is expressed as:

[0046] x(t)=A(t)cos(2πf IF (t)t+φ(t))

[0047] Among them, f IF (t) is the time-varying intermediate frequency (IF) that contains Doppler characteristics caused by target motion, and φ(t) is the instantaneous phase. The collected IF signal provides the data foundation for subsequent sleep behavior recognition.

[0048] Step 2: To reduce the interference of static background on target signal detection, background noise suppression is performed on the intermediate frequency signal. The specific method is sliding window filtering: Let x i (τ) is the time domain intermediate frequency echo of the i-th Chirp signal (τ is the fast time), the sliding window length is N (N = 20), and the background estimation value μ at time t is t (τ) is the local mean of the first N Chirps:

[0049]

[0050] Among them, τ∈[0,T c ], T c is the chirp signal period. From the current Chirp signal x t (τ) minus the background estimate μ t (τ), and the signal that suppresses static clutter is obtained:

[0051]

[0052] After background suppression is completed, the background-removed signal of each chirp is FFT-ed along the fast time axis τ to convert it from the time domain to the frequency domain, generating a range image R(r,t) to characterize the echo intensity distribution at different distance units. Its expression is:

[0053]

[0054] in, represents the Fourier transform performed on the fast time variable τ, and r is the corresponding distance index.

[0055] Based on the range image R(r,t) generated in step 2, the global background estimation method without protection unit is used for target detection. The amplitude spectrum of the entire range image is used as the reference unit, and its mean is calculated as the global background noise estimation value μ bg , the formula is:

[0056]

[0057] Where n represents the total number of reference units, r i Represents the index of the i-th distance unit.

[0058] According to the estimated background value and the set scaling factor α, the global detection threshold is calculated:

[0059] T global =α·μ bg

[0060] Traverse each distance unit, if its reflection intensity R(r,t) satisfies:

[0061] R(r,t)>T global

[0062] If multiple units meet the conditions, the unit with the largest reflection intensity is selected as the final target distance unit r:

[0063]

[0064] Step 4: Perform Fourier transform along the slow time dimension in the distance unit where the target is located to obtain the main frequency distribution of the current signal. Use different window sizes to perform short-time Fourier transform on different main frequency components. After obtaining the short-time Fourier transform results, calculate the instantaneous frequency of STFT. Finally, use synchronous compression transform to redistribute the energy of the STFT results.

[0065] Step 5: Based on the processing of single-frame radar signals in steps 3 and 4, target detection and adaptive synchronous compression transform are repeatedly performed on all radar data frames to obtain Doppler feature enhanced time-frequency map (EDTM). The feature maps of different sleep behaviors are as follows: Figure 4 As shown;

[0066] In order to further verify the performance advantage of the adaptive synchronized compressed transform (ASST) proposed in this paper in sleep behavior recognition, a performance comparison was conducted with the traditional short-time Fourier transform (STFT), continuous wavelet transform (CWT) and synchronized compressed transform (SST).

[0067] In terms of time-frequency energy focusing evaluation, Rényi entropy and main frequency bandwidth are used as quantitative indicators. The lower the Rényi entropy value, the more concentrated the time-frequency energy distribution; the narrower the main frequency bandwidth, the higher the frequency resolution. The results are shown in Table 1:

[0068]

[0069] Table 1

[0070] As can be seen from Table 1, the method of the present invention (ASST) is superior to the traditional time-frequency method in terms of time-frequency energy concentration and frequency resolution, and can more clearly characterize the changes in Doppler characteristics during sleep behavior.

[0071] Furthermore, to verify the stability and robustness of the present invention's Doppler feature extraction under varying signal-to-noise ratios (SNRs), this embodiment simulates SNR variations in millimeter-wave radar applications due to changes in target distance, azimuth, and human posture. Using low SNR settings commonly used in existing literature (-30dB, -20dB, and -10dB), the attenuation of radar echo signals in practical scenarios such as long-range detection and off-axis detection was simulated.

[0072] In order to quantify the extraction performance of weak Doppler features, this paper introduces the instantaneous frequency root mean square error normalization index Q value (defined as the ratio of the square of the instantaneous frequency error to the square of the theoretical frequency), the formula is:

[0073]

[0074] where f d (t) is the instantaneous frequency obtained by identification, f e (t) is the theoretical instantaneous frequency. The smaller the Q value, the higher the instantaneous frequency recognition accuracy and the better the feature extraction effect.

[0075] Experimental results show that under different conditions of signal-to-noise ratios (SNRs) of -30dB, -20dB, and -10dB, the adaptive synchronized compression transform (ASST) method proposed in this paper consistently maintains the lowest Q value and has superior instantaneous frequency extraction capabilities compared to traditional short-time Fourier transform (STFT), continuous wavelet transform (CWT), and synchronized compression transform (SST). A specific comparison is shown in Table 2:

[0076]

[0077] Table 2

[0078] As can be seen from the table, under low signal-to-noise ratio (such as -30dB), the Q value of the ASST method is about 36.7% lower than that of STFT, about 63.5% lower than that of CWT, and about 32.8% lower than that of SST; as the signal-to-noise ratio increases, the ASST method can still maintain the lowest value, showing excellent signal extraction ability.

[0079] Step 6: Feed the EDTM feature map as input data into the designed cascaded CNN-LSTM model. This model uses a supervised learning strategy, utilizing five labeled sleep behavior categories (including sitting up, lying down, rolling over, waving hands, and falling out of bed) as supervisory signals. A cross-entropy loss function is used to calculate the error between the predicted output and the true label. The Adam optimizer is then used to iteratively update the network parameters. Finally, the network weight file with the best performance on the validation set is saved.

[0080] Step 7: Load the optimal model weights trained and saved in step 6, input the EDTM feature map in the validation set into the model for inference and prediction, and obtain the classification results of each sample. To further verify the effectiveness of the model architecture, the present invention systematically compares the cascaded CNN model, LSTM model, and CNN-LSTM model. As can be seen from the comparison results in Table 3, the CNN-LSTM model can fully combine the high-dimensional spatial feature extraction capability of the CNN module and the time series modeling advantages of the LSTM module, thereby significantly improving the classification accuracy and system robustness in complex behavior scenarios.

[0081]

[0082] Table 3

[0083] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A millimeter-wave radar sleep behavior recognition method based on adaptive synchronous compression transform is characterized by: The following steps are involved: Step 1: Initialize the millimeter-wave radar system, set radar operating parameters, and collect intermediate frequency signals of different sleep behaviors to provide data for subsequent behavior recognition; Step 2: Use the sliding window averaging method to suppress dynamic background clutter on the intermediate frequency signal. The mean value within the window is calculated as the background estimate, and a differential operation is performed to remove static clutter, thereby improving the clutter suppression rate while ensuring the integrity of the micro-Doppler characteristics. Step 3: The filtered signal is then subjected to a fast Fourier transform along the fast time axis to obtain a range profile. A global detection threshold is set based on the spectral mean of the range profile, and the presence of a target is determined using the unit average constant false alarm (CA-CFAR) algorithm. After a target is detected, the range cell r in which the target is located is determined by searching for the maximum position of the range profile. Step 4: At the target range unit, perform adaptive synchronous compression transform (ASST) on the range image along the slow time axis to obtain a time-frequency representation with enhanced Doppler characteristics. 4.1: Get the slow time dimension signal sequence x of the target distance unit r r (t), then the signal x r (t) Perform Fourier transform to obtain the frequency distribution X r (f), which is calculated as follows: Among them, t represents the slow time axis, by analyzing the spectrum X r (f) amplitude distribution, determine the main frequency f of the current signal c ; 4.2: According to the frequency range of the signal's main frequency, the window function length in STFT is adaptively adjusted; if the signal's main frequency f c Below the preset frequency domain threshold f th When using the long window function w long (t); otherwise, the short window function w is used short (t), use the selected window function to analyze the signal x r (t) performs STFT operation, which is calculated as follows: Among them, S r (t,ω) is the adaptive time-frequency distribution, ω(t) is the selected window function, and ω is the angular frequency; 4.3: Further analysis of the STFT results S r (t,ω) is processed and its phase derivative with respect to time is calculated to obtain an estimate of the instantaneous frequency The calculation formula is as follows: where arg(S r (t,ω)) is the phase value of the STFT calculation result. According to the estimated instantaneous frequency, the energy of STFT|S r (t,ω)| 2 Reassigned to the estimated instantaneous frequency position to form a synchronous compressed time-frequency representation T r (t,ω), the redistribution process can be expressed as: Where δ(·) is the Dirac function. The energy is redistributed through the synchronized compression transform (SST) to obtain the compressed time-frequency representation, thereby enhancing the micro-Doppler feature expression of the target. Step 5: Steps 3 and 4 constitute a processing flow for a single frame of radar signal. This processing flow is repeated for all collected radar data frames, and the adaptive synchronous compressed time-frequency representation is extracted frame by frame. The time-frequency representations of each frame are sequentially spliced ​​in time order to generate the Doppler feature enhanced time-frequency map EDTM; Step 6: Input EDTM into the cascaded CNN-LSTM network and train the model to recognize sleep behavior; Step 7: Use the trained model to identify five sleep behaviors in the validation set, including sitting up, lying down, turning over, waving hands, and falling out of bed.

2. The millimeter wave radar sleep behavior recognition method based on adaptive synchronous compression transform according to claim 1 is characterized in that: The millimeter-wave radar initialization in step 1 includes setting the transmission frequency, bandwidth, sampling rate, and number of transceiver antennas; collecting intermediate frequency signals of five sleep behaviors, namely, sitting up, lying down, turning over, waving hands, and falling out of bed, directly below the radar, and ensuring that the acquisition time for each action is sufficient to cover the entire action cycle to ensure signal integrity.

3. The millimeter wave radar sleep behavior recognition method based on adaptive synchronous compression transform according to claim 1 is characterized in that: In step 2, when processing the collected intermediate frequency signal, a sliding window method is first used to calculate the local background average value within the previous fixed window for each chirp signal Chirp, and the local background average value is subtracted from the current Chirp to effectively suppress static interference. After completing the background suppression, the filtered intermediate frequency signal is subjected to a one-dimensional Fourier transform along the fast time axis to convert the time domain signal into the frequency domain, thereby generating a range image.

4. The millimeter wave radar sleep behavior recognition method based on adaptive synchronous compression transform according to claim 1 is characterized in that: In step 3, the entire spectrum of the range profile of the current radar data frame is used as a reference unit, without a protection unit, to prevent the target signal from being filtered by the protection unit. The reference unit spectrum is averaged to obtain a global background noise energy estimate, which is then multiplied by a scaling factor α to obtain a global detection threshold. The detection threshold is used to determine whether the range profile contains a target: if there is a range cell exceeding the threshold, it indicates that there is a target, and the range cell where the target is located is determined by searching for the maximum position of the range profile.

5. The method for millimeter-wave radar sleep behavior recognition based on adaptive synchronous compression transform according to claim 1, characterized in that: In step 6, the Doppler time map with Doppler feature enhancement is input into the cascaded CNN-LSTM network for behavior recognition training. The process is as follows: EDTM is obtained as a data set, divided into a training set and a test set, and the training set is input into the cascaded CNN-LSTM network for training; wherein the 2D-CNN module consists of three alternating 2D convolutional layers and pooling layers, the kernel size of the convolutional layer is 3, the pooling layer adopts maximum pooling, the kernel size is 2, and the stride is 2, and a ReLU activation function is configured after the convolutional layer to accelerate convergence, and a Dropout layer is introduced to prevent overfitting; subsequently, the spatial features extracted by the 2D-CNN are converted into a one-dimensional vector sequence through a flattening operation; the LSTM module consists of a two-layer LSTM network, the number of hidden units in each layer is 64, and a Dropout layer is introduced to enhance generalization ability to capture the temporal correlation between spatial features; finally, the high-level spatiotemporal features output by the LSTM are input into the classifier.

Images