A fall detection method based on millimeter wave multi-dimensional features and multiple filtering

Abstract

The present invention discloses a fall detection method based on millimeter wave multi-dimensional features and multiple filtering, which uses multi-antenna millimeter wave radar signals to obtain multi-dimensional features such as distance, speed, SNR, vertical angle and phase of human targets, thereby obtaining a high-quality multi-dimensional feature matrix of the fall action, and eliminates false alarm events in fall detection through multiple filtering measures such as pre-filtering, fall detection network judgment, smoothing filtering and post-filtering. The present invention effectively reduces the false alarm rate of fall events. Actual tests show that the correct detection rate of the system is above 95%, and the false alarm rate of the system is below 5%.

Description

Technical Field

[0001] The present invention relates to a fall detection method, in particular to a fall detection method based on millimeter wave multi-dimensional features and multiple filtering. Background Art

[0002] Existing fall detection systems mainly include detection systems based on accelerometers and cameras and detection systems based on millimeter-wave radars.

[0003] The detection system based on accelerometer and camera is that when a person falls, the accelerometer detects that the acceleration of the human body increases sharply, and at the same time, the posture of the human body becomes horizontal or inclined, and the vertical distance between the head joint and the center of the body will also decrease. The specific technical solution is to extract the position of the head joint and the center of the human body in sequence, extract the acceleration of the human body, monitor the vertical distance from the head joint to the center of the human body and the acceleration of the human body, analyze the falling behavior, and judge whether it has recovered from the fall. If it is, there is no need to alarm, otherwise it will alarm. That is, when the accelerometer detects that the human body has generated a large acceleration, and the camera observes that the vertical distance between the head joint and the center of the human body decreases, the system will detect that a fall event has occurred; if the vertical distance between the head joint and the center of the human body begins to increase again within a certain period of time, it means that the person has recovered from the fall state, and there is no need to alarm, otherwise it is considered that the target may have a dangerous fall and the alarm is activated. The shortcomings of the detection system are that the false alarm rate is high in actual application, and the accelerometer must be worn on the human body, which is not convenient for daily use; the camera performs poorly in poor light or in the presence of obstacles, and there is also the problem of leaking privacy in daily life.

[0004] The millimeter-wave radar-based detection system sets up multiple radars to detect falls. There is no need for the human body to wear devices or set up cameras. Compared with the detection system based on accelerometers and cameras, it is not only convenient for daily monitoring but also can avoid privacy leakage, but there is still a problem of high false alarm rate. For example, the Chinese invention patent application with patent application number 202111313188.1 discloses a "fall detection method and system based on millimeter-wave radar and machine learning", which realizes fall detection by extracting multiple features such as Doppler frequency from the radar echo signal and combining it with a machine learning model. The Chinese invention patent application with patent application number 202111354517.7 discloses a "multi-channel ultra-wideband radar bathroom fall detection device and method", which first uses the acceleration information in the echo signal to determine whether a human target exists, and then calculates the discriminant vector to achieve fall detection. The Chinese invention patent application with patent application number 202111256842.X discloses "a fall detection method and radar device", which first extracts features such as speed and distance from the radar signal, and then determines whether the human body falls in the horizontal and vertical directions to achieve fall detection. The above fall detection methods do not fully utilize the multi-dimensional features in the radar signal, and do not perform sufficient pre-filtering and post-filtering processing on the fall detection process. Therefore, the false alarm rate for multiple specific actions is greater than 30%. Summary of the invention

[0005] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a fall detection method based on millimeter wave multi-dimensional features and multiple filtering.

[0006] The technical solution of the present invention is: a fall detection method based on millimeter wave multi-dimensional features and multiple filtering, characterized by following the following steps in sequence:

[0007] Step 1. Obtain millimeter wave radar measurement information of multiple antennas in a continuous T1 time period;

[0008] Step 2. Calculate the distance, speed, SNR, vertical angle and phase of the target point at the current frame time based on the millimeter wave radar measurement information of each antenna;

[0009] Step 3. Fuse the features of multiple target points to obtain the final distance, speed, SNR, vertical angle and phase, that is, obtain the final feature vector at each moment in the T1 time period of each frame;

[0010] Step 4. Based on the final feature vector of each frame in the T1 time period, form M multidimensional feature matrices T1 k , the k is the multidimensional feature matrix T1 k The ordinal number, k = 1, 2, 3, ... M;

[0011] Step 5. Assume that the result of each fall judgment is Rn, where n is the ordinal number of the judgment result, n=1, 2, ..., the number of consecutive fall judgments is m, where m is less than or equal to M, the fall event judgment threshold is Q, where Q is less than m, and k←1, n←1, and then perform the following operations:

[0012] Step 5.1 Calculate the multidimensional feature matrix T1 k The mean and variance of the speed and SNR are determined to determine whether the mean and variance are both less than the threshold. If yes, it is determined to be empty scene data or human static scene data, and the non-fall state is recorded as the fall judgment result Rn, and step 5.5 is performed; if no, step 5.2 is performed;

[0013] Step 5.2. Multidimensional feature matrix T1 k Send it to the fall detection network to classify and judge the fall or non-fall state, and record the classification result as the fall judgment result Rn. Determine whether n is equal to m. If yes, proceed to step 5.3; otherwise, proceed to step 5.5.

[0014] Step 5.3 determines whether the number of fall events in the m fall judgment results is greater than or equal to Q times. If yes, proceed to step 5.4; if no, proceed to step 5.5;

[0015] Step 5.4 obtains the millimeter-wave radar measurement information of multiple antennas in the T2 time period, calculates the distance, speed, SNR, vertical angle and phase of the target point at the current moment; fuses the features of multiple target points to obtain the final distance, speed, SNR, vertical angle and phase, that is, obtains the final feature vector at each moment in the T2 time period; based on the final feature vector at each moment in the T2 time period, forms a multidimensional feature matrix T2S; calculates the mean and variance of the speed and SNR in the multidimensional feature matrix T2S, and determines whether the mean and variance are both less than the threshold. If yes, it is determined to be human static scene data, triggering the fall alarm state, otherwise, proceed to step 5.5;

[0016] Step 5.5: Determine whether k is less than M. If yes, proceed to step 5.6; if no, end;

[0017] Step 5.6 determines whether the difference between M and k is greater than or equal to m. If it is k←k+1, n←1, return to step 5.1. If not, replace the multidimensional feature matrix T1 k+1 —T1 M Save, end.

[0018] The step 2 is performed as follows: first, frequency domain analysis is performed on the multi-antenna radar measurement information, and the frequency domain data of the multi-antennas are arranged according to Range-Antenna-Chirp to obtain a Data Cube; then, a Music beamforming algorithm is used to perform Music beamforming in the vertical direction on the Data Cube to obtain the frequency domain information spectrum Range-Elevation Spectrum of the target at different distances and different vertical angles; then a 2D-CFAR algorithm is performed on the Range-Elevation Spectrum to obtain potential target points and the distance, vertical angle, and SNR information corresponding to each target point; the phase information of the point with the strongest SNR is obtained as the final phase information; then beamforming is performed on the Data Cube along the direction of the target point, and finally a frequency domain analysis based on fast Fourier transform is performed to obtain the speed of each target point.

[0019] The step 3 is performed as follows:

[0020] Step 3.1 Determine the weight α of each target point based on the SNR size of each target point i :

[0021]

[0022] Among them, J represents the number of detected target points;

[0023] Step 3.2: Based on the weight information, the features of fused multi-target points are obtained:

[0024]

[0025] Among them, r, v and ele represent the final distance, speed and vertical angle respectively. j 、v j andele j They represent the distance, speed, and vertical angle of each target point respectively; the final SNR is represented by the average SNR of multiple target points, and the final phase information adopts the phase information of the point with the strongest SNR.

[0026] The present invention uses multi-antenna millimeter wave radar signals to obtain multi-dimensional features such as distance, speed, SNR, vertical angle and phase of human targets, thereby obtaining a high-quality multi-dimensional feature matrix of the fall action, and eliminating false alarm events in fall detection through multiple filtering measures. The specific filtering measures are to first perform pre-filtering processing, that is, to judge whether the current scene is a dynamic scene based on the multi-dimensional feature matrix, and only when it is a dynamic scene, the subsequent fall detection network detection is performed; secondly, the fall detection network is used to make a fall judgment, and a smoothing filtering processing is performed on the judgment result, that is, only when multiple consecutive fall results occur, it indicates that a fall event may have occurred; finally, a post-filtering processing is performed, that is, after a fall event is detected, if the fall feature matrix has been human static scene data for a period of time, the fall alarm state is entered, otherwise the fall alarm state is released. The present invention effectively reduces the false alarm rate of fall events. Actual tests show that the correct detection rate of the system is above 95%, and the false alarm rate of the system is below 5%. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of an embodiment of the present invention.

[0028] Figure 2 2 is an antenna arrangement diagram of an embodiment of the present invention.

[0029] Figure 3 It is a schematic diagram of the multi-dimensional feature extraction route.

[0030] Figure 4 It is a schematic diagram of constructing a characteristic matrix according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] A fall detection method based on millimeter wave multi-dimensional features and multiple filtering of the present invention is as follows Figure 1 As shown, follow the steps below:

[0032] Step 1. Obtain millimeter wave radar measurement information of multiple antennas within a continuous T1 time period (2s);

[0033] The millimeter wave radar with multiple antennas in the embodiment of the present invention is a 3-transmit 4-receive FMCW millimeter wave radar fall detection system, and the system configuration is as follows:

[0034] 1. The system operates in the 60GHz frequency band;

[0035] 2. The system has 3 transmitting antennas (TX1-3) and 4 receiving antennas (RX1-4). The antenna array is arranged as follows: Figure 2 As shown;

[0036] 3. The system sampling rate is 2612ksps and the bandwidth is 4GHz;

[0037] 4. The system's distance resolution is 0.076m, speed resolution is 0.079m / s, maximum measurement distance is 4.9m, and maximum measurement speed is 3.8m / s.

[0038] Step 2. Calculate the distance, speed, SNR, vertical angle and phase of the target point at the current frame time based on the millimeter wave radar measurement information of each antenna;

[0039] Specific steps are as follows Figure 3 As shown: Based on the constant false alarm detection method, the sampling signal of the millimeter wave radar is preprocessed by background elimination and the energy strong point detected by the constant false alarm detection method is beamformed to achieve the perception target focusing.

[0040] First, frequency domain analysis is performed on the multi-antenna radar measurement information, and the frequency domain data of the multiple antennas are arranged according to Range-Antenna-Chirp to obtain the Data Cube; then the Music beamforming algorithm is used to perform Music beamforming in the vertical direction on the Data Cube to obtain the frequency domain information spectrum of the target at different distances and different vertical angles, the Range-Elevation Spectrum; then the 2D-CFAR algorithm is performed on the Range-Elevation Spectrum to obtain potential target points and the distance, vertical angle, and SNR information corresponding to each target point; the phase information of the point with the strongest SNR is obtained as the final phase information; then beamforming is performed on the Data Cube along the direction of the target point, and finally a frequency domain analysis based on fast Fourier transform is performed to obtain the speed of each target point.

[0041] Step 3. Fuse the features of multiple target points to obtain the final distance, speed, SNR, vertical angle and phase, that is, obtain the final feature vector of each frame in each T1 time period;

[0042] Step 3.1 Determine the weight α of each target point based on the SNR size of each target point i :

[0043]

[0044] Among them, J represents the number of detected target points;

[0045] Step 3.2: Based on the weight information, the features of fused multi-target points are obtained:

[0046]

[0047] Among them, r, v and ele represent the final distance, speed and vertical angle respectively. j 、v j andele jThey represent the distance, speed, and vertical angle of each target point respectively; the final SNR is represented by the average SNR of multiple target points, and the final phase information adopts the phase information of the point with the strongest SNR.

[0048] Step 4. Figure 4 As shown, based on the final feature vector of each frame in the T1 time period, 20 multi-dimensional feature matrices T1 are formed k , the k is the multidimensional feature matrix T1 k The ordinal number, k = 1, 2, 3, ... 20;

[0049] Step 5. Assume that the result of each fall judgment is Rn, where n is the ordinal number of the judgment result, n=1, 2, ..., the number of consecutive fall judgments is m=9, the fall event judgment threshold is 7, k←1, n←1, and the following operations are performed:

[0050] Step 5.1 Pre-filtering: Calculate the multi-dimensional feature matrix T1 k The mean and variance of the speed and SNR are determined to determine whether the mean and variance are both less than the threshold (the threshold is set to 0.05 m / s for the mean speed, 0.1 m / s for the variance speed, 0.1 for the mean SNR, and 0.1 for the variance SNR). If yes, it is determined to be empty scene data or human static scene data, and the non-falling state is recorded as the falling judgment result Rn, and step 5.5 is performed; if no, step 5.2 is performed;

[0051] Step 5.2. Fall detection network decision: multi-dimensional feature matrix T1 k Send it to the fall detection network to classify and judge the fall or non-fall state, and record the classification result as the fall judgment result Rn. Determine whether n is equal to m. If yes, proceed to step 5.3; otherwise, proceed to step 5.5.

[0052] The fall detection network of the present invention adopts a 3-layer convolution and fully connected hybrid network, the first two layers are convolution layers, and the third layer is a fully connected layer. The convolution kernel of the first convolution layer is 5×5, and the number of convolution kernels is 16; the convolution kernel of the second convolution layer is 3×3, and the number of convolution kernels is 8; the input of the fully connected layer is the flattened vector of the second convolution output, and the output is a 1×2 state vector; the two states correspond to the probabilities of falling and non-falling states, respectively. If the probability value of the falling state is higher than the probability of the non-falling state, the current judgment is a falling state, otherwise it is a non-falling state. The data set of the fall detection network consists of self-collected fall samples and non-fall samples, and the data set is allocated in a ratio of 7:3, 70% for training and 30% for testing.

[0053] The fall detection network first performs a convolution operation on the multi-dimensional feature matrix of the input dynamic scene to obtain higher-dimensional features. After two-level convolution operations, the fully connected network completes the mapping of high-dimensional features to low-dimensional features to achieve state classification.

[0054] Step 5.3 smoothing filter: determine whether the number of fall events in the m fall judgment results is greater than or equal to Q times, if yes, proceed to step 5.4, if not, proceed to step 5.5;

[0055] Step 5.4 post-filtering stage: obtain the millimeter-wave radar measurement information of multiple antennas within the T2 time period (10s), and calculate the distance, speed, SNR, vertical angle and phase of the target point at the current moment according to the method of the above steps 2-4; fuse the features of multiple target points to obtain the final distance, speed, SNR, vertical angle and phase, that is, obtain the final feature vector at each moment in the T2 time period; based on the final feature vector at each moment in the T2 time period, form a multidimensional feature matrix T2S; calculate the mean and variance of the speed and SNR in the multidimensional feature matrix T2S, and determine whether the mean and variance are both less than the threshold (the threshold is set to a rate mean of 0.05m / s, a rate variance of 0.1m / s, an SNR mean of 0.1, and an SNR variance of 0.1). If yes, it is determined to be human static scene data, that is, the target has not gotten up since falling, and a dangerous fall event has occurred to the target, triggering a fall alarm state; otherwise, proceed to step 5.5;

[0056] Step 5.5: Determine whether k is less than M. If yes, proceed to step 5.6; if no, end;

[0057] Step 5.6 determines whether the difference between M and k is greater than or equal to m. If it is k←k+1, n←1, return to step 5.1. If not, replace the multidimensional feature matrix T1 k+1 —T1 M Store as automatic replacement for next period's data and end.

Claims

1. A fall detection method based on millimeter wave multi-dimensional features and multiple filtering, characterized in that Follow the steps below: Step 1. Obtain millimeter wave radar measurement information of multiple antennas in a continuous T1 time period; Step 2. Calculate the distance, speed, SNR, vertical angle and phase of the target point at the current frame time based on the millimeter wave radar measurement information of each antenna; Step 3. Fuse the features of multiple target points to obtain the final distance, speed, SNR, vertical angle and phase, that is, obtain the final feature vector of each frame in each T1 time period; Step 4. Based on the final feature vector of each frame in the T1 time period, form M multidimensional feature matrices T1 k , the k is the multidimensional feature matrix T1 k The ordinal number, k = 1, 2, 3, ... M; Step 5. Assume that the result of each fall judgment is Rn, where n is the ordinal number of the judgment result, n=1, 2, ..., the number of consecutive fall judgments is m, where m is less than or equal to M, the fall event judgment threshold is Q, where Q is less than m, and k←1, n←1, and then perform the following operations: Step 5.1 Calculate the multidimensional feature matrix T1 k The mean and variance of the speed and SNR are determined to determine whether the mean and variance are both less than the threshold. If yes, it is determined to be empty scene data or human static scene data, and the non-fall state is recorded as the fall judgment result Rn, and step 5.5 is performed; if no, step 5.2 is performed; Step 5.

2. Multidimensional feature matrix T1 k Send it to the fall detection network to classify and judge the fall or non-fall state, and record the classification result as the fall judgment result Rn. Determine whether n is equal to m. If yes, proceed to step 5.3; otherwise, proceed to step 5.

5. Step 5.3 determines whether the number of fall events in the m fall judgment results is greater than or equal to Q times. If yes, proceed to step 5.4; if no, proceed to step 5.5; Step 5.4 obtains the millimeter-wave radar measurement information of multiple antennas in the T2 time period, calculates the distance, speed, SNR, vertical angle and phase of the target point at the current moment; fuses the features of multiple target points to obtain the final distance, speed, SNR, vertical angle and phase, that is, obtains the final feature vector at each moment in the T2 time period; based on the final feature vector at each moment in the T2 time period, forms a multi-dimensional feature matrix T2S; Calculate the mean and variance of the speed and SNR in the multidimensional feature matrix T2S, and determine whether the mean and variance are both less than the threshold. If yes, it is determined to be human static scene data and trigger the fall alarm state. If no, proceed to step 5.5; Step 5.5: Determine whether k is less than M. If yes, proceed to step 5.6; if no, end; Step 5.6 determines whether the difference between M and k is greater than or equal to m. If it is k←k+1, n←1, return to step 5.

1. If not, replace the multidimensional feature matrix T1 k+1 —T1 M Save, end.

2. The fall detection method based on millimeter wave multi-dimensional features and multiple filtering according to claim 1 is characterized in that The step 2 is performed as follows: first, frequency domain analysis is performed on the multi-antenna radar measurement information, and the frequency domain data of the multi-antennas are arranged according to Range-Antenna-Chirp to obtain a Data Cube; then, a Music beamforming algorithm is used to perform Music beamforming in the vertical direction on the Data Cube to obtain the frequency domain information spectrum Range-Elevation Spectrum of the target at different distances and different vertical angles; then a 2D-CFAR algorithm is performed on the Range-Elevation Spectrum to obtain potential target points and the distance, vertical angle, and SNR information corresponding to each target point; the phase information of the point with the strongest SNR is obtained as the final phase information; then beamforming is performed on the Data Cube along the direction of the target point, and finally a frequency domain analysis based on fast Fourier transform is performed to obtain the speed of each target point.

3. The fall detection method based on millimeter wave multi-dimensional features and multiple filtering according to claim 2 is characterized in that The step 3 is performed as follows: Step 3.1 Determine the weight α of each target point based on the SNR size of each target point i : Among them, J represents the number of detected target points; Step 3.2: Based on the weight information, the features of fused multi-target points are obtained: Among them, r, v and ele represent the final distance, speed and vertical angle respectively. j 、v j andele j They represent the distance, speed, and vertical angle of each target point respectively; the final SNR is represented by the average SNR of multiple target points, and the final phase information adopts the phase information of the point with the strongest SNR.

Images