A method for detecting human falls based on millimeter-wave radar

Abstract

The present invention provides a method for detecting human falls based on millimeter-wave radar. This method processes the echo signal of the millimeter-wave radar to obtain the height change parameter ΔH, aspect ratio Q, and Doppler spectrogram of the moving target in the area to be measured. It detects emergencies through the Doppler spectrogram. Once an emergency occurs, it acquires the spectrogram around the emergency, denoises the spectrogram, extracts three features: the extreme frequency amplitude, extreme frequency ratio, and event length of the spectrogram, and jointly inputs two features, the current height change parameter ΔH and aspect ratio Q, into a neural network for training and learning, and then detects falls. It has the advantages of a relatively wide coverage range, low code complexity, fast calculation speed, and low false positive and false negative rates for falls.

Description

Technical Field

[0001] The present invention relates to the field of radar detection, and more specifically, to a method for detecting human falls based on millimeter-wave radar. Background Art

[0002] Falling while moving at home is the most common accident faced by the elderly. Approximately 10%-20% of elderly falls will result in serious injuries such as fractures, head traumas, or even death. For some elderly people, due to the lack of timely rescue after falling, the injuries develop and lead to serious consequences. Detecting elderly falls by installing video surveillance or radar at home is a current research and development direction.

[0003] Currently, there are methods for using the signal energy magnitude, motion posture data, and time-Doppler heat maps detected by radar for convolutional neural network learning to determine falls. However, the features of the above methods are relatively single, and the probability of false judgment or missed judgment is relatively large. The main manifestations are: the problem of low precision and recall rate of detecting human body targets in different postures cannot be effectively solved due to the diversity of human postures and the interference of false targets; the problem of difficult distinction between falls and fall-like behaviors due to the complexity of human behavior actions, resulting in false judgments cannot be solved. Summary of the Invention

[0004] The present invention overcomes the deficiencies of the prior art and provides a method for detecting human falls based on millimeter-wave radar, in the hope of solving the problem of easy false judgment or missed judgment of fall detection in the prior art.

[0005] To solve the above technical problems, an embodiment of the present invention adopts the following technical solutions:

[0006] A method for detecting human falls based on millimeter-wave radar, comprising the following steps:

[0007] Transmit an electromagnetic wave signal through a millimeter-wave radar to a space range to be measured, receive the echo signal, and process the echo signal;

[0008] The processing of the echo signal includes extracting the target, the radial velocity, distance, and angle of the target trace through Fourier transform to obtain the target trace;

[0009] Use a minimum cube to enclose all the traces, vertically cut the minimum cube with a horizontal plane facing the radar to form a minimum rectangle, and calculate the aspect ratio Q of the minimum rectangle, that is, the aspect ratio Q of the target point cloud;

[0010] Record the height change of the center of the minimum rectangle, and calculate the height change amount ΔH every n frames, where n is taken as 5 to 100;

[0011] ΔH = M1 - M2

[0012] M1 is the average value of the height data of the centers of the minimum rectangles of the current m-th frame and the n - 1 frames before the m-th frame;

[0013] M2 is the average value of the height data of the centers of the minimum rectangles of the m - n-th frame and the n - 1 frames before the m - n-th frame;

[0014] For the distance bin where the target is located, by using the joint time-frequency analysis method, the time-varying Doppler frequency at different times is obtained, that is, the Doppler spectrum;

[0015] An emergency is detected through the energy distribution of the Doppler spectrum, and the Doppler spectrograms for 1 second to 5 seconds before and after the occurrence of the emergency are extracted to obtain a spectrogram with a size of 2 seconds to 10 seconds;

[0016] The detected emergency time is usually considered a fall, that is, the position corresponding to the peak;

[0017] The obtained spectrogram is subjected to image segmentation and morphological processing, and three features, namely the extreme frequency amplitude, the extreme frequency ratio, and the event length, are extracted from the processed spectrogram;

[0018] The extreme frequency amplitude, the extreme frequency ratio, the event length, ΔH, and the aspect ratio Q are input into a convolutional neural network for training and learning, and then a fall is detected.

[0019] For a fall, due to the translational movement of the whole body, the high-energy spectrogram is concentrated on the positive or negative frequency, resulting in a relatively high extreme frequency ratio. For other types of movements, such as sitting and standing, they often show a relatively high energy content in both the positive and negative frequency bands because different body parts show different movement patterns, so they correspond to a relatively low extreme frequency ratio. Therefore, the extreme frequency ratio can be used to distinguish a fall from other movements and achieve a better recognition effect.

[0020] A further technical solution is that the millimeter-wave radar uses a 66GHz millimeter-wave broadband radar.

[0021] A further technical solution is that in the described method for detecting human falls based on a millimeter-wave radar, the range of the space to be measured is 4m * 4m.

[0022] A further technical solution is that the echo signal processing further includes the following steps: first, using the position of the traces, then using the trace signal-to-noise ratio information to find the peak point of the signal-to-noise ratio, and then clustering the traces. During the clustering process, some noise points with low signal-to-noise ratios are deleted.

[0023] Through the above processing, the purpose of excluding the interference of some indoor objects can be achieved.

[0024] A further technical solution is that the change amount ΔH of the height is calculated for every n frames, and n is taken as 10.

[0025] A further technical solution is that the Doppler spectrograms 1 second to 5 seconds before and after the occurrence of the emergency are extracted. Specifically, the Doppler spectrograms 2.5 seconds before and after the peak position are taken, and a spectrogram with a size of 5 seconds is obtained.

[0026] A further technical solution is that the method for detecting an emergency by the energy distribution of the Doppler frequency spectrum specifically includes the following steps: by calculating the sum of the squares of the Doppler values in the range of 80 to 100 Hz and the corresponding negative frequency range at each time to form an energy curve, when the peak value appearing in the energy curve exceeds a certain threshold, it is considered that there is an emergency at the peak value.

[0027] A further technical solution is that the image segmentation is threshold segmentation, that is, a suitable threshold is determined to segment the gray time-frequency representation into a target event and a background noise region. The main purpose is to denoise and filter out the interference of other objects again;

[0028] The morphological processing is to bridge the broken segments located in the close positions. The spectrogram may contain weak components, especially near the extreme frequencies, which may generate broken segments after the spectrogram segmentation. The morphological processing can bridge the broken segments located in the close positions. On the other hand, removing the disconnected regions only retains the significant time-frequency regions representing the target activity.

[0029] A further technical solution is that after the fall detection is completed, the target traces are condensed into different regions, and the condensed trace regions are matched to the known tracks according to conditions to obtain the motion track of the target, and the motion state of the personnel in the target region is tracked to determine whether the personnel has fallen.

[0030] By tracking the motion state of the personnel in the target region, it is possible to exclude fall-like actions such as walking normally while bending down, bending down while walking normally, squatting while walking normally, suddenly getting up from a seat, and bending down while walking, reducing false alarms. If it is still judged as a fall, then it is confirmed as a fall.

[0031] Compared with the prior art, the present invention has at least the following beneficial effects: The present invention comprehensively combines the time domain, frequency domain, spatial domain, and Doppler characteristics, provides a comprehensive summary of the fall event behavior, proposes to use the extreme frequency amplitude, extreme frequency ratio, event length, change trend of height, and aspect ratio as features for machine learning, and uses the motion state of the personnel in the target region to reconfirm the fall, greatly reducing the probability of false judgment and missed judgment of the fall. At the same time, the human fall detection method provided by the present invention has the advantages of a wide coverage range, being able to conveniently and accurately monitor the activities of the elderly living alone or patients in the hospital, low code complexity, fast calculation speed, high detection accuracy, etc., and can adapt to many scenarios. Description of the Drawings

[0032] Figure 1 This is a schematic flowchart of an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of point cloud clustering.

[0034] Figure 3 This is a schematic diagram for obtaining the center, width, and height of the point cloud.

[0035] Figure 4 This is a flowchart of the signal processing algorithm and convolutional neural training detection. Specific embodiments

[0036] In order to make the objectives, technical solutions, and advantages of the present invention more clear and understandable, the present invention will be further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] As Figure 1 、 Figure 2 shown, a method for detecting human falls based on a millimeter-wave radar includes the following steps:

[0038] S1. Transmit an electromagnetic wave signal through a millimeter-wave radar to a space to be measured, receive the echo signal, and process the echo signal;

[0039] The millimeter-wave radar uses a 66 GHz millimeter-wave broadband radar, and the space to be measured is 4m * 4m;

[0040] The processing of the echo signal includes the following steps: performing a Fourier transform on the received signal to extract the target, using a CFAR detector to roughly measure the radial velocity, distance, and angle of the target trace, and first using the position of the trace to obtain the target trace;

[0041] S2. Use the signal-to-noise ratio information of the trace to find the peak point of the signal-to-noise ratio, and then cluster the traces. During the clustering process, some noise points with low signal-to-noise ratios are deleted, as Figure 3 shown, so as to achieve the purpose of excluding interference from some indoor objects;

[0042] S3. Use a minimum cube to enclose all the traces, vertically cut the minimum cube with a horizontal plane facing the radar to form a minimum rectangle, and calculate the aspect ratio Q of the minimum rectangle, that is, the aspect ratio Q of the target point cloud;

[0043] As Figure 4As shown in the figure, 01 is the position of the radar, 02 is the smallest cube that can enclose the target point cloud, 03 is the smallest matrix that can contain the target point cloud, and this matrix is the vertical section facing the radar. 04 is the center of the smallest rectangle facing the radar, 05 is the length of the smallest matrix in the Z direction, that is, the height of the smallest rectangle, and 06 is the length of the smallest matrix in the X direction, which is the width of the smallest rectangle.

[0044] S4. Record the height change of the center 04 of the smallest rectangle, and calculate the change amount ΔH of the height every n frames. n can take values from 5 to 100. In this embodiment, n is 10;

[0045] ΔH = M1 - M2

[0046] M1 is the average value of the height data of the center of the smallest rectangle in the current 30th frame and the 9 frames before the 30th frame;

[0047] M2 is the average value of the height data of the center of the smallest rectangle in the 20th frame and the 9 frames before the 20th frame;

[0048] S5. For the distance bin where the target is located, obtain the time-varying Doppler frequency at different times, that is, the Doppler spectrum, through the joint time-frequency analysis method.

[0049] The joint time-frequency analysis method is the short-time Fourier transform to obtain the time-varying Doppler frequency at different times;

[0050] S6. Detect emergencies through the energy distribution of the Doppler spectrum, calculate the distribution curve of the Doppler energy in a specific frequency range, and when a certain peak appears and this peak exceeds a certain threshold, it is considered that there is an emergency at this peak;

[0051] The calculation of the Doppler energy curve is to calculate the sum of the squares of the Doppler values in the range of 80 - 100 Hz and the corresponding negative frequency range at each time to form an energy curve. When the peak appearing in the energy curve exceeds a certain threshold, it is considered that there is an emergency at this peak;

[0052] S7. The detected emergency time is usually considered to be a fall, that is, the position corresponding to the peak. Obtain the Doppler spectrogram for 2.5 seconds before and after the peak position, and obtain a spectrogram with a size of 5 seconds;

[0053] S8. Perform image segmentation and morphological processing on the obtained spectrogram;

[0054] The image segmentation is through appropriate threshold segmentation. Determine a suitable threshold to segment the gray-scale time-frequency representation into the target event and background noise regions; the morphological processing is to bridge the broken segments located in the close positions together.

[0055] Extract three features: extreme frequency amplitude, extreme frequency ratio, and event length from the processed spectrogram;

[0056] When compared to other types of observed motion, critical falls often exhibit a high extreme frequency amplitude, which is the difference between the maximum frequency of the positive frequency range and the minimum frequency of the negative frequency range.

[0057] For falls, due to the translational movement of the whole body, the high-energy spectra are concentrated on the positive or negative frequencies, resulting in a higher extreme frequency ratio. On the other hand, other types of movements, such as sitting and standing, tend to show higher energy content in both positive and negative frequency bands because different body parts show different movement patterns, and therefore correspond to a lower extreme frequency ratio. The extreme frequency ratio is the larger value of the ratio of the maximum frequency in the positive frequency range to the minimum frequency in the negative frequency range and the ratio of the minimum frequency in the negative frequency range to the maximum frequency in the positive frequency range.

[0058] Different movement patterns were compared, usually showing different time spans, where the event length is the difference between the time when the extreme frequency occurs and the time when the event starts.

[0059] S9. The extreme frequency amplitude, extreme frequency ratio, event length, ΔH and aspect ratio Q are input into the convolutional neural network for training and learning, and then the fall is detected.

[0060] like Figure 2 As shown, the neural network is a convolutional neural network with 5 layers, including 3 convolutional layers and two fully connected layers.

[0061] S10, condensing the target point traces into different areas, matching the condensed point trace areas to the known tracks according to conditions, obtaining the motion trajectory of the target, tracking the motion status of the personnel in the target area, and determining whether the personnel have fallen.

[0062] By tracking the movement status of people in the target area, falling actions such as bending over can be excluded to reduce false alarms. If it is still judged as a fall, it will be confirmed as a fall.

[0063] The present invention processes the echo signal of the millimeter-wave radar to obtain the height change parameter ΔH, the aspect ratio Q and the Doppler spectrum of the moving target in the test area, detects emergencies through the Doppler spectrum, and once an emergencies occurs, obtains the spectrum around the emergencies, denoises the spectrum, extracts the three features of the extreme frequency amplitude, the extreme frequency ratio and the event length of the spectrum, and combines the two features of the current height change parameter ΔH and the aspect ratio Q to input into the neural network for training and learning, and then detects falls.

[0064] Although the present invention has been described herein with reference to illustrative embodiments thereof, it should be understood that those skilled in the art can devise many other modifications and embodiments that will fall within the scope and spirit of the principles disclosed in this application. More specifically, within the scope of the disclosure of this application, various variations and improvements can be made to the components and / or layout of the subject combination layout. In addition to the variations and improvements to the components and / or layout, other uses will also be apparent to those skilled in the art.

Claims

1. A method for detecting human falls based on millimeter-wave radar, characterized in that, The steps include: Transmit electromagnetic wave signals to the space range to be measured through a millimeter-wave radar, receive the echo signals, and process the echo signals. The processing of the echo signals includes extracting the target, the radial velocity, distance, and angle of the target trace through Fourier transform to obtain the target trace. Use a minimum cube to enclose all the traces, vertically cut the minimum cube with a horizontal plane facing the radar to form a minimum rectangle, and calculate the aspect ratio Q of the minimum rectangle, that is, the aspect ratio Q of the target point cloud. Record the change in the height of the center of the minimum rectangle, and calculate the change amount ΔH of the height every n frames, where n ranges from 5 to 100. ΔH = M1 - M2 M1 is the average value of the height data of the center of the minimum rectangle in the current m-th frame and the n - 1 frames before the m-th frame. M2 is the average value of the height data of the center of the minimum rectangle in the (m - n)-th frame and the n - 1 frames before the (m - n)-th frame. For the distance bin where the target is located, obtain the time-varying Doppler frequency at different times, that is, the Doppler spectrum, through the joint time-frequency analysis method. Detect emergencies through the energy distribution of the Doppler spectrum, extract the Doppler spectrograms for 1 second to 5 seconds before and after the occurrence of the emergency, and obtain a spectrogram with a size of 2 seconds to 10 seconds. Perform image segmentation and morphological processing on the obtained spectrogram, and extract three features: extreme frequency amplitude, extreme frequency ratio, and event length from the processed spectrogram. Input the extreme frequency amplitude, extreme frequency ratio, event length, ΔH, and aspect ratio Q into a convolutional neural network for training and learning, and then detect falls.

2. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, The millimeter-wave radar uses a 66GHz millimeter-wave broadband radar.

3. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, The space range to be measured is 4m * 4m.

4. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, The processing of the echo signals further includes the following steps: first, use the position of the trace, then use the signal-to-noise ratio information of the trace to find the peak point of the signal-to-noise ratio, and then cluster the traces. During the clustering process, some noise points with low signal-to-noise ratios are deleted.

5. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, For calculating the change amount ΔH of the height every n frames, n is taken as 10.

6. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, The extraction of the Doppler spectrograms for 1 second to 5 seconds before and after the occurrence of the emergency specifically means: take the Doppler spectrograms for 2.5 seconds before and after the peak position to obtain a spectrogram with a size of 5 seconds.

7. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, The method for detecting emergencies through the energy distribution of the Doppler spectrum specifically includes the following steps: calculate the sum of the squares of the Doppler values in the range of 80 - 100Hz and the corresponding negative frequency range at each time to form an energy curve. When the peak value that appears in the energy curve exceeds a certain threshold, it is considered that there is an emergency at the peak position.

8. The method for detecting human falls based on millimeter-wave radar according to claim 1, characterized in that, The image segmentation is threshold segmentation. For the morphological processing, the broken segments located in the close positions are bridged together.

9. The method for detecting human falls based on millimeter-wave radar according to any one of claims 1-8, characterized in that, After completing the fall detection, condense the target traces into different regions, match the condensed trace regions to the known tracks according to the conditions to obtain the movement track of the target, and track the movement state of the personnel in the target region to determine whether the personnel has fallen.

Images