Home elevator person falling detection method and system based on 4D millimeter wave radar

Abstract

The invention relates to the technical field of elevator monitoring, and discloses a home elevator personnel falling detection method and system based on a 4D millimeter wave radar, and the method comprises the steps: obtaining a radar echo signal of any target object in an elevator in real time; determining static point cloud and dynamic point cloud information of the target object; calculating the mean value of all the dynamic point clouds of the target object, forming a target tracking trajectory according to the dynamic point clouds, and selecting the dynamic point clouds larger than the height value of the target object to solve the mean value; when the mean values of the two dynamic point clouds are continuously monotonically decreased within a preset range and the minimum value of the mean values of the two dynamic point clouds is smaller than a first threshold value, calculating the height mean value of the static point clouds of the target object; when the height mean value of the static point cloud of the target object is less than a second threshold for a long time, judging that the target object falls down; and when it is judged that the target object falls down, the radar sends a person trapping signal to the elevator master controller. The detection precision of the falling behavior of people in the elevator is improved, and better environmental adaptability is achieved.

Description

Technical Field

[0001] The present application relates to the field of elevator monitoring technology, and in particular to a method and system for detecting falls in home elevators based on 4D millimeter wave radar. Background Art

[0002] With the arrival of an aging society, especially with the increasing number of elderly people living alone and equipped with home elevators, once a fall or being trapped in the elevator occurs, the consequences are disastrous. Therefore, the safety of home elevators has become increasingly important.

[0003] The elevator safety monitoring system in the existing technology uses cameras, pressure sensors and other monitoring methods. In low light or complex environments, the camera's image capture effect is poor, the environmental adaptability is poor, and the camera's image monitoring has the risk of privacy leakage, which easily causes user concerns. Pressure sensors also have difficulty accurately distinguishing between falls and other behaviors, and the false alarm rate is high. They often cannot identify falls in time and deal with them effectively.

[0004] With respect to the above-mentioned related technologies, the inventors have found that the existing elevator safety monitoring system has a high false alarm rate and is unable to identify falls in a timely manner. Summary of the invention

[0005] In order to improve the detection accuracy of falling behavior and identify falling behavior in time, the present application provides a home elevator personnel fall detection method and system based on 4D millimeter wave radar.

[0006] In a first aspect, the present application provides a method for detecting falls in a home elevator based on a 4D millimeter-wave radar.

[0007] This application is achieved through the following technical solutions:

[0008] A method for detecting falls of people in a home elevator based on 4D millimeter wave radar comprises the following steps:

[0009] Using MIMO technology, the radar echo signal from any target object in the elevator is acquired in real time through multiple antenna channels;

[0010] Based on the radar echo signal, determine the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object;

[0011] Calculate the mean of the dynamic point clouds of all radar echo signals through the three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object, and at the same time, form a target tracking trajectory according to the dynamic point cloud, and select the dynamic point cloud with a height value greater than that of the target object to solve the mean value;

[0012] Determine whether the mean values ​​of the two dynamic point clouds are both continuously and monotonically decreasing within a preset range, and determine whether the minimum mean values ​​of the two dynamic point clouds are both less than a set first threshold;

[0013] When the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, and the minimum means of the two dynamic point clouds are both less than the first threshold, the height mean of the static point cloud of the target object is calculated;

[0014] Determine whether the height average of the static point cloud of the target object is lower than a set second threshold within a preset range;

[0015] If the average height of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds a preset time threshold, the state information of the target object is determined to be a fall.

[0016] In a preferred example, the present application may be further configured as follows: based on the radar echo signal, the step of determining the three-dimensional coordinates, speed and energy value of the static point cloud and the dynamic point cloud of the target object includes:

[0017] After performing a one-dimensional fast Fourier transform on the radar echo signal of the current frame, the mean value of the one-dimensional fast Fourier transform of all Chirp signals of the radar echo signal is subtracted to obtain the first signal of the dynamic point cloud;

[0018] Performing multi-frame progression and storage on the first signal of the dynamic point cloud to obtain the first signal of the static point cloud;

[0019] Performing a two-dimensional fast Fourier transform on the first signals of the acquired static point cloud and the dynamic point cloud to obtain range Doppler maps of the static point cloud and the dynamic point cloud;

[0020] Based on the acquired range Doppler maps of the static point cloud and the dynamic point cloud, combined with the radar echo signals of all antenna channels, the three-dimensional coordinates, velocity and energy values ​​of the static point cloud and the dynamic point cloud of the target object are determined.

[0021] In a preferred example, the present application can be further configured as follows: Based on the acquired range Doppler maps of the static point cloud and the dynamic point cloud, combined with the radar echo signals of all antenna channels, the steps of determining the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object include:

[0022] Obtaining a range Doppler map of a dynamic point cloud from one of the antenna channels of the radar echo signal to perform one-dimensional CFAR detection to obtain the distance and speed of the target object;

[0023] Then, three-dimensional fast Fourier transform and four-dimensional fast Fourier transform are performed on the range Doppler images of the dynamic point clouds of all antenna channels along two antenna directions to obtain the azimuth and elevation angles of the dynamic point clouds of the target object;

[0024] and performing three-dimensional fast Fourier transform and four-dimensional fast Fourier transform on the range Doppler images of the static point clouds of all antenna channels along two antenna directions, respectively, to obtain the azimuth and elevation angles of the static point clouds of the target object;

[0025] The three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object are determined according to the distance, speed, azimuth and pitch angle of the target object.

[0026] In a preferred example, the present application can be further configured as follows: it also includes the following steps:

[0027] Sending the state information of the target object to the elevator main control system;

[0028] When the state information of the target object is falling, the elevator main control system sends a door opening signal to the elevator door opening and closing system to control the elevator door to open.

[0029] In a preferred example, the present application can be further configured as follows: it also includes the following steps:

[0030] The elevator master control system sends a distress signal to the distress device, and starts the emergency call.

[0031] In a preferred example, the present application may be further configured as follows: after determining that the state information of the target object is a fall, the step further includes:

[0032] Counting the duration of the falling state of the target object;

[0033] If the duration exceeds a preset time threshold, a person-trapping signal is sent to the elevator main control system;

[0034] The elevator master control system triggers an instruction to suspend elevator operation and sends a door opening signal to the elevator door opening and closing system, calls a management personnel, and automatically dials a preset emergency contact phone number that matches the target object.

[0035] In a second aspect, the present application provides a home elevator personnel fall detection system based on 4D millimeter wave radar.

[0036] This application is achieved through the following technical solutions:

[0037] A household elevator personnel fall detection system based on 4D millimeter wave radar, comprising:

[0038] A data acquisition module is used to acquire radar echo signals from any target object in the elevator in real time through multiple antenna channels using MIMO technology;

[0039] A dynamic and static point cloud extraction module, used to determine the three-dimensional coordinates, velocity and energy value of the static point cloud and dynamic point cloud of the target object based on the radar echo signal;

[0040] A dynamic point cloud processing module is used to calculate the mean of the dynamic point clouds of all radar echo signals through the three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object, and at the same time, form a target tracking trajectory according to the dynamic point cloud, and select the dynamic point cloud with a height value greater than that of the target object to solve the mean value;

[0041] A dynamic point cloud analysis module, used to determine whether the mean values ​​of the two dynamic point clouds are both continuously and monotonically decreasing within a preset range, and to determine whether the minimum mean values ​​of the two dynamic point clouds are both less than a set first threshold;

[0042] A static point cloud processing module, configured to calculate a height mean of the static point cloud of the target object when the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range and the minimum means of the two dynamic point clouds are both less than the first threshold;

[0043] A static point cloud analysis module, used to determine whether the height average of the static point cloud of the target object is lower than a set second threshold within a preset range;

[0044] The fall detection module is used to determine that the state information of the target object is a fall when the height average of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds a preset time threshold.

[0045] In a third aspect, the present application provides a computer device.

[0046] This application is achieved through the following technical solutions:

[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned methods for detecting falls of people in a home elevator based on a 4D millimeter-wave radar are implemented.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium.

[0049] This application is achieved through the following technical solutions:

[0050] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned methods for detecting a person falling in a home elevator based on a 4D millimeter-wave radar.

[0051] In a fifth aspect, the present application provides a computer program product.

[0052] This application is achieved through the following technical solutions:

[0053] A computer program product includes a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned methods for detecting a person falling in a home elevator based on a 4D millimeter-wave radar.

[0054] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0055] Using MIMO technology, the radar echo signal from any target object in the elevator is obtained in real time through multiple antenna channels, so as to monitor the real-time actions of people in the elevator through 4D millimeter-wave radar technology. 4D millimeter-wave radar is not affected by changes in light, and can still work stably in dim light or complex environments. It can penetrate obstacles and can better adapt to the narrow and complex space in the elevator. Compared with traditional camera monitoring systems, millimeter-wave radar technology performs non-contact monitoring, avoids the risk of privacy leakage, and ensures the privacy protection of personnel. At the same time, 4D millimeter-wave radar can quickly collect target data with better real-time performance. Based on the radar echo signal, the three-dimensional coordinates, speed and energy value of the static point cloud and dynamic point cloud of the target object are determined, and the behavioral characteristics of people in the elevator are accurately extracted from multiple dimensions by combining the dynamic point cloud and static point cloud data processing, which is conducive to more accurately judging whether a fall occurs. At the same time, the processing of point cloud data is more efficient, which is conducive to more quickly judging whether a fall occurs in the elevator. The three-dimensional coordinates, speed and energy value of the static point cloud and dynamic point cloud of the target object are calculated. The method comprises the following steps: calculating the mean of the dynamic point cloud of the wave signal, and at the same time, forming a target tracking trajectory according to the dynamic point cloud, selecting a dynamic point cloud with a larger height value than the target object tracking to solve the mean; judging whether the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, and judging whether the minimum means of the two dynamic point clouds are both less than the set first threshold; when the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, or the minimum means of the two dynamic point clouds are both less than the first threshold, calculating the height mean of the static point cloud of the target object; judging whether the height mean of the static point cloud of the target object is lower than the set second threshold within the preset range; if the height mean of the static point cloud of the target object is lower than the second threshold within the preset range, determining that the state information of the target object is a fall, so as to perform behavior analysis by fusing the dynamic point cloud and the static point cloud data, and detecting the height of the static point cloud and the movement change of the dynamic point cloud to identify the fall behavior; thereby improving the detection accuracy of the fall behavior of people in the elevator, having better environmental adaptability, being able to identify the fall behavior in time, and being conducive to timely response and processing, and having better privacy than other sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the main process of a method for detecting falls in a home elevator based on a 4D millimeter-wave radar is provided as an exemplary embodiment of the present application.

[0057] Figure 2 A dynamic and static point cloud data analysis flow chart of a method for detecting a fall of a person in a home elevator based on a 4D millimeter-wave radar is provided as another exemplary embodiment of the present application.

[0058] Figure 3A structural block diagram of a home elevator personnel fall detection system based on 4D millimeter-wave radar is provided as another exemplary embodiment of the present application.

[0059] Figure 4 A schematic diagram of a 4D millimeter-wave radar installation for a method for detecting a person falling in a home elevator based on a 4D millimeter-wave radar is provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0060] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0063] In order to ensure the safety of the elderly and people with limited mobility when taking elevators, this application uses 4D millimeter-wave radar technology to monitor the real-time movements of people in the elevator, and integrates dynamic point cloud and static point cloud data for behavioral analysis, automatically identifies falls and handles them in a timely manner, providing a new, accurate, efficient and privacy-protected elevator safety monitoring technology, improving the accuracy and real-time performance of fall detection, and thus significantly improving the safety of home elevators.

[0064] An embodiment of the present application provides a method for detecting falls in a home elevator based on a 4D millimeter-wave radar, and the main steps of the method are described as follows.

[0065] Using MIMO technology, the radar echo signal from any target object in the elevator is acquired in real time through multiple antenna channels;

[0066] Based on the radar echo signal, determine the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object;

[0067] Calculate the mean of the dynamic point clouds of all radar echo signals through the three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object, and at the same time, form a target tracking trajectory according to the dynamic point cloud, and select the dynamic point cloud with a height value greater than that of the target object to solve the mean value;

[0068] Determine whether the mean values ​​of the two dynamic point clouds are both continuously and monotonically decreasing within a preset range, and determine whether the minimum mean values ​​of the two dynamic point clouds are both less than a set first threshold;

[0069] When the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, and the minimum means of the two dynamic point clouds are both less than the first threshold, the height mean of the static point cloud of the target object is calculated;

[0070] Determine whether the height average of the static point cloud of the target object is lower than a set second threshold within a preset range;

[0071] If the average height of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds a preset time threshold, the state information of the target object is determined to be a fall.

[0072] In one embodiment, after determining that the state information of the target object is a fall, the method further includes:

[0073] Counting the duration of the falling state of the target object;

[0074] If the duration exceeds a preset time threshold, a person-trapping signal is sent to the elevator main control system;

[0075] The elevator master control system triggers an instruction to suspend elevator operation and sends a door opening signal to the elevator door opening and closing system, calls a management personnel, and automatically dials a preset emergency contact phone number that matches the target object.

[0076] In one embodiment, a method for detecting a person falling in a home elevator based on a 4D millimeter wave radar further includes the following steps:

[0077] Sending the state information of the target object to the elevator main control system;

[0078] When the state information of the target object is falling, the elevator main control system sends a door opening signal to the elevator door opening and closing system to control the elevator door to open.

[0079] In one embodiment, the following steps are also included:

[0080] The elevator master control system sends a distress signal to the distress device, and starts the emergency call.

[0081] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0082] Reference Figure 1 , using MIMO technology, the original echo signal of any target object in the elevator car is collected by the 4D millimeter wave radar, and sent to the controller of the fall system for signal processing, and the intermediate frequency signal is obtained from the echo signal, and then the intermediate frequency signal is sampled to obtain the original ADC time domain data to obtain the intermediate frequency ADC data (digital signal). Among them, the sampling point is set to 512, the Chirp is set to 32, and the 4D millimeter wave radar adopts a four-transmit and four-receive radar configuration.

[0083] MIMO (Multiple-Input, Multiple-Output) technology uses multiple signal segmentation to allow data to be transmitted synchronously via different reflection or penetration paths through multiple transmitting antennas. The receiving end simultaneously receives the echo signal from the transmitting end through multiple receiving antennas, and reconstructs it using DSP, recombining the segmented data based on factors such as time difference, thereby quickly restoring the complete data. MIMO technology can increase the data transmission speed of the wireless network spectrum, improve real-time performance, and do not occupy additional spectrum range, greatly improving spectrum utilization. More importantly, it can also increase the signal reception distance.

[0084] Through high-frequency sampling and multi-channel reception, the fall detection system can obtain rich time domain data, providing an accurate and effective data basis for subsequent signal processing and fall analysis.

[0085] In one embodiment, before processing the radar echo signal, a high-pass filter and a low-pass filter may be used in advance to perform denoising and filtering operations to improve data quality and signal availability.

[0086] The radar data is stored in one frame at 100ms. Each frame outputs static point cloud and dynamic point cloud at the same time. The point cloud data of the human body can be obtained after signal processing of the sampled data.

[0087] Based on the radar echo signal, the three-dimensional coordinates, speed and energy value of the static point cloud and dynamic point cloud of the target object are determined.

[0088] In one embodiment, the step of determining the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object based on the radar echo signal includes:

[0089] After performing a one-dimensional fast Fourier transform on the radar echo signal of the current frame, the mean value of the one-dimensional fast Fourier transform of all Chirp signals of the radar echo signal is subtracted to obtain the first signal of the dynamic point cloud;

[0090] Performing multi-frame progression and storage on the first signal of the dynamic point cloud to obtain the first signal of the static point cloud;

[0091] Performing a two-dimensional fast Fourier transform on the first signals of the acquired static point cloud and the dynamic point cloud to obtain range Doppler maps of the static point cloud and the dynamic point cloud;

[0092] Based on the acquired range Doppler maps of the static point cloud and the dynamic point cloud, combined with the radar echo signals of all antenna channels, the three-dimensional coordinates, velocity and energy values ​​of the static point cloud and the dynamic point cloud of the target object are determined.

[0093] For dynamic point clouds, the one-dimensional FFT mean is stored every two frames, and the storage is continuous, for example, 32 times.

[0094] For static point clouds, the mean of the one-dimensional FFTs of the continuously stored dynamic point clouds is used as the one-dimensional FFT of the static point clouds, and is progressively stored once per frame.

[0095] After performing a one-dimensional FFT on the ADC data, subtracting the mean value of the current frame, and then performing a two-dimensional FFT, an RD map (Range Doppler map) is obtained. The mean value of the current frame is calculated by adding the energy values ​​of all chirps and then dividing it by the total number of chirps to reflect the mean intensity of the current frame.

[0096] In one embodiment, the step of determining the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object based on the acquired range Doppler map of the static point cloud and the dynamic point cloud and combining the radar echo signals of all antenna channels includes:

[0097] The distance Doppler map of the dynamic point cloud is obtained from one of the antenna channels of the radar echo signal for one-dimensional CFAR detection to obtain the distance and speed of the target object; wherein, one-dimensional CFAR (Constant False Alarm Rate) is an algorithm for detecting the presence of a target in a radar echo signal, which can adaptively adjust the detection threshold to maintain a constant false alarm rate, can work in different environments and noise levels, and improves the robustness of target detection, as follows:

[0098] Divide the target radar range Doppler map into multiple range units, select multiple range units as protection units to prevent the target signal from affecting the calculation of the threshold; calculate the average background noise intensity of the area around the protection unit, and use it to design a constant as the threshold value (threshold factor); compare the signal strength of each range unit with the corresponding threshold value, and if it exceeds the threshold value, it is considered as a target signal; obtain the distance index and speed index of the target signal, and then obtain the distance and speed of the target object according to the distance resolution and speed resolution;

[0099] Then, three-dimensional fast Fourier transform and four-dimensional fast Fourier transform are performed on the range Doppler images of the dynamic point clouds of all antenna channels along two antenna directions to obtain the azimuth and elevation angles of the dynamic point clouds of the target object;

[0100] and performing three-dimensional fast Fourier transform and four-dimensional fast Fourier transform on the range Doppler images of the static point clouds of all antenna channels along two antenna directions, respectively, to obtain the azimuth and elevation angles of the static point clouds of the target object;

[0101] The three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object are determined according to the distance, speed, azimuth and pitch angle of the target object.

[0102] By performing one-dimensional CFAR on the RD map of an antenna channel, the distance and speed information of the radar point cloud of the target object is obtained, and then three-dimensional FFT and four-dimensional FFT are performed on each radar point cloud of the target object along the two antenna directions to obtain the azimuth and elevation angles of the dynamic point cloud and the static point cloud. The three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object can be obtained through the distance, speed, azimuth and elevation angle information of the radar point cloud of the target object. The specific solution process is as follows.

[0103] Find the range gate index n corresponding to the target peak according to the range spectrum after FFT R , calculate the target distance:

[0104] R=n R ΔR (1)

[0105] Where ΔR is the range resolution, defined as:

[0106]

[0107] Find the Doppler gate index n corresponding to the target peak according to the Doppler spectrum after FFT v , calculate the target speed:

[0108] v=n v ·Δv (3)

[0109] The velocity resolution (Δv) describes the minimum speed interval that the radar system can distinguish when measuring speed. The calculation formula is as follows:

[0110]

[0111] Where T c is the duration of a single chirp, N chirpIt is the number of chirps in a frame. The static point cloud is obtained by increasing the duration of a single chirp, thereby reducing the velocity resolution and making it more sensitive to target recognition.

[0112] The three-dimensional coordinates (x, y, z) can be calculated by the distance R, azimuth φ and pitch angle θ of the target point cloud. The calculation formula is as follows:

[0113] x=R·cos(φ)·cos(θ) (5)

[0114] y=R·cos(φ)·sin(θ) (6)

[0115] z=R·sin(φ) (7)

[0116] The three-dimensional coordinates (x, y, z) of the dynamic point cloud and the static point cloud can be obtained by the above formula.

[0117] Finally, the energy values ​​of the static point cloud and dynamic point cloud of the target object can be determined based on the three-dimensional coordinates and velocity.

[0118] By obtaining static point cloud and dynamic point cloud data of the human body, the static point cloud reflects the posture of the person after stabilization, while the dynamic point cloud captures the changing process of the person's movement. The combination of the two can provide richer information for the accurate identification of falling behavior and provide accurate data support for subsequent behavior analysis and fall judgment.

[0119] In one embodiment, by setting the thresholds of the CFAR and maximum value search algorithms, interferences in the environment can be filtered out, further improving data quality and effectiveness.

[0120] Reference Figure 2 , perform fall detection on the static point cloud and dynamic point cloud data of the target object, including:

[0121] Obtain dynamic point cloud data, static point cloud data and height value of the target object (i.e. tracking target data);

[0122] Based on the three-dimensional coordinates, velocity and energy values ​​of the static point cloud and dynamic point cloud of the target object, the mean of the dynamic point cloud of all radar echo signals is calculated. At the same time, the target tracking trajectory is formed according to the dynamic point cloud, and the dynamic point cloud with a height value greater than that of the target object is selected to solve the mean value;

[0123] Dynamic point cloud forms a height set {z1,z2,...,z M}, calculate the average value of all dynamic point clouds, the formula is:

[0124]

[0125] Among them, z represents the mean height of the dynamic point cloud, and M represents the total number of dynamic point clouds.

[0126] For the selected dynamic point cloud, extract the height z value of all points to form a height set Calculate the average value greater than the tracking height value of the target object, the formula is:

[0127]

[0128] Among them, z max Represents the mean height of the dynamic point cloud, M max Indicates the number of dynamic point clouds that are greater than the tracked height value of the target object.

[0129] Determine whether the mean values ​​of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, and determine whether the minimum mean values ​​of the two dynamic point clouds are both less than the set first threshold value; such as continuously determining whether the mean values ​​of all dynamic point clouds are monotonically decreasing for 10 frames and the minimum value (lowest height value) of the mean values ​​of all dynamic point clouds is less than the set first threshold value, and continuously determining whether the mean value of the dynamic point cloud higher than the tracking target is monotonically decreasing for 10 frames and the minimum value (lowest height value) of the mean value of the dynamic point cloud higher than the tracking target is less than the set first threshold value;

[0130] When the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, and the minimum means of the two dynamic point clouds are both less than the first threshold, that is, it is continuously judged that the means of all dynamic point clouds are monotonically decreasing in 10 frames and their minimum values ​​are less than the set first threshold, and it is continuously judged that the means of the dynamic point clouds higher than the tracked target are monotonically decreasing in 10 frames and their minimum values ​​are less than the set first threshold, the height mean of the static point cloud of the target object is calculated; for the screened static point clouds, the height z values ​​of all points are extracted to form a height set {z1, z2, ..., z N}, calculate the average value of all height values ​​z, the formula is:

[0131]

[0132] in, represents the mean height of the static point cloud, and N represents the total number of static point clouds.

[0133] When it is continuously judged that the mean of all dynamic point clouds is not monotonically decreasing in 10 frames or its lowest value is not less than the set first threshold, the mean of the dynamic point clouds of all radar echo signals continues to be calculated; and, when it is continuously judged that the mean of the dynamic point cloud higher than the tracked target is not monotonically decreasing in 10 frames or its lowest value is not less than the set first threshold, the dynamic point cloud with a height value larger than the target object continues to be taken to solve the mean; until the means of the two dynamic point clouds are continuously monotonically decreasing within the preset range, and the minimum means of the two dynamic point clouds are both less than the first threshold, the height mean of the static point cloud of the target object is calculated.

[0134] Determine whether the height mean (average) of the static point cloud of the target object is lower than the set second threshold within a preset range; such as continuously determining whether the mean of the static point cloud is less than the set second threshold within 600 frames;

[0135] If the average height of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds the preset time threshold, the state information of the target object is determined to be a fall.

[0136] When the height average of the static point cloud of the target object is not lower than the second threshold within the preset range, continue to calculate the mean of the dynamic point cloud of all radar echo signals for judgment, and continue to take the dynamic point cloud solution mean value larger than the height value of the target object for judgment.

[0137] On the basis of traditional radar fall detection, this application introduces a dynamic and static point cloud behavior logic analysis module, which integrates dynamic point cloud and static point cloud data to conduct in-depth analysis of the movement trajectory and posture changes of the person, so as to accurately determine whether a fall has occurred. When encountering other actions similar to fall behaviors, such as suddenly losing balance or falling after leaning on an elevator, the behavior logic judgment module can distinguish these easily confused actions based on the analysis results, identify and eliminate false alarms of non-fall behaviors, reduce the false alarm rate, and ensure that the fall system can stably, accurately, and efficiently identify fall behaviors in complex environments.

[0138] In one embodiment, before analyzing the target object point cloud data, DBSCAN clustering is performed first, which is conducive to faster analysis of the target point cloud data and can effectively distinguish the target object from noise points. The G_TRACK algorithm is then used to track the clustering results to obtain the movement trajectory of the person.

[0139] Reference Figure 1 Finally, when the target object's status information is determined to be a fall, the duration of the target object's fall is counted. The radar works for 100ms per frame, and the duration is counted by accumulating the number of frames. If the duration exceeds the preset time threshold, a person-trapped signal is sent to the elevator main control system. The elevator main control system triggers the elevator operation pause command and sends a door-opening signal to the elevator door opening and closing system, calls the management personnel, and automatically dials the preset emergency contact number that matches the target object to promptly handle elevator entrapment and fall events.

[0140] When the system detects that a single target has fallen and has been in the elevator for a long time, such as more than 5 minutes, the trapping mechanism is automatically triggered and a trapping signal is sent to the elevator main control system. After receiving the trapping signal, the elevator main control system will immediately take corresponding emergency measures, including suspending the elevator operation, opening the elevator door, etc., to ensure the safety of the personnel; at the same time, the system will automatically call the emergency contact number pre-matched with the target object, and notify the relevant management personnel to rush to the scene to ensure timely response to avoid personnel being unable to receive timely rescue due to trapping accidents or other emergencies. The trapping mechanism can minimize the danger of elevators caused by failures or other emergencies, thereby enhancing the safety of elevator use.

[0141] In one embodiment, the duration of the target object's fall state is counted; if the duration exceeds a preset time threshold, that is, when the system detects that the target object has been trapped in the elevator for a long time, a person entrapment signal is sent to the elevator main control system; the elevator main control system triggers an elevator operation pause instruction and sends a door opening signal to the elevator door opening and closing system, calls the management personnel, and automatically dials the preset emergency contact number that matches the target object to promptly handle elevator entrapment and fall incidents.

[0142] In summary, a method for detecting falls in home elevators based on 4D millimeter-wave radar utilizes MIMO technology to obtain radar echo signals from any target object in the elevator in real time, so as to monitor the real-time movements of people in the elevator through 4D millimeter-wave radar technology. 4D millimeter-wave radar is not affected by changes in light, and can still work stably in dim light or complex environments, and can penetrate obstacles, so it can better adapt to the narrow and complex space in the elevator. Compared with traditional camera monitoring systems, millimeter-wave radar technology performs contactless monitoring, avoids the risk of privacy leakage, and ensures the protection of personnel privacy. At the same time, 4D millimeter-wave radar can quickly collect target data with better real-time performance. Based on the radar echo signal, the three-dimensional coordinates, speed and energy value of the static point cloud and dynamic point cloud of the target object are determined, so as to combine the dynamic point cloud and static point cloud data to accurately extract the behavioral characteristics of people in the elevator from multiple dimensions, which is conducive to more accurately judging whether a fall occurs. At the same time, the processing of point cloud data is more efficient, which is conducive to more quickly judging whether a fall occurs in the elevator. The three-dimensional coordinates of the static point cloud and dynamic point cloud of the target object are obtained by the three-dimensional coordinates of the static point cloud and dynamic point cloud of the target object. The target object is a static point cloud of the elevator. ...

[0143] This application detects whether the height change of a person exceeds a set threshold. If so, it enters the fall monitoring mode and continues to monitor the height change of the person to automatically identify the fall behavior; and if the standing state is not restored within the set time, a trapped person signal is output to the elevator main control system.

[0144] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0145] The embodiment of the present application also provides a home elevator personnel fall detection system based on 4D millimeter wave radar, which corresponds one-to-one to a home elevator personnel fall detection method based on 4D millimeter wave radar in the above embodiment. The home elevator personnel fall detection system based on 4D millimeter wave radar includes:

[0146] A data acquisition module is used to acquire radar echo signals from any target object in the elevator in real time through multiple antenna channels using MIMO technology;

[0147] A dynamic and static point cloud extraction module, used to determine the three-dimensional coordinates, velocity and energy value of the static point cloud and dynamic point cloud of the target object based on the radar echo signal;

[0148] A dynamic point cloud processing module is used to calculate the mean of the dynamic point clouds of all radar echo signals through the three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object, and at the same time, form a target tracking trajectory according to the dynamic point cloud, and select the dynamic point cloud with a height value greater than that of the target object to solve the mean value;

[0149] A dynamic point cloud analysis module, used to determine whether the mean values ​​of the two dynamic point clouds are both continuously and monotonically decreasing within a preset range, and to determine whether the minimum mean values ​​of the two dynamic point clouds are both less than a set first threshold;

[0150] A static point cloud processing module, configured to calculate a height mean of the static point cloud of the target object when the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range and the minimum means of the two dynamic point clouds are both less than the first threshold;

[0151] A static point cloud analysis module, used to determine whether the height average of the static point cloud of the target object is lower than a set second threshold within a preset range;

[0152] The fall detection module is used to determine that the state information of the target object is a fall when the height average of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds a preset time threshold.

[0153] Reference Figure 3 ,A 4D millimeter-wave radar-based home elevator personnel fall detection system collects radar echo signals from any target object in the elevator in real time by setting a radar RF module and a radar MCU module.

[0154] The radar RF module uses a 4D millimeter-wave radar sensor and a four-transmit and four-receive antenna with MIMO technology to obtain the echo signal of people in the elevator in real time. After the transceiver antenna receives the echo signal, it converts the intermediate frequency signal into ADC data and transmits it to the MCU chip for processing.

[0155] The radar MCU module (fall system controller) integrates FFT and CFAR algorithms, which can efficiently process radar echo signals and extract information such as the three-dimensional coordinates, speed, energy value, etc. of personnel. Through data processing algorithms, the system can calculate the length of time a person has stayed and detect falls.

[0156] A household elevator personnel fall detection system based on 4D millimeter wave radar then sends the personnel status information to the elevator main control system through a communication module (such as 485) and a 485 communication protocol.

[0157] The elevator master control system is electrically connected to the radar MCU module, the elevator door opening and closing system, and the emergency call device, and is responsible for receiving the trigger signal and issuing control commands. When a person falls, the elevator master control system sends a door opening signal to the elevator door opening and closing system, and transmits the emergency call signal to the emergency call device to start the emergency call.

[0158] When the radar MCU module detects that a person has fallen and maintained this state for a certain period of time, the fall system will output a signal that a person is trapped, control the emergency call device to work in conjunction with the elevator door opening and closing system, automatically open the elevator door, and make an emergency call to relevant personnel.

[0159] Reference Figure 4 , a radar of a home elevator personnel fall detection system based on 4D millimeter wave radar can be installed on the top of the elevator door, with a tilt angle of 30 degrees to ensure coverage of a wide monitoring area and to monitor the dynamic behavior of people in the elevator in real time. The radar includes a radar RF module and a radar MCU module. The radar configuration parameters are shown in Table 1 below.

[0160] Table 1

[0161]

[0162] Initialize the radar.

[0163] If no one enters the elevator, no personal information will be detected and no distress signal will be output.

[0164] When a person enters the elevator, the radar detects the target object, emits a millimeter wave signal and receives the reflected signal, obtains the radar echo signal of the person in the elevator, and monitors the change of the radar echo signal of the person in real time. The radar MCU module extracts features and analyzes and judges the received radar echo signal to ensure efficient and accurate fall detection.

[0165] If the person leaves within the set threshold time and the set altitude change threshold is not reached, no distress signal will be output.

[0166] Once a person falls or stays for a long time, the fall system will promptly notify the elevator main control system and take emergency measures, such as stopping the elevator and opening the elevator door, and automatically calling the emergency contact and notifying the management staff, greatly improving the safety of home elevators.

[0167] For the specific limitations of a home elevator personnel fall detection system based on 4D millimeter-wave radar, please refer to the above limitations of a home elevator personnel fall detection method based on 4D millimeter-wave radar, which will not be repeated here.

[0168] Each module in the above-mentioned household elevator personnel fall detection system based on 4D millimeter wave radar can be implemented in whole or in part by software, hardware and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0169] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, any of the above-mentioned methods for detecting falls in home elevators based on 4D millimeter-wave radars is implemented.

[0170] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the above-mentioned methods for detecting personnel falls in a home elevator based on a 4D millimeter-wave radar is implemented.

[0171] In one embodiment, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned methods for detecting people falling in a home elevator based on 4D millimeter wave radar.

[0172] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0173] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A method for detecting falls in home elevators based on 4D millimeter wave radar, characterized in that: The following steps are included: Using MIMO technology, the radar echo signal from any target object in the elevator is acquired in real time through multiple antenna channels; Based on the radar echo signal, determine the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object; Calculate the mean of the dynamic point clouds of all radar echo signals through the three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object, and at the same time, form a target tracking trajectory according to the dynamic point cloud, and select the dynamic point cloud with a height value greater than that of the target object to solve the mean value; Determine whether the mean values ​​of the two dynamic point clouds are both continuously and monotonically decreasing within a preset range, and determine whether the minimum values ​​of the mean values ​​of the two dynamic point clouds are both less than a set first threshold; When the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range, and the minimum means of the two dynamic point clouds are both less than the first threshold, the height mean of the static point cloud of the target object is calculated; Determine whether the average height of the static point cloud of the target object is lower than a set second threshold within a preset range; If the average height of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds a preset time threshold, the state information of the target object is determined to be a fall.

2. The method for detecting falling of a person in a home elevator based on 4D millimeter wave radar according to claim 1 is characterized in that: The step of determining the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object based on the radar echo signal comprises: After performing a one-dimensional fast Fourier transform on the radar echo signal of the current frame, the mean value of the one-dimensional fast Fourier transform of all Chirp signals of the radar echo signal is subtracted to obtain the first signal of the dynamic point cloud; Performing multi-frame progression and storage on the first signal of the dynamic point cloud to obtain the first signal of the static point cloud; Performing a two-dimensional fast Fourier transform on the first signals of the acquired static point cloud and the dynamic point cloud to obtain range Doppler maps of the static point cloud and the dynamic point cloud; Based on the acquired range Doppler maps of the static point cloud and the dynamic point cloud, combined with the radar echo signals of all antenna channels, the three-dimensional coordinates, velocity and energy values ​​of the static point cloud and the dynamic point cloud of the target object are determined.

3. The method for detecting falls of people in a home elevator based on 4D millimeter wave radar according to claim 2, characterized in that: The steps of determining the three-dimensional coordinates, velocity and energy value of the static point cloud and the dynamic point cloud of the target object based on the acquired range Doppler map of the static point cloud and the dynamic point cloud and combining the radar echo signals of all antenna channels include: Obtaining a range Doppler map of a dynamic point cloud from one of the antenna channels of the radar echo signal to perform one-dimensional CFAR detection to obtain the distance and speed of the target object; Then, three-dimensional fast Fourier transform and four-dimensional fast Fourier transform are performed on the range Doppler images of the dynamic point clouds of all antenna channels along two antenna directions to obtain the azimuth and elevation angles of the dynamic point clouds of the target object; and performing three-dimensional fast Fourier transform and four-dimensional fast Fourier transform on the range Doppler images of the static point clouds of all antenna channels along two antenna directions, respectively, to obtain the azimuth and elevation angles of the static point clouds of the target object; The three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object are determined according to the distance, speed, azimuth and pitch angle of the target object.

4. The method for detecting falls of people in a home elevator based on 4D millimeter wave radar according to any one of claims 1 to 3, characterized in that: The following steps are also included: Sending the state information of the target object to the elevator main control system; When the state information of the target object is falling, the elevator main control system sends a door opening signal to the elevator door opening and closing system to control the elevator door to open.

5. The method for detecting falling of a person in a home elevator based on 4D millimeter wave radar according to claim 4, characterized in that: The following steps are also included: The elevator master control system sends a distress signal to the distress device, and starts the emergency call.

6. The method for detecting falling of a person in a home elevator based on 4D millimeter wave radar according to claim 1, characterized in that: After determining that the state information of the target object is a fall, the method further includes: Counting the duration of the falling state of the target object; If the duration exceeds a preset time threshold, a person-trapping signal is sent to the elevator main control system; The elevator master control system triggers an instruction to suspend elevator operation and sends a door opening signal to the elevator door opening and closing system, calls a management personnel, and automatically dials a preset emergency contact phone number that matches the target object.

7. A household elevator personnel fall detection system based on 4D millimeter wave radar, characterized in that: include, A data acquisition module is used to acquire radar echo signals from any target object in the elevator in real time through multiple antenna channels using MIMO technology; A dynamic and static point cloud extraction module, used to determine the three-dimensional coordinates, velocity and energy value of the static point cloud and dynamic point cloud of the target object based on the radar echo signal; A dynamic point cloud processing module is used to calculate the mean of the dynamic point clouds of all radar echo signals through the three-dimensional coordinates, speed and energy values ​​of the static point cloud and the dynamic point cloud of the target object, and at the same time, form a target tracking trajectory according to the dynamic point cloud, and select the dynamic point cloud with a height value greater than that of the target object to solve the mean value; A dynamic point cloud analysis module, used to determine whether the mean values ​​of the two dynamic point clouds are both continuously and monotonically decreasing within a preset range, and to determine whether the minimum mean values ​​of the two dynamic point clouds are both less than a set first threshold; A static point cloud processing module, configured to calculate a height mean of the static point cloud of the target object when the means of the two dynamic point clouds are continuously and monotonically decreasing within a preset range and the minimum means of the two dynamic point clouds are both less than the first threshold; A static point cloud analysis module, used to determine whether the height average of the static point cloud of the target object is lower than a set second threshold within a preset range; The fall detection module is used to determine that the state information of the target object is a fall when the height average of the static point cloud of the target object is lower than the second threshold within a preset range and the duration exceeds a preset time threshold.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 6 when being executed by a processor.

Images