Human body tumble detection method based on multi-branch attention network and ultra wide band radar

Abstract

The invention discloses a human body tumble detection method based on a multi-branch attention network and an ultra wide band radar. The method comprises the following steps: acquiring a human body ultra wide band radar echo signal in a target area through an ultra wide band radar; preprocessing the human body ultra-wideband radar echo signal to obtain a corresponding time Doppler map, a distance Doppler map and a distance time map; according to the time Doppler graph, the distance Doppler graph, the distance time graph and a pre-trained multi-branch attention network, determining a first human body tumble detection result, a second human body tumble detection result and a third human body tumble detection result of the target area; and determining a final human body tumble detection result. According to the embodiment of the invention, human body tumble detection can be carried out based on the multi-branch attention network and the ultra-wideband radar, and tumble detection can be comprehensively and accurately carried out on the human body in the designated area from multiple dimensions of the time Doppler diagram, the distance Doppler diagram and the distance time diagram.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a human fall detection method based on a multi-branch attention network and an ultra-wideband radar. Background Art

[0002] In order to ensure the safety of the human body in the designated area, a fall detection is usually performed on the human body in the designated area when the human body is in the designated area to determine whether the human body in the designated area has fallen.

[0003] In the related art, a commonly used fall detection scheme is: obtaining a single posture feature of a human body in a specified area determined according to a radar echo signal of a human body in the specified area, and then performing fall detection on the human body in the specified area based on the single posture feature to determine whether the human body in the specified area has fallen. The fall detection scheme in the related art performs fall detection on the human body in the specified area based on a single posture feature, and cannot comprehensively and accurately perform fall detection on the human body in the specified area to determine whether the human body in the specified area has fallen. Summary of the invention

[0004] The present invention provides a method for detecting a human fall based on a multi-branch attention network and an ultra-wideband radar, so as to solve the problem that the fall detection scheme in the related art performs fall detection on a human body in a specified area based on a single posture feature, and is unable to comprehensively and accurately perform fall detection on a human body in the specified area and determine whether a human body in the specified area has fallen.

[0005] According to one aspect of the present invention, a method for detecting a human fall based on a multi-branch attention network and an ultra-wideband radar is provided, comprising:

[0006] Acquire an ultra-wideband radar echo signal of a human body in the target area by placing an ultra-wideband radar in the target area;

[0007] Preprocessing the human body ultra-wideband radar echo signal in the target area to obtain a time Doppler map, a range Doppler map and a range-time map corresponding to the human body ultra-wideband radar echo signal in the target area;

[0008] Determine a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network; wherein the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a range Doppler map detection branch model, and a range-time map detection branch model;

[0009] A final human fall detection result of the target area is determined according to the first human fall detection result, the second human fall detection result and the third human fall detection result.

[0010] According to another aspect of the present invention, a human fall detection device based on a multi-branch attention network and an ultra-wideband radar is provided, comprising:

[0011] A signal acquisition module, used to acquire an ultra-wideband radar echo signal of a human body in the target area through an ultra-wideband radar placed in the target area;

[0012] A signal processing module is used to pre-process the human body ultra-wideband radar echo signal in the target area to obtain a time Doppler map, a range Doppler map and a range-time map corresponding to the human body ultra-wideband radar echo signal in the target area;

[0013] A network processing module, used to determine a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network; wherein the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a range Doppler map detection branch model, and a range-time map detection branch model;

[0014] A result determination module is used to determine a final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result and the third human fall detection result.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor;

[0017] and a memory communicatively coupled to the at least one processor;

[0018] In which, the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the human fall detection method based on multi-branch attention network and ultra-wideband radar described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the human fall detection method based on a multi-branch attention network and an ultra-wideband radar as described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention is to obtain the ultra-wideband radar echo signal of the human body in the target area through an ultra-wideband radar placed in the target area, pre-process the ultra-wideband radar echo signal of the human body in the target area, and obtain the time Doppler map, distance Doppler map and distance time map corresponding to the ultra-wideband radar echo signal of the human body in the target area; then, according to the time Doppler map, the distance Doppler map, the distance time map and the pre-trained multi-branch attention network, determine the first human body fall detection result, the second human body fall detection result and the third human body fall detection result in the target area; the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a distance Doppler map detection branch model and a distance time map detection branch model; finally, according to The first human fall detection result, the second human fall detection result and the third human fall detection result are used to determine the final fall detection result of the target area, which solves the problem that the fall detection scheme in the related technology performs fall detection on the human body in the specified area based on a single posture feature, and cannot comprehensively and accurately perform fall detection on the human body in the specified area to determine whether the human body in the specified area falls. The human body in the specified area can be automatically detected based on a multi-branch attention network and an ultra-wideband radar to determine whether the human body in the specified area falls. The human body in the specified area can be comprehensively and accurately detected from multiple dimensions of time Doppler map, distance Doppler map and distance-time map to determine whether the human body in the specified area falls.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a method for detecting a human fall based on a multi-branch attention network and an ultra-wideband radar is provided in Embodiment 1 of the present invention.

[0024] Figure 2 A schematic diagram of a human fall detection process based on a multi-branch attention network and an ultra-wideband radar provided in Embodiment 1 of the present invention.

[0025] Figure 3A flowchart of a method for detecting a human fall based on a multi-branch attention network and an ultra-wideband radar is provided in Embodiment 2 of the present invention.

[0026] Figure 4 A schematic diagram of the structure of a human fall detection device based on a multi-branch attention network and an ultra-wideband radar provided in Example 3 of the present invention.

[0027] Figure 5 A schematic diagram of the structure of an electronic device for implementing a human fall detection method based on a multi-branch attention network and an ultra-wideband radar according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "target", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise", "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of human body information or user personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0030] Embodiment 1

[0031] Figure 1 The flowchart of a human fall detection method based on a multi-branch attention network and an ultra-wideband radar is provided in the first embodiment of the present invention. This embodiment is applicable to the case of detecting a human fall in a specified area. The method can be performed by a human fall detection device based on a multi-branch attention network and an ultra-wideband radar. The fall detection device can be implemented in the form of hardware and / or software, and the fall detection device can be configured in an electronic device. Figure 1As shown, the method includes:

[0032] Step 101: Acquire an ultra-wideband radar echo signal of a human body in a target area through an ultra-wideband radar placed in the target area.

[0033] Optionally, the target area may be an area where it is necessary to detect whether a human body has fallen at the current moment. The human body in the target area is a human body located in the target area. An ultra-wideband radar is placed in the target area. The ultra-wideband radar placed in the target area can continuously transmit electromagnetic wave pulse signals to the human body in the target area. The emitted electromagnetic wave pulse signal is reflected by the human body in the target area and then received by a receiver of the pulse radio ultra-wideband radar, and the received signal is the ultra-wideband radar echo signal of the human body in the target area. Acquiring the ultra-wideband radar echo signal of the human body in the target area through the ultra-wideband radar placed in the target area includes: receiving the ultra-wideband radar echo signal of the human body in the target area sent by the ultra-wideband radar placed in the target area.

[0034] Step 102: preprocessing the human body ultra-wideband radar echo signal in the target area to obtain a time Doppler map, a range Doppler map, and a range-time map corresponding to the human body ultra-wideband radar echo signal in the target area.

[0035] Optionally, after each receipt of an ultra-wideband radar echo signal of the human body in the target area sent by an ultra-wideband radar placed in the target area, the ultra-wideband radar echo signal of the human body in the target area is preprocessed to obtain a time Doppler map, a range Doppler map and a range-time map corresponding to the ultra-wideband radar echo signal of the human body in the target area.

[0036] Optionally, a time-Doppler map (TDM) is an image obtained by processing a radar echo signal received after being reflected by a target, and is used to display the speed change of a target at different time points. The waveform on the time-Doppler map can represent the speed change of the target at different time points. The time-Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area is a time-Doppler map obtained after preprocessing the ultra-wideband radar echo signal of the human body in the target area. The time-Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area is an image obtained by processing the ultra-wideband radar echo signal of the human body received after being reflected by the human body in the target area, and is used to display the speed change of the human body in the target area at different time points. The waveform on the time-Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area can represent the speed change of the human body in the target area at different time points. The time-Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area can characterize the speed change of the human body in the target area at different time points.

[0037] Optionally, a Range-Doppler Map (RDM) is an image obtained by processing a radar echo signal received after being reflected by a target, and is used to display the distance and speed information of a target in a graphical form. The position of the peak in the Range-Doppler Map represents the distance and speed of the target. The intensity value of the peak in the Range-Doppler Map represents the intensity of the radar echo signal after being reflected by the target. The Range-Doppler Map can characterize the motion characteristics of the target. The Range-Doppler Map corresponding to the ultra-wideband radar echo signal of the human body in the target area is a Range-Doppler Map obtained after preprocessing the ultra-wideband radar echo signal of the human body in the target area. The Range-Doppler Map corresponding to the ultra-wideband radar echo signal of the human body in the target area is an image obtained by processing the ultra-wideband radar echo signal of the human body received after being reflected by the human body in the target area, and is used to display the distance and speed information of a human body in the target area in a graphical form. The position of the peak in the Range-Doppler Map corresponding to the ultra-wideband radar echo signal of the human body in the target area represents the distance and speed of a human body in the target area. The intensity value of the peak in the range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area represents the intensity of the human ultra-wideband radar echo signal after being reflected by the human body in the target area. The range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area can characterize the motion characteristics of the human body in the target area.

[0038] Optionally, a Range-Time Map (RTM) is an image obtained by processing a radar echo signal received after being reflected by a target, and is used to display how the distance between a target and a radar changes over time. The Range-Time Map can characterize how the distance between a target and a radar changes over time. The Range-Time Map corresponding to the ultra-wideband radar echo signal of a human body in a target area is a range-time map obtained after preprocessing the ultra-wideband radar echo signal of a human body in a target area. The Range-Time Map corresponding to the ultra-wideband radar echo signal of a human body in a target area is an image obtained by processing an ultra-wideband radar echo signal of a human body received after being reflected by a human body in a target area, and is used to display how the distance between a human body in a target area and a radar changes over time. The Range-Time Map corresponding to the ultra-wideband radar echo signal of a human body in a target area can characterize how the distance between a human body in a target area and an ultra-wideband radar changes over time.

[0039] Optionally, the ultra-wideband radar echo signal of the human body in the target area is preprocessed to obtain a time Doppler map, a range Doppler map and a distance-time map corresponding to the ultra-wideband radar echo signal of the human body in the target area, including: preprocessing the ultra-wideband radar echo signal of the human body in the target area through a time Doppler map generation component to obtain a time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area; preprocessing the ultra-wideband radar echo signal of the human body in the target area through a range Doppler map generation component to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area; preprocessing the ultra-wideband radar echo signal of the human body in the target area through a range-time map generation component to obtain a distance-time map corresponding to the ultra-wideband radar echo signal of the human body in the target area.

[0040] Optionally, the time Doppler map generation component may be a component provided in an electronic device for preprocessing the ultra-wideband radar echo signal of the human body to obtain a time Doppler map corresponding to the ultra-wideband radar echo signal of the human body. The electronic device may input the ultra-wideband radar echo signal of the specified human body into the time Doppler map generation component. The time Doppler map generation component preprocesses the input ultra-wideband radar echo signal of the specified human body to obtain a time Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body, and then outputs a time Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body. The electronic device obtains the time Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body output by the time Doppler map generation component, thereby preprocessing the ultra-wideband radar echo signal of the specified human body through the time Doppler map generation component to obtain a time Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body. The time Doppler map generation component may be written by a technician and provided in an electronic device.

[0041] Optionally, the ultra-wideband radar placed in the target area will send the ultra-wideband radar echo signal of the human body in the target area to the electronic device. The electronic device inputs the received ultra-wideband radar echo signal of the human body in the target area into the time Doppler map generation component. The time Doppler map generation component preprocesses the input ultra-wideband radar echo signal of the human body in the target area to obtain a time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and then outputs a time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area. The electronic device obtains the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area output by the time Doppler map generation component, thereby preprocessing the ultra-wideband radar echo signal of the human body in the target area through the time Doppler map generation component to obtain a time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area.

[0042] Optionally, the input human ultra-wideband radar echo signal within the target area is preprocessed to obtain a time Doppler map corresponding to the human ultra-wideband radar echo signal within the target area, including: discretizing the human ultra-wideband radar echo signal within the target area to obtain the human ultra-wideband radar echo signal after discretization; performing clutter suppression on the human ultra-wideband radar echo signal after discretization by using a moving average algorithm to obtain the human ultra-wideband radar echo signal after clutter suppression; performing short-time Fourier transform on the human ultra-wideband radar echo signal after clutter suppression to obtain the time Doppler map corresponding to the human ultra-wideband radar echo signal within the target area.

[0043] Optionally, the range Doppler map generation component may be a component provided in an electronic device for preprocessing the ultra-wideband radar echo signal of the human body to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the human body. The electronic device may input the ultra-wideband radar echo signal of the specified human body into the range Doppler map generation component. The range Doppler map generation component preprocesses the input ultra-wideband radar echo signal of the specified human body to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body, and then outputs a range Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body. The electronic device obtains the range Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body output by the range Doppler map generation component, thereby preprocessing the ultra-wideband radar echo signal of the specified human body through the range Doppler map generation component to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the specified human body. The range Doppler map generation component may be written by a technician and provided in an electronic device.

[0044] Optionally, the ultra-wideband radar placed in the target area will send the ultra-wideband radar echo signal of the human body in the target area to the electronic device. The electronic device inputs the received ultra-wideband radar echo signal of the human body in the target area into the range Doppler map generation component. The range Doppler map generation component preprocesses the input ultra-wideband radar echo signal of the human body in the target area to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and then outputs the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area. The electronic device obtains the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area output by the range Doppler map generation component, thereby preprocessing the ultra-wideband radar echo signal of the human body in the target area through the range Doppler map generation component to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area.

[0045] Optionally, preprocessing is performed on the ultra-wideband radar echo signal of the human body in the target area to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, including: discretizing the ultra-wideband radar echo signal of the human body in the target area to obtain the ultra-wideband radar echo signal of the human body after discretization; performing clutter suppression on the ultra-wideband radar echo signal of the human body after discretization by using a phasor mean cancellation algorithm to obtain the ultra-wideband radar echo signal of the human body after clutter suppression; and performing discrete Fourier transform on the ultra-wideband radar echo signal of the human body after clutter suppression along a slow time dimension to obtain a range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area.

[0046] Optionally, the distance-time diagram generation component may be a component provided in an electronic device for preprocessing a human ultra-wideband radar echo signal to obtain a distance-time diagram corresponding to the human ultra-wideband radar echo signal. The electronic device may input a specified human ultra-wideband radar echo signal into the distance-time diagram generation component. The distance-time diagram generation component preprocesses the input specified human ultra-wideband radar echo signal to obtain a distance-time diagram corresponding to the specified human ultra-wideband radar echo signal, and then outputs a distance-time diagram corresponding to the specified human ultra-wideband radar echo signal. The electronic device obtains the distance-time diagram corresponding to the specified human ultra-wideband radar echo signal output by the distance-time diagram generation component, thereby preprocessing the specified human ultra-wideband radar echo signal through the distance-time diagram generation component to obtain a distance-time diagram corresponding to the specified human ultra-wideband radar echo signal. The distance-time diagram generation component may be written by a technician and provided in an electronic device.

[0047] Optionally, the ultra-wideband radar placed in the target area will send the ultra-wideband radar echo signal of the human body in the target area to the electronic device. The electronic device inputs the received ultra-wideband radar echo signal of the human body in the target area into the distance-time graph generation component. The distance-time graph generation component preprocesses the input ultra-wideband radar echo signal of the human body in the target area to obtain a distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area, and then outputs a distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area. The electronic device obtains the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area output by the distance-time graph generation component, thereby preprocessing the ultra-wideband radar echo signal of the human body in the target area through the distance-time graph generation component to obtain a distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area.

[0048] Optionally, preprocessing the human body ultra-wideband radar echo signal in the target area to obtain a distance-time diagram corresponding to the human body ultra-wideband radar echo signal in the target area includes: discretizing the human body ultra-wideband radar echo signal in the target area to obtain the human body ultra-wideband radar echo signal after discretization; suppressing clutter on the human body ultra-wideband radar echo signal after discretization by a phasor mean cancellation algorithm to obtain the human body ultra-wideband radar echo signal after clutter suppression; preprocessing the human body ultra-wideband radar echo signal after clutter suppression to obtain the distance-time diagram corresponding to the human body ultra-wideband radar echo signal in the target area.

[0049] Step 103: determine a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network.

[0050] Among them, the pre-trained multi-branch attention network includes a time Doppler image detection branch model, a distance Doppler image detection branch model and a distance time image detection branch model.

[0051] Optionally, a pre-trained multi-branch attention network is provided in the electronic device. The pre-trained multi-branch attention network is composed of a pre-trained time Doppler image detection branch model, a range Doppler image detection branch model, and a range-time image detection branch model.

[0052] Optionally, according to the time Doppler map, the distance Doppler map, the distance-time map and a pre-trained multi-branch attention network, a first human fall detection result, a second human fall detection result and a third human fall detection result in the target area are determined, including: inputting the time Doppler map into a time Doppler map detection branch model in a pre-trained multi-branch attention network, and obtaining the first human fall detection result in the target area output by the time Doppler map detection branch model; wherein the input of the time Doppler map detection branch model is a time Doppler map corresponding to an ultra-wideband radar echo signal of a human body in the target area, and the output of the time Doppler map detection branch model is the first human fall detection result in the target area, and the first human fall detection result in the target area is a fall state of a human body in the target area determined according to the time Doppler map corresponding to the ultra-wideband radar echo signal of a human body in the target area, and the fall state is falling or not falling.

[0053] Optionally, the pre-trained time Doppler map detection branch model is used to analyze and detect the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and determine the first human body fall detection result in the target area. The human body in the target area is the human body corresponding to the time Doppler map. The input of the time Doppler map detection branch model is the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the output of the time Doppler map detection branch model is the first human body fall detection result in the target area. The first human body fall detection result in the target area is the fall state of the human body in the target area determined according to the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area. The fall state of the human body can be information used to characterize whether the human body is in a fall state. The fall state of the human body is fall or not fall. The fall state of the human body is fall, indicating that the human body is in a fall state and the human body has fallen. The fall state of the human body is not fall, indicating that the human body is not in a fall state and the human body has not fallen.

[0054] Optionally, the electronic device can input the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area into a pre-trained time Doppler map detection branch model. The pre-trained time Doppler map detection branch model will analyze and detect the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, determine the first human fall detection result in the target area, and then output the first human fall detection result in the target area. The first human fall detection result in the target area output by the time Doppler map detection branch model is the fall state of the human body in the target area determined according to the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area. The electronic device can obtain the first human fall detection result in the target area output by the time Doppler map detection branch model.

[0055] Optionally, before obtaining the ultra-wideband radar echo signal of the human body in the target area through an ultra-wideband radar placed in the target area, the method also includes: obtaining a preset number of time Doppler maps and a first human fall detection result corresponding to each of the time Doppler maps, using each of the time Doppler maps and the first human fall detection result corresponding to each of the time Doppler maps as training samples, training the machine learning model, and obtaining a time Doppler map detection branch model.

[0056] Optionally, the preset number can be set according to business needs. Exemplarily, the preset number of time Doppler maps is 2000 time Doppler maps. Each time Doppler map in the preset number of time Doppler maps can be a different time Doppler map obtained by preprocessing the human body ultra-wideband radar echo signal received after reflection from different human bodies. The human body corresponding to each time Doppler map is the human body to which the human body ultra-wideband radar echo signal used to obtain each time Doppler map belongs. The first human body fall detection result corresponding to each time Doppler map is the fall state of the human body corresponding to each time Doppler map when the emitted electromagnetic wave pulse signal is reflected by the human body corresponding to each time Doppler map.

[0057] Optionally, obtaining a preset number of time Doppler images and a first human fall detection result corresponding to each of the time Doppler images includes: obtaining a preset number of time Doppler images sent by a technician and a first human fall detection result corresponding to each of the time Doppler images. The technician may send a preset number of time Doppler images and a first human fall detection result corresponding to each of the time Doppler images to an electronic device through a terminal device. The electronic device may obtain a preset number of time Doppler images sent by the technician and a first human fall detection result corresponding to each of the time Doppler images, and then use each of the time Doppler images and the first human fall detection result corresponding to each of the time Doppler images as training samples to train a machine learning model to obtain a time Doppler image detection branch model. The machine learning model may be a convolutional neural network (CNN).

[0058] Optionally, the pre-trained time Doppler image detection branch model can also be set in the electronic device by a technician. The technician can use a preset number of time Doppler images and the first human fall detection results corresponding to each of the time Doppler images to train the machine learning model to obtain the time Doppler image detection branch model, and then set the time Doppler image detection branch model in the electronic device.

[0059] Optionally, according to the time Doppler map, the range Doppler map, the distance-time map and a pre-trained multi-branch attention network, a first human fall detection result, a second human fall detection result and a third human fall detection result in the target area are determined, including: inputting the range Doppler map into a range Doppler map detection branch model in a pre-trained multi-branch attention network, and obtaining a second human fall detection result in the target area output by the range Doppler map detection branch model; wherein the input of the range Doppler map detection branch model is a range Doppler map corresponding to an ultra-wideband radar echo signal of a human body in the target area, the output of the range Doppler map detection branch model is a second human fall detection result in the target area, and the second human fall detection result in the target area is a fall state of a human body in the target area determined according to the range Doppler map corresponding to the ultra-wideband radar echo signal of a human body in the target area, and the fall state is falling or not falling.

[0060] Optionally, a pre-trained range Doppler map detection branch model is used to analyze and detect the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and determine the second human body fall detection result in the target area. The human body in the target area is the human body corresponding to the range Doppler map. The input of the range Doppler map detection branch model is the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the output of the range Doppler map detection branch model is the second human body fall detection result in the target area. The second human body fall detection result in the target area is the fall state of the human body in the target area determined based on the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area. The fall state of the human body is falling or not falling.

[0061] Optionally, the electronic device can input the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area into a pre-trained range Doppler map detection branch model, and the pre-trained range Doppler map detection branch model will analyze and detect the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, determine the second human fall detection result of the target area, and then output the second human fall detection result of the target area. The second human fall detection result of the target area output by the range Doppler map detection branch model is the fall state of the human body in the target area determined according to the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area. The electronic device can obtain the second human fall detection result of the target area output by the range Doppler map detection branch model.

[0062] Optionally, before obtaining the ultra-wideband radar echo signal of the human body in the target area through an ultra-wideband radar placed in the target area, it also includes: obtaining a preset number of range Doppler maps, and a second human fall detection result corresponding to each of the range Doppler maps, using each of the range Doppler maps and the second human fall detection result corresponding to each of the range Doppler maps as training samples, training the deep learning model, and obtaining a range Doppler map detection branch model.

[0063] Optionally, the preset number can be set according to business needs. Exemplarily, the preset number of range Doppler maps is 2000 range Doppler maps. Each range Doppler map in the preset number of range Doppler maps can be a different range Doppler map obtained by preprocessing the human body ultra-wideband radar echo signal received after reflection from different human bodies. The human body corresponding to each range Doppler map is the human body to which the human body ultra-wideband radar echo signal used to obtain each range Doppler map belongs. The second human body fall detection result corresponding to each range Doppler map is the fall state of the human body corresponding to each range Doppler map when the emitted electromagnetic wave pulse signal is reflected by the human body corresponding to each range Doppler map.

[0064] Optionally, obtaining a preset number of distance Doppler maps, and a second human fall detection result corresponding to each of the distance Doppler maps, includes: obtaining a preset number of distance Doppler maps sent by a technician, and a second human fall detection result corresponding to each of the distance Doppler maps. The technician may send a preset number of distance Doppler maps, and a second human fall detection result corresponding to each of the distance Doppler maps to an electronic device through a terminal device. The electronic device may obtain a preset number of distance Doppler maps, and a second human fall detection result corresponding to each of the distance Doppler maps, sent by the technician, and then use each of the distance Doppler maps and the second human fall detection result corresponding to each of the distance Doppler maps as training samples to train the deep learning model and obtain a distance Doppler map detection branch model.

[0065] Optionally, the pre-trained distance Doppler map detection branch model can also be set by a technician in an electronic device. The technician can use a preset number of distance Doppler maps and a second human fall detection result corresponding to each of the distance Doppler maps to train the deep learning model to obtain the distance Doppler map detection branch model, and then set the distance Doppler map detection branch model in the electronic device.

[0066] Optionally, according to the time Doppler map, the range Doppler map, the distance-time map and a pre-trained multi-branch attention network, a first human fall detection result, a second human fall detection result and a third human fall detection result in the target area are determined, including: inputting the distance-time map into a distance-time map detection branch model in a pre-trained multi-branch attention network, and obtaining a third human fall detection result in the target area output by the distance-time map detection branch model; wherein the input of the distance-time map detection branch model is a distance-time map corresponding to an ultra-wideband radar echo signal of a human body in the target area, the output of the distance-time map detection branch model is the third human fall detection result in the target area, and the third human fall detection result in the target area is a fall state of a human body in the target area determined according to the distance-time map corresponding to the ultra-wideband radar echo signal of a human body in the target area, and the fall state is falling or not falling.

[0067] Optionally, a pre-trained distance-time graph detection branch model is used to analyze and detect the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area, and determine the third human body fall detection result in the target area. The human body in the target area is the human body corresponding to the distance-time graph. The input of the distance-time graph detection branch model is the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the output of the distance-time graph detection branch model is the third human body fall detection result in the target area. The third human body fall detection result in the target area is the fall state of the human body in the target area determined based on the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area. The fall state of the human body is falling or not falling.

[0068] Optionally, the electronic device can input the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area into a pre-trained distance-time graph detection branch model, and the pre-trained distance-time graph detection branch model will analyze and detect the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area, determine the third-person fall detection result in the target area, and then output the third-person fall detection result in the target area. The third-person fall detection result in the target area output by the distance-time graph detection branch model is the fall state of the human body in the target area determined according to the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area. The electronic device can obtain the third-person fall detection result in the target area output by the distance-time graph detection branch model.

[0069] Optionally, before obtaining the ultra-wideband radar echo signal of the human body in the target area through an ultra-wideband radar placed in the target area, it also includes: obtaining a preset number of distance-time graphs, and a third human fall detection result corresponding to each of the distance-time graphs, using each of the distance-time graphs and the third human fall detection result corresponding to each of the distance-time graphs as training samples, training the deep learning model, and obtaining a distance-time graph detection branch model.

[0070] Optionally, the preset number can be set according to business needs. Exemplarily, the preset number of distance-time diagrams is 2000 distance-time diagrams. Each distance-time diagram in the preset number of distance-time diagrams can be a different distance-time diagram obtained by preprocessing the human body ultra-wideband radar echo signal received after reflection from different human bodies. The human body corresponding to each distance-time diagram is the human body to which the human body ultra-wideband radar echo signal used to obtain each distance-time diagram belongs. The third human body fall detection result corresponding to each distance-time diagram is the fall state of the human body corresponding to each distance-time diagram when the emitted electromagnetic wave pulse signal is reflected by the human body corresponding to each distance-time diagram.

[0071] Optionally, obtaining a preset number of distance-time graphs, and a third-person fall detection result corresponding to each of the distance-time graphs, includes: obtaining a preset number of distance-time graphs sent by a technician, and a third-person fall detection result corresponding to each of the distance-time graphs. The technician may send a preset number of distance-time graphs, and a third-person fall detection result corresponding to each of the distance-time graphs to an electronic device through a terminal device. The electronic device may obtain a preset number of distance-time graphs sent by the technician, and a third-person fall detection result corresponding to each of the distance-time graphs, and then use each of the distance-time graphs and the third-person fall detection result corresponding to each of the distance-time graphs as training samples to train a deep learning model to obtain a distance-time graph detection branch model.

[0072] Optionally, the pre-trained distance-time graph detection branch model can also be set in the electronic device by a technician. The technician can use a preset number of distance-time graphs and a third person fall detection result corresponding to each of the distance-time graphs to train the deep learning model to obtain the distance-time graph detection branch model, and then set the distance-time graph detection branch model in the electronic device.

[0073] Step 104: Determine a final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result, and the third human fall detection result.

[0074] Optionally, the final fall detection result of the target area is the final information used to characterize whether the human body in the target area has fallen. The final fall detection result of the target area is fall or no fall. The final fall detection result of the target area is fall, indicating that the human body in the target area has fallen. The final fall detection result of the target area is no fall, indicating that the human body in the target area has not fallen.

[0075] Optionally, the time Doppler graph corresponding to the ultra-wideband radar echo signal of the human body in the target area can characterize the speed change of the human body in the target area at different time points. The distance Doppler graph corresponding to the ultra-wideband radar echo signal of the human body in the target area can characterize the motion characteristics of the human body in the target area. The distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area can characterize the change in the distance between the human body in the target area and the radar. The first human body fall detection result in the target area is the fall state of the human body in the target area determined based on the time Doppler graph corresponding to the ultra-wideband radar echo signal of the human body in the target area. The second human body fall detection result in the target area is the fall state of the human body in the target area determined based on the distance Doppler graph corresponding to the ultra-wideband radar echo signal of the human body in the target area. The third human body fall detection result in the target area is the fall state of the human body in the target area determined based on the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area. Generally, if any two of the first human fall detection result, the second human fall detection result, and the third human fall detection result in the target area are no falls, it indicates that the probability that the human body in the target area has not fallen is relatively high, and it can be determined that the human body in the target area has not fallen. If any two of the first human fall detection result, the second human fall detection result, and the third human fall detection result in the target area are falls, it indicates that the probability that the human body in the target area has fallen is relatively high, and it can be determined that the human body in the target area has fallen.

[0076] Optionally, the final human fall detection result of the target area is determined according to the first human fall detection result, the second human fall detection result and the third human fall detection result, including: if any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are no fall, then the final human fall detection result of the target area is determined to be no fall.

[0077] Optionally, if any two of the first human fall detection result, the second human fall detection result, and the third human fall detection result are no fall, indicating that the probability that the human body in the target area has not fallen is high, it can be determined that the human body in the target area has not fallen, and the final human fall detection result of the target area is determined to be no fall.

[0078] Optionally, the final human fall detection result of the target area is determined according to the first human fall detection result, the second human fall detection result and the third human fall detection result, including: if any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are falls, then the final human fall detection result of the target area is determined to be a fall.

[0079] Optionally, if any two of the first human fall detection result, the second human fall detection result, and the third human fall detection result are falls, it indicates that the probability that the human body in the target area has fallen is relatively high, and it can be determined that the human body in the target area has fallen, and the final human body fall detection result of the target area is determined to be a fall.

[0080] Optionally, after determining the final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result and the third human fall detection result, it also includes: sending the final fall detection result of the target area to a technician's terminal device.

[0081] Optional, Figure 2 A schematic diagram of a human fall detection process based on a multi-branch attention network and an ultra-wideband radar provided in Example 1 of the present invention. The time Doppler map is input into the time Doppler map detection branch model to obtain a first human fall detection result of the target area output by the time Doppler map detection branch model. The distance Doppler map is input into the distance Doppler map detection branch model to obtain a second human fall detection result of the target area output by the distance Doppler map detection branch model. The distance-time map is input into the distance-time map detection branch model to obtain a third human fall detection result of the target area output by the distance-time map detection branch model. Then, according to the first human fall detection result, the second human fall detection result and the third human fall detection result, the final human fall detection result of the target area is determined. The final human fall detection result of the target area is falling or not falling.

[0082] The technical solution of the embodiment of the present invention is to obtain the ultra-wideband radar echo signal of the human body in the target area through an ultra-wideband radar placed in the target area, pre-process the ultra-wideband radar echo signal of the human body in the target area, and obtain the time Doppler map, distance Doppler map and distance time map corresponding to the ultra-wideband radar echo signal of the human body in the target area; then, according to the time Doppler map, the distance Doppler map, the distance time map and the pre-trained multi-branch attention network, determine the first human body fall detection result, the second human body fall detection result and the third human body fall detection result in the target area; the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a distance Doppler map detection branch model and a distance time map detection branch model; finally, according to The first human fall detection result, the second human fall detection result and the third human fall detection result are used to determine the final fall detection result of the target area, which solves the problem that the fall detection scheme in the related technology performs fall detection on the human body in the specified area based on a single posture feature, and cannot comprehensively and accurately perform fall detection on the human body in the specified area to determine whether the human body in the specified area falls. The human body in the specified area can be automatically detected based on a multi-branch attention network and an ultra-wideband radar to determine whether the human body in the specified area falls. The human body in the specified area can be comprehensively and accurately detected from multiple dimensions of time Doppler map, distance Doppler map and distance-time map to determine whether the human body in the specified area falls.

[0083] In an optional implementation of the embodiment of the present invention, optionally, the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area can be expressed as:

[0084]

[0085] Where s(t) is the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area, V TX is the pulse amplitude of the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area, t is the fast time, and f c It is the center frequency of the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area. f B It is the bandwidth of the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area.

[0086] The electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area is reflected by the human body in the target area and received by the radar receiver. The received signal is the ultra-wideband radar echo signal of the human body in the target area. The ultra-wideband radar echo signal of the human body in the target area can be expressed as:

[0087]

[0088] Among them, x m (t) is the human ultra-wideband radar echo signal in the target area. The human ultra-wideband radar echo signal in the target area comes from K directions. K is the total number of directions of the human ultra-wideband radar echo signal in the target area. m，k is the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area and the reflection coefficient in the kth path direction, k=1,2...K, s(t) is the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area, t is the fast time, τ m，n is the delay time of the ultra-wideband radar echo signal of the human body in the target area, and w(t) is the noise from the surrounding environment.

[0089] The human ultra-wideband radar echo signal in the target area is discretized to obtain the human ultra-wideband radar echo signal after discretization. The nth sampling point in the human ultra-wideband radar echo signal after discretization can be expressed as:

[0090]

[0091] in, is the nth sampling point in the discretized human ultra-wideband radar echo signal, m represents the range index, n represents the time index, and β m , k is the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area and the reflection coefficient in the kth path direction, k = 1, 2...K, e is a natural number, j is an imaginary unit, f c is the center frequency of the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area, T m , k is the delay time of the human ultra-wideband radar echo signal in the target area in the kth path direction, s[m, nT s -σ m，n ] is the mth row and nth row in the matrix representation of the electromagnetic wave pulse signal emitted by the ultra-wideband radar placed in the target area to the human body in the target area. s -σ m，n Elements of the column, T s is the sampling interval, τ m，n is the delay time of the ultra-wideband radar echo signal of the human body in the target area, and N[n] is the matrix representation of the noise from the surrounding environment.

[0092] The human ultra-wideband radar echo signal after discretization is subjected to clutter suppression to obtain the human ultra-wideband radar echo signal after clutter suppression. The human ultra-wideband radar echo signal after clutter suppression can be expressed as:

[0093] Y(m,n)=X(m,n)-C(m,n),

[0094] Among them, Y(m, n) is the ultra-wideband radar echo signal of the human body after clutter suppression, X(m, n) is the ultra-wideband radar echo signal of the human body after discretization processing, C(m, n) is the background clutter, m represents the range index, and n represents the time index.

[0095] If the discretized human ultra-wideband radar echo signal is subjected to clutter suppression by the phasor mean cancellation algorithm, the human ultra-wideband radar echo signal after clutter suppression is obtained. The background clutter can be expressed as:

[0096]

[0097] Among them, C(m, n) is the background clutter, X(m, n) is the discretized human ultra-wideband radar echo signal, m represents the range index, n represents the time index, and M is the length of the slow time dimension.

[0098] If the moving average algorithm is used to suppress the clutter of the discretized human ultra-wideband radar echo signal, the human ultra-wideband radar echo signal after clutter suppression is obtained. The background clutter can be expressed as:

[0099] C(m+1,n)=αC(m,n)+(1-α)X(m,n),

[0100] Among them, C(m, n) is the background clutter, X(m, n) is the discretized human ultra-wideband radar echo signal, m represents the range index, n represents the time index, and α is the clutter suppression factor.

[0101] The human ultra-wideband radar echo signal after clutter suppression is subjected to short-time Fourier transform to obtain the time Doppler map corresponding to the human ultra-wideband radar echo signal in the target area. The time Doppler map corresponding to the human ultra-wideband radar echo signal in the target area can be expressed as:

[0102]

[0103] Among them, Y TD (k,f d ) is the time Doppler map corresponding to the human ultra-wideband radar echo signal in the target area, m represents the range index, n represents the time index, Y(m,n) is the human ultra-wideband radar echo signal after clutter suppression, l is the preset length, w(l) is the Gaussian window function with a length of l, and fd is the Doppler frequency, e is a natural number, j is an imaginary unit, and f r is the preset frequency.

[0104] The human ultra-wideband radar echo signal after clutter suppression is discretely transformed along the slow time dimension to obtain the range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area. The range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area can be expressed as:

[0105]

[0106] Among them, Y RD (m, n) is the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, m represents the range index, n represents the time index, Y(m, n) is the ultra-wideband radar echo signal of the human body after clutter suppression, e is a natural number, i is an imaginary unit, k is the number of points in the slow time dimension (Doppler dimension), and M is the length of the slow time dimension.

[0107] The final range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area can be obtained using the feature matrix extraction method. The final range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area can be expressed as:

[0108]

[0109] Among them, Y RD0 (m, n) is the final range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area, Y RD (m, n) is the range Doppler map corresponding to the human body ultra-wideband radar echo signal in the target area obtained by performing discrete Fourier transform on the human body ultra-wideband radar echo signal after clutter suppression along the slow time dimension, α=μ+bσ, μ is the mean of the range Doppler map corresponding to the human body ultra-wideband radar echo signal in the target area obtained by performing discrete Fourier transform on the human body ultra-wideband radar echo signal after clutter suppression along the slow time dimension, σ is the standard deviation of the range Doppler map corresponding to the human body ultra-wideband radar echo signal in the target area obtained by performing discrete Fourier transform on the human body ultra-wideband radar echo signal after clutter suppression along the slow time dimension, and b is a preset coefficient.

[0110] Embodiment 2

[0111] Figure 3 The flowchart of a method for detecting human falls based on a multi-branch attention network and ultra-wideband radar is provided in the second embodiment of the present invention. The embodiment of the present invention can be combined with each optional solution in one or more of the above embodiments. Figure 3 As shown, the method includes:

[0112] Step 201: Acquire an ultra-wideband radar echo signal of a human body in a target area through an ultra-wideband radar placed in the target area.

[0113] Step 202: pre-process the human body ultra-wideband radar echo signal in the target area to obtain a time Doppler map, a range Doppler map, and a range-time map corresponding to the human body ultra-wideband radar echo signal in the target area.

[0114] Step 203: input the time Doppler image into a time Doppler image detection branch model in a pre-trained multi-branch attention network to obtain a first human fall detection result in the target area output by the time Doppler image detection branch model.

[0115] Step 204: input the range Doppler map into a range Doppler map detection branch model in a pre-trained multi-branch attention network to obtain a second human fall detection result of the target area output by the range Doppler map detection branch model.

[0116] Step 205: input the distance-time graph into a distance-time graph detection branch model in a pre-trained multi-branch attention network to obtain a third person fall detection result in the target area output by the distance-time graph detection branch model.

[0117] Step 203, step 204 and step 205 are executed in parallel. After step 203, step 204 and step 205 are executed in parallel, step 206 is executed.

[0118] Step 206: Determine a final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result, and the third human fall detection result.

[0119] Optionally, the final human fall detection result of the target area is determined according to the first human fall detection result, the second human fall detection result and the third human fall detection result, including: if any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are no fall, then the final human fall detection result of the target area is determined to be no fall.

[0120] Optionally, the final human fall detection result of the target area is determined according to the first human fall detection result, the second human fall detection result and the third human fall detection result, including: if any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are falls, then the final human fall detection result of the target area is determined to be a fall.

[0121] The technical solution of the embodiment of the present invention can automatically perform fall detection on a human body in a specified area based on a multi-branch attention network and an ultra-wideband radar to determine whether a human body in the specified area has fallen. The fall detection result of a human body in a target area can be determined based on a time Doppler map, a distance Doppler map, and a distance-time map, respectively. Then, based on each fall detection result, it is determined whether a human body in the target area has fallen. Fall detection can be performed on a human body in a specified area comprehensively and accurately from multiple dimensions to determine whether a human body in the specified area has fallen.

[0122] Embodiment 3

[0123] Figure 4 The schematic diagram of the structure of a human fall detection device based on a multi-branch attention network and an ultra-wideband radar provided in the third embodiment of the present invention. The device can be configured in an electronic device. Figure 4 As shown, the device includes: a signal acquisition module 301, a signal processing module 302, a network processing module 303 and a result determination module 304.

[0124] Among them, the signal acquisition module 301 is used to obtain the ultra-wideband radar echo signal of the human body in the target area through the ultra-wideband radar placed in the target area; the signal processing module 302 is used to pre-process the ultra-wideband radar echo signal of the human body in the target area to obtain the time Doppler map, the distance Doppler map and the distance time map corresponding to the ultra-wideband radar echo signal of the human body in the target area; the network processing module 303 is used to determine the first human fall detection result, the second human fall detection result and the third human fall detection result of the target area according to the time Doppler map, the distance Doppler map, the distance time map and the pre-trained multi-branch attention network; wherein the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a distance Doppler map detection branch model and a distance time map detection branch model; the result determination module 304 is used to determine the final human fall detection result network processing module of the target area according to the first human fall detection result, the second human fall detection result and the third human fall detection result.

[0125] The technical solution of the embodiment of the present invention is to obtain the ultra-wideband radar echo signal of the human body in the target area through an ultra-wideband radar placed in the target area, pre-process the ultra-wideband radar echo signal of the human body in the target area, and obtain the time Doppler map, distance Doppler map and distance time map corresponding to the ultra-wideband radar echo signal of the human body in the target area; then, according to the time Doppler map, the distance Doppler map, the distance time map and the pre-trained multi-branch attention network, determine the first human body fall detection result, the second human body fall detection result and the third human body fall detection result in the target area; the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a distance Doppler map detection branch model and a distance time map detection branch model; finally, according to The first human fall detection result, the second human fall detection result and the third human fall detection result are used to determine the final fall detection result of the target area, which solves the problem that the fall detection scheme in the related technology performs fall detection on the human body in the specified area based on a single posture feature, and cannot comprehensively and accurately perform fall detection on the human body in the specified area to determine whether the human body in the specified area falls. The human body in the specified area can be automatically detected based on a multi-branch attention network and an ultra-wideband radar to determine whether the human body in the specified area falls. The human body in the specified area can be comprehensively and accurately detected from multiple dimensions of time Doppler map, distance Doppler map and distance-time map to determine whether the human body in the specified area falls.

[0126] In an optional implementation manner of an embodiment of the present invention, optionally, the network processing module 303 is specifically used to: input the time Doppler map into the time Doppler map detection branch model in a pre-trained multi-branch attention network, and obtain the first human fall detection result of the target area output by the time Doppler map detection branch model; wherein the input of the time Doppler map detection branch model is the time Doppler map corresponding to the human ultra-wideband radar echo signal in the target area, and the output of the time Doppler map detection branch model is the first human fall detection result in the target area, and the first human fall detection result in the target area is the fall state of the human body in the target area determined according to the time Doppler map corresponding to the human ultra-wideband radar echo signal in the target area, and the fall state is falling or not falling.

[0127] In an optional implementation manner of an embodiment of the present invention, optionally, the network processing module 303 is specifically used to: input the range Doppler map into the range Doppler map detection branch model in a pre-trained multi-branch attention network, and obtain the second human fall detection result of the target area output by the range Doppler map detection branch model; wherein the input of the range Doppler map detection branch model is the range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area, and the output of the range Doppler map detection branch model is the second human fall detection result in the target area, and the second human fall detection result in the target area is the fall state of the human body in the target area determined according to the range Doppler map corresponding to the human ultra-wideband radar echo signal in the target area, and the fall state is falling or not falling.

[0128] In an optional implementation manner of an embodiment of the present invention, optionally, the network processing module 303 is specifically used to: input the distance-time graph into a distance-time graph detection branch model in a pre-trained multi-branch attention network, and obtain a third human fall detection result of the target area output by the distance-time graph detection branch model; wherein the input of the distance-time graph detection branch model is a distance-time graph corresponding to an ultra-wideband radar echo signal of a human body in the target area, and the output of the distance-time graph detection branch model is a third human fall detection result of the target area, and the third human fall detection result of the target area is a fall state of a human body in the target area determined according to the distance-time graph corresponding to the ultra-wideband radar echo signal of a human body in the target area, and the fall state is falling or not falling.

[0129] In an optional implementation of an embodiment of the present invention, optionally, the result determination module 304 is specifically used to: if any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are no fall, then determine that the final human fall detection result of the target area is no fall.

[0130] In an optional implementation manner of an embodiment of the present invention, optionally, the result determination module 304 is specifically used to: if any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are falls, then determine that the final human fall detection result of the target area is a fall.

[0131] In an optional implementation of an embodiment of the present invention, optionally, the human fall detection device based on a multi-branch attention network and an ultra-wideband radar further includes: a first model training module, used to obtain a preset number of time Doppler images, and a first human fall detection result corresponding to each of the time Doppler images, and use each of the time Doppler images and the first human fall detection result corresponding to each of the time Doppler images as training samples to train a machine learning model to obtain a time Doppler image detection branch model; a second model training module, used to obtain a preset number of distance Doppler images, and a first human fall detection result corresponding to each of the time Doppler images. The second human fall detection result corresponding to the distance Doppler map, each of the distance Doppler maps and the second human fall detection result corresponding to each of the distance Doppler maps are used as training samples to train the deep learning model to obtain a distance Doppler map detection branch model; a third model training module is used to obtain a preset number of distance-time maps, and a third human fall detection result corresponding to each of the distance-time maps, each of the distance-time maps and the third human fall detection result corresponding to each of the distance-time maps are used as training samples to train the deep learning model to obtain a distance-time map detection branch model.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] The above-mentioned human fall detection device based on multi-branch attention network and ultra-wideband radar can execute the human fall detection method based on multi-branch attention network and ultra-wideband radar provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the human fall detection method based on multi-branch attention network and ultra-wideband radar.

[0134] Embodiment 4

[0135] Figure 5 The structure diagram of the electronic device 10 that can be used to implement the human fall detection method based on multi-branch attention network and ultra-wideband radar according to an embodiment of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0136] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program built from the storage unit 18 to the random access memory (RAM) 13. In the random access memory (RAM) 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the read-only memory (ROM) 12 and the random access memory (RAM) 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] A number of components in the electronic device 10 are connected to an input / output (I / O) interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0138] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a human fall detection method based on a multi-branch attention network and an ultra-wideband radar.

[0139] In some embodiments, the human fall detection method based on the multi-branch attention network and the ultra-wideband radar can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the read-only memory (ROM) 12 and / or the communication unit 19. When the computer program is built into the random access memory (RAM) 13 and executed by the processor 11, one or more steps of the human fall detection method based on the multi-branch attention network and the ultra-wideband radar described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the human fall detection method based on the multi-branch attention network and the ultra-wideband radar in any other appropriate manner (for example, by means of firmware).

[0140] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The computer program for implementing the human fall detection method based on multi-branch attention network and ultra-wideband radar of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as an independent software package, or completely on a remote machine or server.

[0142] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0145] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A human fall detection method based on multi-branch attention network and ultra-wideband radar, characterized in that: include: Acquiring an ultra-wideband radar echo signal of a human body in the target area by placing an ultra-wideband radar in the target area; Preprocessing the human body ultra-wideband radar echo signal in the target area to obtain a time Doppler map, a range Doppler map and a range-time map corresponding to the human body ultra-wideband radar echo signal in the target area; Determine a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network; wherein the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a range Doppler map detection branch model, and a range-time map detection branch model; A final human fall detection result of the target area is determined according to the first human fall detection result, the second human fall detection result and the third human fall detection result.

2. The human fall detection method based on multi-branch attention network and ultra-wideband radar according to claim 1 is characterized in that: Determining a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network, including: Inputting the time Doppler image into a time Doppler image detection branch model in a pre-trained multi-branch attention network to obtain a first human fall detection result of the target area output by the time Doppler image detection branch model; Among them, the input of the time Doppler map detection branch model is the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the output of the time Doppler map detection branch model is the first human fall detection result in the target area. The first human fall detection result in the target area is the fall state of the human body in the target area determined according to the time Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the fall state is falling or not falling.

3. The human fall detection method based on multi-branch attention network and ultra-wideband radar according to claim 1 is characterized in that: Determining a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network, including: Inputting the range Doppler map into a range Doppler map detection branch model in a pre-trained multi-branch attention network to obtain a second human fall detection result of the target area output by the range Doppler map detection branch model; Among them, the input of the range Doppler map detection branch model is the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the output of the range Doppler map detection branch model is the second human body fall detection result in the target area. The second human body fall detection result in the target area is the fall state of the human body in the target area determined according to the range Doppler map corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the fall state is falling or not falling.

4. The human fall detection method based on multi-branch attention network and ultra-wideband radar according to claim 1 is characterized in that: Determining a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network, including: Inputting the distance-time graph into a distance-time graph detection branch model in a pre-trained multi-branch attention network, and obtaining a third person fall detection result in the target area output by the distance-time graph detection branch model; Among them, the input of the distance-time graph detection branch model is the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the output of the distance-time graph detection branch model is the third human body fall detection result in the target area. The third human body fall detection result in the target area is the fall state of the human body in the target area determined according to the distance-time graph corresponding to the ultra-wideband radar echo signal of the human body in the target area, and the fall state is falling or not falling.

5. The human fall detection method based on multi-branch attention network and ultra-wideband radar according to claim 1 is characterized in that: Determining a final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result, and the third human fall detection result includes: If any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are no fall, then the final human fall detection result of the target area is determined to be no fall.

6. The human fall detection method based on multi-branch attention network and ultra-wideband radar according to claim 1 is characterized in that: Determining a final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result, and the third human fall detection result includes: If any two of the first human fall detection result, the second human fall detection result and the third human fall detection result are falls, then the final human fall detection result of the target area is determined to be a fall.

7. The human fall detection method based on multi-branch attention network and ultra-wideband radar according to claim 1 is characterized in that: Before obtaining the ultra-wideband radar echo signal of the human body in the target area by placing the ultra-wideband radar in the target area, the method further includes: Obtaining a preset number of time Doppler images and a first human fall detection result corresponding to each of the time Doppler images, using each of the time Doppler images and the first human fall detection result corresponding to each of the time Doppler images as training samples, training a machine learning model, and obtaining a time Doppler image detection branch model; Obtaining a preset number of range Doppler images and a second human fall detection result corresponding to each of the range Doppler images, using each of the range Doppler images and the second human fall detection result corresponding to each of the range Doppler images as training samples, training a deep learning model, and obtaining a range Doppler image detection branch model; A preset number of distance-time graphs and a third-person fall detection result corresponding to each of the distance-time graphs are obtained, and each of the distance-time graphs and the third-person fall detection result corresponding to each of the distance-time graphs are used as training samples to train the deep learning model to obtain a distance-time graph detection branch model.

8. A human fall detection device based on multi-branch attention network and ultra-wideband radar, characterized in that: include: A signal acquisition module, used to acquire an ultra-wideband radar echo signal of a human body in the target area through an ultra-wideband radar placed in the target area; A signal processing module is used to pre-process the human body ultra-wideband radar echo signal in the target area to obtain a time Doppler map, a range Doppler map and a range-time map corresponding to the human body ultra-wideband radar echo signal in the target area; A network processing module, used to determine a first human fall detection result, a second human fall detection result, and a third human fall detection result in the target area according to the time Doppler map, the range Doppler map, the range-time map, and a pre-trained multi-branch attention network; wherein the pre-trained multi-branch attention network includes a time Doppler map detection branch model, a range Doppler map detection branch model, and a range-time map detection branch model; A result determination module is used to determine a final human fall detection result of the target area according to the first human fall detection result, the second human fall detection result and the third human fall detection result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the human fall detection method based on a multi-branch attention network and an ultra-wideband radar as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the human fall detection method based on a multi-branch attention network and an ultra-wideband radar as described in any one of claims 1 to 7 when executed.

Images