Method and system for tracking a living object

Abstract

A method for tracking one or more living objects using at least one processor is provided. The method includes: receiving signals transmitted from a plurality of transmitting antennas via a plurality of receiving antennas; identifying a position of each of the one or more living objects relative to a discrete time based on the signals received in a number of sampling periods; determining a movement of each of the one or more living objects based on the identified location; determining one or more vital signs of each of the one or more living objects based on the movement of each of the one or more living objects; and collectively tracking the location and the one or more vital signs of each of the one or more living subjects.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to Singapore Patent Application No. 10202203217Q filed on February 29, 2022, the contents of which are hereby incorporated by reference in their entirety for all purposes. Technical Field

[0002] The present disclosure generally relates to methods and systems for tracking one or more living subjects. Background Art

[0003] In smart home and office environments, the presence, location and behavior of a person are monitored based on his or her vital signs (VS). VS itself can lead to many intelligent functions and therefore there is a large demand for it. For example, by detecting the VS of a person in a bedroom via a sensor embedded in an air conditioner, the sleep quality of the sleeper can be recorded and presented to him or her, and the air conditioner temperature can also be intelligently adjusted based on the detected VS. In the bathroom, fall (behavior) detection with privacy protection is widely needed for accident alarms. In the office, by understanding information about human presence and location based on VS, office resources can be managed more efficiently.

[0004] Currently, VS detection requires contact sensors. It is not user-friendly because wearing sensors may not be convenient for all users in any environment, and transmitting the detected VS to a server requires a wired connection or a wireless transmitter. Another type of VS monitor based on video images requires video capture, which is strongly hindered in bedrooms and prohibited in bathrooms.

[0005] Therefore, there is a need to provide a method and system for tracking one or more living objects, which seeks to overcome or at least improve one or more deficiencies in conventional tracking methods and systems, and more specifically seeks to improve efficiency (e.g., improve convenience) and / or effectiveness (e.g., enhance tracking accuracy). Summary of the invention

[0006] According to a first aspect of the present disclosure, a method for tracking one or more living objects using at least one processor is provided, the method comprising: receiving signals transmitted from multiple transmitting antennas via multiple receiving antennas; identifying the position of each of the one or more living objects relative to discrete time based on the signals received in several sampling periods; determining the movement of each of the one or more living objects based on the identified position; determining one or more vital signs of each of the one or more living objects based on the movement of each of the one or more living objects; and jointly tracking the position and the one or more vital signs of each of the one or more living objects.

[0007] According to a second aspect of the present disclosure, a method for tracking one or more living objects using at least one processor is provided, the method comprising: receiving signals transmitted from multiple transmitting antennas via multiple receiving antennas; sampling the received signals to generate a sequence of received signal values ​​for each pair of transmitting antennas and receiving antennas; generating frequency domain values ​​by applying Fourier transform to the sequence of received signal values; detecting peaks in the frequency domain values ​​relative to discrete time based on signals received in a number of sampling periods; determining a phase disturbance of each detected peak to generate an unwrapped phase angle; performing bandpass filtering on the generated unwrapped phase angle of each peak; and determining that the peak corresponds to a living object if the measured value of the filtered unwrapped phase angle of the peak is higher than a predetermined threshold.

[0008] According to a third aspect of the present disclosure, there is provided a system for tracking a living object, the system comprising: at least one memory; and at least one processor, the at least one processor being communicatively coupled to the at least one memory and configured to execute the method as described herein.

[0009] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium comprising instructions executable by at least one processor to perform the method as described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments of the present disclosure will be better understood and will become readily apparent to those skilled in the art from the following written description, which is given by way of example only, in conjunction with the accompanying drawings, in which: Figure 1 depicts a schematic flow chart of an example method for tracking one or more living objects according to various embodiments of the present disclosure; Figure 2 depicts a schematic flow chart of an example method for tracking one or more living objects according to various embodiments of the present disclosure; Figure 3 depicts a schematic block diagram of an example system for tracking one or more living objects according to various embodiments of the present disclosure; Figure 4 depicts a schematic block diagram of an example system for tracking one or more living objects according to various embodiments of the present disclosure; Figure 5 Describes the various embodiments in which the present disclosure may be implemented or practiced. Figure 3 The system shown, Figure 4 The first system shown or Figure 4 A schematic block diagram of an exemplary computer system of the second system shown; Figure 6 depicts a schematic block diagram of an example method for tracking one or more living objects according to various example embodiments of the present disclosure; Figure 7 depicts a schematic block diagram of an example method for tracking one or more living objects according to various example embodiments of the present disclosure; and Figure 8 A schematic block diagram depicts an example deep learning architecture used in an example method for tracking one or more living objects according to various example embodiments of the present disclosure. DETAILED DESCRIPTION

[0011] The following specific embodiments refer to the accompanying drawings that show the specific details and aspects of the present disclosure in an illustrative manner. One or more aspects are described in sufficient detail to enable those skilled in the art to practice the present disclosure. Without departing from the scope of the present disclosure, other aspects may be utilized and structural, logical and / or electrical changes may be made. Various aspects of the present disclosure are not necessarily mutually exclusive, because some aspects may be combined with one or more other aspects to form new aspects or embodiments. Various aspects are described in conjunction with methods and various aspects are described in conjunction with devices. However, it is understood that the various aspects described in conjunction with methods may be similarly applied to devices, and vice versa.

[0012] It should be understood that the singular terms "a", "an", and "the" include plural referents unless the context clearly indicates otherwise. Similarly, the word "or" is intended to include "and" unless the context clearly indicates otherwise.

[0013] It will be further understood that the terms "comprise" (and any form of "comprise", such as "comprises" and "comprising"), "have" (and any form of "have", such as "has" and "having"), "include" (and any form of "include", such as "includes" and "including"), and "contain" (and any form of "contain", such as "contains" and "containing") are open-ended linking verbs. Thus, a method or apparatus that "comprises," "has," "includes," or "contains" one or more steps or elements possesses those one or more steps or elements, but is not limited to possessing only those one or more steps or elements. Similarly, a method step or apparatus element that "comprises," "has," "includes," or "contains" one or more features possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that is configured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.

[0014] As used herein, phrases in the form “at least one of A or B” may include A or B or both A and B. Correspondingly, phrases in the form “at least one of A or B or C” or including further listed items may include any and all combinations of one or more of the associated listed items.

[0015] The term “exemplary” may be used herein to mean “serving as an example, instance, or illustration.” Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0016] The terms "at least one" and "one or more" may be understood to include a numerical quantity greater than or equal to one (e.g., one, two, three, four, […], etc.). The term "plurality" may be understood to include a numerical quantity greater than or equal to two (e.g., two, three, four, five, […], etc.). The phrase "at least one of..." with respect to a group of elements may be used herein to indicate at least one element from the group consisting of the elements. For example, the phrase "at least one of..." with respect to a group of elements may be used herein to indicate a selection of: one of the listed elements, one of a plurality of listed elements, a plurality of individually listed elements, or a plurality of plural listed elements.

[0017] The words "plurality" and "multiple" in the specification and claims unambiguously refer to quantities greater than one. Therefore, any phrase that unambiguously invokes the above words (e.g., "plurality of (objects)", "plurality of (objects)") referring to quantities of objects unambiguously refers to more than one of the objects. If any, the terms "group", "set", "series", "sequence", "grouping", etc. in the specification and claims refer to quantities equal to or greater than one (i.e., one or more).

[0018] Unless otherwise specified, the terms "first", "second", and "third" as used herein are used to distinguish one element from another similar element and may not necessarily indicate order or relative importance. For example, first transaction data and second transaction data may be used to distinguish two transactions based on two different foreign currency exchanges.

[0019] As used herein, the term "data" may be understood to include information in any suitable analog or digital form, for example, provided as a file, a portion of a file, a collection of files, a portion of a signal or stream, a collection of signals or streams, etc. In addition, the term "data" may also be used to represent a reference to information in the form of, for example, a pointer. However, the term "data" is not limited to the above examples and may take various forms and represent any information understood in the art. As described herein, any type of information may be processed in an appropriate manner, for example, as data, for example, via one or more processors.

[0020] For example, as used herein, the term "processor" or "controller" may be understood as any type of entity that allows processing of data. Data may be processed according to one or more specific functions performed by a processor or controller. In addition, as used herein, a processor or controller may be understood as any type of circuit, for example, any type of analog or digital circuit. A processor or controller may therefore be or include an analog circuit, a digital circuit, a mixed signal circuit, a logic circuit, a processor, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an integrated circuit, an application specific integrated circuit (ASIC), etc., or any combination thereof. Any other type of specific implementation of the corresponding functions described in further detail below may also be understood as a processor, a controller, or a logic circuit. It should be understood that any two (or more) of the processors, controllers, or logic circuits described in detail herein may be implemented as a single entity having equivalent functions, etc., and conversely, any single processor, controller, or logic circuit described in detail herein may be implemented as two (or more) separate entities having equivalent functions, etc.

[0021] The term "memory" as detailed herein may be understood to include any appropriate type of memory or memory device, such as a hard disk drive (HDD), a solid state drive (SSD), flash memory, and the like.

[0022] The term "module" as detailed herein refers to or forms part of or includes: an application specific integrated circuit (ASIC); an electronic circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; other suitable hardware components that provide the functionality; or a combination of some or all of the above, such as in a system on a chip. The term module may include a memory (shared, dedicated, or group) that stores code executed by a processor.

[0023] The distinction between software and hardware implemented data processing can become blurred. The processors, controllers and / or circuits detailed herein may be implemented in software, hardware and / or as a hybrid implementation including software and hardware.

[0024] The term "system" (e.g., transaction facilitator system, computing system, etc.) as detailed herein may be understood as a collection of interacting elements, where, by way of example and not limitation, an element may be one or more mechanical components, one or more electrical components, one or more instructions (e.g., encoded in a storage medium), and / or one or more processors, etc.

[0025] Unless otherwise specifically noted, and as will be apparent from the following, it will be understood that throughout this specification, descriptions or discussions utilizing terms such as "execute," "sample," "generate," "determine," "detect," and the like refer to actions and processes of a computer system or similar electronic device that manipulates data represented as physical quantities within the computer system and transforms it into other data similarly represented as physical quantities within the computer system or other information storage, transmission, or display device.

[0026] Some portions of the present disclosure are presented, either explicitly or implicitly, in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the art of data processing to most effectively convey the substance of their work to others skilled in the art. An algorithm is generally considered here to be a self-consistent sequence of steps leading to a desired result. These steps are steps requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

[0027] In the present disclosure, a multiple-input multiple-output (MIMO) vital sign (VS) radar system is proposed, which can identify multiple human bodies from all major targets in a room, and locate the corresponding human bodies and estimate VS, including the breathing rate and heartbeat of each human body. The proposed system can provide comprehensive detection including target type recognition, positioning and VS estimation. In addition, a unique joint positioning and VS estimation solution can utilize the coherent information between the movement speed of a human body and its VS, resulting in reliable detection.

[0028] In the present disclosure, a mmWave MIMO VS radar is disclosed. The proposed mmWave MIMO VS radar can solve the technical needs and overcome the shortcomings of existing solutions. It is compact enough to be integrated into common consumer electronic appliances in residential homes and offices, such as air conditioners, refrigerators, TVs, computers, etc. It is very convenient for large consumer electronics manufacturers to add value to their existing products by combining with the solution proposed in this article.

[0029] The following examples relate to various aspects of the disclosure.

[0030] Example 1 is a method for tracking one or more living objects using at least one processor and includes: receiving signals transmitted from multiple transmitting antennas via multiple receiving antennas; identifying the position of each of the one or more living objects relative to discrete time based on the signals received in several sampling periods; determining the movement of each of the one or more living objects based on the identified position; determining one or more vital signs of each of the one or more living objects based on the movement of each of the one or more living objects; and jointly tracking the position and the one or more vital signs of each of the one or more living objects.

[0031] In Example 2, the subject matter of Example 1 may optionally include: the one or more vital signs comprising a breathing rate and / or a heartbeat of the one or more living subjects.

[0032] In Example 3, the subject matter of Example 2 may optionally include: the one or more vital signs further comprising a rate of change of the breathing rate and / or a rate of change of the heartbeat of the one or more living subjects.

[0033] In Example 4, the subject matter of Example 1 can optionally include that the movement of each of the one or more living objects includes information related to a speed of each of the one or more living objects.

[0034] In Example 5, the subject matter of Example 4 may optionally include: the information related to the velocity of each of the one or more living objects includes a Doppler velocity of each of the one or more living objects and a direction of arrival (DOA) estimate of each of the one or more living objects.

[0035] In Example 6, the subject matter of Example 2 may optionally include: determining one or more vital signs of each of the one or more living objects includes: performing low-pass filtering to obtain information related to the breathing rate of the one or more living objects and / or performing high-pass filtering to obtain information related to the heartbeat of the one or more living objects; and determining the breathing rate of the one or more living objects by finding a peak in the information related to the breathing rate of the one or more living objects during the sampling period and / or determining the heartbeat of the one or more living objects by finding a peak in the information related to the heartbeat of the one or more living objects during the sampling period.

[0036] In Example 7, the subject matter of Example 6 can optionally include performing the low pass filtering and the high pass filtering at the same frequency.

[0037] In Example 8, the subject matter of Example 6 may optionally include: if the low-pass filtered information is greater than a predetermined ratio of the average value of the low-pass filtered information during the sampling period, determining an intermediate parameter equal to the low-pass filtered information related to the respiratory rate of the one or more living objects, and if the low-pass filtered information is less than or equal to a predetermined ratio of the average value of the low-pass filtered information, determining the intermediate parameter to be zero; and / or if the high-pass filtered information is greater than a predetermined ratio of the average value of the high-pass filtered information during the sampling period, determining an intermediate parameter equal to the high-pass filtered information related to the heartbeat of the one or more living objects, and if the high-pass filtered information is less than or equal to a predetermined ratio of the average value of the high-pass filtered information, determining the intermediate parameter to be zero.

[0038] In Example 9, the subject matter of Example 1 can optionally include detecting a fall by using the convolutional neural network (CNN).

[0039] In Example 10, the subject matter of Example 1 may optionally include: identifying the position of each of the one or more living objects based on the signal received in several sampling periods includes: sampling the received signal to generate a sequence of received signal values ​​for each pair of transmitting antennas and receiving antennas; generating frequency domain values ​​by applying Fourier transform to the sequence of received signal values; generating a radar imaging result by applying a spatial spectrum recovery algorithm to the frequency domain values ​​to generate a first radar imaging result; detecting peaks in the radar imaging result; determining the phase disturbance of each detected peak to generate an unwrapped phase angle; performing bandpass filtering on the generated unwrapped phase angle of each peak; and if the measured value of the filtered unwrapped phase angle of the peak is higher than a predetermined threshold, determining that the peak corresponds to a living object.

[0040] In Example 11, the subject matter of Example 1 may optionally include: displaying the location of each of the one or more living objects in a target location map; and indicating the location of each of the one or more non-living objects in the target location map.

[0041] In Example 12, the subject matter of Example 1 can optionally include: displaying the waveform of the vital sign of the one or more living subjects according to the estimated velocity of the one or more living subjects.

[0042] In Example 13, the subject matter of Example 1 may optionally include: for each of the one or more living objects: simultaneously identifying the position relative to the discrete time based on the signals received in several sampling periods; simultaneously determining the movement based on the identified position; simultaneously determining the one or more vital signs based on the movement; and using Kalman filtering to simultaneously and jointly track the position and the one or more vital signs.

[0043] Example 14 is a method for tracking one or more living objects using at least one processor, the method comprising: receiving signals transmitted from multiple transmitting antennas via multiple receiving antennas; sampling the received signals to generate a sequence of received signal values ​​for each pair of transmitting antennas and receiving antennas; generating frequency domain values ​​by applying Fourier transform to the sequence of received signal values; detecting peaks in the frequency domain values ​​relative to discrete time based on signals received in several sampling periods; determining a phase disturbance of each detected peak to generate an unwrapped phase angle; performing bandpass filtering of the generated unwrapped phase angle of each peak; and determining that the peak corresponds to a living object if the measured value of the filtered unwrapped phase angle of the peak is higher than a predetermined threshold.

[0044] In Example 15, the subject matter of Example 14 may optionally include: before detecting the peak in the frequency value, generating a radar imaging result by applying a spatial spectrum recovery algorithm to the frequency domain value to generate a first radar imaging result; wherein the spatial spectrum recovery algorithm gives a preliminary radar imaging result, and generating the radar imaging result also includes applying a constant false alarm (CFAR) algorithm to the preliminary radar imaging result.

[0045] In Example 16, the subject matter of Example 14 can optionally include determining a Doppler domain value of the frequency domain value.

[0046] In Example 17, the subject matter of Example 14 may optionally include: low-pass filtering the filtered expanded phase angle using a third frequency to generate a raw respiratory rate; modulating the respiratory rate data to be equal to the raw respiratory rate if the raw respiratory rate is greater than a predetermined ratio of an average value of the raw respiratory rate during a sampling period, and determining the respiratory rate data to be zero if the raw respiratory rate is less than or equal to a predetermined ratio of the average value of the raw respiratory rate during the sampling period; and determining the respiratory rate of the living object by finding peaks in the respiratory rate data during the sampling period and setting the size of the number of peaks in the respiratory rate data.

[0047] In Example 18, the subject matter of Example 14 may optionally include: high-pass filtering the filtered expanded phase angle using a fourth frequency to generate a raw heartbeat; modulating the heartbeat data to be equal to the raw heartbeat if the raw heartbeat is greater than a predetermined ratio of an average value of the raw heartbeat during a sampling period, and modulating the heartbeat data to zero if the raw heartbeat is less than or equal to a predetermined ratio of the average value of the raw heartbeat during the sampling period; and determining the heartbeat of a living object by finding peaks in the heartbeat data during the sampling period and setting the size of the number of peaks in the heartbeat data.

[0048] In Example 19, the subject matter of Example 14 may optionally include: forming a zoomed-in range time map and a zoomed-in range Doppler map; and processing the maps by a deep learning structure.

[0049] Example 20 is a system for tracking a living object, the system comprising: at least one memory; and at least one processor, the at least one processor being communicatively coupled to the at least one memory and configured to execute a method according to any one of Examples 1 to 13 or a method according to any one of Examples 14 to 19.

[0050] Example 21 is a non-transitory computer-readable storage medium comprising instructions executable by at least one processor to perform a method according to any one of Examples 1 to 13 or a method according to any one of Examples 14 to 19.

[0051] Figure 1 A schematic flow chart of a method 100 for tracking one or more living objects according to various embodiments of the present disclosure is depicted. The method 100 includes: receiving (at step 102) signals transmitted from multiple transmitting antennas via multiple receiving antennas; identifying (at step 104) the position of each of the one or more living objects relative to discrete time based on the signals received in a number of sampling periods; determining (at step 106) the movement of each of the one or more living objects based on the identified position; determining (at step 108) one or more vital signs of each of the one or more living objects based on the movement of each of the one or more living objects; and using filtering to jointly track (at step 110) the position and one or more vital signs of each of the one or more living objects.

[0052] According to various non-limiting embodiments, the plurality of receiving antennas and the plurality of transmitting antennas may include a radar antenna that receives and transmits radar waves (e.g., microwaves), and thus the signal may be a radar signal (e.g., radar waves). The plurality of receiving antennas may be used as the plurality of transmitting antennas, that is, the plurality of receiving antennas may be the same as the plurality of transmitting antennas. The signals transmitted by the plurality of transmitting antennas may reach an object, which in turn reflects or scatters the signals, and the reflected or scattered signals may be received by the plurality of receiving antennas. The received signal may contain information about the object, including but not limited to the distance and speed of the object.

[0053] According to various non-limiting implementations, the multiple receive antennas and the multiple transmit antennas may follow a multiple-input and multiple-output (MIMO) approach. This may mean using multiple transmit antennas and receive antennas to exploit multipath propagation to multiply the capacity of a radio link.

[0054] According to various non-limiting embodiments, an analog-to-digital (ADC) converter may be used to convert the continuous time and continuous amplitude analog signals from the antenna into discrete time and discrete amplitude digital signals. The digital signals in a number of sampling periods may be processed by the method described herein to obtain the position of each of the one or more living objects relative to the discrete time at the beginning of the number of sampling periods. In other words, the position of each of the one or more living objects relative to the discrete time may be obtained based on the digital signals received in the number of sampling periods.

[0055] According to various non-limiting embodiments, the movement of each of the one or more living objects may be determined based on the identified location (e.g., based on signals received over a number of sampling periods relative to discrete time). In some embodiments, the movement may include information related to the velocity of the living object, for example, the velocity of the living object detected by multiple receiving antennas. In some embodiments, the movement may include information related to Doppler velocity determined by applying a fast Fourier transform (FFT) to the identified location in the Doppler domain and a direction of arrival (DOA) estimate for each of the one or more living objects. The Doppler FFT may be used for target movement velocity and represents velocity relative to the radar rather than velocity along the X-axis and Y-axis.

[0056] According to various non-limiting embodiments, one or more vital signs of each of the one or more living objects may be determined based on the movement of each of the one or more living objects. The one or more vital signs of the living object may include breathing rate, heartbeat, pulse rate, respiratory rate, etc., or the rate of change of the above. The signal may be further processed (e.g., smoothed, modulated) by removing noise and interference. The signal may also be processed by bandpass filtering (e.g., high-pass filtering and / or low-pass filtering) to separate information related to each of the one or more vital signs.

[0057] In various embodiments, determining one or more vital signs of each of the one or more living objects may include: performing low pass filtering to obtain information related to the respiratory rate of the one or more living objects; and determining the respiratory rate of the one or more living objects by finding a peak in the information related to the respiratory rate of the one or more living objects during a sampling period. In addition, determining the one or more vital signs of each of the one or more living objects may include: if the low pass filtered information is greater than a predetermined ratio of the average value of the low pass filtered information during the sampling period, then determining an intermediate parameter equal to the low pass filtered information related to the respiratory rate of the one or more living objects; and if the low pass filtered information is less than or equal to a predetermined ratio of the average value of the low pass filtered information, then determining the intermediate parameter to be zero.

[0058] In various embodiments, determining one or more vital signs of each of the one or more living objects may include: performing high pass filtering to obtain information related to the heartbeat of the one or more living objects; and determining the heartbeat of the one or more living objects by finding a peak in the information related to the heartbeat of the one or more living objects during a sampling period. In addition, determining the one or more vital signs of each of the one or more living objects may include: if the high pass filtered information is greater than a predetermined ratio of the average value of the high pass filtered information during the sampling period, determining an intermediate parameter equal to the high pass filtered information related to the heartbeat of the one or more living objects; and if the high pass filtered information is less than or equal to a predetermined ratio of the average value of the high pass filtered information, determining the intermediate parameter to be zero.

[0059] In various embodiments, the low pass filtering and the high pass filtering may be performed at the same frequency, for example, at a frequency of 0.5 Hz.

[0060] According to various non-limiting embodiments, the position and one or more vital signs of each of one or more living objects may be tracked jointly or jointly. In some embodiments, the movement of each of one or more living objects and / or the rate of change of one or more vital signs may be tracked jointly or jointly. Tracking may include estimating the position and one or more vital signs of each of one or more living objects by estimated variance or uncertainty. In some embodiments, Kalman filtering may be used to track the position and one or more vital signs of each of one or more living objects jointly or jointly. Specifically, Kalman filtering may track position, breathing rate, heartbeat, and movement by tracking the speed of object movement and the rate of change of the breathing rate and heartbeat of one or more living objects relative to subsequent discrete times. That is, Kalman filtering may update position, breathing rate, heartbeat, and movement by tracking the speed of object movement and the rate of change of the breathing rate and heartbeat of one or more living objects relative to consecutive discrete times based on signals received in several sampling cycles starting from consecutive discrete times.

[0061] According to various non-limiting embodiments, method 100 may include detecting falls by using a convolutional neural network (CNN). Method 100 may include tracking sudden changes in the speed of an object moving. A sudden change may mean a change in speed that exceeds a threshold. The threshold may be predetermined or machine-learned based on a received signal.

[0062] In various embodiments, identifying the position of each of the one or more living objects relative to discrete time based on the signal received in several sampling periods may include: sampling the received signal to generate a sequence of received signal values ​​for each pair of transmitting antennas and receiving antennas; generating frequency domain values ​​by applying Fourier transform to the sequence of received signal values; generating a radar imaging result by applying a spatial spectrum recovery algorithm to the frequency domain values ​​to generate a first radar imaging result; detecting peaks in the radar imaging result; determining the phase disturbance of each detected peak to generate an unwrapped phase angle; performing bandpass filtering on the generated unwrapped phase angle of each peak; and if the measured value of the filtered unwrapped phase angle of the peak is higher than a predetermined threshold, determining that the peak corresponds to a living object.

[0063] In various embodiments, method 100 may further include: displaying the location of each of the one or more living objects in a target location map; and indicating the location of each of the one or more non-living objects in the target location map. The target location map may include a radar imaging map.

[0064] In various embodiments, method 100 may further include displaying a waveform of the vital signs of the one or more living objects according to the estimated velocity of the one or more living objects. The waveform of the vital signs of the one or more living objects may be formed by beamforming based on the received signals. The estimated velocity of the one or more living objects may include the jointly or jointly tracked position and one or more vital signs of each of the one or more living objects.

[0065] In various embodiments, the method may include, for each of the one or more living objects: simultaneously identifying the position relative to the discrete time based on the signal received in a number of sampling periods; simultaneously determining the movement based on the identified position; simultaneously determining the one or more vital signs based on the movement; and using Kalman filtering to simultaneously and jointly track the position and the one or more vital signs. "Simultaneously" is intended to include simultaneously tracking each of the one or more objects, for example, the received signal at each discrete time (e.g., the signal received in a number of sampling periods starting from each discrete time) includes information for each of the one or more objects; it is also intended to include tracking each of the one or more objects at consecutive discrete times when the time interval between two immediately consecutive discrete times is set below a threshold.

[0066] Therefore, the method 100 for tracking one or more living objects according to various embodiments of the present disclosure advantageously enables joint tracking of the location and vital signs of one or more living objects in a more effective and efficient manner. As the method and system for tracking one or more living objects are described in more detail according to various embodiments and exemplary embodiments of the present disclosure, these advantages or technical effects and / or other advantages or technical effects will become more apparent to those skilled in the art.

[0067] Figure 2 A schematic flow chart of a method 200 for tracking one or more living objects according to various embodiments of the present disclosure is depicted. The method 200 includes: receiving (at step 202) signals transmitted from a plurality of transmit antennas via a plurality of receive antennas; sampling (at step 204) the received signals to generate a sequence of receive signal values ​​for each pair of transmit antennas and receive antennas; generating (at step 206) frequency domain values ​​by applying a Fourier transform to the sequence of receive signal values; detecting (at step 210) peaks in the frequency domain values ​​relative to discrete times in a number of sampling periods; determining (at step 212) a phase perturbation of each detected peak to generate an unwrapped phase angle; performing (at step 214) bandpass filtering of the generated unwrapped phase angle for each peak; and determining (at step 216) that the peak corresponds to a living object if the measured value of the filtered unwrapped phase angle of the peak is above a predetermined threshold.

[0068] Features described in the context of method 100 may correspondingly apply to the same or similar features in method 200. Furthermore, supplements and / or combinations and / or alternatives as described for features in the context of method 100 may correspondingly apply to the same or similar features in method 200.

[0069] According to various non-limiting embodiments, the plurality of transmit antennas may be the same as the plurality of receive antennas. Thus, the pair of transmit antennas and receive antennas may be the same antenna. That is, when a corresponding transmit antenna in the plurality of transmit antennas is used as a corresponding receive antenna in the plurality of receive antennas, a signal transmitted by the corresponding transmit antenna is received by the corresponding transmit antenna.

[0070] According to various non-limiting embodiments, the received signal may be sampled to generate a sequence of received signal values ​​for each pair of transmit antenna and receive antenna.

[0071] In various embodiments, the sequence of received signal values ​​may be reconstructed into an antenna array signal according to the MIMO principle (e.g., multiple receive antennas and multiple transmit antennas). This may mean that the received signal (e.g., the sequence of received signal values) may form a matrix, where the number of matrix rows is the number of receive antennas and the number of matrix columns is the number of transmit antennas. This may also mean performing vectorization on the matrix of the received signal to obtain a column vector as the reconstructed antenna array signal. The column vector may be obtained relative to samples output by an analog-to-digital conversion (ADC).

[0072] According to various non-limiting embodiments, generating frequency domain values ​​by applying a Fourier transform to a sequence of received signal values ​​may include generating frequency domain values ​​by applying a Fourier transform to a reconstructed antenna array signal. The Fourier transform may include a fast Fourier transform (FFT). A range FFT length may be preset. Applying a Fourier transform to a sequence of received signal values ​​may include applying a row-by-row 1-dimensional (1D) Fourier transform to the sequence of received signal values. That is, a Fourier transform is applied to a column vector of each channel of the antenna array (e.g., each row of a received signal matrix). The frequency domain values ​​may be generated with respect to blocks, each block taking a preset number of sequential samples.

[0073] According to various non-limiting embodiments, before detecting the peak in the frequency domain value, method 200 may further include: generating a radar imaging result by applying a spatial spectrum recovery algorithm to the frequency domain value to generate a first radar imaging result. The radar imaging result may be generated by applying a spatial spectrum recovery algorithm to the frequency domain value to generate a first radar imaging result. The radar imaging result may be generated relative to discrete time based on signals received in a number of sampling periods. The spatial spectrum recovery algorithm may include a process of obtaining spectral information and / or spatial information from the correlation data, such as a Capon method, a steered vector beamforming method.

[0074] In various embodiments, a steering vector may be determined to represent a set of phase delays of an incident wave at each receive antenna. The steering vector may be fixed for multiple receive antennas. The steering vector may correspond to the transmit frequencies of the multiple transmit antennas.

[0075] In various implementations, a frequency domain value matrix may be formed to contain elements of frequency domain values ​​relative to a sampling period.

[0076] In various embodiments, the spatial covariance matrix may be determined relative to discrete time based on signals received over a number of sampling periods.The spatial covariance matrix may be determined as a function of a frequency domain value matrix and a conjugate transpose of the frequency domain value matrix.

[0077] In various embodiments, a first radar imaging result (i.e., a first radar image mapped from the first radar imaging result to a 2D image) may be generated based on the steering vector, the frequency domain value matrix, and the spatial covariance matrix. The first radar imaging result (i.e., the first radar image) may be further generated based on the angular scanning step size of the antenna (e.g., 1 degree). The first radar image may be generated relative to discrete time and may be updated relative to each preset number of sequential samples from all antenna array channels. The size of the first radar image may correspond to the number of antennas and the number of sequential samples, for example, the product of the number of antennas and the number of sequential samples.

[0078] In various embodiments, the first radar imaging result may be further improved to generate a radar imaging result, for example, by a constant false alarm (CFAR) operation. The improvement of the first radar imaging result may include removing unwanted background noise signals by a threshold of the first radar imaging result within the current block size. Similarly, the radar imaging result (i.e., the radar image) may be generated relative to discrete time and may be updated relative to each preset number of sequential samples from all antenna array channels. The size of the radar image may correspond to the number of antennas and the number of sequential samples, for example, the product of the number of antennas and the number of sequential samples.

[0079] According to various non-limiting embodiments, the peak position in the radar image may be identified as a detection object (eg, a target position). The peak position may include the position of a living object and a non-living object.

[0080] According to various non-limiting embodiments, the information associated with the peak position may be further analyzed to determine whether the peak corresponds to a living object. The phase perturbation of each detected peak may be determined to generate an unwrapped phase angle. This may mean analyzing the phase perturbation (e.g., phase delay, phase shift) of the received signal during the sampling period.

[0081] According to various non-limiting embodiments, bandpass filtering using one or more frequencies may be performed for the generated unwrapped phase angle of each peak. Bandpass filtering may be performed using two cutoff frequencies (e.g., a lower limit (e.g., lowest) frequency and an upper limit (e.g., highest) frequency) to filter out unwanted signals, such as from non-living objects, as interference. Interference may include, but is not limited to, low-frequency drift caused by hardware circuits and high-frequency changes caused by non-living moving objects. For example, bandpass filtering may be performed using a first frequency of 0.1 Hz and a second frequency of 4 Hz. This may mean that any signal with a frequency of change below 0.1 Hz or above 4 Hz that is not possible from a living object (e.g., a human body) may be filtered out by bandpass filtering. In other words, after bandpass filtering using two cutoff frequencies (e.g., a lower limit (e.g., lowest) frequency and an upper limit (e.g., highest) frequency), signals related to vital signs of a living object (e.g., a human body) may be retained.

[0082] According to various non-limiting embodiments, if the measured value of the filtered unwrapped phase angle of the peak is above a predetermined threshold, it can be determined that the peak corresponds to a living object.The measured value of the filtered unwrapped phase angle can include taking the norm of the amount of the filtered unwrapped phase angle.

[0083] In various embodiments, method 200 may further include determining a Doppler domain value for the frequency domain value. The Doppler domain value may include a Doppler velocity.

[0084] In various embodiments, method 200 may also include: low-pass filtering the filtered expanded phase angle using a third frequency to generate a raw respiratory rate; modulating the respiratory rate data to be equal to the raw respiratory rate if the raw respiratory rate is greater than a predetermined ratio of the average value of the raw respiratory rate during the sampling period, and determining the respiratory rate data to be zero if the raw respiratory rate is less than or equal to a predetermined ratio of the average value of the raw respiratory rate during the sampling period; and determining the respiratory rate of the living object by finding peaks in the respiratory rate data during the sampling period and setting the size of the number of peaks in the respiratory rate data.

[0085] In various embodiments, method 200 may also include: high-pass filtering the filtered expanded phase angle using a fourth frequency to generate a raw heartbeat; modulating the heartbeat data to be equal to the raw heartbeat if the raw heartbeat is greater than a predetermined ratio of the average value of the raw heartbeat during the sampling period, and modulating the heartbeat data to zero if the raw heartbeat is less than or equal to a predetermined ratio of the average value of the raw heartbeat during the sampling period; and determining the heartbeat of the living object by finding a peak in the heartbeat data during the sampling period and setting the size of the number of the peaks of the heartbeat data.

[0086] In various embodiments, the third frequency may be the same as the fourth frequency. In various embodiments, the third frequency may be less than the fourth frequency, for example, by 0.05 Hz. Thus, applying low-pass filtering to obtain the respiratory rate and applying high-pass filtering to obtain the heartbeat can overcome the overlapping passband problem of the respiratory rate and the heartbeat.

[0087] Although the above methods 100, 200 are illustrated and described as a series of steps or events, it will be understood that any order of such steps or events should not be interpreted in a limiting sense. For example, some steps can occur in different orders and / or occur simultaneously with other steps or events other than those steps or events shown and / or described herein. In addition, all illustrated steps may not be required to realize one or more aspects or embodiments described herein. In addition, one or more steps described herein may be carried out in one or more separate actions and / or stages.

[0088] Figure 3 A schematic block diagram of a system 300 for tracking one or more living objects according to various embodiments of the present disclosure is depicted, the system corresponding to the system 300 described above with reference to FIG. Figure 1 The system 300 includes: at least one receiver 302; and at least one processor 304, the at least one processor being communicatively coupled to the at least one receiver 302 and configured to execute the method 100 for tracking one or more living objects as described above according to various embodiments of the present disclosure. Therefore, the at least one receiver 302 is configured to: receive (at step 102) signals transmitted from multiple transmitting antennas (e.g., receiver 302) via the receiver 302 (e.g., multiple receiving antennas). The signals may include signals including: Figure 3 Information related to all objects 30 of living objects 31, 32, 34, 35 and non-living objects (such as 34) in the region shown. Living objects 31 and / or 35 may move around in the region during the sampling period. It should be understood that Figure 3 Only exemplary areas are shown in which the system 300 is implemented, and any other areas are applicable, for example, a patient room, a bedroom, a bathroom, etc. in a hospital where it is desired to track the vital signs of a living subject.

[0089] Therefore, at least one processor 304 is configured to: identify (at step 104) the position of each of one or more living objects (e.g., living objects 31, 32, 34, 35) relative to discrete time based on signals received in a number of sampling periods; determine (at step 106) the movement of each of the one or more living objects based on the identified position; determine (at step 108) one or more vital signs of each of the one or more living objects based on the movement of each of the one or more living objects; and use filtering to jointly track (at step 110) the position and one or more vital signs of each of the one or more living objects.

[0090] Those skilled in the art will appreciate that the at least one processor 304 may be configured to perform various functions or operations through a set of instructions (e.g., software modules) that can be executed by the at least one processor 304 to perform various functions or operations. Therefore, the processor 304 of the system 300 may include: a data processing module (not shown) configured to process the signals received by the receiver 302 in order to track one or more living objects.

[0091] Figure 4 A schematic block diagram of a system 400 for tracking one or more living objects according to various embodiments of the present disclosure is depicted, the system corresponding to the system described above with reference to FIG. Figure 2 The method 200 for tracking one or more living objects described above. The system 400 includes: at least one memory 402; and at least one processor 404, the at least one processor is communicatively coupled to the at least one memory 402 and is configured to execute the method 200 for tracking one or more living objects as described above according to various embodiments of the present disclosure. Thus, at least one processor 404 is configured to: receive (at step 202) signals transmitted from multiple transmitting antennas via multiple receiving antennas; sample (at step 204) the received signals to generate a sequence of received signal values ​​for each pair of transmitting antennas and receiving antennas; generate (at step 206) frequency domain values ​​by applying a Fourier transform to the sequence of received signal values; detect (at step 210) peaks in the frequency domain values ​​relative to discrete times in a number of sampling periods; determine (at step 212) a phase perturbation of each detected peak to generate an unwrapped phase angle; perform (at step 214) bandpass filtering of the generated unwrapped phase angle for each peak; and if the measured value of the filtered unwrapped phase angle of the peak is above a predetermined threshold, determine (at step 216) that the peak corresponds to a living object.

[0092] Those skilled in the art will appreciate that the at least one processor 404 may be configured to perform various functions or operations through instruction sets (eg, software modules) that can be executed by the at least one processor 404 to perform various functions or operations. Figure 4 As shown, the system 400 may include: an input module (or input circuit) 412, which is configured to perform the above-mentioned receiving (at step 202) signals transmitted from multiple transmitting antennas via multiple receiving antennas; a data sampling module (or data sampling circuit) 414, which is configured to perform the above-mentioned sampling of the received signal (at step 204) to generate a sequence of received signal values ​​for each pair of transmitting antennas and receiving antennas; and a data processing module (or data processing circuit) 416, which is configured to perform the above-mentioned generating (at step 206) frequency domain values ​​by applying Fourier transform to the sequence of received signal values; detecting (at step 210) peaks in the frequency domain values ​​relative to discrete times in a number of sampling periods; determining (at step 212) a phase disturbance of each detected peak to generate an unwrapped phase angle; performing (at step 214) bandpass filtering of the generated unwrapped phase angle of each peak; and if the measured value of the filtered unwrapped phase angle of the peak is higher than a predetermined threshold, determining (at step 216) that the peak corresponds to a living object.

[0093] Those skilled in the art will appreciate that the various modules of the system are not necessarily separate modules, and two or more modules may be implemented by or as one functional module (e.g., circuit or software program) as needed or appropriate without departing from the scope of the present disclosure. For example, two or more modules (e.g., input module 412, data sampling module 414, and data processing module 416) of the system 400 for tracking one or more living objects may be implemented (e.g., compiled together) as one executable software program (e.g., software application or simply "app"), which, for example, may be stored in at least one memory 402 and may be executed by at least one processor 404 to perform various functions / operations as described herein according to various embodiments of the present disclosure.

[0094] Those skilled in the art will appreciate that the system may include additional modules, for example, system 400 may include a display module (not shown) configured to display the location of each of the one or more living objects in a target location map and / or display waveforms of vital signs of the one or more living objects according to the steps as described herein based on the estimated velocity of each of the one or more living objects.

[0095] In various embodiments, the system 300 for tracking one or more living objects may correspond to the system described above with reference to Figure 1The method 100 for tracking one or more living objects described, and therefore, the various functions or operations configured to be performed by the at least one processor 304 may correspond to the various steps or operations of the method 100 for tracking one or more living objects as described above according to various embodiments, and therefore, for the sake of clarity and conciseness, need not be repeated with respect to the system 300 for tracking one or more living objects. In other words, the various embodiments described herein in the context of the method are similarly valid for the corresponding system, and vice versa.

[0096] For example, in various embodiments, the system may also include at least one memory (not shown) storing therein a data processing module corresponding to one or more steps (or operations or functions) of method 100 for tracking one or more living objects as described herein according to various embodiments, and the one or more steps may be executed by at least one processor 304 to perform the corresponding functions or operations as described herein.

[0097] Similarly, in various embodiments, the system 400 for tracking one or more living objects corresponds to the system 400 described above with reference to the present invention according to various embodiments. Figure 2 The method 200 for tracking one or more living objects described, therefore, the various functions or operations configured to be performed by at least one processor 404 may correspond to the various steps or operations of the method 200 for tracking one or more living objects as described above according to various embodiments, and therefore, for the sake of clarity and conciseness, there is no need to repeat with respect to the system 400 for tracking one or more living objects.

[0098] For example, in various embodiments, at least one memory 402 of system 400 may store therein an input module 412, a data sampling module 414, and / or a data processing module 416 corresponding to one or more steps (or operations or functions) of method 200 for tracking one or more living objects as described herein according to various embodiments, which one or more steps may be executed by at least one processor 404 to perform the corresponding functions or operations as described herein.

[0099] According to various embodiments of the present disclosure, a computing system, a controller, a microcontroller, or any other system providing processing capabilities may be provided. Such systems may be employed to include one or more processors and one or more computer-readable storage media. For example, the system 400 for tracking one or more living objects described above may include at least one processor (or controller) 404 and at least one computer-readable storage medium (or memory) 402 as described herein, for example, for various processes performed therein. The memory or computer-readable storage medium used in various embodiments may be a volatile memory, such as a DRAM (dynamic random access memory) or a non-volatile memory, such as a PROM (programmable read-only memory), an EPROM (erasable PROM), an EEPROM (electrically erasable PROM), or a flash memory, such as a floating gate memory, a charge trap memory, an MRAM (magnetoresistive random access memory), or a PCRAM (phase change random access memory).

[0100] In various embodiments, a "circuit" may be understood as any type of logic implementation entity, which may be a dedicated circuit or processor that executes software stored in a memory, firmware, or any combination thereof. Thus, in one embodiment, a "circuit" may be a hardwired logic circuit or a programmable logic circuit, such as a programmable processor, for example a microprocessor (e.g., a complex instruction set computer (CISC) processor or a reduced instruction set computer (RISC) processor). A "circuit" may also be a processor that executes software, such as any type of computer program, such as a computer program using a virtual machine code (e.g., Java). According to various embodiments, any other type of specific implementation of the corresponding function may also be understood as a "circuit". Similarly, a "module" may be part of a system according to various embodiments, and may include a "circuit" as described above, or may be understood as any type of logic implementation entity.

[0101] The present disclosure also discloses various systems (e.g., each of which may also be embodied as a device or apparatus) for performing various operations / functions of the various methods described herein, such as a system 300 for tracking one or more living objects, a system 400 for tracking one or more living objects. Such systems may be specially constructed for the desired purpose, or may include general-purpose computing devices or other devices selectively activated or reconfigured by a computer program stored in a computer. The algorithms presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose machines may be used with computer programs according to the teachings of this article. Alternatively, it may be appropriate to construct more specialized devices to perform various method steps.

[0102] In addition, the present disclosure also discloses computer programs or software / functional modules at least implicitly, because it will be obvious to those skilled in the art that the individual steps of the various methods described herein can be implemented by computer code. The computer program is not intended to be limited to any specific programming language and its specific implementation. It should be understood that various programming languages ​​and their decoding can be used to implement the teachings of the present disclosure contained herein. In addition, the computer program is not intended to be limited to any specific control flow. There are many other variations of computer programs, which can use different control flows without departing from the scope of the present disclosure. It will be understood by those skilled in the art that the various modules described herein (e.g., the data processing module of the system 300, the input module 412, the data sampling module 414, the data processing module 416) can be implemented by a computer program or instruction set executable by a computer processor to perform a desired function. Software modules, or can be hardware modules as functional hardware units designed to perform desired functions. It will also be understood that a combination of hardware modules and software modules can be implemented.

[0103] In addition, two or more steps of the computer program / module or method described herein may be performed in parallel rather than sequentially. Such computer programs may be stored on any computer-readable medium. Computer-readable media may include storage devices, such as disks or optical disks, memory chips, or other storage devices suitable for interfacing with a computer. When loaded and executed on such computers, the computer program effectively generates a system or device that implements the various steps of the method described herein.

[0104] In various embodiments, a computer program product is provided, which is embodied in one or more computer-readable storage media (non-transitory computer-readable storage media), including executable by one or more computer processors to perform the various embodiments as described herein. Figure 1 The instructions of the method 100 described for tracking one or more living objects (e.g., the data processing module of the system 300). Therefore, the various computer programs or modules described herein can be stored in a computer program product, which can be used by the system (such as Figure 3 The system 300 for tracking one or more living objects is shown receiving the information for execution by at least one processor 304 of the system 300 to perform various functions.

[0105] Similarly, in various embodiments, a computer program product is provided, which is embodied in one or more computer-readable storage media (non-transitory computer-readable storage media), including executable by one or more computer processors to perform the various embodiments as described herein. Figure 2The instructions of the method 200 for tracking one or more objects described herein (e.g., input module 412, data sampling module 414, data processing module 416) can be stored in a computer program product, which can be used by the system described herein (such as Figure 4 The illustrated system 400 for tracking one or more living subjects receives information for execution by at least one processor 404 to perform various functions.

[0106] In various embodiments, system 300 and / or system 400 may each be implemented by any computer system (e.g., a desktop or portable computer system (e.g., a mobile device)) including at least one processor and at least one memory, such as, for example only and not limitation. Figure 5 Schematically illustrated example computer system 500. Various methods / steps or functional modules may be implemented as software, such as a computer program that executes within computer system 500 and instructs computer system 500 (specifically, one or more processors therein) to perform various functions or operations as described herein according to various embodiments. Computer system 500 may include system unit 502, input devices such as keyboard and / or touch screen 504 and mouse 506, and multiple output devices such as display 508. System unit 502 may be connected to computer network 512 via a suitable transceiver device 514 to enable access to, for example, the Internet or other network systems such as local area network (LAN) or wide area network (WAN). System unit 502 may include a processor 518, random access memory (RAM) 520, and read-only memory (ROM) 522 for executing various instructions. System unit 502 may also include several input / output (I / O) interfaces, such as I / O interface 524 to display device 508 and I / O interface 526 to keyboard 504. The components of the system unit 502 typically communicate via an interconnect bus 528 and in a manner known to those skilled in the art.

[0107] In order to easily understand the present disclosure and put it into practice, various example embodiments of the present disclosure will be described below only by way of example and not limitation. However, it will be understood by those skilled in the art that the present disclosure can be embodied in various forms or configurations and should not be construed as being limited to the example embodiments set forth below. On the contrary, these example embodiments are provided to make the present disclosure thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0108] Figure 6 6 is a schematic block diagram of a method 600 for tracking one or more living objects according to various example embodiments of the present disclosure. Figure 6The proposed VS detection and tracking method 600 shown may include comprehensive radar detection. The proposed method 600 may be a joint breathing rate, heartbeat and person position estimation and tracking. The method 600 may utilize the coherent information in these signals to increase the estimation reliability and accuracy. It may also overcome the overlapping passband problem of fixed breathing rate and heartbeat bandpass filters (BPFs).

[0109] At step 601, a plurality of receiving antennas receive signals transmitted from a plurality of transmitting antennas. The receiving antennas and the transmitting antennas may include radar antennas.

[0110] At step 603, multiple transmissions (T x ) antenna and multiple receiving (R x ) antennas to reconstruct the antenna array signal according to the multiple-input and multiple-output (MIMO) principle. ij ) is sampled to generate each pair of transmit antennas (T x ) and the receiving antenna (R x ) of the received signal values ​​(V(n)). For example, by using the basic reconstruction algorithm, the reconstructed antenna array signal is Where n represents the nth sample of the analog-to-digital converter (ADC) output, Vec(·) represents the vectorized operation, and And where r ij R x The corresponding value at antenna j is T x The received signal transmitted by antenna i. There are a total of T Tx antennas and R Rx antennas.

[0111] At step 605, frequency domain values ​​are generated by applying a Fourier transform (e.g., a Fast Fourier Transform (FFT)) to the sequence of received signal values. The range FFT length is set to M. From each channel (column vector) V of the antenna array, a block of M sequential samples of the chirp duration is taken and an FFT is performed. The kth range FFT output is in(·) (i) represents the ith channel from the antenna array, v (i) is the signal from channel i of V. k represents the kth block. is a row-by-row 1D-FFT operator and is the k-th range FFT input block.

[0112] At step 607, a radar imaging result is generated by applying a spatial spectrum recovery algorithm to the frequency domain values ​​to generate a first radar imaging result. This can be performed between antenna array channels. The spatial spectrum recovery algorithm may include steered vector beamforming, Capon, etc. The Capon method is described below as an example of spatial spectrum recovery.

[0113] The matrix H is formed as follows: Where Z is the time window size. (i) (k,m) is the k-th range FFT output R (i) The mth element of (k). S is set to the steering vector as shown in (6) below. The steering vector may represent the set of phase delays experienced by an input wave evaluated at a set of array elements (antennas). The kth radar image is where ∈ is the angular scan step size, which is set to 1 degree, and C(k,m)= H(k,m)H H (k,m) (8) is the spatial covariance matrix computed over Z sampling periods, and (·) H represents the conjugate transpose. image(k) is the preliminary result of radar imaging. The size of the radar image is It is updated with respect to every M sequential samples from all array channels.

[0114] Therefore, a radar image can be formed based on the frequency domain values ​​of the sequence of received signal values ​​from the receiving radar antenna. A spatial spectrum recovery algorithm can be applied to the frequency domain values ​​for radar imaging as a function of the phase delay set of received signal values ​​(e.g., steering vectors), and a spatial covariance matrix calculated over Z sampling periods based on the frequency domain values ​​and the conjugate transpose of the frequency domain values ​​to generate a preliminary result of radar imaging.

[0115] At step 609, a constant false alarm (CFAR) algorithm may be applied to the preliminary radar imaging results generated by the spatial spectrum recovery algorithm. As described herein, a unit average method is presented below as an example of a CFAR algorithm. Other CFAR algorithms may include complex algorithms that use more accurate and non-Gaussian statistical models of background noise to determine thresholds in different application scenarios.

[0116] In the 181×M matrix, scan from left to right, element by element and row by row, starting from the top left. For each element, image(k) The threshold c i,j is calculated as where Δ is the block size and is The number of events, abs() means taking the absolute value of each element, and mean() is the average operation of all elements. or / and When , the corresponding element is padded with zeros and ones. Zero padding can be performed to obtain faster speed and / or improved accuracy. Then the CFAR output image is an image with each element CFAR (k).

[0117] image CFAR (k) is the output of the radar imaging with unwanted background noise removed. It is a 181×M matrix and is updated with respect to every M sequential samples from all antenna array channels.

[0118] Therefore, another imaging function is applied to the preliminary results of radar imaging. The other imaging function may include determining the absolute value of an element in the preliminary results and determining the average of the absolute values ​​as a threshold. The absolute value may be determined relative to several elements (e.g., not all elements in the preliminary results) within a predetermined block size. For better results (e.g., in the case of identifying living objects), a larger block size may be preferred. The other imaging function may also include: if the absolute value of the element is greater than the threshold, determining another element equal to the absolute value of the element, and if the absolute value of the element is less than or equal to the threshold, determining another element equal to zero. The CFAR output image may be improved by removing unwanted background noise corresponding to zero-valued additional elements.

[0119] At step 610 , a radar image (ie, a CFAR output image) is output.

[0120] At step 611, in image CFAR (k) Perform 2D peak search in the matrix. image CFAR The peak position in (k) is the detection target position. image CFAR The detected peak in (k) can also be the peak in image(k). Set the number of detected targets to X. Detecting X targets is represented by P1, ... P n ,…,P X , and image CFAR P in (k) n The position of (k) is represented as <x n (k),y n (k)>, where x n (k)=range(Pn (k))sin(DoA(P n (k))), y n (k)=range(P n (k))cos(DoA(P n (k))). DoA represents the direction of arrival estimation. Therefore, as shown in (10), the position of the detected target can be obtained by performing the direction of arrival (DoA) estimation on the CFAR output image.

[0121] At step 612, a target location map showing the target location (eg, mapped by peak finding) is output.

[0122] At step 613, the phase perturbation (ie, v′) of each detected peak at each discrete time k in a plurality of detection times (Z sampling periods) is determined. n (k)) to generate the unwrapped phase angle. The phase perturbation at each detected target position along time index k in image(k) is calculated as follows: v n (k)=∠(S H (DoA(P n (k)))X n (k)) (11) Where ∠(·) represents the phase angle of the complex number “·”, and X n (k) = [r (1) (k,range(P n (k))),…,r (TR) (k,range(P n (k)))] T S represents the position of the X target relative to discrete time k. H is the conjugate transpose of the steering factor S in equation (6).

[0123] The sliding window is expanded by the window size Z as follows: v′ n (k) = V z , (12) where V z is the zth element of the vector V and V=unwrap([v n (k),…,v n (k+Z-1)]), (13) and where unwrap(·) means unwrapping the radian phase angle in a vector.

[0124] Thus, determining the phase perturbation of each detected peak may include determining the phase perturbation for each of the X targets (p1, ...P n ,…,PX ) is a DoA estimate of the position of ). Determining the phase disturbance of each detected peak may also include determining as a phase disturbance for each of the X targets (P1, ...P n ,…,P X )’s position, each of the X targets (P1,…P n ,…,P X ) and the guidance factor S H The conjugate transpose of the phase angle of the function, and the unfolding of the phase angle with respect to frequency (e.g., restoring the physical continuity of the phase diagram).

[0125] At step 615, bandpass filtering (BPF) is performed on the generated unwrapped phase angle of each peak. x (k) Bandpass filtering can be performed using the first frequency f1 = 0.1 Hz and the second frequency f2 = 4 Hz: v″ n (k) = BPF(v′ n (k)) (14)

[0126] At step 615a, if the measured value of the filtered unwrapped phase angle of the peak is above a predetermined threshold (γ), then it is determined that the peak corresponds to a living object. n (k) (e.g., n = 1 to X), if ||V″ n (k)||>γ, (15) in V″ n (k) = [v" n (k),…,v″ n (k+Z-1)] (16) and γ is the VS threshold. Then the target n is marked as a living object (step 618), otherwise the target n is marked as a non-living object (step 616). In the positioning map, all target positions (P1, ...P n ,…,P X ) and label its corresponding type (living object or non-living object).

[0127] Thus, the measurement of the peak filtered unwrapped phase angle may include taking the norm of the magnitude of the filtered unwrapped phase angle relative to a discrete time k during the Z sampling periods.

[0128] At step 606, a Doppler domain value (eg, moving object speed) is determined for the frequency domain value: Where [°] TDenotes the transposed matrix "°". Then, the Doppler velocities of targets 1 to X are determined by their Doppler FFTs at their ranges, respectively. The Doppler FFT results can be used as in equation (27)

[0129] At steps 617 and 620, for the BPF signals of all living object targets, the third frequency f is used to perform the L =0.5Hz low pass filter (LPF) and using the fourth frequency f H =0.5Hz high pass filter (HPF). The output is B n,m (k) = LPF(v″ n (k)), (18) and H n,m (k) = HPF(v″ n (k)), (19) Where n is the index of the target, m is the index of the living object target, and it is assumed that X′ living object targets are detected.

[0130] make and in and mean(·) means averaging across all elements of the vector “·”. Then, the initial detection of the number of respirations and heartbeats during the Z sampling period is and Where findpeaks(·) means finding peaks in the vector “·”, and size(·) means the number of elements in the vector “·”.

[0131] Therefore, if the original respiratory rate (B n,m (k)) is greater than the original respiration rate (B) during the sampling period (Z) n,m The average value of (k) The respiratory rate data (B′ n,m (k)) can be modulated to be equal to the original breathing rate (B n,m (k)). If the original respiration rate (B n,m(k)) is less than or equal to the original respiration rate during the sampling period (B n,m The average value of (k) of a predetermined ratio (e.g., 0.5), the respiratory rate (B n ' ,m (k)) can be modulated to zero. The breathing rate of the living subject The determination may be made by finding peaks in the respiration rate data during a sampling period and setting the size of the number of peaks in the respiration rate data.

[0132] Therefore, if the original heartbeat (H n,m (k)) is greater than the original heartbeat (H) during the sampling period (Z) n,m The average value of (k) The heartbeat data (H′ n,m (k)) can be modulated to be equal to the original heartbeat (H n,m (k)). If the original heartbeat (H n,m (k)) is less than or equal to the original heartbeat during the sampling period (H n,m The average value of (k) The heart rate (H′ n,m (k)) can be modulated to zero. Heartbeat of a living subject The determination may be made by finding peaks in the heartbeat data during a sampling period and setting the magnitude of the number of peaks in the heartbeat data.

[0133] It should be understood that although the predetermined ratio is shown as 0.5 in equations (20) and (21), other ratios may be determined as required, such as 0.45, 0.4 or 0.6, or within the range of 0.45 to 0.55.

[0134] The determined breathing rate and heartbeat are output to counter 1 (step 622) and counter 2 (step 619), respectively.

[0135] At step 623, for each living subject target n, Kalman filtering is used to perform joint position, respiration rate, and heartbeat tracking.

[0136] Let the vehicle's state vector be Where k represents the discrete time k, x n (k) and y n (k) represents the position of target n projected onto the X-axis and Y-axis at discrete time k, respectively. The duration between any discrete time k and k-1 is β, which is Z times the sampling duration, and They are the speeds at which the target moves projected onto the X-axis and Y-axis respectively. and is the breathing rate and heartbeat of the living object target n initially detected by the radar sensor relative to discrete time k, and and is the rate of change of the breathing rate and heartbeat of the living object n relative to discrete time k. The state vector These elements (including x n (k) and y n (k) and and and ) can be updated in the following equation (27). Since the state vector includes heartbeat, respiratory rate, their rate of change, and the object position movement speed, the state vector can establish the relationship between the movement speed and the vital sign value, and data fusion is achieved by tracking the state vector including the object speed. In other words, by tracking the state vector including multiple interrelated data sets, the proposed method provides more accurate and consistent detection.

[0137] Therefore, the state vector relative to discrete time k+1 is expressed as: in

[0138] The control vector is and Where D n is the target P detected by the radar in equation (17) n Doppler velocity, DoA(P n ) is detected by radar n DoA. And in is the measurement vector from the radar at discrete time k.

[0139] Recursive process: Predictive (a priori) state estimates: Predicted (a priori) estimated covariance: P(k|k-1) = FP(k-1|k-1)F T +Q Measure the pre-fit residuals: Innovation (or pre-fit residual) covariance: s(k) = HP(k|k-1)H T +R Optimal Kalman gain: Kal(k) = P(k|k-1)H T s(k) -1 Updated (a posteriori) state estimate: Updated (a posteriori) estimated covariance: P(k|k) = (I-Kal(k)H)P(k|k-1)

[0140] Then, the Kalman tracking output of target n is middle.

[0141] Thus, method 600 can track the position of one or more living objects (e.g., objects 1, ..., n, ..., X) (the position of object n projected onto the X-axis and Y-axis at discrete time k) and the respiration rate and heartbeat of the one or more living objects relative to discrete time k ( and In addition, the method 600 can also track the speed of the target moving projected onto the X-axis and the Y-axis ( and ) to track the movement of one or more moving living objects. Method 600 can also track the rate of change of the breathing rate and heartbeat of the living object target n relative to the discrete time k ( and ).

[0142] At step 625, the positions of one or more living subject targets and the estimated VS including breathing rate and heartbeat are output, for example to display the positions in a target position map.

[0143] At step 626, the one or more non-living object targets are indicated by showing the locations of the one or more non-living object targets.

[0144] At step 627, the waveforms of the breathing rate and the heartbeat of the one or more living subject targets are reconstructed based on the estimated rates of the one or more living subject targets, respectively.

[0145] At step 629, waveforms of the breathing rate and heartbeat of one or more living subject targets are displayed.

[0146] Figure 7A schematic block diagram of a method 700 for tracking one or more living objects according to various example embodiments of the present disclosure is depicted. Method 700 may include similar steps to method 600, for example, steps 701, 703, 705, 706, 707, 709, 711, 713, 715, and 717 are similar to steps 601, 603, 605, 606, 607, 609, 611, 613, 615, and 615a, and therefore similar steps do not need to be discussed. Features described in the context of method 600 may be correspondingly applicable to the same or similar features in method 700. In addition, supplements and / or combinations and / or alternatives as described for features in the context of method 600 may be correspondingly applicable to the same or similar features in method 700.

[0147] At step 719, a zoomed-range Doppler composite map is formed.

[0148] At step 721, a zoomed-in time composite graph is formed.

[0149] At step 723, the Figure 8 The described deep learning structure is used to analyze the zoomed-in range Doppler composite image from step 719 and the zoomed-in range time composite image from step 721.

[0150] Fall detection can be achieved by inference by the designed AI model instead of manually setting the criteria. Before the AI ​​model infers a fall or no fall, the AI ​​model can be trained with a real data set including radar signals with fall or normal conditions. After training, the AI ​​model can directly infer whether there is a fault with the new data received.

[0151] At step 725, the fall detection result is output.

[0152] The method 700 may be applied in fall detection as a prerequisite to first detecting a human subject before activating a fall detection determination. That is, when the MIMO VS radar system detects a target as a human subject and locates the human subject, fall detection may be activated at a specific location of the human subject. The method 700 may help alleviate the need for wearable devices, sensors, or video monitoring for detecting falls, where such sensors are not suitable, for example, in wet environments, or in showers where video monitoring raises privacy issues.

[0153] According to various non-limiting embodiments, if the position in the DoA range plane <DoA(P n ),range(P n )>, target P n If the subject is identified as a living object, the magnified composite image time range, time Doppler, and range Doppler can be formed as follows.

[0154] Target P in the DoA range plane n The waveform at the ADC sample k around the position is in is the steering vector (i.e., the same as equation (6)), and And r (i) (k,m) is the range FFT output R at sample k of antenna channel i as defined in equation (3) (i) (k) is the mth element, and Δ r (even number) is the range window of the graph. Then the target P n The range time graph is where abs(·) represents the absolute value of each element of the matrix “·”, and the target P n The range of the Doppler map is in is a column-by-column 1D-FFT operator. In the present disclosure, Z is selected to be 12800 and Δ r Selected as 40.

[0155] Fall detection can be formulated as a classification problem. There can be two categories: positive fall and negative fall. A deep learning structure can determine which category it is given a radar image. As an example, a convolutional neural network (CNN) is used as Figure 8 The deep learning structure 800 is shown. The parameters of CNN are set in Figure 8. The input layer is a 1-dimensional (1D) convolutional layer with 2 channel inputs (801, 802) and 6 channel outputs. The filter kernel size is 5, the stride is 1, and the padding is 2 (step 803). This is followed by a ReLU activation (step 805) and a maximum pooling with a kernel size of 2 and a stride of 2 (step 807). Layer 2 is also a 1D convolutional layer with 6 channel inputs and 12 channel outputs. The filter kernel size is 5, the stride is 1, and the padding is 2 (step 809). This is followed by a ReLU activation (step 811) and a maximum pooling with a kernel size of 2 and a stride of 2 (step 813). Layer 3 is also a 1D convolutional layer with 12 channel inputs and 24 channel outputs. The filter kernel size is 5, the stride is 1, and the padding is 2 (step 815). This is followed by a ReLU activation (step 817) and a maximum pooling with a kernel size of 2 and a stride of 2 (step 819). Layer 4 is still a 1D convolutional layer with 24 channel input and 24 channel output. The filter kernel size is 5, the stride is 1, and the padding is 2 (step 821). This is followed by a ReLU activation (step 823) and a maximum pooling with a kernel size of 2 and a stride of 2 (step 825).

[0156] After the 4 convolutional layers, at step 827, a dropout operation is performed to enhance reliability and avoid overfitting during training.

[0157] After the dropout operation, at step 829, layer 5 is a fully connected layer with 32800×24 inputs and 1024 outputs. Note that both the range time and range Doppler maps are of size 41×12800. After the max pooling in the 4 convolutional layers, the size of each channel in the latest convolutional layer is The final output layer is also a fully connected layer with 1024 inputs and 2 outputs (step 831). The full detection is output at step 832.

[0158] It may be necessary to train the deep learning architecture before actual detection. Since this is a classification problem, the criterion of the loss function may be cross entropy (step 833). The optimizer used for back propagation may be Adam (step 835).

[0159] The method 700 based on the above embodiment can advantageously first identify whether the detected target is a human body according to vital signs, then locate the human body, and then perform fall event detection at the human body position. Therefore, it can eliminate false alarms caused by non-living body movement interference. In addition, it can also locate the accident point and make multiple fall accident detection possible.

[0160] Although the above methods 600, 700 are illustrated and described as a series of steps or events, it will be understood that any order of such steps or events should not be interpreted in a limiting sense. For example, some steps can occur in different orders and / or occur simultaneously with other steps or events other than those steps or events shown and / or described herein. In addition, it may not be necessary for all illustrated steps to realize one or more aspects or embodiments described herein. In addition, one or more steps described herein may be carried out in one or more separate actions and / or stages.

[0161] Although the embodiments of the present disclosure have been particularly shown and described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made thereto without departing from the scope of the present disclosure as defined by the appended claims. Therefore, the scope of the present disclosure is indicated by the appended claims, and therefore, all changes within the meaning and equivalent range of the claims will be included therein.

Claims

1. A method for tracking one or more living objects using at least one processor, the method comprising: receiving, via the plurality of receiving antennas, signals transmitted from the plurality of transmitting antennas; identifying a position of each of the one or more living objects relative to discrete time based on signals received during a number of sampling periods; determining movement of each of the one or more living objects based on the identified positions; determining one or more vital signs of each of the one or more living subjects based on the movement of each of the one or more living subjects; as well as The position and one or more vital signs of each of the one or more living subjects are collectively tracked. 2 . The method of claim 1 , wherein the one or more vital signs include a breathing rate and / or a heartbeat of the one or more living subjects.

3. The method according to claim 2, wherein the one or more vital signs further include a rate of change of the breathing rate and / or a rate of change of the heartbeat of the one or more living subjects. 4 . The method of claim 1 , wherein the movement of each of the one or more living objects comprises information related to a speed of each of the one or more living objects.

5. The method of claim 4, wherein the information related to the velocity of each of the one or more living objects comprises a Doppler velocity of each of the one or more living objects and a direction of arrival (DOA) estimate of each of the one or more living objects.

6. The method of claim 2, wherein determining one or more vital signs of each of the one or more living subjects comprises: performing low pass filtering to obtain information related to the breathing rate of the one or more living subjects and / or performing high pass filtering to obtain information related to the heartbeat of the one or more living subjects; as well as The breathing rate of the one or more living objects is determined by finding a peak in the information related to the breathing rate of the one or more living objects during the sampling period and / or the heartbeat of the one or more living objects is determined by finding a peak in the information related to the heartbeat of the one or more living objects during the sampling period. The method according to claim 6 , wherein the low-pass filtering and the high-pass filtering are performed at the same frequency.

8. The method according to claim 6, further comprising: determining an intermediate parameter equal to the low pass filtered information associated with a respiration rate of the one or more living subjects if the low pass filtered information is greater than a predetermined ratio of an average of the low pass filtered information during the sampling period, and determining the intermediate parameter to be zero if the low pass filtered information is less than or equal to a predetermined ratio of an average of the low pass filtered information; and / or If the high-pass filtered information is greater than a predetermined ratio of the average value of the high-pass filtered information during the sampling period, an intermediate parameter equal to the high-pass filtered information associated with the heartbeat of the one or more living objects is determined, and if the high-pass filtered information is less than or equal to a predetermined ratio of the average value of the high-pass filtered information, the intermediate parameter is determined to be zero.

9. The method according to claim 1, further comprising: Falls are detected by using Convolutional Neural Network (CNN).

10. The method of claim 1, wherein identifying the position of each of the one or more living objects relative to discrete time based on signals received during a number of sampling periods comprises: sampling the received signal to generate a sequence of received signal values ​​for each pair of transmit antenna and receive antenna; generating frequency domain values ​​by applying a Fourier transform to the sequence of received signal values; generating a radar imaging result by applying a spatial spectrum recovery algorithm to the frequency domain values ​​to generate a first radar imaging result; Detecting a peak in the radar imaging result; determining a phase perturbation for each detected peak to generate an unwrapped phase angle; performing bandpass filtering of the generated unwrapped phase angle of each peak; as well as If the measured value of the filtered unwrapped phase angle of the peak is above a predetermined threshold, it is determined that the peak corresponds to a living object.

11. The method according to claim 1, further comprising: displaying the location of each of the one or more living objects in a target location map; as well as The location of each of the one or more non-living objects is indicated in the target location map.

12. The method according to claim 1, further comprising: The waveforms of the vital signs of the one or more living subjects are displayed according to the estimated rates of the one or more living subjects.

13. The method of claim 1, further comprising, for each of the one or more living objects: simultaneously identifying a position relative to the discrete time based on signals received during a plurality of sampling periods; simultaneously determining the movement based on the identified position; simultaneously determining the one or more vital signs based on the movement; and The position and the one or more vital signs are tracked simultaneously and jointly using a Kalman filter.

14. A method for tracking one or more living objects using at least one processor, the method comprising: receiving, via the plurality of receiving antennas, signals transmitted from the plurality of transmitting antennas; sampling the received signal to generate a sequence of received signal values ​​for each pair of transmit antenna and receive antenna; generating frequency domain values ​​by applying a Fourier transform to the sequence of received signal values; detecting peaks in the frequency domain values ​​relative to discrete time based on signals received during a number of sampling periods; determining a phase perturbation for each detected peak to generate an unwrapped phase angle; performing bandpass filtering of the generated unwrapped phase angle of each peak; as well as If the measured value of the filtered unwrapped phase angle of a peak is above a predetermined threshold, it is determined that the peak corresponds to a living object.

15. The method according to claim 14, before detecting the peak value in the frequency domain value, the method further comprises: generating a radar imaging result by applying a spatial spectrum recovery algorithm to the frequency domain values ​​to generate a first radar imaging result; The spatial spectrum recovery algorithm provides a preliminary radar imaging result, and generating the radar imaging result further comprises applying a constant false alarm rate (CFAR) algorithm to the preliminary radar imaging result.

16. The method according to claim 14, further comprising: A Doppler domain value of the frequency domain value is determined.

17. The method according to claim 14, further comprising: low pass filtering the filtered unwrapped phase angle using a third frequency to generate a raw respiration rate; modulating the respiration rate data to be equal to the raw respiration rate if the raw respiration rate is greater than a predetermined ratio of an average value of the raw respiration rate during the sampling period, and determining the respiration rate data to be zero if the raw respiration rate is less than or equal to a predetermined ratio of an average value of the raw respiration rate during the sampling period; as well as The respiration rate of the living subject is determined by finding peaks in the respiration rate data during the sampling period and setting a size for the number of peaks in the respiration rate data.

18. The method according to claim 14, further comprising: performing high pass filtering using a fourth frequency on the filtered unwrapped phase angle to generate a raw heartbeat; If the original heartbeat is greater than a predetermined ratio of an average value of the original heartbeat during a sampling period, the heartbeat data is modulated to be equal to the original heartbeat, and if the original heartbeat is less than or equal to a predetermined ratio of an average value of the original heartbeat during the sampling period, the heartbeat data is modulated to zero; as well as The heartbeat of the living subject is determined by finding peaks in the heartbeat data during the sampling period and setting the size of the number of peaks of the heartbeat data.

19. The method according to claim 14, further comprising: forming an amplified range time map and an amplified range Doppler map; as well as These graphs are processed by deep learning structures.

20. A system for tracking a living subject, the system comprising: at least one memory; and At least one processor is communicatively coupled to the at least one memory and is configured to perform the method of any one of claims 1 to 13 or the method of any one of claims 14 to 19.

21. A non-transitory computer-readable storage medium comprising instructions executable by at least one processor to perform the method of any one of claims 1 to 13 or the method of any one of claims 14 to 19.

Images