Radar-based tracker for target sensing

Abstract

Embodiments of the present disclosure relate to radar-based trackers for target sensing. In an embodiment, a method for tracking a target includes receiving data from a radar sensor of a radar, processing the received data to detect a target, identifying a first geometric feature of a first detected target at a first time step, the first detected target being associated with a first track, identifying a second geometric feature of a second detected target at a second time step, determining an error value based on the first geometric feature and the second geometric feature, and associating the second detected target with the first track based on the error value.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to electronic systems and methods, and in particular embodiments, to radar-based trackers for target sensing. BACKGROUND

[0002] Applications in millimeter-wave frequency regimes have gained significant interest in the past few years due to rapid advances in low-cost semiconductor technologies such as silicon germanium (SiGe) and fine-geometry complementary metal-oxide-semiconductor (CMOS) processes. The availability of high-speed bipolar and metal-oxide-semiconductor (MOS) transistors has led to growing demand for integrated circuits for millimeter-wave applications at, for example, 24 GHz, 60 GHz, 77 GHz, and 80 GHz, and beyond 100 GHz. Such applications include, for example, automotive radar systems and multi-gigabit communication systems.

[0003] In some radar systems, a distance between a radar and a target is determined by transmitting a frequency-modulated signal, receiving a reflection (also referred to as a return) of the frequency-modulated signal, and determining the distance based on a time delay and / or a frequency difference between transmission and reception of the frequency-modulated signal. Accordingly, some radar systems include a transmit antenna to transmit a radio frequency (RF) signal, and a receive antenna to receive a reflected RF signal, and associated RF circuitry to generate the transmitted signal and receive the RF signal. In some cases, multiple antennas can be used to implement directional beams using phased array techniques. Multiple-input multiple-output (MIMO) configurations with multiple chipsets can also be used to perform coherent and non-coherent signal processing. SUMMARY

[0004] According to embodiments, a method for tracking a target includes receiving data from a radar sensor of a radar, processing the received data to detect a target, identifying a first geometric feature of a first detected target at a first time step, the first detected target being associated with a first track, identifying a second geometric feature of a second detected target at a second time step, determining an error value based on the first geometric feature and the second geometric feature, and associating the second detected target with the first track based on the error value.

[0005] According to an embodiment, a radar system comprises: a millimeter wave radar sensor comprising: a transmit antenna configured to transmit a radar signal; a first receive antenna and a second receive antenna configured to receive a reflected radar signal; an analog-to-digital converter (ADC) configured to generate raw digital data at an output of the ADC based on the reflected radar signal; and a processing system configured to process the raw digital data for: detecting a target, identifying a first geometric feature of a first detected target at a first time step, identifying a second geometric feature of a second detected target at a second time step, determining an error value based on the first feature and the second feature, and associating the second detected target to a first track associated with the first detected target based on the error value.

[0006] According to an embodiment, a method comprises: receiving data from a radar sensor of a radar; processing the received data to detect humans; clustering the detected humans into a cluster of cells using k-means clustering to generate a plurality of clusters; identifying a first geometric feature of a first cluster of the plurality of clusters at a first time step; identifying a second geometric feature of a second cluster of the plurality of clusters at a second time step, wherein the first time step and the second time step are consecutive time steps; determining an error value based on the first feature and the second feature; and associating the second cluster to a first track associated with the first cluster based on the error value. BRIEF DESCRIPTION OF DRAWINGS

[0007] For a more complete understanding of the present application, and for further features and advantages thereof, reference is made to the following description taken in conjunction with the accompanying drawings, in which:

[0008] Figure 1 A schematic diagram of a millimeter wave radar system according to an embodiment of the application is shown;

[0009] Figure 2 A chirp sequence transmitted by a transmitter antenna of Figure 1 is shown according to an embodiment of the application;

[0010] Figure 3 A flowchart of an embodiment method for personnel tracking according to an embodiment of the application is shown;

[0011] Figure 4 Target association using template matching according to an embodiment of the application is shown;

[0012] Figure 5 Target association using template matching using a distributed radar system according to an embodiment of the application is shown;

[0013] Figure 6 A flowchart of an embodiment method for performing data association using template matching according to an embodiment of the application is shown;

[0014] Figure 7A flowchart of an embodiment of a method for extracting empirical mode decomposition (EMD) features and calculating the error associated with the extracted EMD features is shown according to an embodiment of the present invention;

[0015] Figure 8 The diagram illustrates a waveform of a data signal decomposed into multiple intrinsic frequency components by EMD according to an embodiment of the present invention.

[0016] Figure 9 A flowchart illustrating an embodiment of a method for extracting SIFT features and calculating the error associated with the extracted SIFT features according to an embodiment of the present invention;

[0017] Figure 10A and 10B The corresponding histograms of the input radar image and the gradient with SIFT features according to an embodiment of the present invention are shown respectively;

[0018] Figure 11 A flowchart of an embodiment of a method for calculating Wasserstein distance between clusters at continuous time steps, according to an embodiment of the present invention, is shown;

[0019] Figure 12 The illustration shows radar images with two clusters at consecutive time steps according to an embodiment of the present invention; and

[0020] Figure 13 A method for performing data association using template matching according to an embodiment of the present invention is shown, such as when applied to... Figure 4 Examples.

[0021] Unless otherwise stated, corresponding reference numerals and symbols in different figures generally refer to corresponding parts. The figures are drawn to clearly illustrate relevant aspects of the preferred embodiments and are not necessarily drawn to scale. Detailed Implementation

[0022] The manufacture and use of the disclosed embodiments are discussed in detail below. However, it should be understood that the present invention provides many applicable inventive concepts that can be implemented in various specific environments. The specific embodiments discussed are merely illustrative of specific ways of manufacturing and using the invention and do not limit the scope of the invention.

[0023] The following description illustrates various specific details that provide a thorough understanding of several exemplary embodiments. Embodiments may be obtained without one or more specific details or by other methods, components, materials, etc. In other instances, known structures, materials, or operations have not been shown or described in detail so as not to obscure different aspects of the embodiments. References to “embodiment” in this specification indicate that a particular configuration, structure, or feature described with respect to that embodiment is included in at least one embodiment. Therefore, phrases such as “in one embodiment” that may appear at different points in this specification do not necessarily refer precisely to the same embodiment. Furthermore, in one or more embodiments, particular formations, structures, or features may be combined in any suitable manner.

[0024] Embodiments of the invention will be described in a specific context, describing a millimeter-wave radar-based tracker for personnel sensing. Embodiments of the invention can also be used to track other targets (e.g., animals, vehicles, robots, etc.) and / or can operate under conditions different from millimeter-wave radar.

[0025] In embodiments of the invention, millimeter-wave radar is used to track human targets based on features extracted from detected targets. In some embodiments, some or all of the extracted features used to associate the target with a trajectory are not based on a motion model. Therefore, some embodiments are advantageously able to detect human targets without knowing their (actual or predicted) motion and / or location. Therefore, some embodiments are advantageously able to use low frame rates to track human targets. In some embodiments, using low frame rates advantageously allows for power savings, which can extend battery life in battery-powered applications, and / or can advantageously allow compliance with regulatory requirements (such as FCC requirements) associated with the maximum duty cycle (maximum frame rate) used for radar operation without sacrificing tracking performance.

[0026] In some embodiments, features such as distance, Doppler velocity, and / or angle are additionally tracked and also used to associate detected targets with trajectories based on motion models, which can advantageously increase tracking performance. For example, distance and Doppler velocity at previous time steps can be used to predict the position of the target at future time steps, and such information can be used to increase confidence that the target assignment is correct (e.g., by using a gating region of the expected position of the target at future time steps).

[0027] Radar, such as millimeter-wave radar, can be used to detect and track humans. For example, Figure 1 A schematic diagram of a millimeter-wave radar system 100 according to an embodiment of the present invention is shown. The millimeter-wave radar system 100 includes a millimeter-wave radar sensor 102 and a processing system 104.

[0028] During normal operation, the millimeter-wave radar sensor 102 operates as a frequency-modulated continuous wave (FMCW) radar sensor and transmits multiple TX radar signals 106, such as chirps, into scene 120 using a transmitter (TX) antenna 114. The radar signals 106 are generated using RF and analog circuitry 130. The radar signals 106 can be in the range of 20 GHz to 122 GHz. Objects in scene 120 may include, for example, one or more humans, which may be moving or stationary. Other objects may also be present in scene 120, such as furniture, machinery, mechanical structures, walls, etc.

[0029] Radar signal 106 is reflected by an object in scene 120. The reflected radar signal 108 (also known as the echo signal) is received by receiver (RX) antennas 116a and 116b. RF and analog circuitry 130 processes the received reflected radar signal 108 in a manner known in the art, using, for example, a bandpass filter (BPF), a low-pass filter (LPF), a mixer, a low-noise amplifier (LNA), and / or an intermediate frequency (IF) amplifier, to generate an analog signal x. outa (t) and x outb (t).

[0030] Use ADC112 to convert the analog signal x outa (t) and x outb (t) is converted into raw digital data x out_dig (n). Processing system 104 processes raw digital data x out_dig (n) to detect humans and their locations, and to track the detected humans.

[0031] although Figure 1 A radar system with two receiver antennas 116 is shown, but it should be understood that more than two receiver antennas 116, such as three or more, may also be used.

[0032] although Figure 1 A radar system with a single transmitter antenna 114 is shown, but it should be understood that more than one transmitter antenna 114, such as two or more, may also be used.

[0033] Controller 110 controls one or more circuits of millimeter-wave radar sensor 102, such as RF and analog circuitry 130 and / or ADC 112. For example, controller 110 may be implemented as custom digital or mixed-signal circuitry. Controller 110 may also be implemented in other ways, such as using a general-purpose processor or controller. In some embodiments, processing system 104 implements part or all of controller 110.

[0034] Processing system 104 can be implemented using a general-purpose processor, controller, or digital signal processor (DSP) that includes, for example, combinational circuitry coupled to memory. In some embodiments, processing system 104 can be implemented as an application-specific integrated circuit (ASIC). In some embodiments, processing system 104 can be implemented using, for example, an ARM, RISC, or x86 architecture. In some embodiments, processing system 104 may include an artificial intelligence (AI) accelerator. Some embodiments may use a combination of a hardware accelerator and software running on a DSP or general-purpose microcontroller. Other implementations are also possible.

[0035] In some embodiments, part or all of the millimeter-wave radar sensor 102 and processing system 104 may be implemented within the same integrated circuit (IC). For example, in some embodiments, part or all of the millimeter-wave radar sensor 102 and processing system 104 may be implemented in corresponding semiconductor substrates integrated in the same package. In other embodiments, part or all of the millimeter-wave radar sensor 102 and processing system 104 may be implemented in the same monolithic semiconductor substrate. Other implementations are also possible.

[0036] As a non-limiting example, RF and analog circuit 130 can be implemented, for example, as Figure 1 As shown. During normal operation, VCO 136 generates radar signals, such as linear frequency chirps (e.g., from 57 GHz to 64 GHz, or from 76 GHz to 77 GHz), which are transmitted by transmitting antenna 114. VCO 136 is controlled by PLL 134, which receives a reference clock signal (e.g., 80 MHz) from reference oscillator 132. PLL 134 is controlled by a loop including frequency divider 138 and amplifier 140.

[0037] The TX radar signal 106 transmitted by transmitting antenna 114 is reflected by an object in scene 120 and received by receiving antennas 116a and 116b. The echoes received by receiving antennas 116a and 116b are mixed with a copy of the signal transmitted by transmitting antenna 114 using mixers 146a and 146b, respectively, to generate corresponding intermediate frequency (IF) signals x. IFa (t), x IFb (t) (also known as beat frequency signal). In some embodiments, the beat frequency signal x IFa (t), x IFb (t) has a bandwidth between 10 kHz and 1 MHz. Beat frequency signals with bandwidths below 10 kHz or above 1 MHz are also possible.

[0038] The beat frequency signal x was filtered using the corresponding low-pass filters (LPF) 148a and 148b. IFa (t), x IFb(t) is filtered and then sampled by ADC112. ADC112 advantageously enables sampling of the filtered beat frequency signal x at a sampling frequency much lower than the frequency of the signal received by receiving antennas 116a and 116b. outa (t), x outb (t) Sampling is performed. Therefore, in some embodiments, the use of FMCW radar advantageously allows for a compact and low-cost implementation of the ADC112.

[0039] In some embodiments, the original digital data x out_dig (n)(which in some embodiments includes a filtered beat frequency signal x) outa (t) and x outb The digital version of (t) (e.g., temporarily) is stored, for example, in N of each receiver antenna 116. c ×N s In the matrix, where N c It is the number of chirps considered in the frame, and N s It is the number of transmitted samples for each chirped signal, for further processing by the processing system 104.

[0040] In some embodiments, ADC112 is a 12-bit ADC with multiple inputs. ADCs with higher resolution (e.g., 14 bits or higher) or lower resolution (e.g., 10 bits or lower) may also be used. In some embodiments, one ADC may be used per receiver antenna. Other implementations are also possible.

[0041] Figure 2 A sequence of chirps 106 transmitted by the TX antenna 114 according to an embodiment of the present invention is shown. Figure 2 As shown, the chirp 106 is organized across multiple frames and can be implemented as an upward chirp. Some embodiments may use a downward chirp or a combination of upward and downward chirps, such as an upward-downward chirp and a downward-upward chirp. Other waveform shapes may also be used.

[0042] like Figure 2 As shown, each frame may include multiple chirps 106 (often also referred to as pulses). For example, in some embodiments, the number of pulses in a frame is 16. Some embodiments may include more than 16 pulses per frame, such as 20 pulses, 32 pulses or more, or less than 16 pulses per frame, such as 10 pulses, 8 pulses, 4 or less. In some embodiments, each frame includes only a single pulse.

[0043] The frame repeats every FT time. In some embodiments, the FT time is 50 ms. Different FT times can be used, such as more than 50 ms, such as 60 ms, 100 ms, 200 ms or more, or less than 50 ms, such as 45 ms, 40 ms or less.

[0044] In some embodiments, the FT time is selected such that the time between the start of the last chirp of frame n and the start of the first chirp of frame n+1 is equal to the PRT. Other embodiments may use or result in different timings.

[0045] The time between frame chirps is typically referred to as the Pulse Repeat Time (PRT). In some embodiments, the PRT is 5 ms. Different PRTs may also be used, such as less than 5 ms, such as 4 ms, 2 ms or less, or more than 5 ms, such as 6 ms or more.

[0046] The duration of a chirp (from start to finish) is typically referred to as the chirp time (CT). In some embodiments, the chirp time can be, for example, 64 μs. Higher chirp times, such as 128 μs or higher, can also be used. Lower chirp times can also be used.

[0047] In some embodiments, the chirp bandwidth can be, for example, 4 GHz. Higher bandwidths (such as 6 GHz or higher) or lower bandwidths (such as 2 GHz, 1 GHz or lower) are also possible.

[0048] In some embodiments, the sampling frequency of the millimeter-wave radar sensor 102 may be, for example, 1 MHz. Higher sampling frequencies (such as 2 MHz or higher) or lower sampling frequencies (such as 500 kHz or lower) are also possible.

[0049] In some embodiments, the number of samples used to generate the chirp may be, for example, 64 samples. Higher numbers of samples (e.g., 128 samples) or higher, or lower numbers of samples (e.g., 32 samples) or lower may also be used.

[0050] Figure 3 A flowchart of an embodiment method 300 for personnel tracking according to an embodiment of the present invention is shown. Method 300 may be implemented by a processing system 104.

[0051] During steps 302a and 302b, raw ADC data x is received, for example, from millimeter-wave radar sensor 102. out_dig (n). As shown in the figure, the original ADC data x out_dig (n) includes data from multiple antennas (e.g., Figure 3 Individual baseband radar data (2 in the example shown).

[0052] During steps 304a and 304b, signal conditioning, low-pass filtering, and background removal are performed on the raw ADC data of the corresponding antenna 116. Raw ADC data x out_dig (n) The radar data is filtered, the DC component is removed to remove, for example, Tx-Rx self-interference, and optionally, interference coloring noise is pre-filtered. Filtering may include removing data outliers that have values ​​that are significantly different from those in other adjacent range-gate measurements. Therefore, the filtering is also used to remove background noise from the radar data.

[0053] During steps 306a and 306b, a 2D Moving Target Indication (MTI) filter is applied to the data generated during steps 304a and 304b, respectively, to remove responses from static targets. The MTI filter can be performed by subtracting the average along the fast time (inter-chirp time) to remove transmitter-receiver leakage in the first few range windows of the disturbance, and then subtracting the average along the slow time (inter-chirp time) to remove reflections from static objects (or zero-Doppler targets).

[0054] During steps 308a and 308b, a series of FFTs are performed on the filtered radar data generated during steps 306a and 306b, respectively. The first window FFT has a chirp length calculated along each waveform for each of a predetermined number of chirs in the signal frame. The FFT for each waveform of the chirp can be referred to as a “range FFT”. A second FFT is calculated across each range interval over multiple consecutive periods to extract Doppler information. After each 2DFFT is performed during steps 308a and 308b, a range-Doppler image is generated, respectively.

[0055] During step 310, the minimum variance distortionless response (MVDR) technique (also known as Capon) is used to determine the angle of arrival based on range and Doppler data from different antennas. A range-angle image (RAI) is generated during step 310. In some embodiments, a range-Doppler-angle data cube is generated during step 310.

[0056] During step 312, an ordered statistical (OS) constant false alarm rate (OS-CFAR) detector is used to detect targets. The CFAR detector generates a detection image, where, for example, based on the power level of the RAI, points above the threshold are marked as targets ("-") and points below the threshold are marked as non-targets ("zero").

[0057] In some embodiments, targets present in the detected image generated during step 312 are clustered in step 314, for example, based on similar feature properties associated with the detected targets, such as Empirical Mode Decomposition (EMD) and / or Scale Invariant Feature Transform (SIFT). In some embodiments, other types of features of the detected targets (e.g., motion model-based features based on distance, Doppler, and / or angle) may also be used to cluster units together. In some embodiments, metrics such as correlation and / or Wasserstein distance may be used to determine the similarity between clusters. In some embodiments, feature-based clustering is performed using k-means clustering, in which targets are grouped (clustered) into a k-cluster with the nearest mean of such (e.g., combined) features based on similar features.

[0058] For example, in some embodiments, the feature vector contains multiple features associated with the EMD (e.g., the number of intrinsic mode functions (IMFs) and / or IMFs associated with the EMD, and / or the magnitude M(m,n) and / or phase φ(m,n) associated with the SIFT), where each channel describes the type of feature (e.g., IMF, number of IMFs, magnitude M(m,n), and / or phase φ(m,n)). Each channel can be described as a Gaussian distribution (mean and variance over the available vectors of the same feature). A weighted sum over all the different Gaussian distributions on the channels is obtained to provide a descriptor for each cell, where the descriptor is associated with all feature types and can be a value or vector indicating the characteristics (features) of the associated clusters, and can be used to determine how similar the clusters are. Such descriptors are used for clustering, for example, using the k-means clustering algorithm.

[0059] In some embodiments, density-based spatial clustering with the application of the noise (DBSCAN) algorithm can also be used to associate targets with clusters during step 314. The output of DBSCAN groups the detected points into specific targets. DBSCAN is a popular unsupervised algorithm that uses the minimum point and minimum distance criteria to cluster targets and can be implemented in any manner known in the art. Other clustering algorithms may also be used.

[0060] Therefore, in some embodiments, clustering results in radar images (e.g., RAI or RDI) or data cubes being divided into groups of cells with similar descriptors. In some embodiments, each cluster corresponds to (e.g., a potential) detection target. Because the extension of features is not necessarily uniform, in some embodiments, each cluster is not necessarily equal. Therefore, in some embodiments, radar images or data cubes are divided into cell clusters, but each cell cluster does not necessarily have the same size (e.g., not the same number of cells / sub-cells).

[0061] During step 316, the detected (clustered) targets are associated with corresponding trajectories. As will be described in more detail later, in some embodiments, the detected targets are associated with corresponding trajectories (during step 318) using feature-based template matching. For example, in some embodiments, geometric features are used for template matching during step 318. Geometric features can be understood as features that represent fragmentation changes in the rotation of an identifiable target, as well as changes in the target's centroid's distance, Doppler velocity, and angle. In some embodiments, geometric features may include physical geometric features, such as the physical edges of the target (e.g., from radar images). In some embodiments, additionally or alternatively, geometric features may include measures (e.g., vectors, functions, or groups of functions) based on relationships between cells of the raw data (e.g., data cubes) (e.g., relationships between distance cells, Doppler velocity cells, and / or angle cells). Examples of such measures include functions extracted using the functional decomposition of the data cube, the gradient of the data cube, and / or statistical properties of the data cube (such as the histogram / PDF of the data cube). Examples of geometric features include EMD features and SIFT features.

[0062] In some embodiments, geometric features allow for the identification of targets without relying on a motion model. In some embodiments, geometric features allow for the differentiation between tracked targets.

[0063] In some embodiments, geometric features, such as EMD and / or SIFT, are tracked for the target. For each cluster unit (for each detected target), a feature vector with values ​​for each feature associated with the cluster unit is generated for each time step i. The detected target at time step i+1 (e.g., using Hungarian assignment) is assigned to a corresponding trajectory based on the similarity between feature vectors (e.g., based on the error between feature vectors). For example, in some embodiments, a similarity metric is identified between feature clusters at consecutive time steps (e.g., i and i+1), and an assignment that minimizes the error between feature clusters (e.g., increases correlation) is selected for trajectory assignment.

[0064] In some embodiments, the data association step (316) may additionally include a data association method that does not rely on feature-based template matching.

[0065] In some embodiments, the data allocation from the detected target (cluster) to the trajectory depends on the geometric features of the cluster and does not depend on (or only depends on) the actual physical location and / or velocity of the detected target.

[0066] During step 320, trajectory filtering is formed, for example, for the target being tracked over time. For example, in some embodiments, an unscented Kalman filter is used to perform tracking filtering during step 320. For instance, in some embodiments, features (e.g., SIFT, EMD, distance, Doppler, angle, deep learning-based parameters, and / or other parameters associated with the trajectory) are additional features, for example, used to perform data association (which can also be tracked by the Kalman filter). The unscented Kalman filter can also track the localization of each trajectory and can rely on the trajectory history of such localization to enhance data association. The Kalman filter can be implemented in any manner known in the art.

[0067] It should be understood that while a target may be identified using template matching (during step 316) that may not include spatial and / or motion information (e.g., range, Doppler, angle), such localization information can still be tracked during step 320. Therefore, in some embodiments, feature-based template matching (step 318) is an enabler for digital association in an environment, such as low frame rate and / or multi-target scenarios, and / or where relying solely on localization information may be difficult in distributed radar implementations.

[0068] During step 324, trajectory management tasks are performed, such as generating and terminating trajectories. For example, during step 324, tracking initialization, re-initialization, and / or tracking termination may be performed, for example, based on whether the detected target is no longer in the field of view (in scene 120) or has re-entered the field of view.

[0069] In some embodiments, steps 316, 320, and 324 may be implemented in a different order. For example, in some embodiments, trajectory initialization may be performed before step 316 (during step 324).

[0070] Figure 4 This illustrates a target association using template matching according to an embodiment of the present invention. Template matching (e.g., such as...) Figure 4 (As shown in the diagram) can be performed, for example, during step 316.

[0071] like Figure 4 As shown, humans A and B move within field of view 401 over time (from time step i to time step i+1). In some embodiments, the time between time step i and i+1 can be, for example, 66 ms (for a 15 frames per second radar system). In some embodiments, a faster frame rate can be used. As will be described in more detail later, in some embodiments, a slower frame rate, such as a 10 frames per second radar system (where the time between time step i and i+1 can be, for example, 100 ms) or slower, can be advantageously used while maintaining the ability to effectively track the target.

[0072] like Figure 4 As shown, humans A and B are located at positions 402 and 404 respectively at time step i. At time step i+1, humans A and B are located at positions 406 and 408 respectively. Since at time step i+1, the detected target at position 408 is closer to position 402, and the detected target at position 406 is closer to position 404, the traditional association method using a probabilistic data association filter (PDAF) may associate the detected human B (at position 408) with the trajectory of human A, and the detected human A (at position 406) with the trajectory of human B. Figure 4 As shown, by relying on features not based on (or not only based on) the motion model, feature-based assignment advantageously allows for correct tracking assignment without increasing the duty cycle (without reducing the size of the time step i, making the target detectable more closely in time). Conversely, in some embodiments, target detection is assigned based on, for example, the level of correlation between features across consecutive time steps for tracking. Therefore, some embodiments advantageously allow for tracking humans at low duty cycles (such as 10 frames per second or slower).

[0073] Because some embodiments rely on geometric features not based on motion models (such as EMD and SIFT) to track targets, some embodiments are advantageously suited for tracking targets using distributed radar, where humans can move from the fields of view of different radars, and where the radars may lack information about the target's movement outside the fields of view of these radars. For example, Figure 5 This illustration demonstrates target association using template matching with a distributed radar system 500, which employs two radar systems, according to an embodiment of the present invention. In some embodiments, each radar system used in the distributed radar system 500 is implemented as a millimeter-wave radar system 100. Template matching can be performed, for example, during step 316 of each radar in the distributed radar system 500, such as... Figure 4 As shown in the diagram (and as explained in more detail later). In some embodiments, the distributed radar system 500 may include more than two radars, such as three, five, eight or more.

[0074] like Figure 5As shown, over time, humans A and B move between fields of view 501 and 503 (from time step i to time step i+1). For example, humans A and B are located at positions 502 and 504, respectively, at time step i. At time step i+1, humans A and B are located at positions 506 and 508, respectively. Since the radar with field of view 501 detects a single target (human A at position 502) at time step i and a single target (human B at position 508) at time step i+1, and since the radar with field of view 503 detects a single target (human B at position 504) at time step i and a single target (human A at position 506) at time step i+1, it is possible to associate the detected human B (at position 508) with the trajectory of human A, and the detected human A (at position 506) with the trajectory of human B using conventional association methods of PDAF. Figure 5 As shown, feature-based assignment advantageously allows for correct trajectory allocation when a target moves between the fields of view of different radars in a distributed radar system. In some embodiments, a controller 510 shared among the radars of the distributed radar system 500 is used to identify the characteristics of the tracked target, and the common controller 510 can be used to perform tracking functions. For example, in some embodiments, the common controller 510 can perform steps 316, 320, and 324, while the processing system 104 of each radar 100 of the distributed radar system 500 performs steps 302, 304, 306, 308, 310, 312, and 314.

[0075] In some embodiments, controller 510 may be implemented as part of processing system 104 of one of the radars 100 of distributed radar system 500. In some embodiments, controller 510 may be implemented externally to processing system 104 of radar system 100 of distributed radar system 500, for example, as a general-purpose processor, controller, or digital signal processor (DSP) having combinational circuitry coupled to memory. In some embodiments, processing system 104 may be implemented as application-specific integrated circuit (ASIC). In some embodiments, processing system 104 may be implemented using, for example, ARM, RISC, or x86 architectures. In some embodiments, processing system 104 may include an artificial intelligence (AI) accelerator. Some embodiments may use a combination of hardware accelerators and software running on a DSP or general-purpose microcontroller. Other implementations are also possible.

[0076] For example, in some embodiments, the characteristics of the detected target, along with spatial and movement parameters, are passed to a central processing unit 510 (external to each radar of the distributed radar system 500) for data association and tracking. In some embodiments, one of the processing systems 104 of the distributed radar system 500 may operate as the central processing unit 510, for example, for data association and tracking.

[0077] Figure 6 A flowchart of an embodiment method 600 for performing data association using template matching according to an embodiment of the present invention is shown. Step 316 can be performed as method 600.

[0078] like Figure 6 As shown, for each of the L clusters (e.g., identified during step 314), features associated with time step i are extracted during steps 602 and 604, and compared with, for example, features of the L clusters at time step i+1 during steps 606 and 610 to generate L error vectors. In some embodiments, step 606 may be considered as part of step 610 (e.g., as in the manner in which step 610 is implemented).

[0079] During step 612, an allocation is performed between the L clusters at time step i and the L clusters at time step i+1, for example, to minimize the error between error vectors (e.g., to minimize the sum of the errors in the error vectors between the allocated clusters). In some embodiments, a Hungarian allocation is used during step 612 to associate the clusters at time step i with the clusters at time step i+1. In some embodiments, each cluster at time step i+1 is associated with a trajectory corresponding to the cluster associated at time step i.

[0080] In some embodiments, applying the Hungarian assignment includes:

[0081] Calculate the cost matrix C, where c i,j It is a cluster p at time step i based on a metric F (e.g., correlation, variance, Wasserstein distance, Euclidean distance, mean squared error, etc.). i Cluster y at time step i+1 i The cost between them can be given by the following formula:

[0082] c i,j =F(p) i ,y j (1)

[0083] Find the assignment matrix A that minimizes the element-wise product between C and A, for example, by:

[0084]

[0085] The vector y is reordered according to the vector y in the assignment matrix A, and, for example, the clusters from vector p are assigned sequentially to the clusters from the ordered vector y.

[0086] In some embodiments, the number of clusters at time steps i and i+1 is different (e.g., due to the disappearance of previously detected targets or the arrival of new targets in the field of view). In some such embodiments, assignment is performed to minimize the error between vectors at each time step, and the error of the additional vector that is not associated with the corresponding vector at another time step is assigned a default error. In some embodiments, for each unassigned cluster or target, an error counter is used to count how many times an unsigned target exists, and when the counter reaches a predetermined threshold (e.g., 5, 6, etc.), the corresponding trajectory is canceled.

[0087] In some embodiments, a motion model relying on features such as distance, Doppler, and / or angle is used to associate a detected target with a trajectory, which can advantageously improve tracking performance. For example, distance and Doppler velocity at previous time steps can be used to predict the target's position at future time steps, and such information can be used to increase the confidence that the target assignment is correct (e.g., by using a gating region of the target's expected position at future time steps), and wherein the confidence level is used to associate the target with the trajectory. Therefore, in some embodiments, associating the target with the trajectory also depends on a motion-based model, such as one based on distance, Doppler, and / or angle.

[0088] Figure 7 A flowchart of an embodiment method 700 for extracting EMD features (step 702) and calculating the error associated with the extracted EMD features (step 710) according to an embodiment of the present invention is shown. In some embodiments, step 602 may be performed as step 702, and a portion of step 610 associated with comparing EMD features may be performed as step 710.

[0089] During step 702, EMD feature extraction is performed. EMD can be understood as decomposing signal data into intrinsic mode functions (IMFs) of instantaneous frequencies included in the original signal data. The disjointed signal components form the basis of the original data signal, which is either fully or nearly orthogonal. In some embodiments, EMD is performed on the original data associated with a cluster cell (e.g., a Doppler signal). For example, in some embodiments, EMD is performed on the original data associated with a specific cluster cell at time step i (e.g., from a data cube).

[0090] During step 724, IMFs with energy values ​​higher than a predetermined value are identified. During step 726, the identified IMFs are sorted (e.g., in ascending order). During step 728, the error between the sorted identified IMFs at time i (for each cluster unit) and the sorted IMFs for each cluster at time i+1 is calculated to generate (e.g., L) error values ​​for each cluster unit at time i (which can be used as part of the error vector in step 610).

[0091] In some embodiments, the error between IMFs at time steps i and i+1 is determined using, for example, mean squared error. In some embodiments, the number of IMFs above a threshold can also be used to determine the error between clusters at time steps i and i+1.

[0092] In some embodiments, the EMD characteristics (e.g., sorted IMF) of all clusters at time step i are compared with those of all clusters at time step i+1, and the cluster pairs that result in the lowest mean squared error at time steps i and i+1 are associated.

[0093] Figure 8 The diagram illustrates a waveform obtained by decomposing a data signal into multiple intrinsic frequency components using EMD according to an embodiment of the present invention.

[0094] exist Figure 8 In one embodiment, three of the ten modes (IMF1, IMF2, and IMF3) have energies above a predetermined threshold.

[0095] Figure 9 A flowchart of an embodiment method 900 for extracting SIFT features (step 904) and calculating the error associated with the extracted SIFT features (step 910) according to an embodiment of the present invention is shown. In some embodiments, step 604 may be performed as step 904, and a portion of step 610 associated with comparing SIFT features may be performed as step 910.

[0096] During step 922, SIFT feature extraction is performed. SIFT can be understood as a modal identification method for detecting features of radar images (e.g., RDI, RAI), which are invariant to scaling, rotation, translation, and geometric distortion or any affine distortion of the image. For example, in some embodiments, SIFT feature extraction is obtained by calculating the gradient between different units of the image. For example, for each unit in a cluster, the magnitude M(m,n) and phase φ(m,n) can be determined by applying equations 3 and 4:

[0097]

[0098]

[0099] Where X(m,n) is a sub-unit of the clustered unit of the radar image, and m and n are the positions of the sub-unit.

[0100] In some embodiments, SIFT feature extraction is performed on, for example, RDI, RAI, or data cubes (e.g., from steps 308a, 308b, and / or 310) in the region associated with a specific cluster unit at time step i, for example, by using Equations 3 and 4.

[0101] During step 928, the error between the magnitude and / or phase of the cluster unit at time i and the magnitude and / or phase of each other cluster at time i+1 is calculated to generate L error values ​​(which can be used as part of the error vector in step 610). For example, in some embodiments, a correlation value r can be used, which can be given by the following formula.

[0102]

[0103] Where x is the magnitude M or phase φ vector at time i. y is the average value of the magnitude M or phase φ vector at time i, and y is the magnitude M or phase φ at time i+1. It is the average of the magnitude M or phase φ vector at time i+1. In embodiments where both magnitude M and phase φ are used, Equation 5 is applied to each metric (magnitude M and phase φ), and the (e.g., weighted) average between the two correlations r obtained using Equation 5 is used as a single metric associated with the SIFT characteristics of the clustering unit.

[0104] In some embodiments, for example, by using Equation 5, the correlation r between SIFT features is calculated between all clusters at time step i and all clusters at time step i+1. In some embodiments, the clusters at time steps i and i+1 that have the highest correlation and are greater than a predetermined correlation threshold are associated. In some embodiments, the predetermined correlation threshold is between 0.5 and 0.9.

[0105] Figure 10A and 10B The corresponding histograms of the input radar image and the gradient with SIFT features according to an embodiment of the present invention are shown respectively.

[0106] Figure 11 A flowchart of an embodiment method 1100 for calculating the Wasserstein distance between clusters at time steps i and i+1 according to an embodiment of the present invention is shown. In some embodiments, step 606 may be performed as step 1101.

[0107] During step 1101, the Wasserstein distance is calculated between clusters at time steps i and i+1. The Wasserstein distance (also known as land movement distance, Wasserstein metric, or Kantorovich-Rubinstein metric) is a mathematical function that calculates the distance between two probability distributions in addition to their similarity. It should be understood that the term "distance" used with respect to the Wasserstein metric refers to the distance between distributions and is not necessarily a physical distance. For example, the Wasserstein metric can be used to determine the distance between SIFT features of two clusters and / or EMD features of two clusters. The Wasserstein metric can be a physical distance in other scenarios, such as when used relative to RAI or RDI data.

[0108] In some embodiments, the raw data associated with each cluster (e.g., vectors of different feature values, such as SIFT and / or EMD, and such as distance, Doppler, and / or angle) is modeled (approximated) as a Gaussian distribution with a corresponding mean μ and standard deviation σ. For example, in an embodiment having a cluster (P1) with a mean μ1 and standard deviation σ1 at time step i and a cluster (P2) with a mean deviation μ2 and standard deviation σ2 at time step i+1, the Wasserstein metric can be calculated as:

[0109]

[0110] in It is an L2 norm.

[0111] In some embodiments, such as by using Equation 6, the Wasserstein metric is calculated between all clusters at time step i and all clusters at time step i+1. In some embodiments, the clusters with the lowest Wasserstein distance at time steps i and i+1 are associated.

[0112] Figure 12 An image 1201 showing two clusters at times i and i+1 is illustrated according to an embodiment of the present invention. Each cluster unit 1202, 1204, 1212, and 1214 is approximated as a Gaussian distribution. In some embodiments, all possible Wasserstein distances between time steps i and i+1 are determined, which in this embodiment are four Wasserstein distances, i.e., W 1202,1212 (Between clusters 1202 and 1212), W 1202,1214 (between clusters 1202 and 1214), W 1204,1212(between clusters 1204 and 1212) and W 1204,1214 (Between clusters 1204 and 1214). Equation 6 can be used to calculate each Wasserstein distance (W). 1202,1212 W 1202,1214 W 1204,1212 W 1204,1214 In some embodiments, clusters with the minimum distance are associated.

[0113] In some embodiments, image 1201 is a representation of features (e.g., SIFT, EMD) and Wasserstein metrics to determine the similarity between features. In some embodiments, image 1201 is a radar image (e.g., RAI, RDI), and the Wasserstein metric is used for motion-based tracking (e.g., using Euclidean distance).

[0114] In some embodiments, a single type of feature is used during template matching. For example, in some embodiments, step 316 is performed by executing steps 702 and 710, such as by minimizing the total mean squared error to allocate clusters at time steps i and i+1. In some embodiments, step 316 is performed by executing steps 904 and 910 and allocating clusters at time steps i and i+1, such as by maximizing the correlation between clusters. In some embodiments, step 316 is performed by executing step 1101 based solely on EMD features or solely on SIFT features, and by allocating clusters at time steps i and i+1, such as by minimizing the Wasserstein distance (e.g., minimizing the sum of Wasserstein distances between matched clusters).

[0115] Figure 13 A method for performing data association using template matching according to an embodiment of the present invention is shown, such as when applied to appendices. Figure 4 Examples.

[0116] As shown in the figure Figure 13 A non-limiting example is shown, wherein EMD feature extraction (step 602) and SIFT feature extraction (step 604) extract geometric features for clusters 402 and 404 (at time step i) and clusters 406 and 408 (at time step i+1), and for each extracted geometric feature, a Wasserstein metric is computed between each cluster detected at time step i (402 and 404) and each cluster detected at time step i+1 (406 and 408) (step 606).

[0117] like Figure 13As shown, for each cluster identified at time step i (402 and 404) and time step i+1 (406 and 408), IMFs above a predetermined threshold are identified. The magnitude M and phase φ associated with the SIFT features are also extracted for each cluster identified at time step i (402 and 404) and time step i+1 (406 and 408). Features V associated with clusters 402, 404, 406, and 408 are generated, respectively. 402 V 404 V 406 and V 408 The resulting vector. Feature V 402 V 404 V 406 and V 408 The vector is modeled as a Gaussian distribution, and for each feature (in this example, IMF, M, or φ, etc.), the Wasserstein distance is calculated between each cluster identified at time step i (402 and 404) and each cluster identified at time step i+1 (406 and 408). Therefore, for each possible allocation between the clusters identified at time steps i and i+1 (in this example, 402 / 406, 402 / 408, 404 / 406, and 404 / 408), an error vector (in this embodiment, the Wasserstein distance, although other metrics such as correlation could also be used) is generated.

[0118] Then, a similarity metric is generated, for example, by using the (e.g., weighted) average of the errors within each error vector in the error vector (Wasserstein distance in this embodiment). Thus, the similarity metric can be used as a measure of how similar two clusters are. An assignment is then performed (e.g., using a Hungarian assignment) to minimize the total error. For example, in this example, the error D... 402_406 and D 404_408 The sum is lower than the error D 404_406 and D 402_408 The sum of, and therefore clusters 402 and 406 are matched and clusters 404 and 408 are matched (also as in Figure 4 (As shown in the image).

[0119] In some embodiments, such as in Figure 6In this approach, more than one type of feature can be used to perform template matching. In such embodiments, it is possible that a first type of feature (e.g., EMD) indicates a first match (e.g., cluster A at time step i will match cluster A' at time step i+1, and cluster B at time step i will match cluster B' at time step i+1), and a second type of feature (e.g., SIFT) indicates a second, different match (e.g., cluster A at time step i will match cluster B' at time step i+1, and cluster B at time step i will match cluster A' at time step i+1). In some such embodiments, the average error can be used to determine the final cluster assignment to a particular trajectory. In some embodiments, a weighted average (with, for example, predetermined coefficients) can be used instead of an average. In some embodiments, the function F in Equation 1 can be modified to account for, for example, averaging the weight coefficients or applying them to the corresponding error measure associated with the corresponding feature.

[0120] like Figure 6 As shown, template matching can be performed using one or more of EMD and SIFT features. Some embodiments may include different features for template matching. For example, some embodiments may rely on deep learning-based feature extraction and correlation using deep convolutional neural networks (DCNNs) to process data cubes (distance-Doppler-angle data cubes, e.g., output from step 310) to extract geometric features to be used during the template matching step, in order to determine the similarity between clusters by determining the correlation between such deep learning-based geometric features and / or Wasserstein (e.g., during steps 606 and / or 610).

[0121] For example, in some embodiments, the output of the Lth layer of the DCNN can be derived from (W l H l D l ) is given, where W l H l It represents the width and height of each feature map, and D... l This is the size / number of feature maps in layer L. In some embodiments, instead of a single layer, multiple layer outputs can be processed to extract features, for example, layer L and layer L+1.

[0122] In some embodiments, the DCNN is trained to learn different geometric features by using supervised learning with a dataset that includes multiple (e.g., human) targets performing multiple activities (e.g., walking, running, idling, etc.).

[0123] In some embodiments, in addition to one or more of EMD, SIFT, and / or deep learning-based geometric features, template matching is also performed on a motion model that depends on one or more of distance, Doppler, and angle.

[0124] Exemplary embodiments of the invention are summarized herein. Other embodiments may also be understood from the entire specification and claims submitted herein.

[0125] Example 1. A method for tracking a target, the method comprising: receiving data from a radar sensor of a radar; processing the received data to detect a target; identifying a first geometric feature of a first detected target at a first time step, the first detected target being associated with a first trajectory; identifying a second geometric feature of a second detected target at a second time step; determining an error value based on the first geometric feature and the second geometric feature; and associating the second detected target with the first trajectory based on the error value.

[0126] Example 2. As in Example 1, the method of associating the second detection target with the first trajectory includes: associating the second detection target with the first trajectory when the error value is below a predetermined threshold.

[0127] Example 3. According to the method of either Example 1 or 2, wherein associating the second detected target with the first trajectory includes: associating the second detected target with the first trajectory using a Hungarian assignment.

[0128] Example 4. A method according to any one of Examples 1 to 3, wherein: identifying the first geometric feature includes: performing a first empirical mode decomposition (EMD) on the received data associated with the first detection target, identifying first intrinsic mode functions (IMFs) above a predetermined threshold from the first EMD, and sorting the first IMFs; identifying the second geometric feature includes: performing a second EMD on the received data associated with the second detection target, identifying second IMFs above a predetermined threshold from the second EMD, and sorting the second IMFs; and determining the error value includes determining the mean squared error based on the sorted first IMFs and second IMFs.

[0129] Example 5. A method according to any one of Examples 1 to 4, wherein identifying the first geometric feature includes performing SIFT feature extraction on a first radar image associated with a first time step based on received data to extract a first magnitude or a first phase associated with a first detected target, wherein identifying the second geometric feature includes performing SIFT feature extraction on a second radar image associated with a second time step based on received data to extract a second magnitude or a second phase associated with a second detected target, and wherein determining the error value includes determining the correlation between the first magnitude and the second magnitude or between the first phase and the second phase.

[0130] Example 6. According to the method of any one of Examples 1 to 5, wherein determining the error value includes determining the correlation between the first quantity and the second quantity, and between the first phase and the second phase.

[0131] Example 7. A method according to any one of Examples 1 to 6, wherein identifying the first geometric feature includes approximating the data associated with the first detection target as a Gaussian distribution with a first mean and a first standard deviation at a first time step, wherein identifying the second geometric feature includes approximating the data associated with the second detection target as a Gaussian distribution with a second mean and a second standard deviation at a second time step, and wherein determining the error value includes determining the error value based on the first mean, the second mean, the first standard deviation, and the second standard deviation.

[0132] Example 8. The method according to any one of Examples 1 to 7 further includes: using k-means clustering to cluster the detection targets into clusters of units to generate multiple clusters, wherein the first detection target and the second detection target are the first cluster and the second cluster among the multiple clusters.

[0133] Example 9. According to one of Examples 1 through 8, the method further includes using an unscented Kalman filter to track the first detected target.

[0134] Example 10. According to the method of any one of Examples 1 to 9, wherein tracking the first detected target includes location information of tracking the first detected target over time.

[0135] Example 11. The method according to any one of Examples 1 to 10 further includes: identifying a first distance and a first Doppler velocity associated with a first detection target at a first time step; and identifying a second distance and a second Doppler velocity associated with a second detection target at a second time step, wherein associating the second detection target with the first trajectory is also based on the first distance and the second distance, as well as the first Doppler velocity and the second Doppler velocity.

[0136] Example 12. According to the method of any one of Examples 1 to 11, where the first detection target and the second detection target are human targets.

[0137] Example 13. The method is based on any of Examples 1 to 12, where the first time step and the second time step are consecutive time steps.

[0138] Example 14. The method according to any one of Examples 1 to 13 further includes: transmitting a radar signal using a transmitter antenna of a radar; receiving a reflected radar signal using a receiver antenna of a radar; and generating digital data from the received reflected radar signal using an analog-to-digital converter (ADC), wherein receiving data from the radar includes receiving data from the ADC, and wherein transmitting the radar signal includes transmitting the radar signal at a frame rate of 10 frames per second or slower.

[0139] Example 15. The method according to any of Examples 1 to 14, where the radar is a millimeter-wave radar.

[0140] Example 16. The method according to any one of Examples 1 to 15 further includes: receiving additional data from another radar sensor of another radar; processing the received additional data to detect another target; identifying another geometric feature of the other detected target at a third time step; determining another error value based on the first geometric feature and the other geometric feature; and associating the other detected target with a first trajectory associated with the first detected target based on the other error value.

[0141] Example 17. A radar system includes: a millimeter-wave radar sensor, the millimeter-wave radar sensor including: a transmitting antenna configured to transmit a radar signal; a first receiving antenna and a second receiving antenna configured to receive a reflected radar signal; an analog-to-digital converter (ADC) configured to generate raw digital data at the output of the ADC based on the reflected radar signal; and a processing system configured to process the raw digital data for: detecting a target; identifying a first geometric feature of the first detected target at a first time step; identifying a second geometric feature of a second detected target at a second time step; determining an error value based on the first and second geometric features; and associating the second detected target with a first trajectory associated with the first detected target based on the error value.

[0142] Example 18. A radar system according to Example 17, wherein the transmitting antenna is configured to transmit radar signals at a rate of 10 frames per second or slower.

[0143] Example 19. A method comprising: receiving data from a radar sensor of a radar; processing the received data to detect humans; clustering the detected humans into clusters of units using k-means clustering to generate multiple clusters; identifying a first geometric feature of a first cluster among the multiple clusters at a first time step; identifying a second geometric feature of a second cluster among the multiple clusters at a second time step, wherein the first time step and the second time step are consecutive time steps; determining an error value based on the first geometric feature and the second geometric feature; and associating the second cluster with a first trajectory associated with the first cluster based on the error value.

[0144] Example 20. According to the method of Example 19, wherein: identifying the first geometric feature includes: performing a first empirical mode decomposition (EMD) on received data associated with a first cluster, identifying a first intrinsic mode function (IMF) above a predetermined threshold from the first EMD, and sorting the first IMF; identifying the second geometric feature includes: performing a second EMD on received data associated with a second cluster, identifying a second IMF above a predetermined threshold from the second EMD; and determining an error value includes determining an error value based on the first IMF and the second IMF.

[0145] Example 21. According to one of Examples 19 or 20, identifying the first and second geometric features includes using a deep convolutional neural network based on a data cube derived from the received data.

[0146] Although the invention has been described with reference to illustrative embodiments, this description is not intended to be limiting. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of the invention, will be apparent to those skilled in the art from this specification. Therefore, the appended claims are intended to cover any such modifications or embodiments.

Claims

1. A method for tracking a target, the method comprising: Receive data from the radar's radar sensors; Process the received data to detect the target; A first geometric feature of a first detected target is identified at a first time step, the first detected target being associated with a first trajectory, wherein identifying the first geometric feature includes: Perform a first empirical mode decomposition (EMD) on the received data associated with the first detection target. The first intrinsic mode function (IMF) is identified from the first EMD that is higher than a predetermined threshold, and Sort the first IMF; The second geometric feature of the second detected target is identified at the second time step, wherein identifying the second geometric feature includes: Perform a second EMD on the received data associated with the second detection target; The second IMF is identified from the second EMD that is higher than the predetermined threshold, and Sort the second IMF; An error value is determined based on the first geometric feature and the second geometric feature, wherein determining the error value includes determining the mean square error based on the sorted first IMF and second IMF; and The second detected target is associated with the first trajectory based on the error value.

2. The method of claim 1, wherein associating the second detected target with the first trajectory comprises: When the error value is lower than a predetermined threshold, the second detection target is associated with the first trajectory.

3. The method of claim 1, wherein associating the second detected target with the first trajectory comprises: The second detected target is associated with the first trajectory using a Hungarian assignment.

4. The method of claim 1, wherein identifying the first geometric feature further comprises performing SIFT feature extraction on a first radar image associated with the first time step based on the received data to extract a first magnitude or a first phase associated with the first detected target, wherein identifying the second geometric feature further comprises performing SIFT feature extraction on a second radar image associated with the second time step based on the received data to extract a second magnitude or a second phase associated with the second detected target, and wherein determining the error value comprises determining the correlation between the first magnitude and the second magnitude or between the first phase and the second phase.

5. The method of claim 4, wherein determining the error value comprises determining the correlation between the first quantity and the second quantity, and between the first phase and the second phase.

6. The method of claim 1, wherein identifying the first geometric feature further comprises approximating the data associated with the first detection target as a Gaussian distribution having a first mean and a first standard deviation at the first time step, wherein identifying the second geometric feature further comprises approximating the data associated with the second detection target as a Gaussian distribution having a second mean and a second standard deviation at the second time step, and wherein determining the error value comprises determining the error value based on the first mean, the second mean, the first standard deviation, and the second standard deviation.

7. The method according to claim 1, further comprising: The detection targets are clustered into clusters of units using k-means clustering to generate multiple clusters, wherein the first detection target and the second detection target are the first and second clusters among the multiple clusters.

8. The method of claim 1, further comprising using an unscented Kalman filter to track the first detected target.

9. The method of claim 8, wherein tracking the first detection target includes tracking the location information of the first detection target over time.

10. The method of claim 1, further comprising: The first time step identifies the first distance and the first Doppler velocity associated with the first detected target; as well as The second time step identifies a second distance and a second Doppler velocity associated with the second detected target, wherein associating the second detected target with the first trajectory is also based on the first distance and the second distance, as well as the first Doppler velocity and the second Doppler velocity.

11. The method of claim 1, wherein the first detection target and the second detection target are human targets.

12. The method of claim 1, wherein the first time step and the second time step are continuous time steps.

13. The method according to claim 1, further comprising: The radar transmitter antenna is used to transmit radar signals; The radar receiver antenna is used to receive reflected radar signals; as well as Digital data is generated from the received reflected radar signal using an analog-to-digital converter (ADC), wherein receiving the data from the radar includes receiving the data from the ADC, and wherein transmitting the radar signal includes transmitting the radar signal at a frame rate of 10 frames per second or slower.

14. The method according to claim 1, wherein the radar is a millimeter-wave radar.

15. The method according to claim 1, further comprising: Receive additional data from another radar sensor of another radar; Process the received additional data to detect additional targets; At the third time step, another geometric feature of another detected target is identified; Another error value is determined based on the first geometric feature and the other geometric feature; as well as The other detected target is associated with the first trajectory associated with the first detected target based on the other error value.

16. A radar system, comprising: Millimeter-wave radar sensors, including: The transmitting antenna is configured to transmit radar signals; A first receiving antenna and a second receiving antenna, configured to receive reflected radar signals; An analog-to-digital converter (ADC) is configured to generate raw digital data at the output of the ADC based on the reflected radar signal; and A processing system is configured to process the raw digital data for: Detection target, A first geometric feature of the first detected target is identified at a first time step, wherein identifying the first geometric feature includes: Perform a first empirical mode decomposition (EMD) on the received data associated with the first detection target. The first intrinsic mode function (IMF) is identified from the first EMD that is higher than a predetermined threshold, and The first IMF is sorted; the second geometric features of the second detected target are identified at a second time step, wherein identifying the second geometric features includes: Perform a second EMD on the received data associated with the second detection target; The second IMF is identified from the second EMD that is higher than the predetermined threshold, and The second IMF is sorted; an error value is determined based on the first and second geometric features, wherein determining the error value includes determining the mean squared error based on the sorted first and second IMFs, and The second detected target is associated with a first trajectory associated with the first detected target based on the error value.

17. The radar system according to claim 16, wherein, The transmitting antenna is configured to transmit radar signals at a rate of 10 frames per second or slower.

18. A method comprising: Receive data from the radar's radar sensors; Process the received data to detect humans; The detected humans are clustered into clusters of units using k-means clustering to generate multiple clusters. A first geometric feature of a first cluster among the plurality of clusters is identified at a first time step, wherein identifying the first geometric feature includes: Perform a first empirical mode decomposition (EMD) on the received data associated with the first cluster. The first intrinsic mode function (IMF) is identified from the first EMD that is higher than a predetermined threshold, and Sort the first IMF; A second geometric feature of a second cluster among the plurality of clusters is identified at a second time step, wherein the first time step and the second time step are consecutive time steps, and identifying the second geometric feature includes: Perform a second EMD on the received data associated with the second cluster; The second IMF is identified from the second EMD that is higher than the predetermined threshold; and Sort the first IMF; An error value is determined based on the first geometric feature and the second geometric feature, wherein determining the error value includes determining the error value based on the first IMF and the second IMF; and The second cluster is associated with the first trajectory associated with the first cluster based on the error value.

19. The method of claim 18, wherein identifying the first geometric feature and the second geometric feature comprises: A deep convolutional neural network based on a data cube derived from the received data is used.

Images