Methods for efficient radar pre-processing - Patents.com

Abstract

A system and method for pre-processing data for further processing by a machine learning model, the pre-processing comprising: generating sample data from transmitted and received signals over a period of time, allocating the sample data to a number of (L) range bins, selecting a first subset of the (M) range bins, generating evaluation data based on an evaluation of the sample data of the first subset of the (M) range bins against one or more criteria, selecting a second subset of the (N) range bins based on the evaluation data, generating calculated data based on the sample data of the second subset of the (N) range bins, and providing the evaluation data and the calculated data to a machine learning model having temporal dynamics for further processing.

Description

Technical Field

[0001]

[0001] This disclosure generally relates to preprocessing of data for efficient automatic target recognition. More specifically, without limitation, the present invention relates to a system and method for preprocessing received radar signals for the detection or recognition of objects, people, or gestures using a machine learning model with temporal dynamics.

Background Art

[0002]

[0002] Radar uses radio waves to determine the distance (ranging), angle (azimuth and elevation), and radial velocity of an object relative to a radar system site. Radar is typically used to detect and track aircraft in flight, spacecraft, missiles, etc., and to map weather formations and terrain. More recently, small radar systems have also been used to detect the movement of objects such as hand gestures for hands-free control of devices such as televisions.

[0003]

[0003] A radar system consists of a transmitter that generates electromagnetic waves in the radio wave or microwave region, a transmitting antenna, a receiving antenna (often the same antenna is used for transmitting and receiving), a receiver, and a processor for determining the characteristics of the object. The radio waves (pulses or continuous) from the transmitter are reflected from the object and return to the receiver, providing information about the position and speed of the object.

[0004]

[0004] Automatic target recognition using machine learning models has been used for gesture recognition for hands-free control of devices. In previously proposed systems, preprocessing of the data for input to such machine learning models has involved the generation of spectrograms of data derived from the received radar signals. Examples of spectrograms used for such applications include range-Doppler, micro-Doppler, range-angle, and angle-Doppler. These spectrograms comprise a rectangular map of relevant features versus the time axis.

[0005]

[0005] In the case of range-Doppler, creating a spectrogram requires calculating range profiles and Doppler (radial velocity) profiles to generate a range-Doppler surface with a large set of data representing changes in features such as range and velocity over time. This involves storing and processing a certain number of samples into a joint 2D representation of selected features. This spectrogram-based approach requires a large computational and memory load that remains constant regardless of the input data, and imposes a constant consumption of memory and power on the data preprocessing system. The same limitations also apply when extending from 2D to multidimensional representations, such as range-Doppler-angle.

[0006]

[0006] Document WO2022031299A1 discloses techniques and apparatus for implementing a smart device-based radar system capable of performing angular position estimation. A machine learning module analyzes complex range data generated to estimate the angular position of an object. The machine learning module is implemented using a multi-stage architecture. In the local stage, the machine learning module divides the complex range data into different range intervals and processes subsets of the complex range data separately using individual branching modules. In the global stage, the machine learning module merges the feature data generated from the individual branching modules using a symmetric function to generate angular position data.

[0007]

[0007] Document US2021156987A1 discloses a radar system comprising a transmitter configured to transmit electromagnetic waves, a receiver configured to receive reflected electromagnetic waves, and a processor configured to extract the relative velocity of at least one forward object to the radar system based on the electromagnetic waves received by the receiver, the processor further configured to locally adjust each resolution for scanning a forward region based on the relative velocity of at least one forward object. [Overview of the project]

[0008]

[0008] To address the shortcomings of the prior art described above, the present invention provides a "requirement" mechanism that enables intelligent adjustment of data preprocessing parameters to focus on the data of interest and conserve memory and power consumption.

[0009]

[0009] The proposed innovation disentangles features that would typically correspond to the axes of a spectrogram used in conventional systems. In the spectrogram approach, selected features (detection, range, angle, velocity, etc.) are computed for all sample data. In the proposed approach, each feature is computed over a distinct selection of sample data, with one feature computed for a first subset of samples and another for a second set of samples. The selected samples may or may not be consecutive, allowing for fine-tuning of the processing cost per feature and the range bin sampling resolution. Furthermore, the number of samples selected for a feature can be flexibly changed at any given time, enabling dynamic adjustment of processing costs according to the expected benefit at any given time.

[0010]

[0010] In one embodiment, a method is provided for preprocessing data for further processing by a machine learning model. The preprocessing comprises generating sample data from signals transmitted and received over a time period; allocating the sample data to a plurality of (L) range bins; selecting a first subset of (M) range bins; generating evaluation data based on an evaluation of the sample data of the first subset of (M) range bins against one or more criteria; selecting a second subset of (N) range bins based on the evaluation data; generating computed data based on the sample data of the second subset of (N) range bins; and providing the evaluation data and computed data to a machine learning model with temporal dynamics for further processing.

[0011]

[0011] The received signal may consist of signals received from multiple receiving antennas, and sample data may be generated and preprocessed separately for each signal from each receiving antenna. Generating sample data may involve generating an intermediate frequency signal from the transmitted and received signals over a certain time period, and applying a Fourier transform to the intermediate frequency signal. The intermediate frequency signal may be generated by mixing the transmitted and received signals.

[0012]

[0012] Each range bin may contain sample data for each subrange of the frequency range of the transmitted signal. In this way, range bins may contain sample data arranged according to the range to which the sample data pertains. The size and total number of (L) range bins can be predetermined or adjusted during the operation of the system.

[0013]

[0013] The data provided to a machine learning model may be limited to evaluation data for a first subset of (M) range bins and computed data for a second subset of (N) range bins, but other data may also be provided to the machine learning model, such as sample data for the first subset of (M) range bins and / or the second subset of (N) range bins, and other data computed or derived from the sample data for the first subset of (M) range bins and / or the second subset of (N) range bins.

[0014]

[0014] A first subset of the (M) range bins for a time period can be selected based on sample data generated during the previous time period. In this way, the selection of the first subset of the (M) range bins is dynamic and a function of the previous sample data, and it may be possible to change each time period.

[0015]

[0015] The criteria used to determine the second subset of the (N) range bins may include whether the sample data in each range bin indicates that the object of interest was detected. Thus, the selection of the second subset of the (N) range bins is dynamic, a function of evaluation data, and each time period may be changeable. Furthermore, since the selection of the second subset of the (N) range bins is performed based on different data than the selection of the first subset of the (M) range bins, the selection of the second subset of the (N) range bins may be performed independently of the selection of the first subset of the (M) range bins.

[0016]

[0016] The evaluation of sample data from a first subset of (M) range bins may involve comparing the size of the sample data in the range bins to a threshold, which may be adjusted based on a calculated probability of false positives.

[0017]

[0017] The calculated data may be arrival angle data that can be calculated based on a first sample data generated from a first received signal from a first receiving antenna and a second sample data generated from a second received signal from a second receiving antenna.

[0018]

[0018] The transmitted and received signals may be radar signals, such as frequency-modulated continuous-wave radar signals, and the time duration may be determined based on the duration of the chirp in the transmitted signal.

[0019]

[0019] The method may further comprise repeating preprocessing for a series of consecutive time periods to provide a machine learning model with a series of evaluation and computed data associated with a series of consecutive time periods for further processing. The machine learning model may be adapted to perform object, person, or gesture detection or recognition based on the sample and computed data provided for the sequence of time periods.

[0020]

[0020] In another embodiment, a system is provided for preprocessing data for further processing by a machine learning model with temporal dynamics. The system comprises an input circuit for receiving transmitted and received signals, and a processor, the processor configured to generate sample data from transmitted and received signals over a time period, allocate the sample data to a plurality of (L) range bins, select a first subset of (M) range bins, generate evaluation data based on an evaluation of the sample data of the first subset of (M) range bins against one or more criteria, select a second subset of (N) range bins based on the evaluation data, and generate computed data based on the sample data of the second subset of (N) range bins. The system also includes an output circuit adapted to provide the evaluation data and computed data to a machine learning model with temporal dynamics for further processing.

[0021]

[0021] The processor may comprise a microprocessor, ASIC, FPGA, or a combination of these and other circuits. The system may also comprise memory, which may be implemented as blocks of memory or as distributed memory elements. The system may further comprise a sensor comprising one or more transmitting antennas and one or more receiving antennas, the sensor generating received signals from one or more receiving antennas. The system may further comprise a machine learning model. The system, including the sensor and the machine learning model, may be integrated onto a single semiconductor chip.

[0022]

[0022] The processor may be configured to select a first subset of (M) range bins based on sample data generated during a previous time period. The processor may be configured to select a second subset of (N) range bins based on whether the sample data of each range bin indicates that an object of interest has been detected. The evaluation of the sample data of the first subset of (M) range bins may comprise comparing the magnitude of the sample data of the range bins with a threshold value, and the processor may be configured to adjust the threshold value based on the calculated probability of false detection.

[0023]

[0023] The processor may be configured to repeatedly generate evaluation data and calculated data for a series of consecutive time periods for providing to a machine learning model for further processing a series of evaluation data and calculated data associated with the series of consecutive time periods.

[0024]

[0024] Next, embodiments are described by way of example only with reference to the accompanying schematic drawings, where corresponding reference numerals indicate corresponding parts.

Brief Description of the Drawings

[0025] [Figure 1]

[0025] FIG. 1 is a schematic diagram of a radar system that may be used with the present invention. [Figure 2a]

[0026] FIG. 2a is a schematic diagram showing data of a first subset of M range bins generated for each time period over a certain time period. [Figure 2b] FIG. 2b is a schematic diagram showing data of a first subset of M range bins generated for each time period over a certain time period. [Figure 2c] FIG. 2c is a schematic diagram showing data of a first subset of M range bins generated for each time period over a certain time period. [Figure 3]

[0027] Figure 3 is a flowchart illustrating an example of data preprocessing for a system like the one shown in Figure 1. [Figure 4]

[0028] Figure 4 is a schematic diagram showing the data of a first subset of M range bins and the data of a second subset of N range bins, generated for each time period over a certain time period. [Modes for carrying out the invention]

[0026]

[0029] These figures are for illustrative purposes only and do not serve as a limitation of the scope or protection defined by the claims.

[0027]

[0030] The following describes specific embodiments in further detail. However, it should be understood that these embodiments should not be construed as limiting the scope of protection of this disclosure.

[0028]

[0031] Figure 1 is a schematic diagram of a radar system 100 comprising one or more transmitting antennas 101 and one or more receiving antennas 102, which are coupled to a transmitting / receiving circuit 103 and together form a sensor 105. This system transmits radio waves, known as radar signals, in a predetermined direction. When an object 110 is within the system's detection range, the transmitted signal is typically reflected or scattered by the object, and a portion of the signal is received by the receiving antennas 102. The received signal can be processed to obtain range (distance to the object), angle (azimuth and elevation), and radial velocity (Doppler).

[0029]

[0032] The system 100 in Figure 1 includes a data preprocessing system 106, which includes a memory 107 for storing signal data and processed data, and a processor 108 for performing data preprocessing. The data preprocessing system 106 may be configured to provide preprocessed data to a machine learning model 109 for further processing in order to infer the characteristics of an object from the received signal, such as its position, displacement, velocity, number, material, shape, and the category to which it belongs, and to infer the type of gesture being performed.

[0030]

[0033] The data preprocessing system 106 and the machine learning model 109 may be implemented as hardware circuits, software, or a combination of hardware circuits and software, for example, using a microprocessor, ASIC, FPGA, or a combination of these and other circuits. The processor 108 may similarly be implemented in various different ways using a microprocessor, ASIC, FPGA, or a combination of these and other circuits. The memory 107 may be implemented as a block of memory or as distributed memory elements.

[0031]

[0034] The data preprocessing system 106 and the machine learning model 109 may be implemented as separate units (e.g., in separate integrated circuits) or as a single integrated unit (e.g., both implemented by an integrated circuit on a single semiconductor chip). A complete system may be implemented using the sensor 105 (including the transmitting antenna 101 and the receiving antenna 102), the data preprocessing system 106, and the machine learning model 109, all of which are integrated on a single semiconductor chip.

[0032]

[0035] Different types of radar signals, such as pulsed or continuous-wave radar signals, may be used, whether frequency-modulated or unmodulated. Various types of frequency modulation of the signal may be used, for example, the signal frequency may vary in a sawtooth, triangular, or segmented linear waveform. Although embodiments of the present invention are described in the context of continuous-wave radar signals, the present invention is equally applicable to other types of systems and other types of signals.

[0033]

[0036] Figure 3 is a flowchart illustrating an example of data preprocessing for a system like the one shown in Figure 1.

[0034]

[0037] In step 301, the signal is received for data preprocessing. In the case of a radar system, the received signal is essentially an attenuated and delayed version of the transmitted signal. The received signal is preferably generated from each of two or more receiving antennas to facilitate the calculation of the angle of arrival of the received signal. The radar system comprises one or more transmitting antennas and one or more receiving antennas, preferably three or more receiving antennas having a pair of receiving antennas separated in the x-axis direction and a pair of receiving antennas separated in the y-axis direction. The signal received from each receiving antenna is preferably processed on a separate channel.

[0035] [Generate Range Profile]

[0038] In step 302, a range profile is generated from the transmitted and received signals over a certain period of time. The received signal from the first antenna is first processed in the first channel to obtain data suitable for further processing. For example, an intermediate frequency (IF) signal representing the difference between the transmitted and received signals is typically derived. This signal may be generated by mixing (e.g., multiplying) the transmitted and received signals, and various known preprocessing steps may be performed as needed, such as noise reduction, signal filtering and / or scaling, and conversion of the signal from analog to digital format.

[0036]

[0039] IF signal data is divided into time periods to generate a range profile. For example, a time period could be the duration of one "chirp" in a frequency-modulated continuous wave (FMCW) radar, where the frequency of the transmitted signal is swept through its frequency range. One chirp in an FMCW radar corresponds to a time period in which the frequency of the transmitted signal changes over its frequency range, for example, a period in which the frequency is swept from the lowest frequency to the highest frequency for frequency modulation following a sawtooth function. When the transmitted signal is not frequency-modulated, the time period may be determined based on a suitable time window for the transmitted signal. The range profile represents the received signal data from the shortest range (lowest frequency of the IF signal) to the longest range (highest frequency of the IF signal).

[0037]

[0040] The range profile comprises sample data generated from the IF signal and allocated to range bins. The frequency of the IF signal is a function of the range (e.g., the distance between the radar system and the detected object reflecting the transmitted signal), but its frequency can also vary to some extent depending on other factors such as the velocity of the detected object and changes in the density of the medium through which the signal passes.

[0038]

[0041] Sample data can be calculated from the IF signal for each time period, for example, by applying a Fourier transform to the IF signal over a given time period, in order to generate a range profile containing sample values ​​representing the IF signal amplitude in different ranges over a given time period. Fast Fourier Transform (FFT) or Discrete Fourier Transform (DFT) algorithms may be used, or other algorithms such as Goertzel or the Multiple Signal Classification (MUSIC) algorithm may be used to generate the range profile. The generated sample data may take the form of real, imaginary, or complex numbers.

[0039]

[0042] The time period used to calculate the sample data may be selected as the duration of each chirp in a frequency-modulated radar, or as a suitable time window in a non-frequency-modulated radar. The sample data for each time period forms a range profile. The sample data may be preprocessed, for example, by applying a Moving Target Indication (MTI) technique to the data to remove or reduce sample data associated with stationary objects.

[0040] [Allocation of sample data to range bins]

[0043] In step 303, the sample data is allocated to multiple (L) range bins. The transmitted and received signals vary within a certain frequency range, which can be subdivided into several frequency subranges forming range bins, each range bin containing sample data for its respective frequency subrange. The sample data for each range profile is allocated to several (L) range bins, each range bin containing sample data for its respective frequency subrange. The size and total number of (L) range bins can be predetermined or adjusted during system operation.

[0041]

[0044] The signal received from the second receiving antenna may also be processed in parallel on the second channel, as described above, and this process includes generating a second IF signal, generating second sample data from the second IF signal, and allocating the second sample data to a second range bin. Similarly, signals received from any further receiving antennas may also be processed in parallel on further channels in a similar manner to obtain sample data that is allocated to a range bin for each such receiving antenna. Thus, in the case of a system with three receiving antennas, sample data is generated and allocated to the corresponding range bins for the three received signals from the three receiving antennas.

[0042] [Selection of the first subset of sample data of interest]

[0045] In step 304, a first subset of (M) range bins is selected. The size, number, and position of the first subset of range bins can be predetermined or adjusted dynamically during the operation of the system. The first subset of range bins can be updated for each time period, for example, for each chirp in a frequency-modulated radar or for each time window in a radar without frequency modulation. The first subset of (M) range bins can include all (L) range bins (M = L) or a selection of fewer range bins than all (M < L). The selected range bins can be consecutive (adjacent) range bins or non-consecutive ones covering different parts of the detection range of the system.

[0043]

[0046] FIG. 2a is a schematic diagram showing a first subset of range bins numbered from 1 to M. The range bins are generated for each time period to form a sequence of subsets of M range bins, and each range bin has associated sample data.

[0044]

[0047] This step operates as a first selection of data of interest, so that subsequent processing of the data focuses in an intelligent way on data that is particularly useful for related applications, such as object or gesture recognition. The selection of range bins can be based on the range profile (sample data) of the previous time period, or sample data processed earlier within the range profile of the current time period, or other data providing an indication that the range bins selected in the current time period contain sample data of interest. Thus, the selection of data is dynamic, a function of the actual data, and can be changed for each time period.

[0045]

[0048] Selection may be made, for example, to reduce the number of range bins that need to be processed further if no object was detected in a previous time period, or to increase the number of range bins if an object was detected in a previous time period. Selection may also be made to select only range bins associated with a particular range if an object was detected within that range in a previous time period. If an object was detected within a particular range in a previous time period (e.g., the most recent time period or several previous time periods), it is likely that the object will be detected within the same or a similar range in the current time period.

[0046]

[0049] Therefore, selecting a range bin focuses the processing resources spent by the system on the range of interest, reduces the total processing resources consumed, and thus saves power.

[0047] [Selective preprocessing (generation of evaluation data)]

[0050] In step 305, selective preprocessing is performed on a first subset of the (M) range bins. This may involve generating evaluation data based on an evaluation of sample data from the first subset of the (M) range bins against one or more criteria. The criteria used for this evaluation may be based on any one or more features of the sample data, e.g., range (distance from the radar system to the detected object), angle (azimuth and / or elevation between the radar system and the detected object), detection (whether the object is detected within the range encompassed by the range bins), radial velocity (velocity of the detected object to and from the radar system), the rate of change of one of these features, or other values ​​derived from one of these features. The evaluation may also involve multiple tests against multiple criteria.

[0048]

[0051] For example, selection may be based on whether the sample data in a range bin indicates that an object of interest has been detected within the range covered by the range bin (i.e., the relevant feature is "detection"). This may be done based on a detection threshold, where the size of the sample data in each range bin is compared to the detection threshold. In this example, the criterion for selection might be whether the size of the signal data in a range bin exceeds the detection threshold; if so, that range bin is selected for selective preprocessing.

[0049]

[0052] Figures 2a and 2c are schematic diagrams showing a first subset of range bins numbered 1 through M, generated for each time period over a certain time period. Figure 2a shows a situation where the first range bin (bin 1) in each time period satisfies the relevant criteria, for example, when an object is detected within the range represented by the first range bin. Figure 2b shows a situation where the first range bin (bin 1) in the first time period satisfies the relevant criteria, and the last range bin (bin M) in the second time period also satisfies the relevant criteria. Figure 2c shows a situation where both the first range bin (bin 1) in the first time period and the first and last range bins (bin 1 and bin M) in the second time period satisfy the relevant criteria.

[0050]

[0053] The criteria used in the evaluation process can be predetermined and static, for example, using a static threshold determined experimentally by a hyperparameter grid search involving the selection of a threshold and the evaluation of the results. Alternatively, the criteria can be dynamic, for example, using an adaptive threshold adjusted for each time period based on the calculated probability of false positives using methods such as constant false positive rate (CFAR), cell-averaged CFAR, ordinal statistical CFAR, or other methods.

[0051]

[0054] Selective preprocessing and generation of evaluation data are performed only for the data of interest, i.e., only for the first subset of the (M) range bins, and not for the other range bins that were not selected for the first subset. Selective preprocessing generates evaluation data for use by machine learning models focused on the data of interest. In addition, the evaluation data is used for further selection of the data of interest to enable further reduction of the required data processing. In this way, the amount of preprocessing is further reduced to focus the processing resources spent by the system on the data of interest, thereby reducing the processing resources consumed and saving power.

[0052] [Data selection for further selective preprocessing]

[0055] Step 306 involves further selection of the data of interest. A second subset of the (N) range bins is selected based on the evaluation data generated for the first subset of the (M) range bins. This selection process may result in selecting all range bins that meet the criteria, selecting a certain number of range bins that meet the criteria (e.g., selecting the first three range bins that meet the first selection criterion), or applying a different selection method to the first set of range bins based on the evaluation data.

[0053]

[0056] For example, if evaluation data indicates whether an object of interest was detected in each range bin of a first subset of range bins, the selection process may involve selecting all range bins in which the object was detected, or selecting the first three range bins (with the lowest range values) in which the object was detected, or selecting the three range bins with the highest signal magnitudes among the range bins in which the object was detected, or applying some other preferred selection rule.

[0054]

[0057] The number of range bins in the second subset of (N) range bins can vary between 0, indicating that none of the range bins contain the data of interest and therefore no range bins are selected, and M, indicating that all of the range bins in the first subset of range bins contain the data of interest and therefore all range bins are selected. However, the criterion used for selection preferably results in fewer range bins being selected to conserve processing resources.

[0055]

[0058] This step acts as a second selection of data of interest, and as a result, subsequent processing of the data is further focused in an intelligent way on data that is particularly useful for the relevant application. The second selection of data is also dynamic, a function of sample data, and each time period can be changed. The first and second selections of data of interest (steps 304 and 306) can be adjusted independently, with the first selection based on the range profile (sample data) of the previous time period, while the second selection is based on evaluation data derived from the sample data selected by the first selection.

[0056] [Selective preprocessing (generation of computed data)]

[0059] In step 307, selective preprocessing of a second subset of the (N) range bins is then performed. Selective preprocessing may include, for example, generating computed data from the sample data of the second subset of the range bins for input to a machine learning model. This selection of sample data / range bins is done to focus on the data of interest, i.e., the data that will be most useful in subsequent data processing by the machine learning model for the relevant application, e.g., for object or gesture detection or recognition.

[0057]

[0060] For example, selective preprocessing may include calculating arrival angle data for data samples in a second subset of the (N) range bins selected by the selection process. The arrival angle data may be calculated from the sample data in each range bin, for example, by calculating the phase difference between signals received by two receiver antennas.

[0058]

[0061] The second selective preprocessing and generation of the computed data is performed only for the data of further interest, i.e., only for the second subset of the (N) range bins, and not for the other range bins that were not selected for the second subset. Thus, the selective preprocessing generates computed data for use by the machine learning model, focused on the data of further interest. This, as before, reduces the amount of preprocessing that focuses the processing resources consumed by the system, saving power.

[0059]

[0062] Since the first and second selections, which determine the first and second subsets of the range bin, are based on different data, the first and second selections of the data of interest (steps 304 and 306) and the first and second selective preprocessing (steps 305 and 307) can be coordinated independently.

[0060]

[0063] The dimensions of each feature can be adjusted independently. This means that when a feature is used as the basis for range bin selection, the number of range bins, and therefore the size of the dataset provided as input for further processing (e.g., as input to a machine learning model), will change depending on the adjustment of the criteria (e.g., threshold) associated with that feature. If more than one feature is used for range bin selection, the criteria for each feature can be adjusted independently.

[0061]

[0064] This adjustment of the criteria for a feature may be a function of the current or previous values ​​of the feature, or a function of different features, or a function of the current or previous values ​​of a combination of features.

[0062]

[0065] For example, when the selection process is based on object detection features that use a detection threshold, the appearance or disappearance of an object within the detection range of the radar signal can result in a change in the number of range bins selected. When an object appears within the detection range, the number of range bins with sample data exceeding the detection threshold increases, and the number of range bins selected may increase accordingly. Conversely, when an object disappears from the detection range, the number of range bins selected by the data selection process may decrease accordingly.

[0063]

[0066] Similarly, when the selection feature is a range or angle of interest, the presence of one or more objects within that range or angle of interest may result in a change in the number of range bins selected.

[0064]

[0067] As the number of range bins selected by the data selection process changes, the number of data samples undergoing data preprocessing also changes accordingly.

[0065]

[0068] The number of samples used to calculate the angle of arrival is dynamically optimized; if fewer than N samples have values ​​exceeding the detection threshold, the same number of fewer than N samples is used for calculating the angle of arrival. Therefore, the number of samples is subject to dynamic changes between 0 and N.

[0066] [Additional data selection and selective preprocessing]

[0069] The flowchart in Figure 3 illustrates an example involving two data sample selections (generating two subsets of a range bin) and two selective preprocessing steps. However, additional data selections and selective preprocessing may also be performed. Additional data selections may be based on different characteristics than those used for the first two data selections, and additional selective preprocessing may involve calculating different values ​​and / or performing different processing than in the first two selective preprocessing steps.

[0067] [Output of evaluation data and calculated data]

[0070] In step 309, the system provides preprocessed data, comprising evaluation data associated with a first subset of (M) range bins and computed data associated with a second subset of (N) range bins, for input to a machine learning model with temporal dynamics. For example, the evaluation data may comprise detection data indicating whether an object was detected within the range encompassed by each range bin in the first subset of (M) range bins, and the computed data may comprise angle of arrival data for each range bin in the second subset of (N) range bins. If additional data selection and selective preprocessing steps are included, the evaluation data and / or computed data resulting from these additional steps may also be provided to the machine learning model.

[0068]

[0071] The system repeatedly performs the data preprocessing steps in Figure 3 to generate a set of evaluation data and calculated data for each time period in the sequence of time periods. The resulting sets of sample data and calculated data are sequentially provided to a machine learning model for further processing.

[0069]

[0072] Figure 4 is a schematic diagram showing evaluation data 404 of a first subset of M range bins and computed data 405 of a second subset of N range bins, generated for each time period t. The evaluation data 404 and computed data 405 can be provided directly to the machine learning model 109 or encoded for processing by the machine learning model.

[0070]

[0073] For example, in the case of evaluation data with detection, the sample data is processed to remove static detection (e.g., using a moving target designation technique to distinguish moving targets from clutter), and the magnitude of the processed sample data is binarized to determine the detected / non-detected values. In the case of calculated data with arrival angles, the calculated data may be quantized; for example, azimuth data may be quantized as left-center-right, and elevation data may be quantized as up-center-down.

[0071]

[0074] Machine learning models are configured to model dynamically changing time data. For example, spiking neural networks (SNNs), recurrent neural networks (RNNs), or long-term short-term memory (LSTM) networks may be used.

[0072]

[0075] While embodiments are described in the context of systems for detecting or recognizing objects, people, or gestures, the present invention may also be applicable to applications that benefit from the described types of preprocessing of data for input to machine learning models.

[0073]

[0076] Although the embodiments are described in the context of radar signals, the present invention may also be applicable to other types of signals that generate data for input to machine learning models and would benefit from the data preprocessing described.

[0074]

[0077] While specific embodiments of the present invention have been described herein, the present invention is not limited to these embodiments. Two or more of the above embodiments may be combined in any suitable manner, and the elements and components of the described embodiments may be replaced or modified using substitutes readily apparent to those skilled in the art. The following is a direct reproduction of the claims as originally filed. [C1] A method for preprocessing data for further processing by a machine learning model, wherein the method is: To generate sample data from signals transmitted and received over a certain period of time, The sample data is allocated to multiple (L) range bins, Selecting a first subset of the (M) range bins, Based on the evaluation of the sample data of the first subset of the (M) range bins for one or more criteria, evaluation data is generated. Based on the aforementioned evaluation data, a second subset of the (N) range bins is selected, To generate calculated data based on the sample data of a second subset of the (N) range bins, For further processing, the evaluation data and the calculated data are provided to a machine learning model that has temporal dynamics. A method for providing this. [C2] The method according to C1, wherein a first subset of the (M) range bins for the time period is selected based on sample data generated during the previous time period. [C3] The method of C1 or 2, wherein the criterion used to determine a second subset of the (N) range bins is whether the sample data for each range bin indicates that an object of interest has been detected. [C4] The method according to any one of C1 to C3, wherein the evaluation of sample data of a first subset of the (M) range bins comprises comparing the size of the sample data of the range bins with a threshold. [C5] The method according to C4, wherein the threshold is adjusted based on the calculated probability of a false positive. [C6] The calculated data is the angle of arrival data, as described in any one of C1 to C5. [C7] The method according to C6, wherein the angle of arrival data is calculated based on a first sample data generated from a first received signal from a first receiving antenna and a second sample data generated from a second received signal from a second receiving antenna. [C8] The method according to any one of C1 to C7, further comprising repeating the preprocessing step described in C1 for a series of consecutive time periods in order to provide a series of evaluation data and calculated data associated with a series of consecutive time periods to the machine learning model for further processing. [C9] The method according to any one of C1 to C8, wherein the transmitted signal and the received signal are radar signals. [C10] The method according to any one of C1 to C9, wherein the transmitted signal is a frequency-modulated continuous-wave radar signal, and the time period is determined based on the duration of the chirp of the transmitted signal. [C11] The method according to any one of C1 to C10, wherein the machine learning model is adapted to perform object or gesture detection or recognition based on the sample data and the calculated data provided for a sequence of time periods. [C12] A system (100) for preprocessing data for further processing by a machine learning model (109) having temporal dynamics, wherein the system is configured to receive transmitted and received signals, To generate sample data from signals transmitted and received over a certain period of time, The sample data is allocated to multiple (L) range bins, Selecting a first subset of the (M) range bins, Based on the evaluation of the sample data of the first subset of the (M) range bins for one or more criteria, evaluation data is generated. Based on the aforementioned evaluation data, a second subset of the (N) range bins is selected, To generate calculated data based on the sample data of a second subset of the (N) range bins, A processor (108) configured to perform the following: System (100) is configured to provide the evaluation data and the computed data to a machine learning model having temporal dynamics for further processing. [C13] The system (100) according to C12, wherein the processor (108) is configured to select a first subset of the (M) range bins based on sample data generated during a previous time period. [C14] The system (100) according to C12 or C13, wherein the processor (108) is configured to select a second subset of the (N) range bins based on whether the sample data in each range bin indicates that an object of interest has been detected. [C15] The system (100) according to any one of C12 to C14, wherein the processor (108) is configured to repeatedly generate the evaluation data and calculated data for a series of consecutive time periods in order to provide the machine learning model with a series of evaluation data and calculated data associated with a series of consecutive time periods for further processing. [C16] A system (100) according to any one of C12 to C15, further comprising a sensor (105) including one or more transmitting antennas (101) and one or more receiving antennas (102) integrated on a single semiconductor chip, and a machine learning model (109).

Claims

1. A method (300) for preprocessing data for further processing by a machine learning model (109), wherein the method is: (302) Generating sample data from signals transmitted and received over a certain period of time, (303) Allocating the sample data to multiple (L) range bins, Selecting a first subset of the (M) range bins (304), Based on the evaluation of the sample data of the first subset of the (M) range bins for one or more criteria, the evaluation data (404) is generated (305), Based on the aforementioned evaluation data (404), a second subset of the (N) range bins is selected (306), Based on the sample data of the second subset of the (N) range bins, the calculated data (405) is generated (307), For further processing, the evaluation data (404) and the calculated data (405) are provided to a machine learning model (109) having temporal dynamics (308), A method for providing this.

2. The method according to claim 1, wherein a first subset of the (M) range bins for the aforementioned time period is selected based on sample data generated during the previous time period.

3. The method according to claim 1 or 2, wherein the criterion used to determine a second subset of the (N) range bins comprises whether the sample data for each range bin indicates that an object of interest has been detected.

4. The method according to claim 1 or 2, wherein the evaluation of sample data from a first subset of the (M) range bins comprises comparing the size of the sample data in the range bins to a threshold, preferably the threshold is adjusted based on a calculated probability of false detection.

5. The method according to claim 1 or 2, wherein the criteria used for the evaluation are based on any one or more features of the sample data, preferably each feature is calculated over a distinct selection of the sample data, and more preferably one feature is calculated over the first subset of the sample and another feature is calculated over the second set of the sample, in order to allow for fine-tuning of the processing cost per feature and the range bin sampling resolution.

6. The method according to claim 1 or 2, wherein the calculated data (405) is arrival angle data, and preferably the arrival angle data is calculated based on a first sample data generated from a first received signal from a first receiving antenna and a second sample data generated from a second received signal from a second receiving antenna.

7. The method according to claim 1 or 2, further comprising repeating the preprocessing step according to claim 1 for a series of consecutive time periods in order to provide a series of evaluation data (404) and calculated data (405) associated with a series of consecutive time periods to a machine learning model (109) for further processing.

8. The method according to claim 1 or 2, wherein the transmitted signal and the received signal are radar signals.

9. The method according to claim 1 or 2, wherein the transmitted signal is a frequency-modulated continuous-wave radar signal, and the time period is determined based on the duration of the chirp in the transmitted signal.

10. The method according to claim 1 or 2, wherein the machine learning model (109) is adapted to perform object or gesture detection or recognition based on the sample data and calculated data (405) provided for a sequence of time periods.

11. A system (100) for preprocessing data for further processing by a machine learning model (109) having temporal dynamics, wherein the system is configured to receive transmitted and received signals, and the system To generate sample data from signals transmitted and received over a certain period of time, The sample data is allocated to multiple (L) range bins, Selecting a first subset of the (M) range bins, Based on the evaluation of the sample data of the first subset of the (M) range bins for one or more criteria, evaluation data (404) is generated. Based on the aforementioned evaluation data (404), a second subset of the (N) range bins is selected, Based on the sample data of the second subset of the (N) range bins, the calculated data (405) is generated. A processor (108) configured to perform the following: The system (100) is configured to provide the evaluation data (404) and the calculated data (405) to a machine learning model (109) having temporal dynamics for further processing.

12. The system (100) according to claim 11, wherein the processor (108) is configured to select a first subset of the (M) range bins based on sample data generated during a previous time period.

13. The system (100) according to claim 11 or 12, wherein the processor (108) is configured to select a second subset of the (N) range bins based on whether the sample data in each range bin indicates that an object of interest has been detected.

14. The system (100) according to claim 11 or 12, wherein the processor (108) is configured to repeatedly generate the evaluation data (404) and calculated data (405) for a series of consecutive time periods in order to provide the machine learning model (109) with the evaluation data (404) and calculated data (405) associated with a series of consecutive time periods for further processing.

15. The system (100) according to claim 11 or 12, further comprising a sensor (105) including one or more transmitting antennas (101) and one or more receiving antennas (102) integrated on a single semiconductor chip, and a machine learning model (109).

Images