Apparatus, method, radar system and electronic device

Abstract

The invention relates to a device, a method, a radar system and electronic equipment. There is presented an apparatus comprising processing circuitry configured to acquire radar data indicative of a received radar signal of a radar sensor, acquire data indicative of a detection area in which a person to be detected is located, modify the radar data to shield an undesired area outside the detection area, and transmit the modified radar data to the radar sensor. And determining the number of people within the detection area using the trained neural network based on the modified radar data.

Description

Technical Field

[0001] The present disclosure relates to radar-based detection of people. Examples relate to an apparatus, a method, a radar system, and an electronic device. Background Art

[0002] Conventional radar-based methods for detecting people face the problem of fixed detection parameters. These limitations can lead to inaccurate detection, missed events, or false alarms. Therefore, there is a need for improved radar sensing. Summary of the invention

[0003] This need is met by the subject matter of the independent claims.

[0004] Some aspects of the present disclosure relate to an apparatus that includes a processing circuit device configured to obtain radar data indicating a received radar signal of a radar sensor, obtain data indicating a detection area where a person to be detected is located, modify the radar data to mask undesirable areas outside the detection area, and determine a number of people within the detection area using a trained neural network based on the modified radar data.

[0005] Some aspects of the present disclosure relate to a radar system comprising the apparatus described herein and a radar sensor, wherein the radar sensor is configured to generate radar data based on received radar signals.

[0006] Some aspects of the present disclosure relate to an electronic device that includes the radar system described herein and a control circuit device configured to control the electronic device based on a determined number of people.

[0007] Some aspects of the present disclosure relate to a method that includes acquiring radar data indicative of received radar signals of a radar sensor, acquiring data indicative of a detection zone in which a person to be detected is located, modifying the radar data to mask undesirable areas outside the detection zone, and determining a number of people within the detection zone using a trained neural network based on the modified radar data. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Some examples of apparatus and / or methods will be described below, by way of example only, and with reference to the accompanying drawings, in which:

[0009] Figure 1 An example of an apparatus is illustrated;

[0010] Figure 2 An example of a method for pre-processing radar data is illustrated;

[0011] Figure 3a and Figure 3billustrates examples of range-Doppler and range-angle representations of radar data;

[0012] Figures 4a to 4c An example of microscopic motion data is illustrated;

[0013] Figure 5 An example of a radar system is illustrated;

[0014] Figure 6 An example of an electronic device is illustrated;

[0015] Figure 7 An example of a method is illustrated; and

[0016] Figure 8 An example of a detection area within the field of view of a radar sensor is illustrated. DETAILED DESCRIPTION

[0017] Some examples are now described in more detail with reference to the accompanying drawings. However, other possible examples are not limited to the features of these embodiments described in detail. Other examples may include modifications of features and equivalents and substitutes of features. In addition, the terms used to describe some examples herein should not be limited to other possible examples.

[0018] Throughout the description of the figures, the same or similar reference numerals refer to the same or similar elements and / or features, which may be implemented in the same or modified form while providing the same or similar functions. The thickness of the lines, layers and / or regions in the figures may also be exaggerated for illustration.

[0019] Unless otherwise explicitly defined in individual cases, when two elements A and B are combined using "or", it should be understood that all possible combinations are disclosed, namely only A, only B, and A and B. As alternative expressions for the same combination, "at least one of A and B" or "A and / or B" can be used. The same applies to combinations of more than two elements.

[0020] If singular forms such as "a", "an" and "the" are used, and the use of only a single element is not explicitly or implicitly defined as mandatory, additional examples may also use several elements to implement the same functionality. If a function is described below as being implemented using multiple elements, additional examples may use a single element or a single processing entity to implement the same functionality. It is also to be understood that the terms "includes", "including", "comprises" and / or "comprising" when used describe the presence of specified features, wholes, steps, operations, processes, elements, parts and / or combinations thereof, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, processes, elements, parts and / or combinations thereof.

[0021] Figure 1 An example of the apparatus 100 is shown. The apparatus 100 is considered in the context of a radar sensor. For example, the apparatus 100 may be integrated into a system including the following reference Figure 5 The device 100 may be in the radar system of the radar sensor, or may be outside the radar system. In the former case, the device 100 may be outside the radar sensor or (eg partially or completely) integrated into the radar sensor.

[0022] Apparatus 100 includes processing circuitry 110 and, optionally, interface circuitry 120. If present, interface circuitry 120 may be communicatively coupled (e.g., via a wired or wireless connection) to processing circuitry 110, e.g., for data exchange between interface circuitry 120 and processing circuitry 110.

[0023] Processing circuit device 110 can be, for example, a single dedicated processor, a single shared processor, or multiple individual processors (some or all of which can be shared), digital signal processor (DSP) hardware, application specific integrated circuit (ASIC), microcontroller, or field programmable gate array (FPGA). Processing circuit device 110 can optionally be coupled to read-only memory (ROM), random access memory (RAM), and / or non-volatile memory, for example, for storing software.

[0024] The interface circuit device 120 can be any device or component for transferring or exchanging data. For example, the interface circuit device 120 can be a set of electronic components, circuits and / or subsystems for interacting between different interface entities (such as devices, systems or components). It can include voltage level converters, buffers, amplifiers, filters, converters, multiplexers, demultiplexers and / or various other electronic components. These components can be configured to make signals suitable for communication, convert data formats, match impedances, ensure signal integrity, etc.

[0025] The processing circuitry 110 is configured to acquire radar data indicative of received radar signals of the radar sensor and to acquire data indicative of a detection zone in which a person to be detected is located.

[0026] A received radar signal refers to a reflection of a radio frequency signal detected by a radar sensor, i.e., after the radio frequency signal has been bounced or scattered by an object in the surrounding environment. For example, a radar sensor may be configured to transmit a radio frequency signal into the field of view of the radar sensor and generate radar data based on the received reflections. For example, the radar data may indicate or be a sample of an intermediate frequency (IF) signal or a baseband signal generated by the radar sensor. For example, an IF signal may be created by mixing the reflections with a local oscillator signal of a specific frequency. A baseband signal may be generated by down-converting or demodulating the received reflections to a baseband frequency. A radar sensor (or processing circuit device 110) may determine radar data by, for example, sampling a received radar signal (e.g., an IF signal) and performing additional processing on the radar data. The radar sensor (or processing circuit device 110) may optionally modify the sampled signal in a preprocessing step, for example, for noise reduction, DC removal (direct current), etc. In some examples, the device 100 may include a memory configured to store radar data.

[0027] A detection zone refers to a demarcated area in space to which detection of a person should be limited. For example, a detection zone may be an area of ​​interest (e.g., a volume or area) in which a person should be detected within the field of view of a radar sensor. Detections of persons in areas of no interest (i.e., outside the detection zone) may be intended to be suppressed or discarded, resulting in a digital focus on the detection zone. By demarcating this specific detection zone, the device 100 is able to focus efforts and resources on the detection zone, thereby optimizing target detection, tracking, and data collection. This may allow for efficient use of its resources and enhanced situational awareness in a designated area.

[0028] For example, the data may indicate the range, location, size and / or shape of the detection area. For example, the detection area may be smaller than the field of view of the radar sensor, i.e., the implementation of the detection area may narrow the area where the person to be detected is located. In some examples, the data may indicate at least one of a distance interval and an angle interval where the person to be detected is located. The distance interval may specify the distance range in which the device 100 is intended to search for a target. It may be defined by a minimum and maximum distance. The angle interval may define the angular field of view in which the device 100 is intended to search for a target. It may be defined by a minimum and maximum angle. The angle may be measured relative to a reference axis of the radar sensor, which may be a visual axis direction or another reference direction. The angle may refer to an azimuth, an elevation, a Doppler angle, or a combination thereof.

[0029] In some examples, processing circuit device 110 is configured to obtain data by receiving user input from a user of device 100. For example, device 100 can be communicatively coupled to a user interface that is configured to generate a signal indicative of the user input based on the user's interaction with the user interface. This user interface can be, for example, a command line interface, a graphical user interface, a voice user interface, a gesture-based interface, etc. The user input can enable a user-specific and adjustable detection area. It can allow the user to customize the detection area according to a specific environment or preference.

[0030] Optionally or alternatively, processing circuit device 110 may automatically determine the detection area, for example based on the environment of device 100. For example, processing circuit device 110 may obtain environmental data indicating the type of environment (crowded or quiet environment, public space or home environment), and may adjust the detection area accordingly, for example based on predefined logic. For example, processing circuit device 110 may reduce the detection area in a crowded environment compared to a quiet environment. This may prevent undesired detection of people in the detection area in crowded places. For example, the detection of the type of environment may be based on sensor data or computer vision.

[0031] If the apparatus 100 includes an interface circuit device 120, the interface circuit device 120 may be configured to receive radar data and / or data. In the case of receiving radar data, the interface circuit device 120 may be communicatively coupled to a radar sensor or to a storage device storing radar data. In the case of receiving data, the interface circuit device 120 may be communicatively coupled to a user interface that generates data in response to interaction with a user, to a device that automatically determines a detection zone, and / or to a storage device storing radar data.

[0032] Depending on the specific implementation, the device 100 may omit the interface circuit device 120. For example, the processing circuit device 110 may determine the radar data, for example if the processing circuit device 110 is integrated into a radar sensor. Optionally or alternatively, the processing circuit device 110 may determine the data, for example if the processing circuit device 110 is integrated into a device including a user interface or it determines the detection area itself.

[0033] Alternatively, processing circuit device 110 may partially determine the data and / or radar data. For example, processing circuit device 110 may determine a first portion of the radar data, and at least one external processing circuit device may determine at least a second portion of the radar data. Processing circuit device 110 and external processing circuit device may be connected, for example, in a distributed computing environment to jointly determine the radar data. In this case, processing circuit device 110 may be integrated into the radar sensor or may be external to the radar sensor. Processing circuit device 110 may receive the second portion of the radar data, for example through an interface to an external processing circuit device (such as interface circuit device 120), and also process the first portion and the second portion of the radar data, as described below.

[0034] In another alternative, processing circuit device 110 is partially integrated into the radar sensor and partially external to the radar sensor. In such a case, interface circuit device 120 is optional. Processing circuit device 110 may, for example, include a first part (first processing circuit device) integrated into the radar sensor and a second part (second processing circuit device) external to the radar sensor. In this case, as described below, the determination and / or further processing of the radar data may be performed in a distributed manner by the first part and the second part of processing circuit device 110.

[0035] The processing circuit device 110 is configured to modify the radar data to shield the undesirable area (e.g., distance interval and / or angle interval) outside the detection area. For example, the undesirable angle and / or undesirable distance can be shielded using digital processing technology. For example, by setting an angle threshold or gate, the part of the radar data falling outside the detection area can be suppressed or discarded, thereby shielding the undesirable angle. In the case of shielding distance, a distance threshold setting (gating) or thresholding can be applied, involving setting a specific distance threshold or threshold. The part of the radar data corresponding to the radar echo falling outside this distance threshold or below the threshold is regarded as an undesirable signal and can be attenuated or discarded. The distance and / or angle can be excluded by filtering technology, for example, in the distance or angle domain applied to the radar data. A digital filter such as a low-pass filter or a distance threshold can be designed to pass the part of the radar data within the desired distance, while attenuating or removing the other parts of the radar data outside the distance.

[0036] In general, processing circuit device 110 can be configured to modify radar data by at least one of: attenuating, removing and zeroing the portion of radar data corresponding to the undesired area. For example, processing circuit device 110 can determine which portions of radar data correspond to radar returns from the undesired area, such as through range and / or angle processing, and reject (mask) the determined portions. In the case of zeroing, the portion of radar data can be replaced with a zero or null value, or with any baseline or neutral value that does not trigger the detection of a person.

[0037] Examples of detection areas and examples of undesirable areas are shown below. Figure 8 Describing.

[0038] Processing circuit device 110 is also configured to use trained neural network 115 to determine the number of people in the detection area based on the modified radar data. Trained neural network 115 can be any type of neural network, such as a convolutional neural network, a deep neural network, a generative neural network. For example, when the modified radar data or data derived therefrom is input into neural network 115, trained neural network 115 can output the number of people (prediction of the number of people).

[0039] The trained machine learning model 115 should be understood as a set of rules and / or data structures representing a statistical model that can be used to determine the number of people without the use of explicit instructions, but relying on models and reasoning. The data structure and / or rule set represents learned knowledge that is acquired based on training performed by a machine learning algorithm using training data. For example, to perform a determination of the number of people, a transformation of modified radar data inferred from an analysis of the training data can be used.

[0040] Training data refers to information or examples used to train the machine learning model 115. It can include a collection of input data fed into the training framework for training the machine learning model 115, and optionally corresponding labels, outputs or target values ​​(e.g., in the case of supervised learning). The training data proposed here is based on example radar data or pre-processed radar data. The purpose of such training data can be to enable the machine learning model 115 to learn and generalize patterns, relationships and rules from the provided examples.

[0041] The training of the machine learning model 115 can be performed using any machine learning or training algorithm. For example, the machine learning model 115 can be trained using supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning, feature learning, association rules.

[0042] For example, the trained machine learning model 115 can be an artificial neural network (ANN). ANN is a system inspired by biological neural networks such as those found in the retina or brain. ANN includes multiple interconnected nodes and multiple connections between nodes, so-called edges. There are usually three types of nodes, namely input nodes that receive input values ​​(such as modified radar data), hidden nodes that are (only) connected to other nodes, and output nodes that provide output values ​​(such as indicating the number of people determined). Each node can represent an artificial neuron. Each edge can transfer information from one node to another. The output of a node can be defined as a (nonlinear) function of its input (such as the sum of its inputs). The input of a node can be used in a function based on the "weight" of the edge or the node that provides the input. The weights of nodes and / or edges can be adjusted during the learning process. In other words, the training of the ANN can include adjusting the weights of the nodes and / or edges of the ANN, that is, to achieve the desired output for a given input. Alternatively, the machine learning model 115 can be a support vector machine, a random forest model, a gradient boosting model, a Bayesian network, etc.

[0043] For example, processing circuit device 110 may execute or run trained neural network 115 to determine the number of people. For example, model parameters, learned model distribution, and / or the architecture of trained machine learning model 115 may be loaded into a memory that is coupled to processing circuit device 110. Processing circuit device 110 may run an inference phase in which processing circuit device 110 passes modified radar data (or data derived therefrom) through trained machine learning model 115 to obtain the number of people.

[0044] The inference phase may involve performing calculations using, for example, model parameters, activation functions, and / or other operations defined by the model architecture. Depending on the model architecture, these calculations may be performed using linear algebra operations, such as matrix multiplication, convolution, or element-wise operations. The trained machine learning model 115 may include several layers, and at one or more of the layers, an activation function may be applied to introduce nonlinearity and enable the trained machine learning model 115 to interpret complex patterns in the modified radar data. Common activation functions include a rectified linear unit, a sigmoid, or a tanh function.

[0045] The calculations may be performed, for example, in a forward propagation manner (feedforward), where data flows through the trained machine learning model 115 in a predefined direction. For example, the modified radar data may be provided to the input layer of the trained machine learning model 115, and the data of the calculated interim results may be propagated through the downstream layers of the trained machine learning model 115 until reaching the output layer. The output layer may then output the determined number of people. These outputs may be interpreted, post-processed, or used for further downstream tasks performed by the processing circuit device 110.

[0046] The device 100 can provide improved human detection that focuses on the detection area and has improved computational efficiency. Unlike conventional systems, the device 100 can enable the detection area to dynamically adapt to the environment of the device 100. For example, the device 100 can be used for bystander detection, which can become less stringent in crowded environments (trains, cafes, etc.), and in less crowded environments, it can be assumed that people farther away from the device 100 may be potential bystanders (people looking at the screen). For example, a radar sensor can detect people within a certain distance (e.g., 5 meters) based on its configuration. In some environments, the detection area can be limited to 1 meter, such as on a train.

[0047] In the case of bystander detection, the device 100 may be coupled to or integrated into a screen or a computer including a screen. Examples of such configurations are described below. Figure 6 Provide explanation.

[0048] Thus, the device 100 may provide a number of advantages, including improved accuracy in bystander identification, adaptability to changing environmental conditions, reduced false alarms, and improved real-time performance. The device 100 may find application in various fields, such as security systems, crowd monitoring, behavioral analysis, and surveillance, where accurate bystander detection is essential.

[0049] In some cases, the machine learning model 115 may require a fixed input size or data dimension, such as in the case of a convolutional neural network. Therefore, the processing circuit device 110 can be configured to modify the radar data to exhibit a predefined data size, such as to exhibit compatibility with the input requirements of the machine learning model 115. For example, if the portion of the radar data corresponding to the detection area is smaller than the predefined data size, the processing circuit device 110 can fill the data area. Alternatively, if the radar data is larger than the predefined data size, the processing circuit device 110 can cut off the portion of the radar data corresponding to the undesired area outside the detection area. Therefore, the device 100 can provide reduced data variability and simplified data processing for consistent use of the machine learning model 115.

[0050] In the following, techniques are described for preprocessing radar data to improve shielding of undesired areas and use of the machine learning model 115. The preprocessing can generate improved inputs for the machine learning model 115.

[0051] For example, the processing circuit device 110 can be configured to determine a range-angle representation (graph) of the radar data, and modify the radar data by modifying the range-angle representation. The range-angle representation can be a multi-dimensional representation of the radar data, in which the signal parts are organized according to their distance and angle relative to the radar sensor, for example, a certain dimension of the data matrix can represent the distance, and another dimension can represent the angle. The distance-angle representation can provide a spatial arrangement of the data components. If the detection area is defined by distance and / or angle, the distance-angle representation can facilitate adaptive shielding of undesirable areas. In addition, the distance-angle representation can be an ideal input format for the machine learning model 115, because the (motion) patterns and characteristics of potential targets can be directly derived from it.

[0052] The range-angle representation can be created using range and channel processing. For example, the processing circuit device 110 can be configured to determine the range-angle representation by determining the range-Doppler (range-speed) representation of the radar data, and perform channel processing on the range-Doppler representation. The range-Doppler representation (graph) can be a multidimensional representation of radar data, which organizes the data points of the radar data according to the measured distance and speed (e.g., using binning). In the range-Doppler representation, the distance can be arranged along the first axis, and the speed can be arranged along the second axis. Then, the data points of the radar data can be assigned to the bins spanned by the first axis and the second axis. Therefore, the target can be represented as a point or intensity change on the range-Doppler representation. The range-Doppler representation can also facilitate the analysis of moving targets, including the distance, speed and direction of their movement. Therefore, the machine learning model 115 can have higher accuracy in distinguishing stationary clutter or interference from moving objects of interest.

[0053] For example, processing circuitry 110 may be configured to determine the range-Doppler representation by performing a (eg, fast) Fourier transform, any other suitable transform (z-transform, etc.), or correlation on the radar data.

[0054] In some examples, the processing circuit device 110 may be configured to determine the distance-angle representation using coherent integration. Coherent integration may involve combining multiple coherent radar signals to improve the signal-to-noise ratio (SNR) and detection sensitivity. Coherence may refer to maintaining a constant phase relationship between several radar signals (e.g., chirps or pulses) and accumulating received radar echoes over multiple coherent pulses. Instead of processing each radar signal separately, the signals received from multiple pulses may be coherently combined or added together. Coherent summation may take into account the phase and amplitude information of the received signal to enhance the desired target signal while reducing the effects of random noise or interference. Coherent integration may improve the SNR by aligning and enhancing the coherent echoes from the target while averaging out the random noise or interference. As more and more coherent pulses are integrated, the desired target signal accumulates and becomes stronger, while the random noise may be offset due to its random phase relationship. By integrating the signal over a longer period of time, coherent integration may allow detection of weaker targets that may not be distinguishable in a single echo.

[0055] In order to detect people, micro and macro information in the radar data can be analyzed. For example, the processing circuit device 110 can be configured to determine macro motion data indicating macro motion within the field of view of the radar sensor based on the radar data and micro motion data indicating micro motion within the field of view of the radar sensor based on the radar data. By considering micro motion data that captures fine-grained kinematics and subtle changes, the machine learning model 115 can be enabled to gain detailed insights into the dynamics of objects and search for characteristic patterns of people. Micro motion data can provide detailed information about small-scale movements, changes, and interactions that may not be obvious in macro motion analysis. Micro motion analysis can also achieve higher accuracy in capturing and quantifying motion. It can allow detection and measurement of breathing movements, heartbeat movements, or subtle gestures.

[0056] Macro-motion data can focus on the overall movement or global behavior of the target. By combining macro-motion analysis with micro-motion analysis, the machine learning model 115 can obtain a deeper understanding of the target type. Micro-motion data can help reveal the underlying details, interactions, and mechanisms that lead to the observed macro-level behavior. This overall understanding may be valuable for the detection and counting of people performed by the machine learning model 115.

[0057] The processing circuit device 110 can also be configured to determine the corresponding distance-angle representation of the macro-motion data and the micro-motion data, and modify the radar data by modifying the corresponding distance-angle representation of the macro-motion data and the micro-motion data. That is, there can be two processing pipelines for separately processing the micro-motion data and the macro-motion data. Separate processing can provide improved resource utilization: micro-data analysis may require more computing resources due to its detailed nature, while macro-data analysis can be performed in a resource-saving manner. By utilizing the appropriate processing level for each data type, computing efficiency and resource utilization can be optimized. The separate processing of micro- and macro-motion data can also allow the application of specialized algorithms, models or methods customized for each data type.

[0058] For example, the processing circuit device 110 can be configured to determine the range-angle representation of the macro-motion data using digital beamforming. Digital beamforming can be a specific type of channel processing, that is, processing and combining different antenna channels of a radar sensor (e.g., with an antenna array). Digital beamforming can involve manipulating the phase and amplitude of the corresponding digitized received radar signal from each antenna element. Digital beamforming can also involve applying Fourier transforms, algorithms (such as delay superposition or adaptive beamforming), etc. Digital beamforming can simplify the processing of macro-motion data while improving beamforming accuracy and control.

[0059] In the case of micro-motion data, the processing circuit device 110 can be configured, for example, to use the Capon method to determine the distance-angle representation of the micro-motion data. The Capon method, i.e., Capon's minimum variance method or minimum variance distortion-free response (MVDR) beamforming, can achieve more accurate angle determination of micro-motion data. The Capon method can obtain a distance-angle representation from radar data and based on an antenna array of a radar sensor with a known geometric structure or spatial arrangement. Radar data can be used to estimate a sample covariance matrix. This matrix can represent the statistical relationship between signal portions received by different antenna elements. The inverse of the estimated covariance matrix can be determined and used in subsequent steps to estimate weights for different directions of angles corresponding to different antennas of the antenna array. The Capon method can also estimate the spatial spectrum of the received radar signal, for example, by representing the power distribution as a function of the direction of arrival of the signal portion. The weight of the antenna array can be determined by increasing the output power of the signal while minimizing the noise power. The Capon method may be particularly useful for high resolution of micro-motion data and for adaptive zeroing of undesirable areas.

[0060] To analyze motion in radar data, a moving target indicator may be used. For example, processing circuit device 110 may be configured to filter radar data using a moving target indicator and to modify the radar data by modifying the filtered radar data. For example, processing circuit device 110 may sample two or more (e.g., consecutive) echoes represented in the radar data. The sample collected with the first echo may be rotated 180 degrees and added to the next sample. If the object moves at the position corresponding to the two samples, the resulting signal may survive this process due to constructive interference. By using a moving target indicator, device 100 may effectively distinguish moving targets from clutter, thereby achieving improved detection and tracking through machine learning model 115.

[0061] Figure 2 An example of a pre-processing chain 200 is illustrated. The pre-processing chain 200 may be performed by an apparatus described herein, such as the apparatus 100, in particular by the processing circuitry 110. The pre-processing chain 200 includes acquiring radar data 210 indicative of a received radar signal of a radar sensor. Figure 2 In the example of FIG. 4 , the radar data is raw ADC (analog-to-digital converter) data, that is, data sampled by the ADC.

[0062] The pre-processing chain 200 further comprises performing a (fast) Fourier transformation 220 on the radar data, thereby obtaining a range representation (range profile) of the radar data. A macroscopic Doppler filter 230 and a microscopic Doppler filter are then applied to the range representation, respectively. In this way, macroscopic motion data indicating macroscopic motion in the field of view of the radar sensor and microscopic motion data indicating microscopic motion in the field of view of the radar sensor are determined.

[0063] The corresponding range-angle representations 240, 245 of the macro-motion data and the micro-motion data are then determined in the following manner: The pre-processing chain 200 includes determining corresponding range-Doppler representations 250, 255 of the macro-motion data and the micro-motion data, respectively. For each of the multiple channels of radar data, a corresponding range-Doppler representation is determined. The range-Doppler representations are then filtered using a moving target indicator (MTI) 260, 265 in both processing pipelines 240, 245. Digital beamforming (DBF) 270 is used to determine the range-angle representation of the macro-motion data, while the Capon method 275 is used to determine the range-angle representation of the micro-motion data. Then, in both cases, coherent integration 280, 285 is applied to the channel-processed data.

[0064] The distance-angle representation can then be modified to mask out undesired areas outside the detection area, and the modified distance-angle representation can be input into a trained machine learning model that outputs a prediction of the number of people within the detection area.

[0065] Figure 3a and Figure 3b An example of a range-Doppler representation 300a of radar data and an example of a range-angle representation 300b of radar data are respectively illustrated.

[0066] For purposes of illustration, the range-Doppler representation 300a and the range-angle representation 300b are illustrated as two-dimensional data structures. The range-Doppler representation 300a spans the range dimension over the Doppler dimension, while the range-angle representation 300b spans the range dimension over the angle dimension.

[0067] Both the range-Doppler representation 300a and the range-angle representation 300b are modified to mask undesired regions 310a, 310b in the radar data, respectively, such as by cutting off or zeroing a certain distance or a certain distance and angle. For example, a user may define (via user input) a detection range and / or angle, thereby defining detection regions 320a and 320b for the range-Doppler representation 300a and the range-angle representation 300b, respectively.

[0068] Only the detection regions 320a, 320b may be "visible" for detecting targets.The masked range-Doppler representation 300a and range-angle representation 300b may be input to a trained machine learning model, as described above.

[0069] Figures 4a to 4c Examples of microscopic motion data 400a, 400b, and 400c are shown in FIG. Figure 4a In the example of FIG. 4 , the range-Doppler representation of radar data is shown. The microscopic motion data 400 b and 400 c are shown in FIG. Figure 4b and Figure 4c is illustrated as a range-angle representation of radar data.

[0070] The micro-motion data 400a, 400b, 400c are modified to mask the undesired areas in the radar data. The corresponding desired distances 410a, 410b, and 410c for detecting targets are defined. Figure 4c In the example of FIG. 4 , a desired angle interval from a first angle 420 - 1 to a second angle 420 - 2 in which a target is to be detected is defined.

[0071] The undesired area is cut out from the detection of the target. The rest of the micro-motion data 400a, 400b, 400c is the detection area 430a, 430b, 430c, respectively. The micro-motion data 400a, 400b, 400c can be input to a trained machine learning model, as described above.

[0072] Figure 5 An example of a radar system 500 is illustrated. Radar system 500 includes an apparatus 510 described herein, such as apparatus 100. Radar system 500 also includes a radar sensor 520. Radar sensor 520 is configured to generate radar data based on received radar signals. For example, radar sensor 520 may be a FMCW (Frequency Modulated Continuous Wave) radar sensor.

[0073] Although device 510 and radar sensor 520 are Figure 5 5. Although depicted as a separate box in FIG. 5, in other examples, device 510 may be partially or fully included in radar sensor 520, thereby correspondingly including all or part of the processing circuit device of device 510 (e.g., processing circuit device 110).

[0074] If the device 510 is only partially included in the radar sensor 520, the radar system 500 may include distributed processing circuit devices that perform corresponding parts of the processing steps, for example in the form of a first processing (sub-)circuit device included in the radar sensor 520 and a second processing (sub-)circuit device that is external to the sensor and communicates with the first processing circuit device via an interface circuit device (e.g., interface circuit device 120), for example for exchanging data between the first processing circuit device and the second processing circuit device.

[0075] If device 510 is integrated in radar sensor 520 , processing circuitry and radar sensor 520 may be integrated together in a single semiconductor chip or in more than one semiconductor chip.

[0076] If device 510 is not included in radar sensor 520 , the processing circuitry may take the form of circuitry external to radar sensor 520 and may be communicatively coupled thereto via interface circuitry.

[0077] Further details and aspects of the radar system 500 are described in conjunction with the proposed technology or one or more of the examples described above, for example, with reference to Figure 1 The radar system 500 may include one or more additional optional features corresponding to one or more aspects of the proposed technology or one or more of the examples described above.

[0078] Radar system 500 may provide improved human detection that is focused on a detection zone and has increased computational efficiency. Unlike conventional systems, radar system 500 may enable the detection zone to dynamically adapt to the environment of device 510.

[0079] Figure 6 An example of an electronic device 600 is shown. The electronic device 600 may be a consumer device, such as a laptop, a smartphone, a display, etc. The electronic device 600 includes the radar system 610 described above, such as the radar system 500. The electronic device 600 also includes a control circuit device 620 configured to control the electronic device 600 based on the determined number of people.

[0080] For example, the electronic device 600 may have a bystander detection function. In this case, the control circuit device 620 may alert the user of the electronic device 600, or take action if the radar system 610 detects the presence of a potentially unauthorized individual or bystander. Such mechanisms for generating an alert and / or taking action may be implemented regardless of whether the electronic device 600 exhibits such a bystander detection function. For example, the control circuit device 620 may trigger an alert, such as sounding an alarm, sending a notification to a user, etc. Optionally or alternatively, the control circuit device 620 may activate security measures to mitigate the risk of bystanders, such as dimming a display connected to the electronic device 600, hiding confidential data on the display, etc.

[0081] Further details and aspects of the electronic device 600 are described in conjunction with the proposed technology or one or more of the above examples, for example, with reference to Figure 1 The electronic device 600 may include one or more additional optional features corresponding to one or more aspects of the proposed technology or one or more examples described above.

[0082] The electronic device 600 can provide improved human detection that is focused on a detection area and has improved computational efficiency. Unlike conventional systems, the electronic device 600 can enable the detection area to dynamically adapt to the environment of the device 600.

[0083] Figure 7 An example of a method 700 is illustrated. The method 700 may be performed by an apparatus described herein, such as the apparatus 100.

[0084] The method 700 includes acquiring 710 radar data indicative of received radar signals of a radar sensor, acquiring 720 data indicative of a detection zone in which a person to be detected is located, and modifying 730 the radar data to mask an undesired area outside the detection zone 730. The method 700 also includes determining 740 a number of people within the detection zone based on the modified radar data using the trained neural network.

[0085] Further details and aspects of the method 700 are described in conjunction with the proposed technology or one or more of the above examples, for example, with reference to Figure 1 The method 700 may include one or more additional optional features corresponding to one or more aspects of the proposed technology or one or more examples described above.

[0086] The method 700 may provide improved human detection that is focused on a detection area and has improved computational efficiency. Unlike conventional systems, the method 700 may enable dynamic adaptation of the detection area.

[0087] Figure 8 An example of a field of view 800 of a radar sensor 810 is illustrated. The field of view 800 refers to the maximum angle interval and the maximum distance (range) in which the radar sensor 810 can detect and measure targets with a predefined accuracy. It represents the total span of angles and distances over which the radar sensor 810 can operate effectively and provide accurate information. For example, these angles and distances are measured from the phase center of the antenna of the radar sensor 810. Therefore, the field of view 800 can correspond to the angle and distance coverage of the radar sensor 810.

[0088] An apparatus described herein (e.g., apparatus 100) may acquire data indicating a detection zone 820 within the field of view 800 of a radar sensor 810 in which a person is to be detected. The detection zone 820 is a sub-portion of the field of view 800. Figure 8 In the example of FIG. 8 , the detection area 820 is reduced to a smaller distance and angle than the field of view 800 . The portion of the field of view 800 that is excluded from the detection area 820 is referred to as an undesired area 830 .

[0089] In the following, some examples of the proposed concepts are presented:

[0090] The example (Example 1) relates to an apparatus comprising a processing circuit device configured to obtain radar data indicating a received radar signal of a radar sensor, obtain data indicating a detection area where a person to be detected is located, modify the radar data to shield an undesired area outside the detection area, and determine the number of people within the detection area using a trained neural network based on the modified radar data.

[0091] Another example (eg, Example 2) relates to the previous example (eg, Example 1) or any other example, and further includes data indicating at least one of a distance interval and an angle interval in which a person to be detected is located.

[0092] Another example (eg, Example 3) relates to the previous example (eg, one of Examples 1 or 2) or any other example, and also includes the processing circuit device being configured to modify the radar data to exhibit a predefined data size.

[0093] Another example (e.g., Example 4) relates to a previous example (e.g., one of Examples 1 to 3) or any other example, and also includes the processing circuit device being configured to modify the radar data by at least one of: attenuating, removing, and zeroing portions of the radar data corresponding to the undesired area.

[0094] Another example (eg, Example 5) relates to the previous example (eg, one of Examples 1 or 4) or any other example, and further includes the processing circuit device being configured to obtain data by receiving user input from a user of the device.

[0095] Another example (e.g., Example 6) relates to a previous example (e.g., one of Examples 1 or 5) or any other example, and also includes the processing circuit device being configured to determine a range-angle representation of the radar data and modify the radar data by modifying the range-angle representation.

[0096] Another example (e.g., Example 7) relates to a previous example (e.g., one of Examples 1 to 6) or any other example, and also includes a processing circuit device configured to determine macro-motion data indicating macro-motion in the field of view of the radar sensor based on the radar data and micro-motion data indicating micro-motion in the field of view of the radar sensor based on the radar data, determine corresponding distance-angle representations of the macro-motion data and the micro-motion data, and modify the radar data by modifying the corresponding distance-angle representations of the macro-motion data and the micro-motion data.

[0097] Another example (eg, Example 8) relates to the previous example (eg, Example 7) or any other example, and also includes the processing circuit device being configured to determine the range-angle representation of the macroscopic motion data using digital beamforming.

[0098] Another example (eg, Example 9) relates to a previous example (eg, Example 7 or 8) or any other example, and further includes the processing circuit device being configured to determine a distance-angle representation of the micro-motion data using Capon's method.

[0099] Another example (e.g., Example 10) relates to a previous example (e.g., one of Examples 6 to 9) or any other example, and also includes a processing circuit device configured to determine a range-angle representation by determining a range-Doppler representation of the radar data, and performing channel processing on the range-Doppler representation.

[0100] Another example (eg, Example 11) relates to the previous example (eg, Example 10) or any other example, further comprising the processing circuit device being configured to determine the range-Doppler representation by performing a Fourier transform on the radar data.

[0101] Another example (eg, Example 12) relates to the previous example (eg, one of Examples 6 to 11) or any other example, and also includes the processing circuit device being configured to determine the distance-angle representation using coherent integration.

[0102] Another example (e.g., Example 13) relates to a previous example (e.g., one of Examples 1 to 12) or any other example, and also includes a processing circuit device configured to filter radar data using a moving target indicator and modify the radar data by modifying the filtered radar data.

[0103] An example (eg, Example 14) relates to a radar system comprising an apparatus according to any one of the previous examples and a radar sensor, wherein the radar sensor is configured to generate radar data based on a received radar signal.

[0104] Examples (eg, Example 15) relate to an electronic device comprising a radar system according to Example 14 and a control circuit device configured to control the electronic device based on a determined number of people.

[0105] An example (e.g., Example 16) relates to a method comprising obtaining radar data indicating a received radar signal of a radar sensor, obtaining data indicating a detection area in which a person to be detected is located, modifying the radar data to mask an undesired area outside the detection area, and determining the number of people within the detection area using a trained neural network based on the modified radar data.

[0106] Another example (eg, Example 17) relates to a non-transitory machine-readable medium having a program stored thereon, the program having program code for performing the method of Example 16 when the program is executed on a processor or programmable hardware.

[0107] Another example (eg, Example 18) relates to a program having a program code for executing the method of Example 16 when the program is executed on a processor or programmable hardware.

[0108] The aspects and features described with respect to a specific example in the previous examples may also be combined with one or more of the other examples to replace the same or similar features of the other examples or to additionally introduce features into the other examples.

[0109] The example may also be or relate to a (computer) program, which includes a program code, which is used to perform one or more of the above methods when the program is executed on a computer, a processor or other programmable hardware component. Therefore, the steps, operations or processes of the different methods in the above methods may also be performed by a programmed computer, a processor or other programmable hardware component. The example may also cover a program storage device, such as a digital data storage medium, which is readable by a machine, a processor or a computer and encodes and / or contains machine-executable, processor-executable or computer-executable programs and instructions. For example, the program storage device may include or may be a digital storage device, a magnetic storage medium (such as a disk and a tape), a hard drive or an optically readable digital data storage medium. Other examples may also include a computer, a processor, a control unit, a (field) programmable logic array ((F)PLA), a (field) programmable gate array ((F)PGA), a graphics processor unit (GPU), an application-specific integrated circuit (ASIC), an integrated circuit (IC) or a system on a chip (SoC) system, which is programmed to perform the steps of the above methods.

[0110] It should also be understood that the disclosure of several steps, processes, operations or functions disclosed in the specification or claims should not be interpreted as implying that these operations must be in the described order, unless explicitly stated in individual cases or necessary for technical reasons. Therefore, the above description does not limit the execution of several steps or functions to a certain order. In addition, in other examples, a single step, function, process or operation may include and / or be decomposed into several sub-steps, sub-functions, sub-processes or sub-operations.

[0111] If certain aspects have been described with respect to a device or system, these aspects should also be understood as descriptions of corresponding methods. For example, a block, device, or functional aspect of a device or system may correspond to a feature of a corresponding method, such as a method step. Therefore, aspects described with respect to a method should be understood as descriptions of properties or functional features of a corresponding block, corresponding element, corresponding device, or corresponding system.

[0112] The following claims are hereby incorporated into the Detailed Description, where each claim may stand alone as a separate example. It should also be noted that although in the claims a dependent claim refers to a specific combination with one or more other claims, other examples may also include a combination of a dependent claim with the subject matter of any other dependent or independent claim. Such combinations are explicitly proposed herein, unless otherwise stated in individual cases where a specific combination is not intended. In addition, features of a claim should also be included by any other independent claim, even if the claim is not directly defined as dependent on the other independent claim.

Claims

1. A device comprising a processing circuit device (110), wherein the processing circuit device (110) is configured to: acquiring radar data indicative of a received radar signal of the radar sensor; acquiring data indicating a detection area (820) where a person to be detected is located; modifying the radar data to mask undesired areas (830) outside the detection area (820); as well as Based on the modified radar data, a trained neural network (115) is used to determine the number of people within the detection area (820).

2. The device (100) according to claim 1, wherein the data indicates at least one of a distance interval and an angle interval in which the person to be detected is located.

3. The apparatus (100) of any preceding claim, wherein the processing circuitry (110) is configured to modify the radar data to exhibit a predefined data size.

4. The apparatus (100) of any preceding claim, wherein the processing circuitry (110) is configured to modify the radar data by at least one of: attenuating, removing, and zeroing a portion of the radar data corresponding to the undesired region (830).

5. The apparatus (100) according to any one of the preceding claims, wherein the processing circuit means (110) is configured to obtain the data by receiving user input from a user of the apparatus (100).

6. The apparatus (100) of any preceding claim, wherein the processing circuitry (110) is configured to determine a range-angle representation of the radar data and to modify the radar data by modifying the range-angle representation.

7. The apparatus (100) according to any one of the preceding claims, wherein the processing circuit arrangement (110) is configured to: determining macro-motion data indicative of macro-motion in a field of view of the radar sensor based on the radar data and micro-motion data indicative of micro-motion in the field of view of the radar sensor based on the radar data; determining corresponding distance-angle representations of the macro-motion data and the micro-motion data; as well as The radar data is modified by modifying the macro-motion data and the corresponding range-angle representations of the micro-motion data.

8. The apparatus (100) of any one of claims 6 or 7, wherein the processing circuit means (110) is configured to determine the range-angle representation by determining a range-Doppler representation of the radar data and to perform channel processing on the range-Doppler representation.

9. The apparatus (100) of any one of claims 6 to 8, wherein the processing circuitry (110) is configured to determine the distance-angle representation using coherent integration.

10. The apparatus (100) of any preceding claim, wherein the processing circuitry (110) is configured to filter the radar data using a moving target indicator and to modify the radar data by modifying the filtered radar data.

11. A radar system (500), comprising: The device (510) according to any of the preceding claims; as well as The radar sensor (520), wherein the radar sensor is configured to generate the radar data based on the received radar signal.

12. An electronic device (600), include: The radar system (610) of claim 11; and The control circuit device (620) is configured to control the electronic device (600) based on the determined number of people.

13. A method (700), comprising: acquiring ( 710 ) radar data indicative of a received radar signal of a radar sensor; acquiring (720) data indicating a detection area where a person to be detected is located; modifying (730) the radar data to mask undesired areas outside the detection area; as well as Based on the modified radar data, a number of people within the detection area is determined (740) using a trained neural network.

14. A non-transitory machine-readable medium having a program stored thereon, the program having program codes for executing the method according to claim 13 when the program is executed on a processor or programmable hardware.

15. A program having a program code for executing the method according to claim 13 when the program is executed on a processor or programmable hardware.

Images