Method and apparatus for training generative models to generate synthetic radar data

Abstract

The invention relates to a method and apparatus for training a generative model to generate synthetic radar data. A method is presented that includes obtaining a trained generative model, and generating synthetic radar data using the trained generative model. The synthetic radar data is synthetic raw radar data sampled chirp.

Description

Technical Field

[0001] The present disclosure relates to the generation of synthetic radar data. Examples relate to methods and apparatus, methods and apparatus for training a generative model to generate synthetic radar data, and synthetic radar data. Background Art

[0002] Many radar devices perform radar tasks based on machine learning models such as deep neural networks. Such machine learning models require a lot of data in their training phase, which increases the workload of creating models and adapting them to specific radar devices or radar tasks. Therefore, improved generation of training data may be needed. Summary of the invention

[0003] This need is met by the independent claim. Advantageous embodiments are described by the dependent claims.

[0004] Some aspects of the present disclosure relate to a method that includes obtaining a trained generative model and generating synthetic radar data using the trained generative model, the synthetic radar data being synthesized raw radar data of a sampled chirp.

[0005] Some aspects of the present disclosure relate to a method for training a generative model to generate synthetic radar data, the synthetic radar data being synthesized raw radar data of a sampled chirp, the method comprising training the generative model based on real raw radar data.

[0006] Some aspects of the present disclosure relate to an apparatus that includes processing circuitry configured to obtain a trained generative model and generate synthetic radar data using the trained generative model, the synthetic radar data being synthetic raw radar data of a sampled chirp.

[0007] Some aspects of the present disclosure relate to an apparatus for training a generative model to generate synthetic radar data, the synthetic radar data being synthesized raw radar data of a sampled chirp, the apparatus comprising a processing circuit apparatus configured to train the generative model based on real raw radar data.

[0008] Some aspects of the present disclosure relate to a radar system comprising an apparatus as described herein, and a radar sensor configured to generate real raw radar data.

[0009] Some aspects of the present disclosure relate to synthetic radar data that can be obtained by methods as described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Some examples of apparatus and / or methods will be described hereinafter, by way of example only, and with reference to the accompanying drawings, in which:

[0011] Figure 1 An example of an apparatus is shown;

[0012] Figure 2 An example of real raw radar data is shown;

[0013] Figure 3 An example of an apparatus for training a generative model to generate synthetic radar data is shown;

[0014] Figure 4 An example of a generative adversarial network is shown;

[0015] Figure 5 An example of a diffusion model is shown;

[0016] Figure 6 An example of a method is shown;

[0017] Figure 7 An example of synthetic radar data is shown;

[0018] Figure 8 An example of a method for training a generative model to generate synthetic radar data is shown;

[0019] Fig. 9 An example of a radar system is shown; and

[0020] Fig.10a and Fig.10b An example of a method for training a machine learning model to perform radar tasks is shown. DETAILED DESCRIPTION

[0021] Some examples are now described in more detail with reference to the accompanying drawings. However, other possible examples are not limited to the features of these embodiments described in detail. Other examples may include modifications of features and equivalents and substitutes of features. In addition, the terms used herein to describe certain examples should not limit other possible examples.

[0022] In the description of the entire drawings, the same or similar reference numerals refer to the same or similar elements and / or features, which may be implemented in an equivalent or modified form while providing the same or similar functions. For clarity, the thickness of the lines, layers and / or regions in the drawings may also be exaggerated.

[0023] When two elements A and B are combined using "or", this should be understood to disclose all possible combinations, i.e. only A, only B, and A and B, unless otherwise explicitly defined in individual cases. As an alternative wording for the same combination, "at least one of A and B" or "A and / or B" can be used. This is equally applicable to combinations of more than two elements.

[0024] If singular forms such as "a", "an", and "the" are used, and the use of only a single element is not explicitly or implicitly defined as mandatory, additional examples may also use several elements to achieve the same function. If the function is described below as being implemented using multiple elements, additional examples may use a single element or a single processing entity to achieve the same function. It is further understood that the terms "comprise", "comprising", "including", and / or "comprising" when used describe the presence of specified features, integers, steps, operations, processes, elements, parts, and / or groups thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, parts, and / or groups thereof.

[0025] Figure 1 An example of an apparatus 100 is shown. The apparatus 100 includes a processing circuit device 110. Optionally, the apparatus 100 includes an interface circuit device 120. Where the interface circuit device 120 is present, the interface circuit device 120 may be communicatively coupled to the processing circuit device 110 (e.g., via a wired or wireless connection), e.g., for data exchange between the interface circuit device 120 and the processing circuit device 110. The interface circuit device 120 may be any device or means for communicating or exchanging data.

[0026] Processing circuit device 110 can be, for example, a single dedicated processor, a single shared processor, or multiple individual processors (some or all of which can be shared), digital signal processor (DSP) hardware, application specific integrated circuit (ASIC), microcontroller, or field programmable gate array (FPGA). Processing circuit device 110 can optionally be coupled to read-only memory (ROM), random access memory (RAM), and / or non-volatile memory, for example, for storing software.

[0027] The processing circuit device 110 is configured to obtain the trained generative model 130. For example, where the apparatus 100 includes the interface circuit device 120, the interface circuit device 120 may be configured to receive data indicating the trained generative model 130, for example, from an external device that has trained the underlying generative model. The interface circuit device 120 may be communicatively coupled to the external device or a storage device storing the trained generative model 130. Additionally or alternatively, the processing circuit device 110 may obtain the trained generative model 130 by training or partially training the generative model, thereby generating the trained generative model 130. Further details on optional training of the generative model are described below.

[0028] Processing circuitry 110 is configured to generate synthetic radar data using trained generative model 130. The synthetic radar data is synthetic raw radar data of a sampled chirp. For example, the synthetic raw radar data may simulate raw data of a sampled chirp originating from a radar sensor.

[0029] For example, processing circuit device 110 may execute or run trained generative model 130 to generate synthetic radar data. For example, model parameters, learned model distribution and / or trained generative model 130 architecture may be loaded into a memory coupled to processing circuit device 110.

[0030] Processing circuit device 110 may run an inference phase in which processing circuit device 110 passes input data (e.g., noisy input data) through trained generative model 130 to obtain synthetic radar data. For example, processing circuit device 110 may generate noisy input data, for example, using a noise source, or receive noisy input data from an external source. Alternatively, processing circuit device 110 may run an inference phase without any input data, i.e., the inference phase may involve sampling from a learned model distribution to generate new data points, i.e., synthetic radar data.

[0031] The inference phase may involve performing calculations using, for example, model parameters, activation functions, and / or other operations defined by the model architecture. These calculations may be performed using linear algebra operations such as matrix multiplication, convolution, or element-wise operations depending on the model architecture. The trained generative model 130 may include multiple layers, and in one or more layers, an activation function may be applied to introduce nonlinearity and enable the trained generative model 130 to generate complex patterns in the synthetic radar data. Common activation functions include rectified linear unit functions, sigmoid functions, or hyperbolic tangent (tanh) functions.

[0032] The computations may be performed, for example, in a forward propagation manner (feedforward), where data flows through the trained generative model 130 (network) in a predefined direction. For example, input data may be provided to an input layer of the trained generative model 130, and data of intermediate results of the computations may be propagated through subsequent downstream layers of the trained generative model 130 until an output layer is reached. The output layer may then output synthetic radar data. These outputs may be interpreted, post-processed, or used for further downstream tasks, for example, the synthetic radar data may be converted (e.g., by labeling) into training data or prepared for use in training a machine learning model 140, as further explained below.

[0033] In some examples, one or more layers of the trained generative model 130 have learnable parameters, such as weights or biases. During inference, the processing circuit device 110 can use these parameters to calculate the weighted sum of the inputs of the layer. This can then be followed by the application of an activation function. The weights represent the strength of the connections between neurons in the network of the trained generative model 130, while the bias controls the offset of the activation function.

[0034] Depending on the model architecture, the generative model can be trained, for example, on a latent space or under specific input conditions. In the former case, during inference, random or specific values ​​can be sampled from the latent space (a lower dimensional representation of the input data), and the trained generative model 130 can generate corresponding outputs or samples in the original data space, thereby producing synthetic radar data. In the latter case, the trained generative model 130 can generate samples conditioned on the input conditions. For example, the trained generative model 130 can generate synthetic radar data using specified attributes or class labels.

[0035] To sample the synthetic radar data, processing circuitry 110 may use any sampling technique, such as random sampling, uniform sampling, Gaussian sampling, categorical sampling, top-k sampling, and the like.

[0036] The trained generative model 130 is a trained machine learning model that has learned the basic probability distribution of radar data. It can, for example, generate new samples (forming sampling chirps) from noise data input, which are similar to the radar data on which the machine learning model is trained. The trained generative model 130 can, for example, directly output synthetic radar data. The generative model can model the joint probability distribution of input variables (e.g., real raw radar data) and output variables (e.g., synthetic raw radar data), rather than focusing only on predicting outputs given inputs as in the discriminative model. This can improve the generation of real synthetic radar data. The automatic generation of radar data can speed up and simplify the process of ground truth collection. For example, the device 100 can improve the training of a machine learning model to be trained to perform a radar task by providing more training data with less effort.

[0037] Since the trained generative model 130 can be parameterized to generate new data points (sampled chirps) that are similar to the original (real) radar data for training with specific statistical variations, the trained generative model 130 can generate realistic but diverse synthetic radar data that follows specific statistical properties. This can enable improved training of machine learning models with respect to real radar environments outside of the laboratory, which can also improve the performance of radar tasks.

[0038] In addition, conventional methods may determine processed radar data rather than raw radar data. Synthetic raw radar data may refer to synthetically generated uninterpreted measurements, similar to, for example, an intermediate frequency (IF) signal of a radar sensor. Synthetic raw radar data may, for example, represent data points (samples) of a synthetic received chirp or data points (samples) of a correlation between a synthetic chirp and a reference frequency. Synthetic raw radar data may be treated like a real manually recorded radar signal.

[0039] Unlike synthetic raw radar data, the processed radar data may be, for example, an interpretation of the radar data, such as a range-velocity representation (range-Doppler image, RDI) or a micro-RDI of the radar data, which indicates the distance and velocity of a moving object in the field of view of the radar sensor. Conventional methods may generate only an RDI or a micro-RDI. However, in some scenarios, it may be necessary to use the RDI and the micro-RDI as a pair: the RDI may show the macro-movement of an object, while the micro-RDI may show the micro-movement of the same object. In the described scenario, the device 100 may provide a considerable advantage over such conventional methods because it provides synthetic raw radar data, which may then be pre-processed in a preferred manner for training, for example to output a pair of RDI and micro-RDI.

[0040] Furthermore, the preprocessing of the training data may be application specific. Thus, directly outputting the preprocessed training data as in conventional methods may render the data unusable for various applications. Instead, the apparatus 100 may provide synthetic raw radar data that is flexibly adaptable to any desired preprocessing. Thus, no restrictions may be introduced in the preprocessing, and the specific preprocessing method may be changed or adjusted as desired.

[0041] In addition, machine learning model 140 may be trained, for example, by processing circuit device 110 of apparatus 100, another processing circuit device (different from processing circuit device 110) of apparatus 100, and / or a processing circuit device external to apparatus 100. In the former case, processing circuit device 110 may be further configured to train machine learning model 140 based on synthetic radar data to perform radar tasks. For example, synthetic radar data may be used as training data or to derive training data (e.g., RDI, etc.).

[0042] The machine learning model 140 should be understood as a data structure and / or rule set representing a statistical model that can be used to perform radar tasks without using explicit instructions but relying on models and reasoning. The data structure and / or rule set represents learned knowledge acquired based on training performed by a machine learning algorithm using training data. For example, in order to perform radar tasks, transformations of data inferred from analysis of training data can be used. In some examples, the machine learning model 140 can be a deep neural network (DNN).

[0043] Training data refers to information or examples used to train a machine learning model. It can include a collection of input data and optional corresponding labels, outputs or target values ​​(e.g., in the case of supervised learning) fed into a training framework for training the machine learning model 140. The training data presented herein is based on synthetic raw radar data, as further explained below. The purpose of such training data can be to enable the machine learning model 140 to learn and generalize patterns, relationships, and rules from the provided examples.

[0044] The radar task may be any ranging task and / or detection task based on radar data or data derived therefrom. For example, performing a radar task may include gesture sensing, presence sensing, life sensing, signal de-noising, clutter suppression, tracking, or detecting at least one of gesture, motion, micro-motion, macro-motion, and target presence based on additional radar data (i.e., radar data generated by a radar sensor and processed during operation of a trained machine learning model).

[0045] Using a trained machine learning model 140 for radar tasks can provide many advantages over conventional techniques. For example, a trained machine learning model 140 can be good at identifying complex patterns and relationships within data to perform radar tasks, it can continuously adjust and learn based on new data, thereby improving its performance over time, or can reduce the need for human intervention and thereby speed up the response time of performing radar tasks.

[0046] However, the performance of the machine learning model 140 may depend largely on the quality and quantity of the training data (ground truth data). Traditionally, developers may manually operate radar sensors in a laboratory environment to collect a sufficient amount of training data to train the machine learning model 140 to achieve high performance and accuracy. Manual data collection is typically an expensive, time-consuming, and difficult process. In addition, the process may be limited to a laboratory environment, which may not be representative enough of the real world: the training data collected in the laboratory may be one-sided and non-diverse, not covering the variations that occur in practice due to noise or other environmental conditions.

[0047] In contrast, the apparatus 100 can simplify the generation of training data and provide more diverse training data by automatically generating high-fidelity synthetic raw radar data, which can thereby reduce the workload of manual data collection.

[0048] The processing circuit device 110 can train the machine learning model 140 by inputting the training data or (an input portion thereof) into the machine learning model 140, and adjust the internal parameters of the machine learning model 140, so that the difference between the output of the machine learning model 140 and the actual output (for example, the corresponding label of the training data) is reduced.

[0049] Processing circuit device 110 can use any machine learning or training algorithm to train machine learning model 140. For example, supervised learning can be used to train machine learning model 140. In supervised learning, machine learning model 140 is trained using multiple training samples, where each sample can include multiple input data values ​​and multiple expected output values, that is, each training sample is associated with an expected output value. By specifying both the training samples and the expected output values, machine learning model 140 "learns" which output value to provide based on input samples similar to the samples provided during training.

[0050] For example, the training data generated by the apparatus 100 may include a plurality of such training samples used as input data and one or more labels used as desired output data. The labels may be determined manually or automatically. In the latter case, the processing circuit device 110 may be further configured to determine the labeled data by labeling the synthetic radar data or data derived therefrom for the radar task, and to train the machine learning model 140 based on the labeled data. The label may indicate a solution to the target radar task, such as the presence of a target, etc.

[0051] In addition to supervised learning, semi-supervised learning can also be used. In semi-supervised learning, some of the training samples lack corresponding expected output values. Supervised learning can be based on a supervised learning algorithm (e.g., a classification algorithm or a similarity learning algorithm). The classification algorithm can be used to limit the expected output of the trained machine learning model 140 to a finite set of values ​​(category variables), that is, to classify the input into one of the values ​​in the finite set of values ​​(e.g., no target detected, one or more targets detected). The similarity learning algorithm is similar to the classification algorithm, but is based on learning through examples using a similarity function that measures the similarity or correlation between two objects.

[0052] In addition to supervised learning or semi-supervised learning, unsupervised learning can be used to train the machine learning model 140. In unsupervised learning, (only) training data is provided and an unsupervised learning algorithm is used to find structure in the training data, such as training and / or historical radar data (e.g., by grouping or clustering the training data, finding commonalities in the training data). Clustering is the assignment of training data comprising multiple input values ​​into subsets (clusters) such that input values ​​within the same cluster are similar according to one or more (predefined) similarity criteria, but are dissimilar to input values ​​included in other clusters.

[0053] Optionally or alternatively, reinforcement learning can be used to train the machine learning model 140. In reinforcement learning, one or more software actors (referred to as "software agents") are trained to take actions during training. Rewards are calculated based on the actions taken. Reinforcement learning is based on training one or more software agents to select actions that increase cumulative rewards, such that the software agents become better at the tasks they are given (as evidenced by increased rewards).

[0054] In addition, additional techniques may be applied to some of the machine learning algorithms. For example, feature learning may be used. In other words, the machine learning model 140 may be trained at least in part using feature learning, and / or the machine learning algorithm may include a feature learning component. Feature learning algorithms (which may be referred to as representation learning algorithms) may retain information in their inputs, but may also transform the information in a way that makes it useful, typically as a preprocessing step before performing classification or prediction. For example, feature learning may be based on principal component analysis or cluster analysis.

[0055] In some examples, anomaly detection (i.e., outlier detection) can be used to provide identification of input values ​​that raise suspicion by being significantly different from the majority of inputs or training data. In other words, the machine learning model 140 can be trained at least in part using anomaly detection, and / or the machine learning algorithm can include an anomaly detection component.

[0056] In some examples, the machine learning algorithm can use a decision tree as a prediction model. In other words, the machine learning model 140 can be based on a decision tree. In a decision tree, observations about an item (e.g., an input radar data set) can be represented by a branch of the decision tree, and the output value corresponding to the item can be represented by the leaf of the decision tree. Decision trees support discrete values ​​and continuous values ​​as output values. If discrete values ​​are used, the decision tree can be represented as a classification tree, and if continuous values ​​are used, the decision tree can be represented as a regression tree.

[0057] Association rules are another technique that can be used in machine learning algorithms. In other words, the machine learning model 140 can be based on one or more association rules. Association rules are created by identifying relationships between variables in a large amount of data. The machine learning algorithm can identify and / or utilize one or more relationship rules that represent knowledge derived from the data. These rules can be used, for example, to store, manipulate, or apply knowledge.

[0058] For example, the trained machine learning model 140 may be an artificial neural network (ANN). An ANN is a system inspired by biological neural networks (such as those found in the retina or brain). An ANN includes a plurality of interconnected nodes and a plurality of connections (so-called edges) between the nodes. There are usually three types of nodes, input nodes that receive input values ​​(e.g., (additional) radar data and RDI, etc.), hidden nodes that are (only) connected to other nodes, and output nodes that provide output values ​​(e.g., indicating the solution of the radar task). Each node may represent an artificial neuron. Each edge may transfer information from one node to another. The output of a node may be defined as a (nonlinear) function of its input (e.g., the sum of its inputs). The input of a node may be used in a function based on the "weight" of the edge or node that provides the input. The weights of nodes and / or edges may be adjusted during the learning process. In other words, the training of an ANN may include adjusting the weights of the nodes and / or edges of the ANN, i.e., to achieve the desired output for a given input.

[0059] Alternatively, the machine learning model 140 can be a support vector machine, a random forest model, or a gradient boosting model. A support vector machine (i.e., a support vector network) is a model trained by supervised learning and an associated learning algorithm that can be used to analyze data (e.g., in classification or regression analysis). A support vector machine can be trained by providing multiple training input values ​​with training data belonging to one of two categories (e.g., a target is detected or not detected) to an input. A support vector machine can be trained to assign new input values ​​to one of two categories. Alternatively, the machine learning model 140 can be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network can use a directed acyclic graph to represent a set of random variables and their conditional dependencies. Alternatively, the machine learning model 140 can be based on a genetic algorithm, which is a search algorithm and heuristic technique that mimics a natural selection process. In some examples, the machine learning model 140 can be a combination of the above examples.

[0060] In addition to synthetic radar data, real raw radar data can be used for training the machine learning model 140. Therefore, in some examples, the processing circuit device 110 is further configured to train the machine learning model 140 based on real raw radar data to perform radar tasks. For example, the training data (e.g., features and corresponding labels) can be based on synthetic raw radar data, while the real raw radar data can be used to create test data. Test data (e.g., a test set or a validation set) can be a separate data set for evaluating the performance of a (trained, untrained or partially trained) machine learning model. The test data includes input data and also retains the corresponding expected output data. The performance of the machine learning model can be evaluated by predicting the test data and comparing the predicted output with the expected output data. This evaluation can help estimate how well the machine learning model 140 can perform on unseen radar data. By using a separate test data set, potential problems such as overfitting (where the machine learning model 140 performs well on training data but poorly on new data) can be identified. Optionally, in the case where the performance is not good enough for the target application, the processing circuit device 110 can trigger the generation of additional training data.

[0061] Optionally or alternatively, a portion of the training data and / or at least a portion of the real raw radar data is used as validation data. The validation data can be used to fine-tune the hyperparameters of the machine learning model 140 and make decisions during the model development process, such as selecting a model architecture or tuning regularization parameters.

[0062] In some examples, real raw radar data may be the data used to train a generative model, as explained further below.

[0063] The real raw radar data may be radar data recorded by a radar sensor, such as data indicating an IF signal of the radar sensor. For example, the real raw radar data may be multi-dimensional data arranged in an array (e.g., chirps arranged in slow time on fast time). In some examples, the real raw radar data may indicate sampled chirps received from multiple channels of the radar sensor. Figure 2 An example of such real raw radar data is shown. Figure 2 An example of real raw radar data 200 is shown. The real raw radar data 200 indicates three sampled chirp sets 210, 220, 230 received from corresponding channels of a radar sensor. For example, the real raw radar data 200 may be recorded by a 60 GHz (gigahertz) radar sensor having one transmit antenna and three receive antennas. The slow time of the chirp is arranged along a first dimension of each of the sampled chirp sets 210, 220, 230. Figure 2In the example of , each of the sampled chirp sets 210, 220, 230 includes 256 chirps. The fast time (samples) of the chirp are arranged along the first dimension of each of the sampled chirp sets 210, 220, 230. Figure 2 In the example of , each of the sampled chirp sets 210, 220, 230 includes 64 samples per chirp. Therefore, the size of the actual raw radar data may be (3, 256, 64). Figure 2 The shading of the sampled chirp sets 210, 220, 230 shown in FIG. 2 may indicate the signal strength of the received radar signal for each chirp and sample.

[0064] Return to reference Figure 1 In some examples, the apparatus 100 may perform the above-described preprocessing for training the machine learning model 140. For example, the processing circuit device 110 may be configured to extract at least one signal characteristic from the synthetic radar data, and train the machine learning model 140 based on the extracted at least one signal characteristic. The signal characteristic may be any interpretation of the synthetic radar data, such as the size, reflectivity, distance, speed, or angle of the target, and the spectral characteristics or clutter characteristics of the synthetic radar data.

[0065] In some examples, processing circuit device 110 is further configured to determine at least one of a range-velocity representation (e.g., RDI) and a range-angle representation of the synthetic radar data, and train machine learning model 140 based on the determined at least one of the range-velocity representation and the range-angle representation of the synthetic radar data. Processing circuit device 110 may determine the range-velocity representation of the synthetic radar data by, for example, applying a multi-dimensional Fourier transform to the synthetic radar data. Processing circuit device 110 may determine the range-angle representation of the synthetic radar data by range processing (e.g., applying range compression or matched filtering to the synthetic radar data) and subsequent channel processing (e.g., applying phase monopulse or phase interferometry to the synthetic radar data). Then, processing circuit device 110 may train machine learning model 140 by feeding at least one of the range-velocity representation and the range-angle representation of the synthetic radar data as training input into a training framework for training machine learning model 140.

[0066] Additionally or alternatively, processing circuit device 110 may determine any radar data modality or representation of the synthetic radar data and train machine learning model 140 based thereon. The type of representation may depend on the target radar mission and radar sensor configuration. For example, processing circuit device 110 may determine at least one of a range-range rate representation, a polarimetric representation, a holographic representation, a synthetic aperture, and a range-cross-range coherence variation representation of the synthetic radar data.

[0067] In some examples, processing circuit device 110 may be further configured to determine a range-micro-velocity representation and a range-macro-velocity representation of the synthetic radar data, and train machine learning model 140 based on the determined range-micro-velocity representation and the determined range-macro-velocity representation of the synthetic radar data. This may address such scenarios where paired micro-representations and macro-representations are used to perform radar tasks. Device 100 may provide additional advantages in these scenarios, as conventional methods may only provide separate range-macro-velocity representations and range-micro-velocity representations that cannot be paired later.

[0068] Likewise, real raw radar data, when available, may be processed to produce a desired representation for training the machine learning model 140. For example, the processing circuit device 110 may be further configured to determine at least one of a range-velocity representation and a range-angle representation of the real raw radar data, and train the machine learning model 140 based on the determined at least one of the range-velocity representation and the range-angle representation of the real raw radar data. Additionally or alternatively, the processing circuit device 110 may be further configured to determine a range-micro-velocity representation and a range-macro-velocity representation of the real raw radar data, and train the machine learning model 140 based on the determined range-micro-velocity representation and the range-macro-velocity representation of the real raw radar data. This may enable parallel processing of the synthetic radar data and the real raw radar data to train the machine learning model 140.

[0069] When the training of the machine learning model 140 has been completed, a trained machine learning model is obtained. The trained machine learning model can be used, for example, by the device 100, by another device outside the device 100, or by both to jointly perform radar tasks in a distributed processing environment. In the former case, the processing circuit device 110 can be further configured to perform the radar task by analyzing additional radar data using the trained machine learning model. The additional radar data can be real radar data recorded by the radar sensor to perform the radar task. For example, the interface circuit device 120 can receive additional radar data from the radar sensor. Alternatively, the device 100 can be at least partially integrated into the radar sensor and determine the additional radar data based on the radar signal received by the radar sensor.

[0070] Processing circuit device 110 can be configured to perform radar tasks, for example, by detecting at least one of gestures, movements, micro-movements, macro-movements, and target presence based on additional radar data. For example, the additional radar data or data derived therefrom can be fed as input into a trained machine learning model. The trained machine learning model can then output at least one of gestures, movements, micro-movements, macro-movements, and target presence.

[0071] Micro-motion and macro-motion can refer to different types and scales of motion exhibited by a target. Micro-motion can refer to small-scale movements or vibrations of individual components or parts of a target. These movements can be periodic, quasi-periodic, or random. Micro-motion can be detected and analyzed by examining fine Doppler signature features present in additional radar data (radar returns). Macro-motion can refer to the overall or bulk movement of an entire target or object. This can involve translational motion of a target or changes in its position or velocity on a larger scale. Macro-motion can be characterized by range or range-velocity information obtained from additional radars.

[0072] As described above, the trained generative model 130 may be determined by the device 100 itself (e.g., by the processing circuit device 110 or by another processing circuit device of the device 100), by a different another device outside the device 100, or by the device and another device jointly (e.g., based on distributed processing). In the former case, the processing circuit device 110 may be further configured to train the generative model based on real raw radar data (e.g., real raw radar data used to train the machine learning model 140), thereby generating the trained generative model 130.

[0073] refer to Figure 3 Explains more details about how generative models can be trained. Figure 3 An example of an apparatus 300 for training a generative model to generate synthetic radar data is shown. The synthetic radar data is synthetic raw radar data of a sampled chirp, as explained above. The apparatus 300 may be integrated or may be the apparatus 100. Alternatively, the apparatus 300 may be external to the apparatus 100. In the latter case, the apparatus 300 includes an optional interface circuit device 320 configured to send the trained generative model (130) to the apparatus 100.

[0074] The apparatus 300 includes a processing circuit device 310 configured to train a generative model based on real raw radar data. The processing circuit device 310 may be the processing circuit device 110 of the apparatus 100 described above, or may be located outside the processing circuit device 100.

[0075] The real raw radar data used to train the generative model can be reused to train the machine learning model 140, as explained above. The real raw radar data can be multi-dimensional data arranged in an array. In some examples, the real raw radar data can indicate sampled chirps received from multiple channels of a radar sensor.

[0076] The generative model can be trained by using any learning or training method (or training framework). For example, the processing circuit device 310 can use a maximum likelihood estimator, a variational autoencoder, a restricted Boltzmann machine, etc. to train the generative model. In some examples, the processing circuit device 310 is configured to train the generative model using a generative adversarial network (GAN) or a diffusion model. These two training methods may be particularly advantageous for training a generative model to generate synthetic radar data: GAN can train a generative model to provide particularly diverse and varying outputs because GAN teaches complex and multimodal data distributions. This can achieve a realistic recreation of radar data with simulated environmental changes. The diffusion model can provide stable and well-performing training, which can reduce the training workload and increase the predictability and reliability of the training results.

[0077] Figure 4 An example of a GAN 400 is shown. The general structure of the GAN 400 includes two neural networks trained in an adversarial environment. The first network, the (initially untrained) generative model (generator G) 410, is configured to attempt to generate artificial data (synthetic radar data 420) from noise (z) 430, which is a random noise distribution p z (z), which is preferably as close as possible to the real original radar data 440. GAN 400 is configured to learn the data distribution p on the input data (x) (real original radar data 440). g . The "opponent" of the first network, namely the second network 450 (discriminator D), is configured to distinguish between real samples from the original radar data 440 and the synthetic radar data 420 created by G, where D(x) is the probability that the sample is part of the real data set 440 and not from the synthetic data set 420. The generative model 410 is configured to attempt to increase (e.g., maximize) the loss of D by creating samples for the synthetic radar data 420 that are indistinguishable from real samples 440, thereby reducing (e.g., minimizing) log(1-D(G(z))). The discriminator 450 is configured to attempt to reduce (e.g., minimize) the loss D by correctly classifying the samples to be examined. This may cause the two networks 410, 450 to compete with each other in a minimax problem according to Equation 1:

[0078]

[0079] Equation 1, where the term “min G max D V(D,G)” represents a minimax optimization problem, where “min G ” refers to minimizing the objective relative to the generator (G) 410, and “max D ” refers to maximizing the objective with respect to the discriminator (D) 450.

[0080] In Equation 1, the term "V(D, G)" represents a value function that evaluates the performance of the discriminator 450 and the generator 410 in the GAN 400. The value function quantifies the ability of the discriminator 450 to distinguish between real data and synthetic data. The goal of GAN training is to find the optimal values ​​of the generator 410 and the discriminator 450 to minimize the value function.

[0081] The two networks 410, 450 may be interdependent and may be trained simultaneously. The purpose of the GAN 400 may be to balance the performance of both the generative model 410 and the discriminator 450.

[0082] The generative model 410 can learn and improve iteratively using this process. For example, when the convergence criteria (indicating that the model has reached a satisfactory level of training) are met, the training can be completed. These criteria can include stability of loss function values, validation metrics, or model parameters. When these metrics remain stable or show reduced improvements, the training can be stopped. Alternatively, when a predefined level of evaluation metrics is reached, the training can be completed. Evaluation metrics can be used to evaluate the quality of the generated samples. These metrics can include perceptual quality measurements, such as initial scores or Frechet initial distances.

[0083] The trained generative model 410 may then be able to generate synthetic radar data, as described above with reference to Figure 1 as described.

[0084] Return to reference Figure 3 , the processing circuit device 310 can be configured to train a generative model using a style-based GAN. This can be a GAN that increases control over the general style of the synthetic radar data (artificially created samples). A style-based GAN (e.g., StyleGAN) can use a style-based generative model. Based on the architecture of the progressive growth GAN, the generation of synthetic radar data can be trained by gradually increasing the size of the discriminator and the generative model (e.g., by adding layers). The added layers can increase the output resolution of the synthetic radar data. For better reliability and consistency, the style-based GAN can be implemented as StyleGAN2, which can create higher resolution images at a lower computational cost.

[0085] In a style-based implementation, the processing circuit device 310 does not directly input the latent vector z into the generative model. Instead, it can first be mapped to an intermediate space W using a mapping network f comprising eight consecutive fully connected layers. The style y = (y s ,y b ). These styles can be used to control an Adaptive Instance Normalization (AdaIN) operation, where AdaIN is a means for aligning the mean and variance of content features with the mean and variance of style features, and therefore aligning the style with the input data (real raw radar data). AdaIN layers can be placed at the convolutional layers of the upsampling blocks in the generative model (network). Additionally, noise can be sampled and added before input to each AdaIN layer and at each output of the convolutional layer. Noise can allow the generative model to better control local variations in the synthesized radar data, thereby creating more random details in the synthesized image. Adding pixel-by-pixel noise throughout the upsampling process can even increase the performance of the generative model.

[0086] In addition, bilinear upsampling and downsampling or adding hybrid regularization can be implemented in style-based GANs. For example, at inference time, different layers can be fed with varying latent data to create a disentangled latent space, thereby improving the performance of style-based generative models.

[0087] The following additional techniques may be implemented to avoid water droplet specifications in synthetic radar data resulting from the upscaling of feature maps when using style mixing: For example, weighted demodulation may be used instead of normalization techniques. Weighted demodulation may scale the output feature maps to restore the output to unit standard deviation, thereby removing water droplet artifacts from the synthetic radar data.

[0088] Additionally, lazy regularization can be applied to the loss (e.g., once every 16 mini-batches) instead of optimizing the generative model simultaneously. This can reduce computational cost without significantly affecting performance. Applying path length regularization can produce more consistent and reliable behavior of the generative model. In some examples, a skip connection generative model can be used instead of a feed-forward generative model. The discriminator can then use a residual network structure that can improve the Frechet inception distance.

[0089] Figure 5An example of a diffusion model 500 is shown. The diffusion model 500 may be based on non-equilibrium thermodynamics. In a forward diffusion process, noise may be gradually added to a sample of real raw radar data 510, thereby generating real raw radar data with noise 520, until the sample dissolves into pure noise 530. This may be able to create a desired data distribution from any distribution. The generative model of the diffusion model 500 is configured to learn how to reverse the diffusion process, thereby allowing the generative model to generate synthetic raw radar data.

[0090] Equation 2 describes an example of forward diffusion:

[0091]

[0092] Equation 2, where the variance β t A small amount of Gaussian noise is added to the real original radar data x0~q(x0) in the time period T to approximate the latter q(x (1:T) |x0), thereby obtaining multiple samples x1...x with added noise T .

[0093] In Equation 2, the term “q(x (1：T) |x0)" means variables x1 to x T The conditional distribution given an initial value x0. It represents the distribution of the sequence of variables over time. represents the product of the individual conditional distributions at each time step from the first time point t=1 to the later time point T. In other words, it indicates that the distribution at each time step depends on the previous time steps.

[0094] The term “q(x t |x t-1 )" defines each variable x t Given its previous value x t-1 The conditional distribution of the case. It specifies that the conditional distribution follows a Gaussian (normal) distribution. The mean of the Gaussian distribution is √(1-β t )x t-1 , where x t-1 is the previous value, and β t is a scalar factor that determines the variance. The term √(1-β t ) The previous value x t-1 Scaling (1-β t ) times, and β t I means the variance on the diagonal is β t The diagonal covariance matrix of .

[0095] Therefore, the forward diffusion model can assume that for each variable x t The distribution at time t depends on its previous value x t-1, and it follows a Gaussian distribution. The mean of the Gaussian distribution is scaled by √(1-β) based on the previous value. t ) times, and the variance is scaled by a scalar factor β t Sure.

[0096] The diffusion process can be reversed by learning how to predict the transitions in the forward process. For example, according to Equation 3, the distribution p at time T can be obtained by The Markov chain estimates the distribution of the real raw radar data after diffusion processing over time p θ (x 0：T ):

[0097]

[0098] Equation 3, where μ θ (x t , t) is the mean of the data distribution, and Σ θ (x t , t) is the variance of the data distribution. p θ (x 0:T ) represents variables x0 to x T The joint distribution of . It captures the probability distribution of the entire variable sequence. The term “p(x T )" represents the final variable x T The marginal distribution of . It independently describes x T The probability distribution of , regardless of other variables in the sequence. represents the product of the conditional distribution at each time step from t = 1 to T. It indicates that for each variable x t-1 At a given x t The distribution of the cases depends on the next time step.

[0099] The second part of equation 3, p θ (x t-1 |x t ) defines each variable x t-1 At a given x t The conditional distribution of the case. It specifies that the conditional distribution follows a Gaussian (normal) distribution. This symbol indicates that the conditional distribution has a mean and the covariance matrix Gaussian distribution of . Mean and covariance is parameterized by θ, which represents the parameter set. The mean and covariance matrix can depend on x t and t varies.

[0100] By using Gaussian noise By sampling from and applying the inverse diffusion process to it, synthetic raw radar data can be created. To train the generative model of the diffusion model 500, the variational bound can be optimized on the negative log-likelihood according to Equation 4:

[0101]

[0102] Equation 4. The inequality and subsequent formula of Equation 4 represent an evaluation of the negative log-likelihood of the probability model. It compares the negative log-likelihood of the initial variable x0 with the probability of the entire variable sequence x under the true distribution and the proposed distribution. (0:T) The resulting value of L provides a measure of how well the model fits the observed data.

[0103] Equation 4 will be in the true distribution The negative log-likelihood of the initial variable x0 is similar to the proposed distribution q(x (1：T) |x0) is compared with the expected value obtained on the real distribution. This expected value represents the ratio of the entire variable sequence x to the proposed distribution. (0:T) The mean negative log-likelihood of .

[0104] item The expectation is further expanded to include the final variable x T The negative log-likelihood of and the conditional distribution for each time step t and The symbol L represents the negative log-likelihood, which is the value obtained from the above expectation. It is used as the sum of the log-ratios of p θ A measure of how well the probability model represented by σ fits the observed data. The lower the negative log-likelihood value, the better the model fits the data.

[0105] Since the diffusion model 500 does not need to be trained in an adversarial environment, its training may be easier than that of a GAN. However, due to the number of steps required for the forward and backward diffusion processes, it may take longer to train the diffusion model 500 and sample data.

[0106] Once trained, the generative model of the diffusion model 500 may be able to reverse the learned diffusion process, ie, generate synthetic radar data from noise.

[0107] Return to reference Figure 3In some examples, the processing circuit device 310 can be configured to train the generative model using a latent diffusion model. The latent diffusion model can include an autoencoder to reduce the dimensionality of the raw radar data into a latent space. A diffusion process can then be run in the resulting latent space. Random noise can then be sampled and run through an inverse diffusion process to create encoded samples of synthetic raw radar data. A decoder of the autoencoder can decode the lower dimensional samples to create synthetic raw radar data.

[0108] Therefore, the latent diffusion model may not perform either of the two diffusion processes directly in the pixel space. Instead, the latent diffusion model may have a compression part that compresses samples of the real raw radar data into a latent space with a lower dimension, while the diffusion part may run a diffusion operation. In the first part of the training, an autoencoder may be used to encode and decode the real raw radar data. The autoencoder may be pre-trained based on a regularization technique. For example, VQ regularization may be used to create an architecture similar to VQ-GAN. Alternatively, KL regularization may be used, which may produce behavior similar to a variational autoencoder.

[0109] The compression model can encode the input x (real raw radar data) into a lower dimensional latent space z using its encoder E by downsampling the input x. The decoder D can then transform z back into the input space. This approach can provide more flexibility and optimization in how the latent space is created. Additionally, using an autoencoder before running the diffusion operation can reduce complexity, thus creating a more efficient diffusion process.

[0110] In some examples, the processing circuit device 310 can be configured to train the generative model by setting the hyperparameters of the generative model. For example, the hyperparameters may include a learning rate, a batch size, the number of hidden layers, the number of units per layer, a regularization parameter, an activation function, a dropout rate, etc. These values ​​can control the speed of convergence, the complexity of the generative model, the amount of regularization, or the ability of the generative model to generalize. The processing circuit device 310 can, for example, perform tuning of the hyperparameters, that is, it can, for example, iteratively determine the value of the hyperparameter to select a hyperparameter with increased performance. The processing circuit device 310 can perform tuning by using, for example, a grid search, a random search, a Bayesian optimization, etc.

[0111] In other training implementations applying supervised learning, the processing circuit device 310 is further configured to determine the labeled data by labeling the real raw radar data or data derived therefrom, wherein the processing circuit device is configured to train the generative model based on the labeled data.

[0112] In some examples, the processing circuit device 310 is further configured to convert the real raw radar data into a portable network image (PNG) format or a hierarchical data format (HDF). The processing circuit device 310 can be configured to train a generative model based on the converted real raw radar data. This can simplify the training process. For example, an HDF5 file can be used as an input to a GAN or diffusion model. An HDF5 dataset can be viewed as a container that includes the data itself and metadata describing the data, allowing a structure similar to a NumPy array to exist. Using a PNG file, the data can be split into different directories, where a tag as the name of the subdirectory indicates the category of the sub-portion of the data stored therein.

[0113] In order to have more stable and robust training, normalized input may be preferred for training. For example, the processing circuit device 310 may be further configured to normalize the real raw radar data and train the generative model based on the normalized real raw radar data. For example, the processing circuit device 310 may be configured to normalize the real raw radar data by applying minimax normalization to the real raw radar data. Minimax normalization can retain data distribution that is particularly beneficial for GAN and diffusion models, and minimax normalization can further avoid the dominance of features and maintain outlier information. When using normalized data as input for training, the processing circuit device 310 may further normalize the synthesized raw radar data to the value range of the real raw radar data. Alternatively, any other normalization technique may be used, such as feature scaling, robust scaling, or Z-score normalization by range.

[0114] Figure 6 An example of method 600 is shown. For example, method 600 may be performed by an apparatus as described herein (such as apparatus 100). Method 600 includes obtaining 610 a trained generative model, such as reference Figure 4 or Figure 5 The trained generative model described is used, and synthetic radar data is generated 620 using the trained generative model, the synthetic radar data being synthetic raw radar data of the sampled chirp.

[0115] In combination with the proposed technology or one or more of the above examples (e.g., reference Figure 1 ) explains more details and aspects of method 600. Method 600 may include one or more additional optional features corresponding to one or more aspects of the proposed technology or one or more examples described above.

[0116] Figure 7An example of synthetic radar data 700 that can be obtained by a method (such as method 600) for generating training data for training a machine learning model as described herein is shown. Synthetic radar data 700 is synthetic raw radar data of sampled chirps 710, 720, 730. Figure 7 , three sampled chirps 710, 720, 730 are shown stacked on top of each other for illustrative purposes. However, in other examples, synthetic radar data 700 may include any number n≥2 of sampled chirps in any data format or arrangement.

[0117] Synthetic radar data 700 may have several features that make it distinguishable from real radar data. For example, synthetic radar data 700 may have statistical properties, such as correlation between different chirps of synthetic radar data 700, that differ from statistical properties of real radar data by, for example, a predefined value.

[0118] In addition, synthetic radar data 700 may be identified by certain artifacts, inconsistencies, anomalies, or patterns that are unique to the generative model that generated them. For example, artifacts may be quantified by their number in synthetic radar data 700 or their percentage for each predefined number of data points of synthetic radar data 700. For example, synthetic radar data 700 may have a predefined share of artifacts in the data points. Deviations from a known distribution of real radar data or statistically impossible combinations of values ​​in synthetic radar data 700 may also indicate the presence of synthetic data.

[0119] In some examples, synthetic radar data 700 may be identified as synthetic data by accompanying metadata. For example, during the generation of synthetic radar data 700, it may be tagged with metadata that includes information about the generation method used to create synthetic radar data 700, thereby indicating that data 700 originates from a synthetic data generation process. The metadata may also provide detailed information about the source of synthetic radar data 700. In this case, the metadata may indicate that data 700 is derived from noise or generated from scratch (without any data basis). Additionally or alternatively, metadata may provide information about how synthetic radar data 700 is designed to mimic the statistical properties and distribution of real raw radar data. The metadata may include, for example, information about the distribution characteristics of a trained generative model that generates synthetic radar data 700.

[0120] Figure 8An example of a method 800 for training a generative model to generate synthetic radar data is shown. The method 800 may be performed by an apparatus described herein, such as apparatus 300. The synthetic radar data is synthetic raw radar data of a sampled chirp. The method 800 includes obtaining 810 a generative model and training 820 the generative model to generate synthetic radar data based on real raw radar data.

[0121] In combination with the proposed technology or one or more of the above examples (e.g., reference Figure 3 ) explains more details and aspects of method 800. Method 800 may include one or more additional optional features corresponding to one or more aspects of the proposed technology or one or more examples described above.

[0122] Fig. 9 An example of a radar system 900 is shown. The radar system 900 includes an apparatus 910, such as apparatus 300, for training a generative model to generate synthetic radar data as described herein. The radar system 900 further includes a radar sensor 920 configured to generate real raw radar data. The radar sensor 920 can transmit electromagnetic waves in the radio frequency range toward a target or area of ​​interest. The radar sensor 920 can then receive reflected waves or echoes. Information about the echoes, such as signal amplitude and phase, is captured as real raw radar data, thereby providing detailed information about the presence, location, and characteristics of objects within the field of view of the radar sensor.

[0123] Fig.10a and Fig.10b Another example of a method 1000 for generating training data for training a machine learning model to perform a radar task is shown. The method 1000 may be performed by an apparatus as described herein, such as apparatus 100 or 300.

[0124] The method 1000 includes obtaining 1010 real raw radar data and obtaining a trained generative model, for example obtaining 1020 a GAN ( Fig.10a ) or obtain 1025 diffusion models ( Fig.10b ). For example, real raw radar data may be fed into a training framework to train a generative model, thereby producing a trained generative model. The method 1000 further includes generating 1030 synthetic radar data using the trained generative model, the synthetic radar data being the synthesized raw radar data of the sampled chirp. For example, the trained generative model may output the synthetic radar data. The generative portion of the GAN or the diffusion model, i.e., the generator of the GAN and the backward diffusion model in the diffusion model, may be extracted and used to sample the synthetic raw radar data.

[0125] The method 1000 further includes preprocessing 1040 the synthetic radar data and the real raw radar data. For example, the synthetic radar data can be used in a preprocessing pipeline with the real raw radar data to create a range-Doppler image and a range-micro-Doppler image. The method 1000 further includes training 1050 a machine learning model based on the preprocessed synthetic radar data and the preprocessed real raw radar data to perform a radar task. For example, the range-Doppler image and the range-micro-Doppler image can be input into a training framework of the machine learning model. For example, the machine learning model can be a DNN.

[0126] In the following, some examples of the proposed technique are presented:

[0127] An example (eg, Example 1) relates to a method that includes obtaining a trained generative model and generating synthetic radar data using the trained generative model, the synthetic radar data being synthetic raw radar data of a sampled chirp.

[0128] Another example (e.g., Example 2) involves a previous example (e.g., Example 1) or any other example, further including training a machine learning model based on synthetic radar data to perform a radar task.

[0129] Another example (e.g., Example 3) involves the previous example (e.g., Example 2) or any other example, further including training a machine learning model based on real raw radar data to perform a radar task.

[0130] Another example (e.g., Example 4) relates to a previous example (e.g., one of Examples 2 or 3) or any other example, further comprising determining labeled data by labeling synthetic radar data or data derived therefrom for a radar mission, wherein training the machine learning model comprises training the machine learning model based on the labeled data.

[0131] Another example (e.g., Example 5) relates to a previous example (e.g., one of Examples 2 to 4) or any other example, further including extracting at least one signal characteristic from the synthetic radar data, wherein training the machine learning model includes training the machine learning model based on the extracted at least one signal characteristic.

[0132] Another example (e.g., Example 6) involves a previous example (e.g., one of Examples 2 to 5) or any other example, further including determining at least one of a range-velocity representation and a range-angle representation of the synthetic radar data, wherein training the machine learning model includes training the machine learning model based on the determined at least one of the range-velocity representation and the range-angle representation of the synthetic radar data.

[0133] Another example (e.g., Example 7) relates to the previous example (e.g., Example 6) or any other example, further including determining at least one of a range-velocity representation and a range-angle representation of real raw radar data, wherein training the machine learning model includes training the machine learning model based on the determined at least one of the range-velocity representation and the range-angle representation of the real raw radar data.

[0134] Another example (e.g., Example 8) involves a previous example (e.g., one of Examples 2 to 7) or any other example, further including determining a range-micro-speed representation and a range-macro-speed representation of the synthetic radar data, wherein training the machine learning model includes training the machine learning model based on the determined range-micro-speed representation and the determined range-macro-speed representation of the synthetic radar data.

[0135] Another example (e.g., Example 9) relates to the previous example (e.g., Example 8) or any other example, further including determining a range-micro-speed representation and a range-macro-speed representation of real raw radar data, wherein training the machine learning model includes training the machine learning model based on the determined range-micro-speed representation and the range-macro-speed representation of the real raw radar data.

[0136] Another example (e.g., Example 10) relates to a previous example (e.g., one of Examples 1 to 9) or any other example, further comprising training a generative model based on real raw radar data, thereby producing a trained generative model.

[0137] Another example (e.g., Example 11) relates to a previous example (e.g., one of Examples 1 to 10) or any other example, further including performing a radar task by analyzing additional radar data using the trained machine learning model.

[0138] Another example (e.g., Example 12) relates to a previous example (e.g., Example 11) or any other example, further including performing a radar task, performing the radar task including detecting at least one of a gesture, a motion, a micro-motion, a macro-motion, and the presence of a target based on additional radar data.

[0139] One example (eg, Example 13) involves synthetic radar data that can be obtained by a method of a previous example (eg, one of Examples 1 to 12) or any other example.

[0140] Examples (e.g., Example 14) relate to a method for training a generative model to generate synthetic radar data, the synthetic radar data being synthetic raw radar data of sampled chirps, the method comprising training the generative model based on real raw radar data.

[0141] Another example (e.g., Example 15) relates to the previous example (e.g., Example 14) or any other example, further including: training the generative model includes training the generative model using a generative adversarial network GAN or a diffusion model.

[0142] Another example (e.g., Example 16) relates to the previous example (e.g., Example 15) or any other example, further including: training the generative model includes training the generative model using a style-based GAN.

[0143] Another example (e.g., Example 17) relates to the previous example (e.g., Example 15) or any other example, further including: training the generative model includes training the generative model using a latent diffusion model.

[0144] Another example (e.g., Example 18) relates to a previous example (e.g., one of Examples 14 to 17) or any other example, further including: training the generative model includes setting hyperparameters of the generative model.

[0145] Another example (e.g., Example 19) relates to the previous example (e.g., one of Examples 14 to 18) or any other example, further including: the real original radar data is multidimensional data arranged in an array.

[0146] Another example (e.g., Example 20) relates to the previous example (e.g., one of Examples 14 to 19) or any other example, further including: the real raw radar data indicates sampled chirps received from multiple channels of the radar sensor.

[0147] Another example (e.g., Example 21) relates to a previous example (e.g., one of Examples 11 to 17) or any other example, further comprising determining labeled data by labeling real raw radar data or data derived therefrom, wherein training the generative model comprises training the generative model based on the labeled data.

[0148] Another example (e.g., Example 22) relates to the previous example (e.g., Example 21) or any other example, further including converting the real raw radar data into a portable network image format or a layered data format, wherein determining the labeled data includes labeling the converted real raw radar data.

[0149] Another example (e.g., Example 23) relates to a previous example (e.g., one of Examples 14 to 22) or any other example, further including normalizing the real raw radar data, wherein training the generative model includes training the generative model based on the normalized real raw radar data.

[0150] Another example (e.g., Example 24) relates to the previous example (e.g., Example 23) or any other example, further including: normalizing the real raw radar data includes applying minimax normalization to the real raw radar data.

[0151] An example (e.g., Example 25) relates to an apparatus comprising a processing circuit device configured to obtain a trained generative model and to generate synthetic radar data using the trained generative model, the synthetic radar data being synthetic raw radar data of a sampled chirp.

[0152] Another example (e.g., Example 26) relates to the previous example (e.g., Example 25) or any other example, further including: the processing circuit device is further configured to train the machine learning model based on the synthetic radar data to perform the radar task.

[0153] Another example (e.g., Example 27) relates to the previous example (e.g., Example 26) or any other example, further including: the processing circuit device is further configured to train the machine learning model based on real raw radar data to perform radar tasks.

[0154] Another example (e.g., Example 28) relates to a prior example (e.g., one of Example 26 or Example 27) or any other example, further including: the processing circuit device is further configured to determine labeled data by labeling the synthetic radar data or data derived therefrom for the radar mission, wherein the processing circuit device is configured to train the machine learning model based on the labeled data.

[0155] Another example (e.g., Example 29) relates to a previous example (e.g., one of Examples 26 to 28) or any other example, further including: the processing circuit device is further configured to extract at least one signal characteristic from the synthetic radar data, wherein the processing circuit device is configured to train a machine learning model based on the extracted at least one signal characteristic.

[0156] Another example (e.g., Example 30) relates to a previous example (e.g., one of Examples 26 to 29) or any other example, further including: the processing circuit device is further configured to determine at least one of a range-velocity representation and a range-angle representation of the synthetic radar data, wherein the processing circuit device is configured to train a machine learning model based on the determined at least one of the range-velocity representation and the range-angle representation of the synthetic radar data.

[0157] Another example (e.g., Example 31) relates to the previous example (e.g., Example 30) or any other example, further including: the processing circuit device is further configured to determine at least one of the range-velocity representation and the range-angle representation of the real raw radar data, wherein the processing circuit device is configured to train the machine learning model based on the determined at least one of the range-velocity representation and the range-angle representation of the real raw radar data.

[0158] Another example (e.g., Example 32) relates to a previous example (e.g., one of Examples 26 to 31) or any other example, further including: the processing circuit device is further configured to determine a range-micro-speed representation and a range-macro-speed representation of the synthetic radar data, wherein the processing circuit device is configured to train the machine learning model based on the determined range-micro-speed representation and the determined range-macro-speed representation of the synthetic radar data.

[0159] Another example (e.g., Example 33) relates to the previous example (e.g., Example 32) or any other example, further including: the processing circuit device is further configured to determine a range-micro-speed representation and a range-macro-speed representation of the real raw radar data, wherein the processing circuit device is configured to train the machine learning model based on the determined range-micro-speed representation and the range-macro-speed representation of the real raw radar data.

[0160] Another example (e.g., Example 34) relates to a previous example (e.g., one of Examples 25 to 33) or any other example, further including: the processing circuit device is further configured to train the generative model based on real raw radar data, thereby generating a trained generative model.

[0161] Another example (e.g., Example 35) relates to a previous example (e.g., one of Examples 25 to 34) or any other example, further including: the processing circuit device is further configured to perform the radar task by analyzing additional radar data using the trained machine learning model.

[0162] Another example (e.g., Example 36) relates to the previous example (e.g., Example 35) or any other example, further including: the processing circuit device is configured to perform a radar task by detecting at least one of gestures, movements, micro-movements, macro-movements, and the presence of a target based on additional radar data.

[0163] An example (e.g., Example 37) relates to a device for training a generative model to generate synthetic radar data, the synthetic radar data being synthetic raw radar data of sampled chirps, the device comprising a processing circuit device configured to train the generative model based on real raw radar data.

[0164] Another example (e.g., Example 38) relates to the previous example (e.g., Example 37) or any other example, further including: the processing circuit device is configured to train a generative model using a generative adversarial network GAN or a diffusion model.

[0165] Another example (e.g., Example 39) relates to the previous example (e.g., Example 38) or any other example, further including: the processing circuit device is configured to train the generative model using a style-based GAN.

[0166] Another example (eg, Example 40) relates to the previous example (eg, Example 38) or any other example, further comprising: the processing circuit device is configured to train the generative model using the latent diffusion model.

[0167] Another example (e.g., Example 41) relates to a previous example (e.g., one of Examples 37 to 40) or any other example, further including: the processing circuit device is configured to train the generative model by setting hyperparameters of the generative model.

[0168] Another example (e.g., Example 42) relates to the previous example (e.g., one of Examples 37 to 41) or any other example, further including: the real original radar data is multidimensional data arranged in an array.

[0169] Another example (e.g., Example 43) relates to a previous example (e.g., one of Examples 37 to 43) or any other example, further including: the real raw radar data indicates sampled chirps received from multiple channels of the radar sensor.

[0170] Another example (e.g., Example 44) relates to a previous example (e.g., one of Examples 37 to 43) or any other example, further including: the processing circuit device is further configured to determine the labeled data by labeling the real raw radar data or data derived therefrom, wherein the processing circuit device is configured to train the generative model based on the labeled data.

[0171] Another example (e.g., Example 45) relates to a previous example (e.g., one of Examples 37 to 44) or any other example, further including: the processing circuit device is further configured to convert the real raw radar data into a portable network image format or a layered data format, wherein the processing circuit device is configured to train the generative model based on the converted real raw radar data.

[0172] Another example (e.g., Example 46) relates to a previous example (e.g., one of Examples 37 to 45) or any other example, further including: the processing circuit device is further configured to normalize the real raw radar data, wherein the processing circuit device is configured to train the generative model based on the normalized real raw radar data.

[0173] Another example (e.g., Example 47) relates to the previous example (e.g., Example 46) or any other example, further including: the processing circuit device is configured to normalize the real raw radar data by applying minimum-maximum normalization to the real raw radar data.

[0174] An example (e.g., Example 48) relates to a radar system comprising an apparatus of a previous example (e.g., any one of Examples 37 to 47), and a radar sensor configured to generate real raw radar data.

[0175] Another example (e.g., Example 49) involves a non-transitory machine-readable medium having stored thereon a program having program code for performing a method of a previous example (e.g., any one of Examples 1 to 12) when the program is executed on a processor or programmable hardware.

[0176] Another example (eg, Example 50) relates to a program having a program code for performing the method of a previous example (eg, any one of Examples 1 to 12) when the program is executed on a processor or programmable hardware.

[0177] Another example (e.g., Example 51) involves a non-transitory machine-readable medium having a program stored thereon, the program having program code for performing the method of a previous example (e.g., any one of Examples 14 to 24) when the program is executed on a processor or programmable hardware.

[0178] Another example (eg, Example 52) relates to a program having a program code for executing a method of a previous example (eg, any one of Examples 14 to 24) when the program is executed on a processor or programmable hardware.

[0179] The aspects and features described with respect to a particular example in the previous examples may also be combined with one or more additional examples to replace the same or similar features of the additional examples or to additionally introduce these features into the additional examples.

[0180] Examples may also be or relate to (computer) programs, which include program codes that execute one or more of the methods described above when the program is executed on a computer, processor, or other programmable hardware component. Therefore, the steps, operations, or processes of the different methods in the above methods may also be performed by a programmed computer, processor, or other programmable hardware component. Examples may also encompass program storage devices, such as digital data storage media, which are machine, processor, or computer readable, and encode and / or contain machine executable, processor executable, or computer executable programs and instructions. For example, a program storage device may include or may be a digital storage device, a magnetic storage medium (such as a disk and a tape), a hard drive, or an optically readable digital data storage medium. Other examples may also include a computer, a processor, a control unit, a (field) programmable logic array ((F)PLA), a (field) programmable gate array ((F)PGA), a graphics processor unit (GPU), an application specific integrated circuit (ASIC), an integrated circuit (IC), or a system on a chip (SoC) system that is programmed to perform the steps of the above methods.

[0181] It should also be understood that, unless expressly stated in individual cases or required for technical reasons, the disclosure of several steps, processes, operations or functions disclosed in this specification or claims should not be interpreted as implying that these operations must rely on the described order. Therefore, the previous description does not limit the execution of several steps or functions to a specific order. In addition, in other examples, a single step, function, process or operation may include and / or may be decomposed into several sub-steps, sub-functions, sub-processes or sub-operations.

[0182] If some aspects have been described with respect to a device or system, these aspects should also be understood as descriptions of corresponding methods. For example, blocks, devices or functional aspects of a device or system may correspond to features of a corresponding method, such as method steps. Therefore, aspects described with respect to a method should also be understood as descriptions of corresponding blocks, corresponding elements, attributes or functional features of a corresponding device or corresponding system.

[0183] The following claims are hereby incorporated into the detailed description, where each claim may stand alone as a separate example. It should also be noted that although in the claims, a dependent claim refers to a specific combination with one or more other claims, other examples may also include a combination of a dependent claim with the subject matter of any other dependent or independent claim. Unless it is stated in individual cases that a specific combination is not intended, such a combination is explicitly proposed herein. In addition, even if a claim is not directly defined as being dependent on any other independent claim, the features of that claim should also be included in any other independent claim.

Claims

1. A method (600), comprising: Obtaining (610) a trained generative model; as well as Synthetic radar data is generated (620) using the trained generative model, the synthetic radar data being synthesized raw radar data of the sampled chirp.

2. The method (600) of claim 1, further comprising training a machine learning model based on the synthetic radar data to perform a radar task.

3. The method (600) according to claim 2 further includes extracting at least one signal characteristic from the synthetic radar data, wherein training the machine learning model includes training the machine learning model based on the extracted at least one signal characteristic.

4. The method (600) according to any one of claims 2 or 3, further comprising determining at least one of a range-velocity representation and a range-angle representation of the synthetic radar data, wherein training the machine learning model comprises training the machine learning model based on the determined at least one of the range-velocity representation and the range-angle representation of the synthetic radar data.

5. The method (600) according to claim 4 further includes determining at least one of a range-velocity representation and a range-angle representation of real raw radar data, wherein training the machine learning model includes training the machine learning model based on the determined at least one of the range-velocity representation and the range-angle representation of the real raw radar data.

6. The method (600) of any one of claims 2 to 5, further comprising determining a range-micro-velocity representation and a range-macro-velocity representation of the synthetic radar data, wherein training the machine learning model comprises training the machine learning model based on the determined range-micro-velocity representation and the determined range-macro-velocity representation of the synthetic radar data.

7. Synthetic radar data (700) obtainable by a method according to any one of the preceding claims.

8. The synthetic radar data (700) of claim 7, wherein the synthetic radar data (700) includes metadata indicating that the synthetic radar data (700) is synthetic.

9. A method (800) for training a generative model to generate synthetic radar data, the synthetic radar data being synthesized raw radar data of a sampled chirp, the method (800) comprising training (820) the generative model based on real raw radar data.

10. The method (800) of claim 9, wherein training (820) the generative model comprises training the generative model using a generative adversarial network (GAN) or a diffusion model.

11. The method (800) of claim 10, wherein training (820) the generative model comprises training the generative model using a style-based GAN.

12. The method (800) of claim 10, wherein training (820) the generative model comprises training the generative model using a latent diffusion model.

13. An apparatus (100) comprising a processing circuit device (110), the processing circuit device being configured to: obtaining a trained generative model; and The trained generative model is used to generate synthetic radar data, the synthetic radar data being synthesized raw radar data of the sampled chirp.

14. An apparatus (300) for training a generative model to generate synthetic radar data, the synthetic radar data being synthetic raw radar data of sampled chirps, the apparatus (300) comprising a processing circuit device (310) configured to train the generative model based on real raw radar data.

15. A radar system (900), comprising: The device (910) according to claim 14; as well as A radar sensor (920) is configured to generate real raw radar data.

Images