Learning-based radar data processing system

Abstract

The present disclosure provides a system for processing radar data. The system may include a vehicle located in an environment; a radar module associated with the vehicle; and an electronic processor configured to: receive, from the radar module, an input radar signal comprising an indication of an object in the environment; processing the input radar signal by one or more signal processing algorithms to determine a raw radar spectrum; processing the raw radar spectrum through a machine learning computational model to determine a set of output predictions with respect to the environment; determining a representative model of the environment based at least in part on the set of output predictions about the environment; and providing the representative model of the environment to the automatic driving system.

Description

background

[0001] Radio detection and ranging (radar) is used in many applications, including object detection, ranging, direction finding, and mapping. Traditionally, radar has been used in aircraft, satellites, and ships at sea to locate objects and image terrain. In recent years, radar has become increasingly popular in automobiles for applications such as blind spot detection, collision avoidance, and autonomous driving. Unlike optical-based sensors (such as cameras or light detection and ranging (LIDAR) sensors) that are affected by weather and visibility variations, radar operates in low light conditions, darkness, and all types of weather conditions. Overview

[0002] Embodiments of this disclosure generally relate to systems and methods for determining a representative model of an environment by processing input radar signals using a machine learning computational model. In some embodiments, the representative model of the environment includes a three-dimensional (3D) scene representation and scene understanding of the environment.

[0003] This paper recognizes the various limitations of currently available radar systems. To utilize radar, vehicles can be equipped with multiple radar sensors and other types of sensors to detect obstacles and objects in the surrounding environment. However, the multiple radar sensors in current radar systems typically process data independently of each other. This paper provides systems and methods for processing and combining radar data, as well as data received from other sensors (e.g., imaging sensors, LiDAR, etc.). The performance and robustness of radar systems can be improved by combining data from multiple sensors or modules before sensing, detecting, or classifying objects or obstacles in the surrounding environment. Furthermore, the radar systems and methods disclosed herein can be configured to resolve computational ambiguities associated with processing and coherently combining radar data from multiple radar sensors or modules in order to identify nearby objects or obstacles and generate one or more local maps of the surrounding environment.

[0004] Modern vehicles (e.g., autonomous, semi-autonomous, or human-driven vehicles) rely on a combination of cameras and 3D sensors with data-driven methods to measure and extract information about the environment. Information from one or more cameras provides dense semantic information about the environment. 3D distance sensors (e.g., LiDAR or radar) supplement camera data with information such as distance. However, camera and LiDAR technologies are severely limited in adapting to weather conditions, and camera technology is further restricted to daytime applications. Radar is an active sensing 3D sensing technology similar to LiDAR, but it operates in the radio frequency (RF) band and does not have the disadvantages of LiDAR. Furthermore, compared to other 3D sensing technologies such as LiDAR, radar provides ego velocity (longitudinal velocity) and target velocity information. In some embodiments, ego velocity is the speed of the vehicle equipped with the system.

[0005] Radar is typically used in vehicles (e.g., automobiles) to enable specific driving functions, such as maintaining speed and distance to the vehicle ahead (e.g., adaptive cruise control), braking when a collision is possible (e.g., emergency braking assist), or alerting the driver when another vehicle is in a blind spot. To achieve these functions, scene understanding is determined. Scene understanding can include the presence or absence of targets, the location of targets in 3D space, the bounding box of the targets (e.g., their physical extent and size in 3D space), the orientation of the targets, the direction of travel of the targets, the speed of the targets, the category or type of the targets, whether the targets are moving or stationary, the speed of the vehicle equipped with radar sensors, and so on. Radar can also be used to generate scene representations of the environment, such as a free space map of the environment (e.g., a drivable area), which indicates, for example, where objects are present and where they are absent.

[0006] Traditional radar processing assumes the world consists of point-like reflective objects. This assumption allows radar system designers to apply conventional signal processing algorithms (e.g., Fourier transform, window functions, detection, and maximum likelihood estimation) by ignoring the complexities of the real world. An example processing pipeline might first compute a sparse point cloud that conforms to the point-like assumption, and then attempt to predict target attributes from that point cloud representation. However, the information lost during the conversion of radar data into a sparse point cloud significantly limits the performance of downstream sensing applications (e.g., 3D object detection, self-motion, velocity estimation, etc.).

[0007] In contrast, at least one technological advantage includes applying machine learning algorithms to raw radar data without requiring point cloud computation, enabling new applications and improved performance in areas where traditional automotive radar algorithms are difficult to implement. Typically, machine learning algorithms are algorithms trained through an optimization process that adjusts the algorithm's parameters and weights to improve its performance. Examples of machine learning algorithms include deep neural networks, convolutional neural networks, recurrent neural networks, long short-term memory networks, and transformer networks. Compared to traditional radar methods, in vehicle radar implementations, using machine learning algorithms can improve performance, fidelity, and latency in determining target and environmental attributes.

[0008] In some embodiments, the described radar imaging systems and methods evaluate a neural network to determine a desired output from a given set of inputs. In some embodiments, the neural network may be performed before or after conventional radar algorithms, such as signal processing or rule-based perception algorithms. In some embodiments, the neural network is trained to compute attributes from radar signals. In some embodiments, these attributes correspond to attributes of targets in the environment, such as bounding boxes, categories or types, directions of travel, speeds of travel, etc. In other embodiments, the attributes computed by the neural network are intermediate representations that are further processed to compute target-specific attributes.

[0009] During the training phase, in some embodiments, the neural network is evaluated based on input data with known information about the output, and the weights of the neural network are optimized such that the neural network provides an output that best matches these known outputs. In some embodiments, a loss or cost function is used to compare and evaluate the output of the neural network with the desired or known output. In other embodiments, the neural network is evaluated based on input data with unknown information about the output. In such embodiments, the loss function includes radar system information, which is used to optimize the neural network weights by comparing the output with the input.

[0010] In some embodiments, the training phase includes ingesting a large amount of real-world data to optimize the performance of the neural network. During this training phase, the neural network learns the most relevant features from the input data that are best correlated with the desired output and predicts the desired output. In some embodiments, the features learned by the neural network are automatically identified by allowing the training process to converge.

[0011] Training neural networks typically requires managing large amounts of real-world data, where the expected output (i.e., labels) of the neural network may already be determined. However, manually generating labels for real-world data on a large scale is challenging and expensive. Furthermore, in the case of radar data, manual labeling is further complicated by the fact that, unlike camera or LiDAR data, annotating radar spectrum is non-intuitive. Therefore, the automatic labeling method employed in this paper enables the creation of large-scale datasets. In some embodiments, automatic labeling can employ additional sensors (e.g., cameras or LiDAR) to provide 3D scene context for the radar as ground truth labels. Additionally, unsupervised / self-supervised techniques can be used for automatic labeling in radar space.

[0012] It should be understood that the methods according to this disclosure may include any combination of the aspects and features described herein. That is, the methods according to this disclosure are not limited to the combinations of aspects and features specifically described herein, but may also include any combination of the provided aspects and features.

[0013] Details of one or more embodiments of this disclosure are set forth in the accompanying drawings and the following description. Other features and advantages of this disclosure will become apparent from the description, the drawings, and the claims. By incorporating via reference

[0014] All publications, patents, and patent applications mentioned in this specification are incorporated herein by reference to the extent that each individual publication, patent, or patent application is specifically and individually indicated to be incorporated by reference. If any publication, patent, or patent application incorporated by reference contradicts the disclosure contained in this specification, the specification is intended to supersede or supersede any such contradictory material. Brief description of the attached diagram

[0015] A better understanding of the features and advantages of this subject matter will be obtained by referring to the following detailed description and accompanying drawings (also referred to herein as "Figures" and "FIGs"), in which: Figure 1 A system for processing radar data according to some embodiments is described; Figure 2 A system configured to process a subset of radar data from multiple radar modules, according to some embodiments, is described; Figure 3 A non-limiting exemplary computer system is described that can be programmed or otherwise configured to implement the methods or systems disclosed herein; and Figure 4 A flowchart depicts a non-limiting exemplary process that can be implemented by embodiments of the present disclosure. Detailed description

[0016] Although various embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many variations, modifications, and substitutions will occur to those skilled in the art without departing from the present disclosure. It should be understood that various alternatives may be taken to the embodiments of the present disclosure described herein.

[0017] Specific definition Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include the plural pronouns unless the context clearly indicates otherwise. Unless otherwise stated, any reference to “or” herein is intended to cover “and / or.”

[0018] As used herein, the term "real-time" refers to the transmission or processing of data without intentional delay, given the processing limitations of a given system, the time required to accurately acquire data and images, and the rate of change of the data and images. In some examples, "real-time" is used to describe the presentation of information obtained from components of embodiments of this disclosure.

[0019] When the term "at least," "greater than," or "greater than or equal to" precedes the first value in a series of two or more values, the term "at least," "greater than," or "greater than or equal to" applies to each value in the series. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[0020] When the term "not exceeding," "less than," or "less than or equal to" precedes the first value in a series of two or more values, the term "not exceeding," "less than," or "less than or equal to" applies to each value in the series. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[0021] Target detection / classification The described systems and methods can be configured to detect or classify one or more targets in the surrounding environment. Detecting a target may include identifying the presence of a target near a vehicle. Classifying a target may include determining whether the target is stationary or moving, or determining whether the target is located relative to the vehicle in a position that obstructs or partially obstructs the vehicle's path of movement. A target can be any object outside the vehicle. A target can be a living organism or an inanimate object. A target can be a pedestrian, animal, vehicle, building, signpost, sidewalk, sidewalk curb, fence, tree, or any object that may obstruct the vehicle's movement in any given direction. A target can be stationary, moving, or capable of movement.

[0022] The target can be located in front of, behind, or to the side of the vehicle. The target can be located within a range of at least approximately 1 meter (m), 2m, 3m, 4m, 5m, 10m, 15m, 20m, 25m, 50m, 75m, or 100m of the vehicle. The target can be located on land, in water, or in the air. The target can be located on or near the path of the vehicle. The target can be oriented in any direction relative to the vehicle. The target can be oriented facing the vehicle or oriented away from the vehicle at an angle from 0 degrees to approximately 360 degrees. In some cases, the target may include multiple targets external to the land vehicle.

[0023] Targets may have a measurable or detectable spatial arrangement or characteristics. Spatial arrangement information may include information about the target's position, speed, acceleration, or other kinematic properties relative to a land vehicle. Target characteristics may include information about the target's size, shape, orientation, or material properties. Material properties may include the target's reflectivity or radar cross-section. In some cases, target characteristics may include a measurement of the target's angle of arrival relative to a vehicle. The angle of arrival may correspond to the elevation or azimuth angle associated with the input radar signal reflected from the target and received at the vehicle.

[0024] In some embodiments, the target may have a size of at least 0.2 meters, be located laterally to the land vehicle, and be at least about 1 meter away from the land vehicle. In some embodiments, the target may have a size of at least 0.2 meters, be located forward or backward to the land vehicle, and be at least about 1 meter away from the land vehicle. The surrounding environment may be a location or environment in which the vehicle can operate. The surrounding environment may be an indoor or outdoor space. The surrounding environment may be an urban, suburban, or rural environment. The surrounding environment may be a high-altitude or low-altitude environment. The surrounding environment may include environments that provide low visibility (nighttime, heavy precipitation, fog, airborne particles). The surrounding environment may include targets on the vehicle's path of travel. The surrounding environment may include targets outside the vehicle's path of travel.

[0025] Example System Figure 1 An example of a system 100 for processing radar data is depicted. System 100 may include a processor 140 communicatively coupled to a plurality of radar modules 130-1, 1302, 130-3, and so on, up to the nth radar module 130-n, where n can be any integer. In some cases, n may be greater than or equal to 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or greater. Each of the plurality of radar modules may be configured to transmit a first set of signals. The first set of signals may include a plurality of output radar pulses 105. Each of the plurality of radar modules may be configured to receive a second set of signals reflected from a target 102 in the surrounding environment. The second set of signals may be a subset of the first set of signals transmitted by each of the plurality of radar modules and may be generated when a subset of the first set of signals interacts with or is reflected from the target 102. The second set of signals may include a plurality of input radar pulses 106.

[0026] Multiple radar modules 130-n may each include a radar transmitter or a radar receiver. A radar transmitter may include a transmitting antenna. A radar receiver may include a receiving antenna. The transmitting antenna can be any antenna capable of converting electrical signals into electromagnetic waves and transmitting those waves (dipole antenna, directional antenna, patch antenna, sector antenna, Yagi antenna, parabolic antenna, grid antenna). The receiving antenna can be any antenna capable of receiving electromagnetic waves and converting radio frequency radiated waves into electrical signals (dipole antenna, directional antenna, patch antenna, sector antenna, Yagi antenna, parabolic antenna, grid antenna). In some cases, each of the radar modules 130-n may include one or more transmitting antennas or one or more receiving antennas. In some cases, each of the radar modules 130-n may have multiple RX or TX channels. The radar module 130-n can be used to detect one or more targets in the surrounding environment. In some cases, each of the radar modules 130-n may include an imaging device (e.g., a camera).

[0027] Each of the multiple radar modules 130-n can be configured to transmit a first set of radar signals comprising multiple output radar pulses. The multiple output radar pulses can include radar pulses. The radar pulses can be any electromagnetic wave or signal transmitted by the radar module in a frequency range from approximately 1 Hz to approximately 300 GHz. In some cases, the multiple output radar pulses may have frequencies of 24 GHz, 60 GHz, or 79 GHz.

[0028] System 100 may further include a processor 140 operatively coupled to and communicating with each of the plurality of radar modules. Each of the plurality of radar modules may be configured to provide the processor 140 with a second set of radar signals or a plurality of input radar pulses 106 received respectively from each of the plurality of radar modules. In some embodiments, the processor 140 is configured to process the input radar pulses 106 received by the radar modules 103-n via a series of trained algorithms including, for example, machine learning computational models. In some embodiments, the input radar pulses 106 include one or more of the following: raw time-domain signal (fast time x slow time x antenna channel), range-Doppler spectrum (e.g., range x Doppler x (virtual / real) antenna channel), or beam spectrum (e.g., range x Doppler x azimuth x elevation). In some embodiments, the radar signal representation may also be real-valued or complex-valued. In some embodiments, the range-Doppler spectrum is the same as the raw radar spectrum, wherein the data axes are range and Doppler. In some embodiments, the beam spectrum is the same as the original radar spectrum, wherein the data axes are range, Doppler, and azimuth and / or elevation (e.g., azimuth = horizontal angle; elevation = vertical angle).

[0029] In some embodiments, processor 140 is configured to process the input radar pulse 106 by one or more signal processing steps before processing it via a trained neural network. Example signal processing steps include Fourier transform, filtering, interpolation, extrapolation, downsampling, upsampling, resampling, windowing, parameter estimation, subspace decomposition, maximum likelihood estimation, thresholding, and so on. In some embodiments, processor 140 is configured to process the input radar pulse 106 by one or more signal processing steps after processing it via a trained neural network. In some embodiments, processor 140 is configured to include, in addition to the input radar pulse 106, a history of the input radar pulse 106 (e.g., the last N frames) as input to a machine learning computational model. Typically, radar senses the world through snapshots referred to herein as frames. Multiple frames may correspond to consecutive snapshots of the sensor. In some embodiments, the machine learning computational model is configured to learn a temporal fusion of the input radar pulse 106 as history is provided. In some embodiments, radar modules 130-n may have overlapping fields of view. In some embodiments, the machine learning computational model is configured to learn spatial fusion of spatially distributed radar pulses 106 when receiving input radar pulses 106 from multiple radar modules 130-n.

[0030] In some embodiments, processor 140 is configured to provide the output of a series of trained algorithms to an autonomous driving system. In some embodiments, the autonomous driving system includes an emergency braking system, an advanced driver assistance system (ADAS), or an active safety system. In some embodiments, the machine learning computational model is configured to provide one or more of the following as output: point cloud; detection mask (e.g., decomposing and labelling input radar pulses 106 as signal or noise components); data spectrum including, for example, range, Doppler, azimuth, or elevation angle; point cluster; point cluster with estimated trajectory; point-specific features including, for example, range, Doppler, ground speed, ground velocity, azimuth, elevation angle, altitude, or object category (e.g., car, pedestrian, truck, motorcycle, etc.); object-specific features including, for example, type, size, shape, orientation, bounding box, rate, speed, acceleration, turning rate, etc.; vehicle-specific attributes including, for example, turning rate, speed, acceleration, velocity, etc.; environmental attributes including, for example, the safe driving area of ​​the vehicle.

[0031] In some embodiments, processor 140 is configured to generate an occupancy grid for an autonomous driving system using the output of a series of trained algorithms. The occupancy grid can be a visual representation of the surrounding environment in which the radar system operates. The occupancy grid can indicate where one or more objects are detected in space relative to the position or orientation of a vehicle. The occupancy grid can indicate the presence of one or more objects in the surrounding environment and near the vehicle. The occupancy grid can show the position or orientation of one or more objects relative to the position or orientation of the vehicle. The vehicle can be stationary or moving. In some cases, the occupancy grid can be generated or updated based on the movement of the vehicle through the surrounding environment.

[0032] In some embodiments, the machine learning computational model receives input radar pulses 106 as input. The architecture of the machine learning computational model can take any of a variety of forms, such as a neural network model, for example, a convolutional neural network model (CNN), a long short-term memory (LSTM) model, or a visual Transformer (ViT) model. Different kernel sizes (e.g., kernel sizes between 3 and 20) and the number of blocks (e.g., four blocks in the descending portion of the "U" in a U-NET architecture) can be appropriately used or tuned. In some embodiments, the machine learning computational model includes a data-driven machine learning neural network. In some embodiments, the data-driven machine learning neural network includes a multi-level network whose performance improves as it processes more data. In some embodiments, the neural network includes learnable parameters of double precision, single precision, half precision, or 8-bit precision.

[0033] In some embodiments, a machine learning computational model is trained using a body of training data comprising raw radar spectrum data generated manually or otherwise. In some embodiments, the raw radar spectrum data includes: a) one or more of the following quantities: signal amplitude, complex amplitude, phase, power, and / or signal-to-noise ratio; or b) measurements of one or more of range, Doppler, azimuth, and elevation. In some embodiments, the machine learning computational model is retrained upon receiving additional radar pulses. Any suitable training technique for the machine learning computational model can be implemented, such as a gradient descent process using a suitable loss function. In some embodiments, the output of the machine learning computational model is used to retrain the machine learning computational model using random dropout or other regularization methods. In some embodiments, the training data is retained for verification and evaluation of when training is complete. In some embodiments, training is conducted according to a regular time-series schedule (e.g., weekly), after accumulating a specific number (e.g., 200) of confirmed input radar pulses, according to any other suitable schedule, or at the user's command (e.g., via a UI such as…). Figure 3 Under the command received by the user interface (UI) 340, the machine learning computation model is retrained.

[0034] In some embodiments, labels from the following sources are used to train the machine learning computational model: supervised methods utilizing manual labeling (e.g., where a user assigns desired labels to radar data), and supervised learning using automatic labeling methods utilizing other sensors (e.g., cameras, lidar, GPS sensors, or inertial measurement units (IMUs)). In some embodiments, the output from the machine learning computational model is compared with the output from one or more of the sensors listed above. For example, object labels (e.g., people, cars, bicycles) obtained from lidar point clouds and camera images are associated with the radar point cloud and used as training signals by projecting them into the radar space. As another example, to learn how to distinguish between signals and noise, LiDAR can associate radar points with their spatial proximity to assess whether a radar point is a genuine detection or a false alarm.

[0035] In some embodiments, the machine learning computational model is trained using unsupervised / self-supervised methods, such as by learning how to reconstruct the original radar signal from multiple radars or from data at multiple time points. In some embodiments, a mathematical model of radar physics is used to train the machine learning computational model by forcing the feature space learned by the model to be consistent with the physics of radar. For example, an autoencoder-like architecture can be used to implement meaningful latent spatial variables, including estimates of the object's direction of arrival and velocity, as well as our motion. This can be achieved using a physical radar simulation that correlates how the object's 3D position and velocity produce the radar complex-valued beam spectrum as a decoder for the network. In some embodiments, a hybrid of one or more of the above training methods is employed. For example, unsupervised methods can be used to learn the network weights, while supervised methods can be used to fine-tune the network and manually labeled data to capture challenging extreme cases.

[0036] Figure 2 A system configured to process radar data from subsets of multiple radar modules 130-1, 130-2 and 130-3 is shown. Figure 2 The specific number and arrangement of radar modules depicted are merely illustrative, and any number and arrangement of radar modules can be included within a system used to process radar data.

[0037] Multiple radar modules 130-1, 130-2, and 130-3 can be installed on any side of the vehicle 104, or on one or more sides of the vehicle 104 (e.g., the front, rear, lateral, top, or bottom side of the vehicle). The installation locations of the multiple radar modules 130-1, 130-2, and 130-3 can be determined based on driving conditions, vehicle type, usage, etc.

[0038] The front side of the vehicle can be the side facing the general direction of travel of the vehicle 104, while the rear side (or back side) can be the side not facing the general direction of travel of the vehicle 104. The rear side can be opposite to the front side of the vehicle. The front side of the vehicle can point towards the forward direction of travel of the vehicle 104. The rear side of the vehicle can point towards the backward direction of travel of the vehicle 104 (e.g., reversing). The lateral side can include the left or right side of the vehicle. The vehicle 104 can be configured to move or translate orthogonally to the lateral side of the vehicle, or not. In some cases, multiple radar modules 130-1, 1302, and 130-3 can be mounted between two adjacent sides of the vehicle.

[0039] Multiple radar modules 130-1, 130-2, and 130-3 can be oriented to detect one or more targets 102 in front of, behind, to the side of, above, below, or near the vehicle 104. In some cases, each of the multiple radar modules 130-1, 130-2, and 130-3 can be configured to be mounted on the same or different sides of the vehicle 104. For example, one or more radar modules 130-1, 130-2, and 130-3 can be mounted on the top, bottom, front, rear, or side of the vehicle 104. In some cases, each of the multiple radar modules 130-1, 130-2, and 130-3 can be configured to be mounted in the same or different orientations. For example, one or more radar modules 130-1, 130-2 and 130-3 may be oriented to detect one or more targets 102 in front of, behind, to the side of, above or below the vehicle 104.

[0040] Vehicle 104 can be any type of machine used to transport people or goods. Examples of vehicles include, but are not limited to, trucks, bicycles, motorized vehicles, tractors, mining vehicles, rail vehicles, construction facilities, boats, amphibious vehicles, aircraft, spacecraft, etc. Vehicle 104 can be operated by a living being such as an animal (e.g., a human). Vehicle 104 can be stationary, moving, or capable of moving.

[0041] Vehicle 104 can be any suitable land, air, or water vehicle. Land vehicles can be motorized vehicles or any other vehicles using renewable or non-renewable energy sources (e.g., solar, thermal, electrical, wind, oil, etc.) to move on or close to the ground, for example, within 1 meter, 2 meters, or 3 meters of the ground. Air vehicles can be motorized vehicles or any other vehicles using renewable or non-renewable energy sources (solar, thermal, electrical, wind, oil, etc.) to move in the air or in space. Water vehicles can be motorized vehicles or any other vehicles using renewable or non-renewable energy sources (solar, thermal, electrical, wind, oil, etc.) to move on or across water.

[0042] In some embodiments, vehicle 104 is a land vehicle capable of traveling on land. For example, vehicle 104 may be a car. Alternatively or additionally, vehicle 104 may be capable of traveling on or under water. In some embodiments, vehicle 104 is a land vehicle, a boat, an aircraft, or a spacecraft. Vehicle 104 can travel freely above a surface. Vehicle 104 can travel freely in two or more dimensions. Vehicle 104 may primarily travel on one or more roads. In some cases, vehicle 104 is capable of operating in the air or in space. For example, vehicle 104 may be an airplane or a helicopter.

[0043] In some embodiments, vehicle 104 is an unmanned vehicle and can operate without the need for a human operator. In some embodiments, vehicle 104 may be without passengers or an operator on board. In some embodiments, vehicle 104 includes space for passengers to sit in. In some embodiments, vehicle 104 includes space for cargo or objects. In some embodiments, vehicle 104 includes tools that allow the vehicle to interact with its environment (e.g., collect samples, move objects, etc.). In some embodiments, vehicle 104 includes tools that emit objects (e.g., light, sound, liquids, or pesticides) into the surrounding environment.

[0044] In some embodiments, vehicle 104 is an autonomous vehicle. In some embodiments, vehicle 104 may be an autonomous or semi-autonomous vehicle. An autonomous vehicle may be a driverless vehicle. An autonomous vehicle may or may not have passengers or an operator on board. An autonomous vehicle may or may not have space for passengers to sit in. An autonomous vehicle may or may not have space for loading goods or objects. An autonomous vehicle may or may not have tools that allow the vehicle to interact with the environment (e.g., collect samples, move objects). An autonomous vehicle may or may not have objects that can be emitted to disperse into the environment (e.g., light, sound, liquids, pesticides, etc.). An autonomous vehicle can operate without the need for a human operator. An autonomous vehicle may be a fully autonomous vehicle or a partially autonomous vehicle.

[0045] In some embodiments, the vehicle 104 may allow one or more passengers to ride on it. In some embodiments, the vehicle 104 includes space for one or more passengers to ride on the vehicle. In some embodiments, the vehicle 104 includes an interior cabin with space for one or more passengers. In some embodiments, the vehicle 104 includes space for the driver of the vehicle. In some embodiments, the vehicle 104 can be driven by a human operator. In some embodiments, the vehicle 104 operates using an automatic driving system.

[0046] In some embodiments, the vehicle 104 can switch between a manual driving mode and an automatic driving mode. In manual driving mode, a human driver can operate the vehicle. In automatic driving mode, an automatic controller can generate signals to operate the vehicle without human driver intervention. In some embodiments, the vehicle 104 provides driver assistance, under which the driver can primarily drive the vehicle manually, but the vehicle can execute specific automatic procedures or assist the driver in executing specific procedures (e.g., lane changing, merging, stopping, automatic braking). In some embodiments, the vehicle 104 has a default operating mode. For example, manual driving mode or automatic driving mode can be the default operating mode.

[0047] like Figure 2 As shown, multiple radar modules 130-1, 130-2, and 130-3 can be configured to transmit a first set of radar signals, which includes multiple output radar pulses 105-1, 105-2, and 105-3. Multiple radar modules 130-1, 130-2, and 130-3 can be configured to receive a second set of radar signals, which includes multiple input radar pulses 106-1, 106-2, and 106-3. Multiple radar modules 130-1, 130-2, and 130-3 can be operatively coupled to and communicate with a processor 140. Multiple radar modules 130-1, 130-2, and 130-3 can be configured to provide the second set of radar signals to the processor 140 or multiple input radar pulses 106-1, 106-2, and 106-3 received respectively by each of the multiple radar modules. Processor 140 is configured to process input radar pulses 106 received by radar modules 130-1, 130-2, and 130-3 using a machine learning computational model. In some embodiments, the machine learning computational model is local to vehicle 104. In some embodiments, the machine learning computational model is accessed via a cloud or local server. In some embodiments, the machine learning computational model is trained in real time while processor 140 collects data via cloud infrastructure. In some embodiments, the machine learning computational model is trained offline after data is transmitted to a local or cloud data server.

[0048] Computer System Another aspect of this disclosure provides a computer system that is programmed or otherwise configured to implement the methods of this disclosure. Figure 3 A computer system 301 is depicted that is programmed or otherwise configured to implement the platforms, systems, media, and methods of this disclosure. For example, a computing device 510 may be programmed or otherwise configured to process radar data. The computer system 301 may be a user's electronic device or a computer system located remotely relative to that electronic device. The electronic device may be a mobile electronic device.

[0049] Computer system 301 may include a central processing unit (CPU, also referred to herein as a “processor” and “computer processor”) 305, which may be a single-core or multi-core processor, or multiple processors for parallel processing. Computer system 301 also includes memory or memory location 310 (e.g., random access memory, read-only memory, flash memory), electronic storage unit 315 (e.g., hard disk), communication interface 320 for communicating with one or more other systems (e.g., network adapter), and peripheral devices 325 such as cache, other memory, data storage devices, or electronic display adapters.

[0050] The memory 310, storage unit 315, interface 320, and peripheral device 325 communicate with the CPU 305 via a communication bus (solid line) such as a motherboard. The storage unit 315 may be a data storage unit (or data repository) for storing data. The computer system 301 can be operatively coupled to a computer network (“network”) 330 by means of the communication interface 320.

[0051] Network 330 may be the Internet, the Internet of Things, or an extranet, or an intranet or extranet communicating with the Internet. In some cases, network 330 is a telecommunications or data network. Network 330 may include one or more computer servers, which can support distributed computing, such as cloud computing. In some cases, with the help of computer system 301, network 330 can implement a peer-to-peer network, which enables devices coupled to computer system 301 to operate as clients or servers.

[0052] CPU 305 can execute a series of machine-readable instructions, which can be embodied in a program or software. The instructions can be stored in a memory location, such as memory 310. The instructions can be directed to CPU 305, which can then be programmed or otherwise configured to implement the methods of this disclosure. Examples of operations performed by CPU 305 can include fetching, decoding, executing, and writing back. CPU 305 can include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (dedicated processors that execute cryptographic algorithms within hardware), server processors, hardware accelerators, or any other suitable processing device. CPU 305 can be part of a circuit, such as an integrated circuit. One or more other components of system 301 can be included in this circuit. In some cases, the circuit is an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0053] Storage unit 315 may store files, such as drivers, libraries, and saved programs. Storage unit 315 may store user data, such as user preferences and user programs. In some cases, computer system 301 may include one or more additional data storage units located outside of computer system 301, such as on a remote server that communicates with computer system 301 via an intranet or the Internet.

[0054] Computer system 301 can communicate with one or more remote computer systems via network 330. For example, computer system 301 can communicate with a remote computer system belonging to a user (e.g., an end user, consumer, driver, vehicle operator, etc.). Examples of remote computer systems include personal computers (e.g., portable PCs), tablet PCs (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, smartphones (e.g., Apple® iPhone, Android-enabled devices, Blackberry®), or personal digital assistants. Users can access computer system 301 via network 330.

[0055] The methods described herein can be implemented in the form of machine-executable code (e.g., a computer processor) stored in an electronic storage location (such as, for example, memory 310 or electronic storage unit 315) of computer system 301. The machine-executable code, or machine-readable code, can be provided in software form. During use, the code can be executed by processor 305. In some cases, the code can be retrieved from storage unit 315 and stored in memory 310 for rapid access by processor 305. In some cases, electronic storage unit 315 can be excluded, and machine-executable instructions can be stored in memory 310. The code can be pre-compiled and configured for use with a machine having a processor suitable for executing the code, or it can be compiled during runtime. The code can be provided in the form of a programming language, which can be selected to enable the code to be executed in a pre-compiled or as-compiled manner.

[0056] Several aspects of the systems and methods provided herein (such as computer system 301) can be embodied in a programmable manner. These aspects of the technology can be considered “products” or “articles of manufacture,” typically in the form of machine (or processor) executable code or associated data carried or embodied in a machine-readable medium. The machine-executable code can be stored on electronic storage units such as memory (e.g., read-only memory, random access memory, flash memory) or hard disks. “Storage” media can include any or all tangible memory of computers, processors, etc., or related modules thereof, such as various semiconductor memories, tape drives, disk drives, etc., which can provide non-transitory storage for software programming at any time. All or part of the software can sometimes be communicated via the Internet or various other telecommunications networks. For example, such communication can enable the loading of software from one computer or processor to another, such as from a management server or host computer platform to an application server. Therefore, another type of medium that can carry software elements includes physical interfaces between local devices, light waves, radio waves, and electromagnetic waves used over various air links, such as wired and fiber optic terrestrial networks. Physical elements carrying such waves, such as wired or wireless links, optical links, etc., can also be considered as media carrying software. As used herein, unless limited to non-transitory, tangible "storage" media, the term "readable medium" for a computer or machine refers to any medium involved in providing instructions to a processor for execution.

[0057] Therefore, machine-readable media (such as computer-executable code) can take many forms, including but not limited to tangible storage media, carrier media, or physical transmission media. Non-volatile storage media include, for example, optical discs or disks (such as any storage device in any computer, etc.), such as those used to implement databases as shown in the accompanying figures. Volatile storage media include dynamic memory, such as the main memory of a computer platform. Tangible transmission media include coaxial cables; copper wires and optical fibers, including conductors that form buses within a computer system. Carrier transmission media can take the form of electrical or electromagnetic signals, or sound or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Therefore, common forms of computer-readable media include, for example: floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, any other optical media, punched card tapes, any other physical storage media with punched patterns, RAM, ROM, PROM and EPROM, FLASH-EPROM, any other memory chips or cartridges, carrier waves for transporting data or instructions, cables or links for transporting such carrier waves, or any other media from which a computer can read programming code or data. Many of these forms of computer-readable media may involve carrying one or more sequences of one or more instructions to a processor for execution.

[0058] Computer system 301 may include or communicate with electronic display 335, which includes a user interface (UI) 340 for providing, for example, a portal for monitoring one or more objects, obstacles, or targets detected by a radar system. In some cases, the portal may be used to present, view, monitor, or manipulate one or more occupancy grids generated by a processor or multiple radar modules. The portal may be provided via an application programming interface (API). Users or entities may also interact with various elements in the portal via the UI. Examples of UIs include, but are not limited to, graphical user interfaces (GUIs) and web-based user interfaces.

[0059] The methods and systems disclosed herein can be implemented using one or more algorithms. The algorithms can be implemented in software after execution by the central processing unit 305. For example, the algorithm can be configured to execute the process 400 described below.

[0060] Example process Figure 4 A flowchart depicts an example process 400 that can be implemented by embodiments of the present disclosure. Example process 400 can be implemented by components of the described radar signal processing system (such as those described above). Figure 1and Figure 2 This is achieved using the components described in [the document]. Example procedure 400 shows in considerable detail how to process input radar signals using a machine learning computational model to determine a 3D scene representation of the environment.

[0061] For clarity, the following description is generally in... Figures 1 to 3 Example process 400 is described in the context of this document. However, it is to be understood that process 400 may be executed, for example, by any other suitable system, environment, software, and hardware, or by a combination of systems, environments, software, and hardware, as appropriate. In some embodiments, the various operations of process 400 may be performed in parallel, in combination, cyclically, or in any order.

[0062] At 402, an input radar signal is received from a radar module associated with a vehicle located in the environment. The radar signal includes a digitized radar sample, which includes a first indication of an object in the environment. From 402, process 400 proceeds to 404.

[0063] At point 404, the input radar signal is processed by one or more signal processing algorithms to determine the original radar spectrum. From 404, process 400 proceeds to 406.

[0064] At 406, the raw radar spectrum is processed by a machine learning computational model to determine a set of output predictions about the environment. In some embodiments, the input to the machine learning computational model includes the most recent N frames, where N can be 1, 2, 3, 4, 5, 6, 7, 8, or another integer number of frames. From 406, process 400 proceeds to 408.

[0065] At 408, scene representation and scene understanding are determined at least in part based on a set of output predictions about the environment. In some embodiments, scene understanding includes target object categories (vehicles, pedestrians, bicycles, trucks, motorcycles, background, etc.), object bounding boxes, object ground velocities (e.g., two-dimensional (2D) or three-dimensional (3D)), object orientation, object direction of travel, and its own velocity (e.g., 2D or 3D). From 408, process 400 proceeds to 410.

[0066] At point 410, scene representation and scene understanding are provided to the autonomous driving system. From point 410, process 400 ends.

[0067] Non-transitory computer-readable storage medium In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more non-transitory computer-readable storage media encoded with a program, the program including instructions executable by the operating system of an optionally networked computer. In other embodiments, the computer-readable storage medium is a tangible component of a computer. In still other embodiments, the computer-readable storage medium is optionally removable from the computer. In some embodiments, as a non-limiting example, the computer-readable storage medium includes CD-ROMs, DVDs, flash memory devices, solid-state storage, disk drives, tape drives, optical disc drives, cloud computing systems and services, etc. In some cases, the program and instructions are encoded on the medium permanently, substantially permanently, semi-permanently, or non-transitory.

[0068] Computer program In some embodiments, the platforms, systems, media, and methods disclosed herein include at least one computer program or its use. A computer program includes a sequence of instructions executable in the CPU of a computer, written to perform a specified task. Computer-readable instructions can be implemented as program modules that perform a specific task or implement a specific abstract data type, such as functions, objects, APIs, data structures, etc. Based on the disclosure provided herein, those skilled in the art will recognize that computer programs can be written in various versions of various languages.

[0069] The functionality of computer-readable instructions can be combined or assigned as needed in various environments. In some embodiments, a computer program includes a sequence of instructions. In some embodiments, a computer program includes multiple sequences of instructions. In some embodiments, a computer program is provided from one location. In other embodiments, a computer program is provided from multiple locations. In many embodiments, a computer program includes one or more software modules. In many embodiments, a computer program includes, in part or in whole, one or more web applications, one or more mobile applications, one or more standalone applications, one or more web browser plugins, extensions, add-ons, or accessories, or combinations thereof.

[0070] Machine Learning As described above, this paper employs machine learning algorithms to build a model to determine a set of output predictions about the environment. Examples of machine learning algorithms can include Support Vector Machines (SVM), Naive Bayes classification, Random Forest, Neural Networks, Deep Learning, or other supervised or unsupervised learning algorithms for classification and regression. Machine learning algorithms can be trained using one or more training datasets. For example, previously received contextual data can be used to train various algorithms. Furthermore, as described above, these algorithms can be continuously trained / retrained using received real-time user data. In some embodiments, the machine learning algorithms employ regression modeling, where the relationships between variables are determined and weighted. In some embodiments, the machine learning algorithms employ regression modeling, where the relationships between predictor variables and dependent variables are determined and weighted.

[0071] standalone application In some embodiments, a computer program includes a standalone application, which is a program that runs as an independent computer process, rather than an add-on to an existing process (e.g., not a plugin). Those skilled in the art will recognize that standalone applications are often compiled. A compiler is a computer program that translates source code written in a programming language into binary object code (e.g., assembly language or machine code). By way of non-limiting example, suitable programming languages ​​for compilation include C, C++, Objective-C, COBOL, Delphi, Eiffel, Java™, Lisp, Python™, Visual Basic, and VB .NET, or combinations thereof. Compilation is typically performed at least partially to create an executable program. In some embodiments, a computer program includes one or more executable compiled applications.

[0072] Software Module In some embodiments, the platforms, systems, media, and methods disclosed herein include software, server, or database modules, or uses thereof. In view of the disclosure provided herein, software modules are created using techniques known to those skilled in the art, utilizing machines, software, and languages ​​known in the art. The software modules disclosed herein are implemented in a variety of ways. In various embodiments, a software module includes files, code segments, programming objects, programming structures, or combinations thereof. In other embodiments, a software module includes multiple files, multiple code segments, multiple programming objects, multiple programming structures, or combinations thereof. In various embodiments, as non-limiting examples, one or more software modules include web applications, mobile applications, and standalone applications. In some embodiments, a software module is within a computer program or application. In other embodiments, a software module is within more than one computer program or application. In some embodiments, a software module is hosted on a single machine. In other embodiments, a software module is hosted on more than one machine. In other embodiments, a software module is hosted on a cloud computing platform. In some embodiments, a software module is hosted on one or more machines in one location. In other embodiments, a software module is hosted on one or more machines in more than one location.

[0073] While preferred embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. This disclosure is not intended to be limited to the specific examples provided herein. Although this disclosure has been described with reference to the foregoing description, the description and illustration of the embodiments herein are not intended to be construed in a limiting sense. Many variations, alterations, and substitutions will now occur to those skilled in the art without departing from this disclosure. Furthermore, it should be understood that all aspects of this disclosure are not limited to the specific descriptions, configurations, or relative proportions set forth herein, which depend on a variety of conditions and variables. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. Therefore, it is contemplated that this disclosure should also cover any such alternatives, modifications, variations, or equivalents. The appended claims are intended to define the scope of this disclosure and thus cover the methods and structures and their equivalents within the scope of these claims.

[0074] Example The following paragraphs provide various examples of the embodiments disclosed herein.

[0075] Example 1 is a computer-implemented method for processing radar signals. The method is executed by an electronic processor. The method includes: receiving input radar signals, including indications of objects in the environment, from a radar module associated with a vehicle located in the environment; processing the input radar signals by one or more signal processing algorithms to determine an original radar spectrum; processing the original radar spectrum by a machine learning computational model to determine a set of output predictions about the environment; determining a representative model of the environment, at least in part, based on the set of output predictions about the environment; and providing the representative model of the environment to an autonomous driving system.

[0076] Example 2 includes the subject of Example 1 and also specifies that a representative model of the environment includes a scene representation that is generated at least in part using signal processing techniques such as detection, parameter estimation, and model fitting.

[0077] Example 3 includes the subject of Example 1 or 2, and also specifies that a representative model of the environment includes a scene representation that is at least partially generated by a machine learning computational model.

[0078] Example 4 includes the subject of any one of Examples 1 to 3, and also specifies that a representative model of the environment includes a scene representation determined by combining a first scene representation generated using signal processing techniques and a second scene representation generated by a machine learning computational model.

[0079] Example 5 includes the subject of any one of Examples 1 through 4, and also specifies that a representative model of the environment includes a scene representation that includes a parametric model showing spatial relationships.

[0080] Example 6 includes the subject of any one of Examples 1 through 5 and also specifies that a representative model of the environment includes a scene representation that includes one or more of the following: 2D point cloud, 3D point cloud, point cloud with additional metadata such as radial velocity or time, list of object bounding boxes, occupied grid, free space map, drivable area map, range-Doppler spectrum or beam spectrum.

[0081] Example 7 includes the subject of any one of Examples 1 through 6, and further specifies that the method includes determining the context of the current scene in the environment and the properties of objects in the environment by processing the raw radar spectrum by a machine learning computational model, and further specifies that the representative model of the environment includes scene understanding, which has representations of the properties of objects given the context of the current scene.

[0082] Example 8 includes the subject of any one of Examples 1 through 7, and also specifies that a representative model of the environment includes scene understanding, which includes one or more of the following attributes: target object category, object bounding box, object ground velocity, object orientation, object travel direction, or its own velocity.

[0083] Example 9 includes the subject matter of any of Examples 1 through 8, and also specifies that the input radar signal is processed by range and Doppler processing.

[0084] Example 10 includes the subject of any of Examples 1 through 9, and also specifies the use of distance-Doppler spectrum as input to train a machine learning computational model.

[0085] Example 11 includes the subject matter of any of Examples 1 through 10, and also specifies processing of the input radar signal by range and Doppler processing as well as beamforming.

[0086] Example 12 includes the subject of any of Examples 1 through 11, and also specifies that the input to the machine learning computation model includes the beam spectrum.

[0087] Example 13 includes the subject of any one of Examples 1 through 12, and further specifies that the input to the machine learning computation model includes a point cloud, a Cartesian projection of the point cloud, an occupied raster, or a previously determined representative model of the environment.

[0088] Example 14 includes the subject matter of any one of Examples 1 to 13, and further specifies that the radar module is configured to: transmit a first set of signals comprising a plurality of output radar pulses, and receive input radar signals.

[0089] Example 15 includes the subject matter of any one of Examples 1 to 14, and further specifies that the input radar signal is a subset of the first set of signals and includes multiple input radar pulses.

[0090] Example 16 includes the subject of any one of Examples 1 to 15, and further specifies that an input radar signal is generated when a subset of the first set of signals interacts with or is reflected from at least one object in the environment.

[0091] Example 17 includes the subject of any one of Examples 1 through 16, and also specifies that the machine learning computation model is configured to determine the object category label of objects in the environment.

[0092] Example 18 includes themes from any of Examples 1 through 17, and also specifies that object category labels are integrated into the scene representation to provide context about objects and the environment.

[0093] Example 19 includes the subject of any one of Examples 1 to 18, and further specifies that the indication of an object in the environment is a first indication; and further specifies that the method includes receiving image data including a second indication of an object in the environment from a plurality of imaging devices, and processing the raw radar spectrum and image data through a machine learning computational model to produce a set of output predictions about the environment.

[0094] Example 20 includes the subject of any one of Examples 1 to 19, and further specifies that the input to the machine learning computation model includes time-historical data received from at least one of the imaging devices or radar modules.

[0095] Example 21 includes the subject of any one of Examples 1 to 20, and further specifies that the indication of an object in the environment is a first indication; and further specifies that the method includes receiving LIDAR data from a LIDAR sensor that includes a second indication of an object in the environment, and processing the raw radar spectrum and LIDAR data through a machine learning computational model to produce a set of output predictions about the environment.

[0096] Example 22 includes the subject of any one of Examples 1 through 21, and also specifies that the original radar spectrum includes a sequence of radar frames.

[0097] Example 23 includes the subject of any of Examples 1 through 22, and further specifies that the machine learning computation model outputs a set of output predictions based on the radar frame sequence.

[0098] Example 24 includes the subject of any of Examples 1 through 23, and further specifies that the radar frame sequence includes either a continuous time series or a non-continuous time series.

[0099] Example 25 includes the subject of any of Examples 1 through 24, and also specifies that the input to the machine learning computation model includes the most recent N frames of the radar frame sequence.

[0100] Example 26 includes the subject matter of any one of Examples 1 to 25, and further specifies that the method includes: receiving multiple input radar signals, including indications of objects in the environment, from multiple radar modules associated with a vehicle located in the environment, and processing the multiple input radar signals by one or more signal processing algorithms to determine an original radar spectrum.

[0101] Example 27 includes the subject matter of any one of Examples 1 to 26, and further specifies that the radar module includes at least one front corner-facing radar and a forward-facing radar, or at least one rear corner-facing radar and a rear-facing radar.

[0102] Example 28 includes the subject matter of any one of Examples 1 through 27, and further specifies that the radar module includes at least two forward-facing radars or at least two backward-facing radars.

[0103] Example 29 includes the topic of any of Examples 1 through 28, and also specifies the use of a supervised method to train a machine learning computational model using baseline fact labels.

[0104] Example 30 includes the subject of any of Examples 1 through 29, and also specifies training a machine learning computational model using unsupervised or self-supervised methods by learning how to reconstruct the original radar signal from multiple radar modules or from multiple time data points.

[0105] Example 31 includes the subject matter of any one of Examples 1 through 30, and further specifies that the autonomous driving system includes an emergency braking system, ADAS, or active safety system.

[0106] Example 32 includes the subject of any one of Examples 1 through 31, and further specifies that the machine learning computation model includes a neural network model, a CNN model, an LSTM model, or a ViT model.

[0107] Example 33 includes the subject of any one of Examples 1 through 32, and also specifies the vehicle moving through the environment.

[0108] Example 34 includes the subject of any one of Examples 1 through 33, and further specifies that the means of transport includes automated driving vehicles or autonomous vehicles.

[0109] Example 35 includes the subject of any one of Examples 1 through 34, and further specifies that the autonomous driving system is associated with or integrated into a vehicle.

[0110] Example 36 includes the subject matter of any one of Examples 1 through 35, and further specifies that the radar module is installed in a vehicle and includes a radar transmitter and a radar receiver.

[0111] Example 37 is a non-transitory computer-readable medium comprising instructions executable by an electronic processor to perform a set of functions. These functions include: receiving an input radar signal from a radar module containing a first indication of an object in the environment; receiving image data containing a second indication of an object in the environment from multiple imaging devices; processing the image data and raw radar spectrum via a machine learning computational model to determine a set of output predictions about the environment, wherein the raw radar spectrum is determined based on the input radar signal; determining a representative model of the environment based at least in part on the set of output predictions about the environment; and providing the representative model of the environment to an autonomous driving system.

[0112] Example 38 includes the subject of Example 1 and also specifies that a representative model of the environment includes a scene representation that is generated at least in part using signal processing techniques such as detection, parameter estimation, and model fitting.

[0113] Example 39 includes the subject of Example 1 or 38, and also specifies that a representative model of the environment includes a scene representation that is at least partially generated by a machine learning computational model.

[0114] Example 40 includes the subject of any one of Examples 37 to 39, and also specifies that a representative model of the environment includes a scene representation, which is determined by combining a first scene representation generated using signal processing techniques and a second scene representation generated by a machine learning computational model.

[0115] Example 41 includes the subject of any one of Examples 37 to 40, and also specifies that a representative model of the environment includes a scene representation, which includes a parametric model showing spatial relationships.

[0116] Example 42 includes the subject of any one of Examples 37 to 41 and also specifies that a representative model of the environment includes a scene representation that includes one or more of the following: 2D point cloud, 3D point cloud, point cloud with additional metadata such as radial velocity or time, list of object bounding boxes, occupied grid, free space map, drivable area map, range-Doppler spectrum or beam spectrum.

[0117] Example 43 includes the subject of any of Examples 37 through 42, and further specifies a set of features including determining the context of the current scene in the environment and the properties of objects in the environment by processing the raw radar spectrum by a machine learning computational model, and further specifies that the representative model of the environment includes scene understanding, which includes representations of object properties of objects given the context of the current corresponding scene.

[0118] Example 44 includes the subject of any one of Examples 37 through 42, and also specifies that a representative model of the environment includes scene understanding, which includes one or more of the following attributes: target object category, object bounding box, object ground velocity, object orientation, object travel direction, or its own velocity.

[0119] Example 45 includes the subject matter of any of Examples 37 through 43, and also specifies that the input radar signal is processed by range and Doppler processing.

[0120] Example 46 includes the subject of any of Examples 37 through 44, and also specifies the use of distance-Doppler spectrum as input to train a machine learning computational model.

[0121] Example 47 includes the subject matter of any of Examples 37 to 45, and also specifies processing of the input radar signal by range and Doppler processing as well as beamforming.

[0122] Example 48 includes the subject of any of Examples 37 through 46, and also specifies that the input to the machine learning computational model includes the beam spectrum.

[0123] Example 49 includes the subject of any one of Examples 37 to 47, and further specifies that the input to the machine learning computational model includes a point cloud, a Cartesian projection of the point cloud, an occupied raster, or a previously determined representative model of a scene representation of the environment.

[0124] Example 50 includes the subject matter of any one of Examples 37 to 49, and further specifies that the radar module is configured to: transmit a first set of signals comprising a plurality of output radar pulses, and receive input radar signals.

[0125] Example 51 includes the subject matter of any of Examples 37 to 50, and further specifies that the input radar signal is a subset of the first set of signals and includes multiple input radar pulses.

[0126] Example 52 includes the subject of any one of Examples 37 to 51, and further specifies that an input radar signal is generated when a subset of the first set of signals interacts with or is reflected from at least one object in the environment.

[0127] Example 53 includes the subject of any of Examples 37 through 52, and also specifies that the machine learning computation model is configured to determine the object category label of an object in the environment.

[0128] Example 54 includes the subject of any of Examples 37 through 53, and also specifies that object category labels are integrated into the scene representation to provide context about objects and the environment.

[0129] Example 55 includes the subject of any one of Examples 37 to 54, and further specifies that the input to the machine learning computation model includes time-historical data received from at least one of the imaging devices or radar modules.

[0130] Example 56 includes the subject of any one of Examples 37 to 55, and further specifies that the indication of an object in the environment is a first indication; and further specifies a set of functions including receiving LIDAR data from a LIDAR sensor containing a second indication of an object in the environment, and processing the raw radar spectrum and LIDAR data through a machine learning computational model to produce a set of output predictions about the environment.

[0131] Example 57 includes the subject of any of Examples 37 through 56, and also specifies that the original radar spectrum includes a sequence of radar frames.

[0132] Example 58 includes the subject of any of Examples 37 to 57, and further specifies that the machine learning computation model outputs a set of output predictions based on the radar frame sequence.

[0133] Example 59 includes the subject of any of Examples 37 to 58, and further specifies that the radar frame sequence includes either a continuous time series or a non-continuous time series.

[0134] Example 60 includes the subject of any of Examples 37 through 59, and also specifies that the input to the machine learning computation model includes the most recent N frames of the radar frame sequence.

[0135] Example 61 includes the subject matter of any one of Examples 37 to 60, and further specifies a set of functions including: receiving multiple input radar signals from multiple radar modules associated with a vehicle in the environment, including a first indication of an object in the environment, and processing the multiple input radar signals by one or more signal processing algorithms to determine the original radar spectrum.

[0136] Example 62 includes the subject matter of any one of Examples 37 to 61, and further specifies that the radar module includes at least one front-angle radar and a forward-facing radar, or at least one rear-angle radar and a rear-facing radar.

[0137] Example 63 includes the subject matter of any one of Examples 37 to 62, and further specifies that the radar module includes at least two forward-facing radars or at least two backward-facing radars.

[0138] Example 64 includes the topic of any of Examples 37 through 63, and also specifies the use of a supervised method to train a machine learning computational model using baseline fact labels.

[0139] Example 65 includes the subject of any of Examples 37 through 64, and also specifies training a machine learning computational model using unsupervised or self-supervised methods by learning how to reconstruct the original radar signal from multiple radar modules or from multiple time data points.

[0140] Example 66 includes the subject matter of any one of Examples 37 through 65, and further specifies that the autonomous driving system includes an emergency braking system, ADAS, or active safety system.

[0141] Example 67 includes the subject of any of Examples 37 through 66, and further specifies that the machine learning computation model includes a neural network model, a CNN model, an LSTM model, or a ViT model.

[0142] Example 68 includes the subject of any of Examples 37 to 67, and also specifies the vehicle moving through the environment.

[0143] Example 69 includes the subject of any one of Examples 37 to 68, and further specifies that the means of transport includes automated driving vehicles or autonomous vehicles.

[0144] Example 70 includes the subject of any of Examples 37 through 69, and further specifies that the autonomous driving system is associated with or integrated into a vehicle.

[0145] Example 71 includes the subject matter of any of Examples 37 through 70, and further specifies that the radar module is installed in a vehicle and includes a radar transmitter and a radar receiver.

[0146] Example 72 is a non-transitory computer-readable medium comprising instructions executable by an electronic processor to perform a set of functions. The set of functions includes: receiving an input radar signal from a radar module, representing a first indication of an object in the environment; receiving LiDAR data from a LiDAR sensor, representing a second indication of an object in the environment; processing the LiDAR data and the raw radar spectrum through a machine learning computational model to determine a set of output predictions about the environment, wherein the raw radar spectrum is determined based on the input radar signal; determining a representative model of the environment based at least in part on the set of output predictions about the environment; and providing the representative model of the environment to an autonomous driving system.

[0147] Example 73 includes the subject of Example 72 and also specifies that a representative model of the environment includes a scene representation that is generated at least in part using signal processing techniques such as detection, parameter estimation, and model fitting.

[0148] Example 74 includes the subject of Example 72 or 73, and also specifies that a representative model of the environment includes a scene representation that is at least partially generated by a machine learning computational model.

[0149] Example 75 includes the subject of any one of Examples 72 to 74, and also specifies that a representative model of the environment includes a scene representation, which is determined by combining a first scene representation generated using signal processing techniques and a second scene representation generated by a machine learning computational model.

[0150] Example 76 includes the subject of any one of Examples 72 through 75, and also specifies that a representative model of the environment includes a scene representation that includes a parametric model showing spatial relationships.

[0151] Example 77 includes the subject of any one of Examples 72 through 76 and also specifies that a representative model of the environment includes a scene representation that includes one or more of the following: 2D point cloud, 3D point cloud, point cloud with additional metadata such as radial velocity or time, list of object bounding boxes, occupied grid, free space map, drivable area map, range-Doppler spectrum or beam spectrum.

[0152] Example 78 includes the subject of any of Examples 72 to 77, and further specifies a set of functions including determining the context of the current scene in the environment and the properties of objects in the environment by processing the raw radar spectrum by a machine learning computational model, and further specifies that the representative model of the environment includes scene understanding, which includes representations of object properties of objects given the context of the current corresponding scene.

[0153] Example 79 includes the subject of any one of Examples 72 through 78, and also specifies that a representative model of the environment includes scene understanding, which includes one or more of the following attributes: target object category, object bounding box, object ground velocity, object orientation, object travel direction, or its own velocity.

[0154] Example 80 includes the subject matter of any of Examples 72 through 79, and also specifies that the input radar signal is processed by range and Doppler processing.

[0155] Example 81 includes the subject of any of Examples 72 through 80, and also specifies the use of distance-Doppler spectrum as input to train a machine learning computational model.

[0156] Example 82 includes the subject matter of any of Examples 72 to 81, and also specifies processing of the input radar signal by range and Doppler processing as well as beamforming.

[0157] Example 83 includes the subject of any of Examples 72 through 82, and also specifies that the input to the machine learning computation model includes the beam spectrum.

[0158] Example 84 includes the subject of any of Examples 72 to 83, and further specifies that the input to the machine learning computational model includes a point cloud, a Cartesian projection of the point cloud, an occupied raster, or a previously determined representative model of a scene representation of the environment.

[0159] Example 85 includes the subject matter of any one of Examples 72 to 84, and further specifies that the radar module is configured to: transmit a first set of signals comprising a plurality of output radar pulses, and receive input radar signals.

[0160] Example 86 includes the subject matter of any of Examples 72 through 85, and further specifies that the input radar signal is a subset of the first set of signals and includes multiple input radar pulses.

[0161] Example 87 includes the subject of any one of Examples 72 to 86, and further specifies that an input radar signal is generated when a subset of the first set of signals interacts with or is reflected from at least one object in the environment.

[0162] Example 88 includes the subject of any of Examples 72 through 87, and also specifies that the machine learning computation model is configured to determine the object category label of objects in the environment.

[0163] Example 89 includes the subject of any of Examples 72 through 88, and also specifies that object category labels are integrated into the scene representation to provide context about objects and the environment.

[0164] Example 90 includes the subject of any one of Examples 72 to 89, and further specifies that the indication of an object in the environment is a first indication; and further specifies a set of functions including receiving image data of a second indication of an object in the environment from multiple imaging devices, and processing the raw radar spectrum and image data through a machine learning computational model to produce a set of output predictions about the environment.

[0165] Example 91 includes the subject of any one of Examples 72 to 90, and further specifies that the input to the machine learning computation model includes time-historical data received from at least one of the imaging devices or radar modules.

[0166] Example 92 includes the subject of any of Examples 72 through 91, and also specifies that the original radar spectrum includes a sequence of radar frames.

[0167] Example 93 includes the subject of any of Examples 72 to 92, and further specifies that the machine learning computation model outputs a set of output predictions based on the radar frame sequence.

[0168] Example 94 includes the subject of any of Examples 72 to 93, and further specifies that the radar frame sequence includes either a continuous time series or a non-continuous time series.

[0169] Example 95 includes the subject of any of Examples 72 through 94, and also specifies that the input to the machine learning computation model includes the most recent N frames of the radar frame sequence.

[0170] Example 96 includes the subject matter of any one of Examples 72 to 95, and further specifies a set of functions including: receiving multiple input radar signals from multiple radar modules associated with a vehicle in the environment, including a first indication of an object in the environment, and processing the multiple input radar signals by one or more signal processing algorithms to determine the original radar spectrum.

[0171] Example 97 includes the subject matter of any one of Examples 72 to 96, and further specifies that the radar module includes at least one front-angle radar and a forward-facing radar, or at least one rear-angle radar and a rear-facing radar.

[0172] Example 98 includes the subject matter of any one of Examples 72 to 97, and further specifies that the radar module includes at least two forward-facing radars or at least two backward-facing radars.

[0173] Example 99 includes the topic of any of Examples 72 through 98, and also specifies the use of a supervised method to train a machine learning computational model using baseline fact labels.

[0174] Example 100 includes the subject of any of Examples 72 through 99, and also specifies training a machine learning computational model using unsupervised or self-supervised methods by learning how to reconstruct the original radar signal from multiple radar modules or from multiple time data points.

[0175] Example 101 includes the subject matter of any one of Examples 72 through 100, and further specifies that the autonomous driving system includes an emergency braking system, ADAS, or active safety system.

[0176] Example 102 includes the subject of any of Examples 72 through 101, and further specifies that the machine learning computation model includes a neural network model, a CNN model, an LSTM model, or a ViT model.

[0177] Example 103 includes the subject of any of Examples 72 through 102, and also specifies the vehicle moving through the environment.

[0178] Example 104 includes the subject of any one of Examples 72 to 103, and further specifies that the means of transport includes automated driving vehicles or autonomous vehicles.

[0179] Example 105 includes the subject of any of Examples 72 through 104, and further specifies that an autonomous driving system is associated with or integrated into a vehicle.

[0180] Example 106 includes the subject matter of any of Examples 72 through 105, and further specifies that the radar module is installed in a vehicle and includes a radar transmitter and a radar receiver.

[0181] Example 107 is a radar signal processing system comprising a vehicle located in an environment, a radar module associated with the vehicle, and an electronic processor. The electronic processor is configured to receive input radar signals from the radar module, including indications of objects in the environment; process the input radar signals using one or more signal processing algorithms to determine an original radar spectrum; process the original radar spectrum using a machine learning computational model to determine a set of output predictions about the environment; determine a representative model of the environment based at least in part on the set of output predictions about the environment; and provide the representative model of the environment to an autonomous driving system.

[0182] Example 108 includes the subject of Example 107 and also specifies that a representative model of the environment includes a scene representation that is generated at least in part using signal processing techniques such as detection, parameter estimation, and model fitting.

[0183] Example 109 includes the topics of Example 107 or 108, and also specifies that a representative model of the environment includes a scene representation that is at least partially generated by a machine learning computational model.

[0184] Example 110 includes the subject of any one of Examples 107 to 109, and also specifies that a representative model of the environment includes a scene representation determined by combining a first scene representation generated using signal processing techniques and a second scene representation generated by a machine learning computational model.

[0185] Example 111 includes the subject of any one of Examples 107 to 110, and also specifies that a representative model of the environment includes a scene representation that includes a parametric model showing spatial relationships.

[0186] Example 112 includes the subject of any one of Examples 107 to 111 and also specifies that a representative model of the environment includes a scene representation, which includes one or more of the following: 2D point cloud, 3D point cloud, point cloud with additional metadata such as radial velocity or time, list of object bounding boxes, occupied grid, free space map, drivable area map, range-Doppler spectrum or beam spectrum.

[0187] Example 113 includes the subject of any one of Examples 107 to 112, and further specifies that the method includes determining the context of the current scene in the environment and the properties of objects in the environment by processing the raw radar spectrum by a machine learning computational model, and further specifies that the representative model of the environment includes scene understanding, which includes representations of object properties of objects given the context of the current corresponding scene.

[0188] Example 114 includes the subject of any one of Examples 107 through 113, and also specifies that a representative model of the environment includes scene understanding, which includes one or more of the following attributes: target object category, object bounding box, object ground velocity, object orientation, object travel direction, or its own velocity.

[0189] Example 115 includes the subject matter of any of Examples 107 to 114, and also specifies that the input radar signal is processed by range and Doppler processing.

[0190] Example 116 includes the subject of any of Examples 107 through 115, and also specifies the use of distance-Doppler spectrum as input to train a machine learning computational model.

[0191] Example 117 includes the subject matter of any of Examples 107 to 116, and also specifies processing of the input radar signal by range and Doppler processing as well as beamforming.

[0192] Example 118 includes the subject of any of Examples 107 to 117, and also specifies that the input to the machine learning computation model includes the beam spectrum.

[0193] Example 119 includes the subject of any one of Examples 107 to 118, and further specifies that the input to the machine learning computational model includes a point cloud, a Cartesian projection of the point cloud, an occupied raster, or a previously determined representative model of a scene representation of the environment.

[0194] Example 120 includes the subject matter of any one of Examples 107 to 119, and further specifies that the radar module is configured to: transmit a first set of signals including a plurality of output radar pulses, and receive input radar signals.

[0195] Example 121 includes the subject matter of any one of Examples 107 to 120, and further specifies that the input radar signal is a subset of the first set of signals and includes multiple input radar pulses.

[0196] Example 122 includes the subject of any one of Examples 107 to 121, and further specifies that an input radar signal is generated when a subset of the first set of signals interacts with or is reflected from at least one object in the environment.

[0197] Example 123 includes the subject of any one of Examples 107 through 122, and also specifies that the machine learning computation model is configured to determine the object category label of an object in the environment.

[0198] Example 124 includes the subject of any of Examples 107 through 123, and also specifies that object category labels are integrated into the scene representation to provide context about objects and the environment.

[0199] Example 125 includes the subject of any one of Examples 107 to 124, and further specifies that the indication of an object in the environment is a first indication; and further specifies that the method includes receiving image data including a second indication of an object in the environment from multiple imaging devices, and processing the raw radar spectrum and image data through a machine learning computational model to produce a set of output predictions about the environment.

[0200] Example 126 includes the subject of any one of Examples 107 to 125, and further specifies that the input to the machine learning computational model includes time-historical data received from at least one of the imaging devices or radar modules.

[0201] Example 127 includes the subject of any one of Examples 107 to 126, and further specifies that the indication of an object in the environment is a first indication; and further specifies that the method includes receiving LIDAR data from a LIDAR sensor that includes a second indication of an object in the environment, and processing the raw radar spectrum and LIDAR data through a machine learning computational model to produce a set of output predictions about the environment.

[0202] Example 128 includes the subject of any of Examples 107 to 127, and also specifies that the original radar spectrum includes a sequence of radar frames.

[0203] Example 129 includes the subject of any of Examples 107 to 128, and further specifies that the machine learning computation model outputs a set of output predictions based on the radar frame sequence.

[0204] Example 130 includes the subject of any of Examples 107 to 129, and further specifies that the radar frame sequence includes either a continuous time series or a non-continuous time series.

[0205] Example 131 includes the subject of any of Examples 107 to 130, and also specifies that the input to the machine learning computation model includes the most recent N frames of the radar frame sequence.

[0206] Example 132 includes the subject matter of any one of Examples 107 to 131, and further specifies a method comprising: receiving from a plurality of radar modules associated with a vehicle located in the environment a plurality of input radar signals including a first indication of an object in the environment, and processing the plurality of input radar signals by one or more signal processing algorithms to determine an original radar spectrum.

[0207] Example 133 includes the subject matter of any one of Examples 107 to 132, and further specifies that the radar module includes at least one forward-facing radar and a forward-facing radar, or at least one rear-facing radar and a rear-facing radar.

[0208] Example 134 includes the subject matter of any one of Examples 107 to 133, and further specifies that the radar module includes at least two forward-facing radars or at least two backward-facing radars.

[0209] Example 135 includes the topic of any of Examples 107 through 134, and also specifies the use of a supervised method to train a machine learning computational model using baseline fact labels.

[0210] Example 136 includes the subject of any of Examples 107 through 135, and also specifies training a machine learning computational model using unsupervised or self-supervised methods by learning how to reconstruct the original radar signal from multiple radar modules or from multiple time data points.

[0211] Example 137 includes the subject matter of any one of Examples 107 to 136, and further specifies that the autonomous driving system includes an emergency braking system, ADAS, or active safety system.

[0212] Example 138 includes the subject of any of Examples 107 to 137, and further specifies that the machine learning computation model includes a neural network model, a CNN model, an LSTM model, or a ViT model.

[0213] Example 139 includes the subject of any of Examples 107 to 138, and also specifies that the vehicle is moving through the environment.

[0214] Example 140 includes the subject of any one of Examples 107 to 139, and further specifies that the means of transport includes automated driving vehicles or autonomous vehicles.

[0215] Example 141 includes the subject of any one of Examples 107 to 140, and further specifies that an autonomous driving system is associated with or integrated into a vehicle.

[0216] Example 142 includes the subject matter of any one of Examples 107 to 141, and further specifies that the radar module is installed in a vehicle and includes a radar transmitter and a radar receiver.

Claims

1. A computer-implemented method for processing radar signals, the method being executed by an electronic processor and comprising: Receive input radar signals, including indications of objects in the environment, from a radar module associated with a vehicle located in the environment; The input radar signal is processed by one or more signal processing algorithms to determine the original radar spectrum; The raw radar spectrum is processed by a machine learning computational model to determine a set of output predictions about the environment; A representative model of the environment is determined, at least in part, based on the set of output predictions about the environment; as well as A representative model of the environment is provided to the autonomous driving system.

2. The method according to claim 1, wherein, The representative model of the environment includes a scene representation generated at least in part using signal processing techniques such as detection, parameter estimation, and model fitting.

3. The method according to claim 1, wherein, The representative model of the environment includes a scene representation, which is generated at least in part by the machine learning computation model.

4. The method according to claim 1, wherein, The representative model of the environment is determined by combining a first scene representation generated using signal processing techniques and a second scene representation generated by the machine learning computation model.

5. The method according to claim 1, wherein, The representative model of the environment includes a parametric model that shows spatial relationships.

6. The method according to claim 1, wherein, The representative model of the environment includes one or more of the following: a two-dimensional (2D) point cloud, a three-dimensional (3D) point cloud, a point cloud with additional metadata, a list of object bounding boxes, an occupied grid, a free space map, a drivable area map, a range-Doppler spectrum or a beam spectrum, and the additional metadata such as radial velocity or time.

7. The method according to claim 1, further comprising: The raw radar spectrum is processed by the machine learning computation model to determine the context of the current scene in the environment and the attributes of the objects in the environment. The representative model of the environment includes scene understanding, which has representations of the properties of the objects given the context of the current scene.

8. The method according to claim 1, wherein, The representative model of the environment includes one or more of the following attributes: target object category, object bounding box, object ground velocity, object orientation, object travel direction, or its own velocity.

9. The method according to claim 1, wherein, The input radar signal is processed by range and Doppler processing, wherein the range-Doppler spectrum is used as input to train the machine learning computation model.

10. The method according to claim 1, wherein, The input radar signal is processed by range and Doppler processing and beamforming, wherein the input to the machine learning computation model includes the beam spectrum.

11. The method according to claim 1, wherein, The input to the machine learning computation model includes point clouds, Cartesian projections of point clouds, occupancy grids, or a previously determined representative model of the environment.

12. The method according to claim 1, wherein, The radar module is configured as follows: Transmit the first set of signals, which includes multiple output radar pulses, and The input radar signal is received, wherein the input radar signal is a subset of the first set of signals and includes a plurality of input radar pulses, and wherein the input radar signal is generated when the subset of the first set of signals interacts with or is reflected from at least one of the objects in the environment.

13. The method according to claim 1, wherein, The machine learning computation model is configured to determine object category labels for objects in the environment, and wherein the object category labels are integrated into the representative model of the environment to provide context about the objects and the environment.

14. The method according to claim 1, wherein, The indication to an object in the environment is a first indication to an object in the environment. The method further includes: Image data including a second indication of the object in the environment is received from multiple imaging devices, and The raw radar spectrum and the image data are processed by the machine learning computation model to generate the set of output predictions about the environment.

15. The method according to claim 14, wherein, The input to the machine learning computation model includes time-historical data received from at least one of the imaging device or the radar module.

16. The method according to claim 1, wherein, The indication to an object in the environment is a first indication to an object in the environment. The method further includes: Receives LIDAR data from a light detection and ranging (LIDAR) sensor, including a second indication of the object in the environment, and The raw radar spectrum and the LIDAR data are processed by the machine learning computation model to generate the set of output predictions about the environment.

17. The method according to claim 1, wherein, The original radar spectrum includes a radar frame sequence, wherein the machine learning computation model outputs a set of output predictions based on the radar frame sequence, wherein the radar frame sequence includes a continuous time series or a non-continuous time series, and wherein the input of the machine learning computation model includes the most recent N frames of the radar frame sequence.

18. The method according to claim 1, further comprising: Receives multiple input radar signals, including the indication of the object in the environment, from multiple radar modules associated with the vehicle located in the environment, and The multiple input radar signals are processed by one or more signal processing algorithms to determine the original radar spectrum.

19. The method according to claim 18, wherein, The radar module includes at least one front-angle radar and one forward-facing radar, or at least one rear-angle radar and one rear-facing radar.

20. The method according to claim 18, wherein, The radar module includes at least two forward-facing radars or at least two backward-facing radars.

21. The method according to claim 1, wherein, The machine learning computational model is trained using a supervised method with baseline fact labels.

22. The method according to claim 1, wherein, The machine learning computational model is trained using unsupervised or self-supervised methods by learning how to reconstruct the original radar signal from multiple radar modules or from multiple time data points.

23. The method according to claim 1, wherein, The autonomous driving system includes an emergency braking system, an advanced driver assistance system (ADAS), or an active safety system.

24. The method according to claim 1, wherein, The machine learning computational models include neural network models, convolutional neural network (CNN) models, long short-term memory (LSTM) models, or visual Transformer (ViT) models.

25. A non-transitory computer-readable medium comprising instructions executable by an electronic processor to perform a set of functions, said set of functions including: Receive input radar signals from the radar module, including initial indications of objects in the environment; Receive image data from multiple imaging devices, including a second indication of the object in the environment; The image data and raw radar spectrum are processed by a machine learning computational model to determine a set of output predictions about the environment, wherein the raw radar spectrum is determined based on the input radar signal; A representative model of the environment is determined, at least in part, based on the set of output predictions about the environment; and The representative model of the environment is provided to the autonomous driving system.

26. A non-transitory computer-readable medium comprising instructions executable by an electronic processor to perform a set of functions, said set of functions including: Receive input radar signals from the radar module, including initial indications of objects in the environment; Receive LIDAR data from a light detection and ranging (LIDAR) sensor, including a second indication of the object in the environment; The LIDAR data and raw radar spectrum are processed by a machine learning computational model to determine a set of output predictions about the environment, wherein the raw radar spectrum is determined based on the input radar signal; A representative model of the environment is determined, at least in part, based on the set of output predictions about the environment; and The representative model of the environment is provided to the autonomous driving system.

27. A radar signal processing system, comprising: Vehicles located in the environment; A radar module associated with the vehicle; as well as An electronic processor, the electronic processor being configured to: Receive input radar signals from the radar module, including indications of objects in the environment; The input radar signal is processed by one or more signal processing algorithms to determine the original radar spectrum; The raw radar spectrum is processed by a machine learning computational model to determine a set of output predictions about the environment; A representative model of the environment is determined based at least in part on the set of output predictions about the environment; as well as The representative model of the environment is provided to the autonomous driving system.

28. The system according to claim 27, wherein, The vehicle is moving through the environment.

29. The system according to claim 27, wherein, The vehicle includes an automated driving vehicle or an autonomous vehicle, wherein the automated driving system is associated with or integrated into the vehicle.

30. The system according to claim 27, wherein, The radar module is installed on the vehicle and includes a radar transmitter and a radar receiver.

Images