OBJECT DETECTION SYSTEM IN RADAR DATA FRAMES USING A RECURRING CONVOLUTIONARY NEURAL NETWORK

Abstract

The present invention relates to a system for detecting objects in a scene from radar data frames of that scene, said frames being temporally sequential. The object detection system has an encoder-decoder architecture, the encoder, comprising one or more cascaded stages, each stage comprising a convolutional neural network (CNN) and a recurrent neural network (RNN) arranged in series. With each new data frame received, the encoder provides, according to its current internal state (its memory), a set of spatial features and updates its internal state, said internal state being formed by the internal states of the RNNs of the encoder stages. The decoder uses the set of spatial features provided at time t to provide output information relating to the objects present in the scene. Figure to be published with the abbreviation: Fig. 3

Description

Title of the invention: SYSTEM FOR DETECTING OBJECTS IN RADAR DATA FRAMES USING A RECURRENT CONVOLUTIONAL NEURAL NETWORK Technical field

[0001] The present invention relates generally to the field of radar detection as well as that of recurrent convolutional neural networks. It finds particular application in automobile driving assistance devices or even in autonomous driving systems. STATE OF THE PRIOR ART

[0002] Advanced Driver Assistance Systems or AD AS (Advanced Driver Assistance System) have experienced significant growth in recent years. These systems help the driver of a motor vehicle to perceive their environment (obstacles, other vehicles, pedestrians, etc.) and even allow the vehicle to react automatically and in advance of the driver's reflexes.

[0003] AD AS systems generally use different types of sensors such as cameras, lidars and radars on board the vehicle. Cameras provide rich semantic information about the vehicle's environment and lidars provide a high-resolution representation of the objects present in this environment. However, cameras and lidars are sensitive to adverse weather conditions (fog, rain, snow) as well as poor lighting conditions (darkness) which can alter the perception of the environment. Conversely, radars can overcome these conditions due to the millimeter electromagnetic wavelength range in which they operate, the radar beam being able to pass through raindrops or fog. Radar detection devices can accurately calculate distances to obstacles as well as their speeds.The use of 1D or 2D antenna arrays also makes it possible to determine the direction of arrival of the reflected signal and therefore the angle at which an obstacle is seen by the vehicle.

[0004] As a general rule, radar systems do not provide raw data (which would typically be power spectra representing the environment in terms of distance and angle or distance and speed) but themselves directly perform additional signal processing steps applied to the power spectra in order to detect target potentials. These additional steps are, for example, adaptive thresholding operations (CFAR), calculations of distance, angle of arrival, relative speed (Doppler speed), for the target potentials and / or the target tracking.

[0005] Radar systems can thus generate a list of targets (each target being associated for example with a point cloud) and associate with each of them low-level information such as its position, its azimuth, its relative speed, its radar cross section or SER. However, these target lists do not include classification information (pedestrian, obstacle, vehicle for example) and are therefore difficult to use for the purposes of driving assistance. Furthermore, it is impossible to determine whether several points of the same point cloud, detected by such a radar system, actually belong to the same object or whether a detected target is actually a target to be monitored and not an unimportant element (for example a fixed element such as a tree, interference, etc.).

[0006] Radars used in the field of driving assistance or autonomous driving are generally of the FMCW (Frequency Modulated Continuons Wave) type. It is recalled that this type of radar emits chirps with a repetition period P, a chirp, 110, being a linearly frequency modulated wave function, as shown in Fig. 1. At reception, the received signal is mixed at 120 with the transmitted signal, subjected to low-pass filtering, 130, before being converted to digital at 140 and being subjected to an FFT at 150. The FFT can be carried out with a short time step (fast time FFT or range FFT), 151, corresponding to a chirp sampling frequency, to carry out a distance analysis (range spectrum), and for each distance, with a long time step corresponds the recurrence period T (slow time FFT) to carry out a speed analysis (range-Doppler spectrum), 152.When the radar system uses an antenna network for transmission and reception, also called MIMO radar (Multiple Input Multiple Output), the signals received by the different antennas make it possible to obtain a direction of arrival of the signal (DoA) as a function of the distance (range-angle spectrum), 153. We thus obtain a tensor of order 3 of so-called low-level radar data (or raw radar data), this tensor called RAD, which can be projected onto spaces of dimension 2 (or views) to give RD (distance, speed), RA (distance, angle) or AD (angle, speed) frames, the frames succeeding each other at the recurrence period or at a multiple thereof.

[0007] Furthermore, as indicated previously, the radar data at the FFT output can be subject to post-processing, 160, to provide lists of targets, 170.

[0008] Different approaches have been proposed in the state of the art for detecting objects, from raw radar data (RD, RA, AD, or even RAD), sometimes still called pre-CFAR in the literature, using deep neural networks. Most of these approaches use 2D convolutional neural networks, operating on a single data frame (2D CNNs).

[0009] However, these methods fail to correctly detect similar objects (such as pedestrians or cyclists). Thus, methods using 3D convolutional neural networks (3D CNNs), operating on a plurality of consecutive frames have been proposed in the literature. In these networks, convolutions are performed on both the temporal and spatial dimensions in order to reduce the size of the blocks according to these dimensions and retain only the most relevant information.

[0010] For example, a description of such a convolutional neural network operating on raw data can be found in the article by Y. Wang et al. entitled "RODNet: a real-time radar object detection network cross-supervised by camera-radar fused object 3D localization" published in IEEE Journal of Selected Topics in Signal Processing, Vol. 15, No. 4, pp. 954-967.

[0011] However, these approaches have the disadvantage of requiring a large memory to store a plurality N of successive frames.

[0012] Furthermore, object detection is often non-causal, in other words it is necessary to acquire frames subsequent to the occurrence of an event in order to be able to detect this event. These constraints do not allow real-time detection in an embedded system. However, the real-time nature of the detection is a determining criterion in driver assistance systems or autonomous driving systems.

[0013] The object of the present invention is therefore to propose a system for detecting objects in a scene, from radar data frames of this scene, by means of an artificial neural network, which does not have the aforementioned drawbacks, namely in particular which does not require storing a plurality of successive frames, which is causal and allows detection of objects in real time. Statement of the invention

[0014] To meet the above-mentioned objective, according to the present invention the following object detection system is proposed.

[0015] This system is a system for detecting objects in a scene from radar data frames of this scene, said frames succeeding each other in time, characterized in that said system comprises at least one spatio-temporal encoder and at least one decoder, the spatio-temporal encoder being intended to receive said frames and comprising one or more cascaded stages, each stage comprising a convolutional neural network, CNN, and a recurrent neural network, RNN, arranged in series, the encoder being configured to provide, as a function of its internal state, and at each new data frame rk received at a time k, a set Fk of spatial characteristics and to update its internal state as a function of this new frame, said internal state being formed by the internal states of the RNN networks of the stages of the encoder, the decoder using the set ^of spatial characteristics at time k,as well as optionally the internal state updated by the encoder,Hfor , provide output information about objects present in the scene.

[0016] The frames received by the space-time encoder(s) can be of different types.

[0017] In some embodiments, the radar data frames are obtained by projecting a radar measurement tensor into a 2-dimensional space or constitute 3-dimensional tensors.

[0018] In some embodiments, the radar data frames provide (in particular, provide only) two pieces of information, or three pieces of information, among range, angle of arrival and Doppler velocity information.

[0019] Furthermore, the output information may include one or more elementary information items, which are of different natures.

[0020] Thus in certain embodiments, the output information is very simple, and comprises for example only one or more binary information, and / or one or more information in the form of digital quantities (real numbers).

[0021] The output information may indicate, for example, whether a moving object has been detected in the scene (binary information), and possibly the distance at which the detected object is located (information in the form of a digital quantity).

[0022] In certain embodiments, the output information comprises at least one mapping, in particular a mapping giving a probability of presence of the objects at each point of the scene.

[0023] In certain embodiments, the output information comprises a mapping constituting a semantic segmentation of the scene in the form of an image and giving for each pixel of the image a class to which it belongs, among a plurality of possible classes.

[0024] Furthermore, different architectures can be used for the space-time encoder(s) and the decoder(s).

[0025] In some embodiments, the convolutional network of the stage or each stage of the cascade of stages of the encoder is a separable convolutional network, for example an Inverted Residual Bottleneck type network.

[0026] A separable convolution network is a network comprising at least one convolutional layer in which the convolution operations are of the 'depthwise separable convolutions' type, in which the convolution operations in the depth direction ('depthwise convolution') are carried out separately from the convolution operations in the lateral directions ('pointwise convolution').

[0027] Depthwise separable convolutions are described in detail in: Howard, AG, Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T.,... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1704.04861.

[0028] In certain embodiments, the recurrent network of the stage or of each stage of the cascade of stages of the encoder is a separable convolutional network, for example a ConvLSTM network, in particular a Bottleneck LSTM type network.

[0029] In certain embodiments, the encoder comprises, downstream of the cascade of stages, a last stage consisting of a convolutional network with separable convolutions, for example an Inverted Residual Bottleneck type network.

[0030] In some embodiments, the decoder comprises a plurality of cascaded stages, each stage of the decoder being configured to perform a transposed convolution.

[0031] The transmission of certain information from the encoder to the decoder also makes it possible to improve the performance of the network.

[0032] For this reason in certain embodiments, at least one stage of the decoder receives, via a skip-layer connection, a spatial characteristic provided by a corresponding stage of the encoder as well as optionally the internal state of the recurrent neural network belonging to this stage.

[0033] Furthermore, in certain embodiments, the object detection system is configured so as to make the best use of the information acquired by the radar.

[0034] Thus in certain embodiments, the object detection system comprises a plurality of encoders, each encoder being intended to receive radar data frames obtained by projection of a radar measurement tensor (TRAD) onto a distinct space of dimension 2, each encoder being configured to provide, as a function of its internal state, and at each new data frame}k received at a time k, a set of spatial characteristics and to update its internal state as a function of this new frame; and said at least one decoder being configured to use said set of spatial characteristics at time k, as well as optionally the internal states updated by said encoders to provide the output information.

[0035] In certain variants of these embodiments, at least one stage of the decoder receives, via a skip-layer connection, a composite spatial characteristic resulting from the concatenation of spatial characteristics provided by corresponding stages of the same rank of the different coders as well as optionally a composite internal state obtained from the internal states of the recurrent neural networks belonging to the corresponding stages of the same rank of the different coders. Brief description of the drawings

[0036] Other characteristics and advantages of the invention will appear on reading a preferred embodiment of the invention, made with reference to the attached figures among which:

[0037] [Fig. 1] already described schematically represents a known radar system of the state of the art capable of generating radar data frames;

[0038] [Fig.2] schematically represents the progression over time of the func operation of a recurrent and convolutional neural network equipping an object detection system according to an embodiment of the invention;

[0039] [Fig.3] schematically represents the coder-decoder architecture of a object detection system according to a first embodiment of the invention;

[0040] [Fig.4A] illustrates a learning phase of a recurrent and convolutional neural network according to an example of the first embodiment of the invention;

[0041] [Fig.4B] illustrates an inference phase of a recurrent and convolutional neural network according to an example of the first embodiment of the invention;

[0042] [Fig.5] schematically represents the architecture of a detection system of objects according to an example of the first embodiment of the invention;

[0043] [Fig.6] schematically represents the coder-decoder architecture of a object detection system according to a second embodiment of the invention;

[0044] [Fig.7] schematically represents a skip-layer connection between the encoder and decoder of the object detection system in [Fig.6].

[0045] DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS

[0046] We consider in the following a system for detecting objects in frames of raw radar data succeeding one another at a predetermined recurrence period, T. By frames of raw radar data, also referred to hereinafter more simply as radar data frames (or radar spectra), we mean here the projection into a 2-dimensional space of a radar measurement tensor whose components are geometric coordinates or speed values. For example, the measurement tensor may be a RAD tensor (distance, angle, Doppler speed) defined above and the radar data frames may be RD frames (distance, Doppler speed), AD frames (angle, Doppler speed) or RA frames (distance, angle). The raw radar data may be provided by a frequency modulated continuous wave (FMCW) radar as described in the introductory part or of another type, for example a pulsed radar.

[0047] The detection system is intended to provide at each recurrence period and for a given scene output information relating to the objects present in this scene, in particular in the form of a map. Depending on the type of application envisaged, this map may be presented in the form of a confidence map, giving for each point of the scene the probability of presence of an object, or even a semantic segmentation mask, giving for each pixel of an image representative of the scene, the class to which it belongs among a plurality of available classes. For the output information, other formats, in particular other types of mapping of the objects present in the scene may be envisaged by those skilled in the art without departing from the scope of the present invention.

[0048] The object detection system according to the present invention uses a prediction model, M, comprising a space-time encoder, F, and a decoder, D. If we denote Rk — ( rk_N+i, •" rk) a sequence of N radar data frames consecutive where k is a time step index (for example a RA or RD radar spectrum, the prediction model is broken down as follows:

[0049] [Math.l] = (D o E)(Rk) (1)

[0050] where 0 is the functional composition symbol.

[0051] The space-time encoder, E, can be considered as a state machine: it provides, at each instant k, a set Fk of spatial characteristics (features), from the last frame of radar data received E and its current internal state, (its memory) and updates its internal state, which is translated by:

[0052] [Math.2] E(rk, = HJ (2)

[0053] The decoder D uses the internal state updated by the space-time encoder, and all of the spatial characteristics provided at time k, to provide, in the form of a map, information relating to the objects present in the scene (for example a confidence map), Pk, i.e.:

[0054] [Math. 3]

[0055] [Fig.2] schematically represents the operation over time of the prediction model.

[0056] More precisely, two consecutive instants of indices k -1 and k have been represented in this figure. At instant k -1, the space-time encoder E receives the radar data frame rk-\ and, from its internal state (its memory), provides the decoder D with a set of spatial characteristics FkA'. In addition, it updates its internal state from the radar data frame} k-ï and its current state Hk_2 . The updated internal state is also provided to the decoder D. The latter predicts from the spatial characteristics F k_k and the updated encoder state, a map of the objects present in the scene, P^i. The process continues iteratively as illustrated at the following instant, k.

[0057] The idea behind the present invention is to combine in the spatio-temporal encoder at least one convolutional neural network or CNN and one recurrent neural network or RNN. The purpose of the convolutional neural network is to extract “spatial” information in radar data frames, the term “spatial” here referring to the variables represented in these frames (e.g. RA, AD, RD for the first layer) and the recurrent neural network operating on the spatial information thus extracted.

[0058] Advantageously, the space-time encoder may have a plurality of CNN and RNN networks alternating in series. Indeed, a CNN network makes it possible to reduce the dimension of its input so as to keep only a relevant and condensed representation of it, whereas an RNN network makes it possible to jointly manage the spatial and temporal information by retaining the memory of past inputs in its internal state. In the present case, the RNN network makes it possible to analyze the temporal dependence between the radar data frames.

[0059] The encoder-decoder architecture of an object detection system equipped with such a spatio-temporal encoder is represented schematically in [Fig.3].

[0060] The space-time encoder E comprises [Q £ 1 stage(s) in cascade, each stage consisting of a CNN network and an RNN network, arranged in series. The state of the space-time encoder at the given instant is then defined by the respective internal states, at this instant, of the RNN networks of the different stages, in other words:

[0061] [Math.4] ^ = (^1,...,1^ (4)

[0062] where ^2 ^0 are the respective internal states of the RNN networks belonging to the Q stages of the space-time encoder.

[0063] The set of spatial characteristics Fk and the state of the space-time encoder Hk (in other words the states of the RNN networks that it includes, its memory) are provided to the decoder D.

[0064] CNN neural networks may advantageously be separable convolution networks.

[0065] In particular, the CNN neural networks may advantageously be chosen of the Inverted Residual Bottleneck type, hereinafter referred to as IR. A detailed description of such a neural network can be found in the article by M. Sandler et al. entitled “MobileNetV2: Inverted residuals and linear bottlenecks” published in Proc. of the IEEE Conf. on computer vision and pattern recognition, 2018, pp. 4510-4520. The use of IR networks makes it possible to reduce the number of parameters compared to a conventional CNN network as well as the number of MAC (Multiple-ACcumulative) operations since an IT network uses a separable convolution, i.e. a convolution which is decomposed separately into a depthwise convolution (i.e. on a single channel at a time, and a pointwise convolution ( pointwise), that is, on all channels at once.

[0066] Similarly, RNN neural networks may advantageously be separable convolution networks.

[0067] In particular, the RNN networks may be chosen to be of the LSTM (Long Short Term Memory) type and advantageously of the Bottleneck LSTM type. A detailed description of such a neural network can be found in the article by M. Liu et al. entitled “Mobile video object detection with temporally-aware feature maps” published in Proc. of the IEEE Conf. on computer vision and pattern recognition, 2018, pp. 5686-5695.

[0068] Advantageously, a normalization layer may be provided before activation (sigmoid function) at the level of the input, forget and output gates of each LSTM network, the normalization being carried out on all of the spatial characteristics.

[0069] Finally, the first layer of the space-time encoder comprises, as shown in [Fig.3] a classic convolutional neural network to increase the number of channels, each channel corresponding to a block convolution kernel. The CNN-RNN network stages operate on the radar data frames having previously undergone this block convolution step.

[0070] During training, the object detection system is presented sequentially with A' data frames as input to the spatio-temporal encoder and associated prediction results, i.e., mappings of objects actually present in the scene. The model is thus trained to encode a sequence of N data frames to predict at each instant f the corresponding mapping, Pt, i.e. for the decoder:

[0071] [Math.5] D(FM = P t (4)

[0072] where Ft and Ht are respectively the set of characteristics provided by the space-time encoder and its internal state after updating at the moment

[0073] Fig. 4A represents a step of the training phase of an example of an object detection system according to Fig. 3. In the example shown, the space-time encoder comprises Q = 2 cascaded stages, each stage comprising a CNN network (IR network) and an RNN network, arranged in series. The state of the space-time encoder at time k, after updating, is therefore given by A network A block convolutional network is provided upstream of the CNN-RNN stages, and an IR network is arranged downstream of the CNN-RNN cascade. The space-time encoder sequentially receives radar data frames as input, and the decoder makes a prediction Pk based on the last N = 3 frames received, this prediction being confronted with a mapping of existing objects in the scene. The parameters of the different IR and RNN neural networks are updated after calculating the loss function and a backpropagation of the gradient aimed at minimizing the prediction error.

[0074] Once trained, the detection system can be used in an inference phase to predict information relating to the objects present in the scene.

[0075] Fig. 4B represents a step of the inference phase of the previously trained object detection system. At each instant k, the encoder provides from its state before update, fJk = hj) and from the radar data frame that it receives, rk, a set of spatial characteristics as well as the updated state [-[, — , to the decoder D which deduces a map of the objects detected in the scene.

[0076] [Fig.5] schematically represents the encoder-decoder architecture of an object detection system according to an example of the first embodiment of the present invention.

[0077] The architecture of this exemplary embodiment is of the U-net type, a description of which can be found in the article by O. Ronneberger et al. entitled “U-Net: convolutional networks for biomedical image segmentation” published in Proc, of Medical Image Computing and Computer-Assisted Intervention, MICCAI 2015, pp. 234-241,

[0078] This architecture comprises a contraction branch, formed by the space-time encoder E, and an expansion branch, formed by the decoder D.

[0079] The space-time encoder E comprises two stages 520, 530 in cascade (in other words Q = 2), each stage comprising a CNN network (of the Inverted Residual Bottleneck type) and an RNN network (of the Bottleneck LSTM type), in series. The RNN network of the first stage, 520, provides, at each instant k, its internal state fol to the decoder, via a first temporal skip connection and, likewise, the RNN network of the second pair, 530, provides its internal state to the decoder, via a second time connection skips layers.

[0080] The space-time encoder E comprises upstream of the first stage at least one block convolutional neural network, 510, providing as output blocks of size C x H x VF where C is the number of channels (i.e. the number of convolution kernels and therefore of space-time characteristics), H and W are respectively the height and the width of the radar data frames.

[0081] Finally, the space-time encoder comprises, downstream of stage 530, a third and final stage, a CNN network (of the Inverted Residual Bottleneck type), 540.

[0082] The number of channels is doubled between CNN networks of successive stages and the width as well as the width of the blocks are divided by 2. In other words, C = 2C! = 4C2 = 8C3

[0083] The decoder comprises three transposed 2D convolution stages, 550, 560, 570, followed by a convolution stage, 580, allowing to obtain the mapping of the objects which is the output information (for example confidence map or semantic segmentation of the scene image). Each stage of the encoder corresponds to a transposed convolution stage of the decoder: stage 550 corresponds to the CNN network 540, stage 560 corresponds to the CNN-RNN stage, 530, and stage 560 corresponds to the CNN-RNN stage, 520.

[0084] Each stage of the decoder receives the spatial characteristics from the corresponding stage (520,530, 540) of the encoder via a skip connection and, if applicable, the internal state of the RNN belonging to this stage (520,530) via a temporal skip connection or TSC (temporal skip connection). This TSC connection makes it possible to avoid the loss of temporal information along the decoding path. The internal state ( / ^, / 72) of the RNN belonging to a stage of the encoder is kk concatenated to the result of the transposed convolution provided by the stage preceding the corresponding stage of the decoder.

[0085] Thus, for example, the internal state / 72 of the RNN network of the second stage, 530, of the encoder is concatenated or added to the result of the transposed convolution of the first stage, 550, of the decoder. Similarly, the internal state of the CNN network of the first stage, 520, of the k encoder is concatenated or added to the result of the transposed convolution of the second stage, 560 of the decoder.

[0086] [Fig.6] schematically represents the architecture of an object detection system according to a second embodiment of the present invention.

[0087] This second embodiment differs from the first in that it makes it possible to detect the objects present in the scene using the different dimensions of distance, speed and azimuth / angle). This embodiment therefore makes it possible to exploit all of the information present in the radar signal.

[0088] According to a first variant (not illustrated by the figures), successive radar frames rk are received at each time interval by a space-time coder ERAI).

[0089] In this variant, the radar frames rk integrate three pieces of information of distance, speed and angle (R, D, A). These radar frames can for example take the form of a tensor, in which two of the three pieces of information of distance, speed and angle (R, D, A) are coded according to the lateral dimensions (height and width) of the tensor, and the third of these pieces of information is coded according to the depth dimension (the 'channels') of the tensor.

[0090] A second variant is illustrated by [Fig.6]. This variant differs from the first variant in that different views (or projections) of the radar measurement tensor, for example RA, RD and AD frames in the sense defined previously are used to detect the objects present in the scene. This variant therefore exploits separately two by two the dimensions of distance, speed and azimuth / angle, using a plurality of space-time encoders.

[0091] The object detection system according to this variant comprises as many coders as there are views of the measurement tensor, namely, in the illustrated example, three coders ERA, ERD, Ead each having the structure of the first embodiment. More precisely, each of these coders comprises the same number Q of stages, each stage consisting of a CNN network in series with an RNN network. At each instant E, the spatial characteristics, F^D, FAD from the different coders are concatenated in a multi-view latent variable space to form a composite spatial characteristic F1^^, supplied to one or more decoders, here the decoders DRA and E)RD. Similarly, the internal states of the RNN networks of the different stages q — 1, -, Q of the coders E^E^E^ respectively noted are concatenated to form a composite internal state transmitted to the corresponding stage of each of the decoders, as in Fig. 5. Thus, in the example shown, each of the decoders E>ra,Drd can benefit from temporal information coming not only from the corresponding encoder (ERAERD) but from the three encoders (Era,Erd,Ead) associated with the 3 views of the measurement tensor. The composite internal states = - Q are transmitted to the decoders by a hopping temporal connection- layer common to all views, designated in the figure by TMVSC (Temporal Multi-View Skip Connection).

[0092] In the example shown, the decoder allows DRA to predict the position of objects in the scene in an RA space (distance, azimuth) while the decoder Drd allows to predict the positions of these objects, otherwise to represent them in an RD space (distance, speed).

[0093] [Fig.7] represents in more detail the concatenation operation of the composite internal states carried out by the TMVSC connection of [Fig.6].

[0094] The internal states of RNN scales forming part of the stages of rank Q of the respective encoders E^E^E^ are concatenated, if necessary after a subsampling or oversampling step to harmonize their dimensions to C x H x W, resulting in a concatenation result of size 3C x H x W which is then subject to a 2D 1x1 convolution (channelwisepooling) to reduce the number of channels from 3C to C and therefore to use them within the decoder stages as in the first embodiment, as a composite internal state.

Claims

Claims

1. System for detecting objects in a scene from radar data frames of this scene, said frames (; y) succeeding one another in time, said frames (r*) of radar data giving two pieces of information (RA, RD, AD), or three pieces of information (RAD), among information on distance (R), angle of arrival (A) and Doppler speed (D), characterized in that said system comprises at least one spatio-temporal encoder (E ERDEADERA) and at least one decoder (P DRADRD ), the spatio-temporal encoder being intended to receive said frames and comprising a (Q) plurality of cascaded stages (520, 530), each stage comprising a convolutional neural network, CNN, and a recurrent neural network, RNN, arranged in series, the encoder being configured to provide, as a function of its internal state, and at each new data frame rk received at a time k,a set Fk of spatial characteristics and to update its internal state according to this new frame, said internal state being formed by the internal states of the RNN networks of the stages of the encoder, the decoder PPraPrd) using the set ^of spatial characteristics at time E as well as optionally the internal state updated by the encoder,^, to provide output information (Pk) relating to the objects present in the scene.,

2. System for detecting objects in a scene according to claim 1, characterized in that the frames (r0 of radar data are obtained by projection of a radar measurement tensor into a 2-dimensional space or constitute 3-dimensional tensors.

3. System for detecting objects in a scene according to claim 1 or 2, characterized in that said output information (Pk) comprises at least one map, in particular a map giving a probability of presence of the objects at each point of the scene.

4. System for detecting objects in a scene according to any one of claims 1 to 3, characterized in that said output information comprises a mapping constituting a semantic segmentation of the scene in the form of an image and giving for each pixel of the image a class to which it belongs, from among a plurality of possible classes.

5. System for detecting objects in a scene according to one of the preceding claims, characterized in that the convolutional network of a stage or of each stage of the cascade of stages of the encoder is a separable convolutional network, for example an Inverted Residual Bottleneck network.

6. System for detecting objects in a scene according to one of the preceding claims, characterized in that the recurrent network of a stage or of each stage of the cascade of stages of the encoder is a separable convolution network, for example a ConvLSTM network, in particular a Bottleneck LSTM type network.

7. System for detecting objects in a scene according to claim 5 or 6, characterized in that the encoder comprises downstream of the cascade of stages a last stage (540) consisting of a convolutional network with separable convolutions, for example a network of the Inverted Residual Bottleneck type.

8. System for detecting objects in a scene according to one of the preceding claims, characterized in that the decoder comprises a plurality of cascaded stages (550,560,570), each stage of the decoder being configured to perform a transposed convolution.

9. System for detecting objects in a scene according to any one of the preceding claims, characterized in that at least one stage of the decoder (550,560,570) receives, via a skip-layer connection, a spatial characteristic provided by a corresponding stage of the encoder as well as optionally the internal state of the recurrent neural network belonging to this stage.

10. System for detecting objects in a scene according to one of claims 1 to 9, characterized in that it comprises a plurality of coders, each coder being intended to receive radar data frames obtained by projection of a radar measurement tensor (T) onto a distinct space of dimension 2, each coder being configured to provide, as a function of its internal state, and at each new data frame rk received at a time k, a set (Fk>RA; Fk>RD; Fk>AD) of spatial characteristics and to update its internal state as a function of this new frame; and said at least one decoder being configured to use said set of spatial characteristics (FkRA; FkRD jF^) at a time as well as optionally the internal states updated by said coders to provide the output information (Pk).

11. System for detecting objects in a scene according to claim 10, characterized in that at least one stage of the decoder receives, via a skip-layer connection, a composite spatial feature (FkRAI)) resulting from the concatenation of spatial characteristics (FkRA jF^ ;Fk ) provided by corresponding stages of the same rank of the different encoders as well as optionally a composite internal state obtained from the internal states of the networks of kkk recurrent neurons belonging to the corresponding stages of the same rank (^) of the different encoders.

Images