System for detecting objects in radar data frames using a recurrent convolutional neural network

Abstract

The present invention relates to a system for detecting objects in a scene from radar data frames of this scene, the frames being consecutive in time. The object detection system has an encoder-decoder architecture, the encoder, E, comprising one or more cascading stages (Q), each stage comprising a convolutional neural network, CNN, and a recurrent neural network, RNN, arranged in series. With each new data frame received, rk, the encoder provides, depending on its current internal state (its memory), a set Fk of spatial characteristics and updates its internal state, the internal state being formed by the internal states of the RNN networks of the stages of the encoder. The decoder D uses the set of spatial characteristics provided at time k, Fk, to provide output information relating to the objects present in the scene.

Description

[0001] Title: SYSTEM FOR DETECTING OBJECTS IN RADAR DATA FRAMES USING A RECURRENT CONVOLUTIONAL NEURAL NETWORK DESCRIPTION

[0002] TECHNICAL FIELD

[0003] The present invention relates generally to the field of radar detection as well as that of recurrent convolutional neural networks. It finds particular application in automobile driving assistance devices and even in autonomous driving systems.

[0004] STATE OF THE PRIOR ART

[0005] Advanced driver assistance systems (ADAS) have grown significantly in recent years. These systems help the driver of a motor vehicle to perceive their environment (obstacles, other vehicles, pedestrians, etc.) and even allow the vehicle to react automatically and in advance of the driver's reflexes.

[0006] ADAS systems typically use different types of sensors such as cameras, lidars, and radars embedded in the vehicle. Cameras provide rich semantic information about the vehicle's surroundings, and lidars provide a high-resolution representation of objects in that environment. However, cameras and lidars are sensitive to adverse weather conditions (fog, rain, snow) and poor lighting conditions (darkness), which can alter the perception of the environment. Radars, on the other hand, can overcome these conditions due to the millimeter electromagnetic wavelength range in which they operate, as the radar beam can penetrate raindrops or fog. Radar detection devices can accurately calculate distances to obstacles and their speeds.The use of 1D or 2D antenna arrays also makes it possible to determine the direction of arrival of the reflected signal and therefore the angle at which an obstacle is seen by the vehicle.

[0007] Typically, radar systems do not provide raw data (which would typically be power spectra representing the environment in range and angle or range and speed) but directly perform additional signal processing steps applied to the power spectra in order to detect potential targets. These additional steps are, for example, adaptive thresholding operations (CFAR), calculations of range, angle of arrival, relative speed (Doppler velocity), for target potentials and / or target tracking.

[0008] Radar systems can thus generate a list of targets (each target being associated, for example, with a point cloud) and associate with each of them low-level information such as its position, its azimuth, its relative speed, its radar cross section or SER. However, these target lists do not include classification information (pedestrian, obstacle, vehicle, for example) and are therefore difficult to use for the purposes of driving assistance. In addition, it is impossible to determine whether several points in the same point cloud, detected by such a radar system, actually belong to the same object or whether a detected target is actually a target to be monitored and not an unimportant element (for example, a fixed element such as a tree, interference, etc.).

[0009] Radars used in the field of driver assistance or autonomous driving are generally of the FMCW (Frequency Modulated Continuous Wave) type. It is recalled that this type of radar emits chirps with a repetition period P, a chirp, 110, being a linearly frequency modulated wave function, as shown in Fig. 1. At reception, the received signal is mixed at 120 with the transmitted signal, subjected to low-pass filtering, 130, before being converted to digital at 140 and being subjected to an FFT at 150. The FFT can be carried out with a short time step (fast time FFT or range FFT), 151, corresponding to a sampling frequency of the chirp, to carry out a distance analysis (range spectrum), and for each distance, with a long time step corresponds the recurrence period T (slow time FFT) to carry out a speed analysis (range-Doppler spectrum), 152.When the radar system uses an antenna network for transmission and reception, also called MIMO radar (Multiple Input Multiple Output), the signals received by the different antennas make it possible to obtain a direction of arrival of the signal (DoA) as a function of the distance (rangeangle spectrum), 153. We thus obtain a tensor of order 3 of so-called low-level radar data (or raw radar data), this tensor called RAD, which can be projected onto spaces of dimension 2 (or views) to give RD (distance, speed), RA (distance, angle) or AD (angle, speed) frames, the frames succeeding each other at the recurrence period or at a multiple thereof.

[0010] Furthermore, as previously indicated, the radar data output from FFT can be post-processed, 160, to provide target lists, 170.

[0011] Different approaches have been proposed in the state of the art to detect objects from raw radar data (RD, RA, AD, or even RAD), sometimes still referred to as pre-CFAR in the literature, using deep neural networks. Most of these approaches use 2D convolutional neural networks, operating on a single data frame (2D CNNs).

[0012] However, these methods fail to correctly detect similar objects (such as pedestrians or cyclists). Thus, methods using 3D convolutional neural networks (3D CNNs), operating on a plurality of consecutive frames, have been proposed in the literature. In these networks, convolutions are performed on both temporal and spatial dimensions in order to reduce the block size along these dimensions and retain only the most relevant information. For example, a description of such a convolutional neural network operating on raw data can be found in the article by Y. Wang et al. entitled "RODNet: a real-time radar object detection network cross-supervised by camera-radar fused object 3D localization" published in IEEE Journal of Selected Topics in Signal Processing, Vol. 15, No. 4, pp. 954-967.

[0013] However, these approaches have the disadvantage of requiring a large memory to store a plurality N of successive frames.

[0014] Furthermore, object detection is often non-causal, meaning that it is necessary to acquire frames after the occurrence of an event in order to detect this event. These constraints do not allow real-time detection in an embedded system. However, the real-time nature of detection is a determining criterion in driver assistance systems or autonomous driving systems.

[0015] The object of the present invention is therefore to propose a system for detecting objects in a scene, from radar data frames of this scene, by means of an artificial neural network, which does not have the aforementioned drawbacks, namely in particular which does not require storing a plurality of successive frames, which is causal and allows detection of objects in real time.

[0016] STATEMENT OF THE INVENTION

[0017] To meet the above stated objective, according to the present invention the following object detection system is proposed.

[0018] This system is a system for detecting objects in a scene from radar data frames of this scene, said frames succeeding one another in time, said radar data frames giving two pieces of information, or three pieces of information, among distance, angle of arrival and Doppler speed information, characterized in that said system comprises at least one spatio-temporal encoder and at least one decoder, the spatio-temporal encoder being intended to receive said frames and comprising a plurality of cascaded stages, each stage comprising a convolutional neural network, CNN, and a recurrent neural network, RNN, arranged in series, the encoder being configured to provide, depending on its internal state, and at each new data frame r k received at a time k, a set F kof spatial characteristics and to update its internal state according to this new frame, said internal state being formed by the internal states of the RNN networks of the stages of the encoder, the decoder using the set F k spatial characteristics at time k, as well as optionally the internal state updated by the encoder, H k , to provide output information relating to the objects present in the scene.

[0019] The frames received by the space-time encoder(s) can be of different types.

[0020] These radar data frames are typically obtained by projecting a radar measurement tensor into a 2-dimensional space or constitute 3-dimensional tensors.

[0021] In the object detection system according to the present disclosure, the radar data frames provide (in particular, provide only) two pieces of information, or three pieces of information, among distance information, angle of arrival information and Doppler velocity information.

[0022] Furthermore, the output information may include one or more elementary information, which are of different natures.

[0023] Thus in certain embodiments, the output information is very simple, and comprises for example only one or more binary information, and / or one or more information in the form of digital quantities (real numbers).

[0024] The output information may indicate, for example, whether a moving object has been detected in the scene (binary information), and possibly the distance at which the detected object is located (information in the form of a digital quantity). In certain embodiments, the output information comprises at least one map, in particular a map giving a probability of presence of the objects at each point of the scene.

[0025] In some embodiments, the output information comprises a mapping constituting a semantic segmentation of the scene in image form and giving for each pixel of the image a class to which it belongs, among a plurality of possible classes.

[0026] Furthermore, different architectures can be used for the space-time encoder(s) and the decoder(s).

[0027] In some embodiments, the convolutional network of a stage or each stage of the cascade of stages of the encoder is a separable convolutional network, for example an Inverted Residual Bottleneck network.

[0028] A separable convolutional network is a network comprising at least one convolutional layer in which the convolution operations are of the type 'depthwise separable convolutions', in which the convolution operations in the depth direction ('depthwise convolution') are performed separately from the convolution operations in the lateral directions ('pointwise convolution').

[0029] Depthwise separable convolutions are described in detail in: Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

[0030] In some embodiments, the recurrent network of a stage or each stage of the cascade of stages of the encoder is a separable convolutional network, for example a ConvLSTM network, in particular a Bottleneck LSTM type network.

[0031] In some embodiments, the encoder comprises, downstream of the cascade of stages, a final stage consisting of a convolutional network with separable convolutions, for example an Inverted Residual Bottleneck type network.

[0032] In some embodiments, the decoder comprises a plurality of cascaded stages, each stage of the decoder being configured to perform a transposed convolution.

[0033] The transmission of certain information from the encoder to the decoder also helps improve network performance.

[0034] For this reason in certain embodiments, at least one stage of the decoder receives, via a skip-layer connection, a spatial characteristic provided by a corresponding stage of the encoder as well as optionally the internal state of the recurrent neural network belonging to this stage.

[0035] Furthermore, in some embodiments, the object detection system is configured to make the best use of the information acquired by the radar.

[0036] Thus in certain embodiments, the object detection system comprises a plurality of encoders, each encoder being intended to receive radar data frames obtained by projection of a radar measurement tensor (T RAD ) sur u n distinct space of dimension 2, each encoder being configured to provide, according to its internal state, and at each new data frame r k received at a time k, a set of spatial characteristics and to update its internal state according to this new frame; and said at least one decoder being configured to use said set of spatial characteristics at time k, as well as optionally the internal states updated by said coders to provide the output information.

[0037] In certain variants of these embodiments, at least one stage of the decoder receives, via a skip-layer connection, a composite spatial characteristic resulting from the concatenation of spatial characteristics provided by corresponding stages of the same rank of the different encoders as well as optionally a composite internal state obtained from the internal states of the recurrent neural networks belonging to the corresponding stages of the same rank of the different encoders.

[0038] BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Other characteristics and advantages of the invention will appear on reading a preferred embodiment of the invention, made with reference to the attached figures among which:

[0040] [Fig. 1] already described schematically represents a radar system known from the state of the art capable of generating radar data frames;

[0041] [Fig. 2] schematically represents the sequence over time of the operation of a recurrent and convolutional neural network equipping an object detection system according to an embodiment of the invention;

[0042] [Fig. 3] schematically represents the coder-decoder architecture of an object detection system according to a first embodiment of the invention;

[0043] [Fig. 4A] illustrates a learning phase of a recurrent and convolutional neural network according to an example of the first embodiment of the invention;

[0044] [Fig. 4B] illustrates an inference phase of a recurrent and convolutional neural network according to an example of the first embodiment of the invention

[0045] [Fig. 5] schematically represents the architecture of an object detection system according to an example of the first embodiment of the invention;

[0046] [Fig. 6] schematically represents the encoder-decoder architecture of an object detection system according to a second embodiment of the invention; [Fig. 7] schematically represents a skip-layer connection between the encoder and the decoder of the object detection system of Fig. 6. DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS

[0047] In the following, we consider a system for detecting objects in raw radar data frames following one another at a predetermined recurrence period, T . By raw radar data frames, also referred to hereinafter more simply as radar data frames (or radar spectra), we mean here the projection into a 2-dimensional space of a radar measurement tensor whose components are geometric coordinates or speed values. For example, the measurement tensor could be a RAD (distance, angle, Doppler speed) tensor defined above and the radar data frames could be RD (distance, Doppler speed), AD (angle, Doppler speed) or RA (distance, angle) frames. The raw radar data could be provided by a frequency modulated continuous wave (FMCW) radar as described in the introductory part or of another type, for example a pulsed radar.

[0048] The detection system is intended to provide at each recurrence period and for a given scene output information relating to the objects present in this scene, in particular in the form of a map. Depending on the type of application envisaged, this map may be presented in the form of a confidence map, giving for each point of the scene the probability of presence of an object, or a semantic segmentation mask, giving for each pixel of an image representative of the scene, the class to which it belongs among a plurality of available classes. For the output information, other formats, in particular other types of mapping of the objects present in the scene may be envisaged by those skilled in the art without departing from the scope of the present invention.

[0049] The object detection system according to the present invention uses a prediction model, M, comprising a spatio-temporal encoder, E, and a decoder, D. If we denote R k = (r k-N+1 , r k-N+2+i , ■ ■ , r k ~) a sequence of N consecutive radar data frames where k is a time step index (e.g. an RA or RD radar spectrum, the prediction model is decomposed as follows:

[0050] [Math. 1]

[0051] W ) - ( « K)(^) ( 1) where ° is the functional composition symbol.

[0052] The space-time encoder, E, can be considered as a state machine: it provides, at each instant k, a set F k of spatial features, from the last received radar data frame r k and its current internal state, (its memory) and updates its internal state, which we translate as: [Math.2]

[0053] The decoder D uses the internal state updated by the space-time encoder, H k , and the set of spatial characteristics provided at time k, F k , to provide, in the form of a map, information relating to the objects present in the scene (for example a confidence map), P k , or: [Math. 3] o0W -) = A (3)

[0054] Fig. 2 schematically represents the operation over time of the prediction model.

[0055] More precisely, two consecutive instants with indices k — 1 and k have been represented in this figure. At instant k — 1, the space-time encoder E receives the radar data frame and, from its internal state H k-2 (its memory), provides the decoder D with a set of spatial characteristics Ffc-i. In addition, it updates its internal state from the radar data frame r fc-x and its current state H k-2• The updated internal state is also provided to the decoder D . This predicts from the spatial characteristics F k- and the updated encoder state, H k-lr a mapping of the objects present in the scene, P k-1 The process continues iteratively as shown at the next time, k.

[0056] The idea behind the present invention is to combine in the spatio-temporal encoder at least one convolutional neural network or CNN and one recurrent neural network or RNN. The purpose of the convolutional neural network is to extract “spatial” information in the radar data frames, the term “spatial” here referring to the variables represented in these frames (e.g. RA, AD, RD for the first layer) and the recurrent neural network operating on the spatial information thus extracted.

[0057] Advantageously, the space-time encoder may have a plurality of CNN and RNN networks alternating in series. Indeed, a CNN network makes it possible to reduce the dimension of its input so as to keep only a relevant and condensed representation of it, whereas an RNN network makes it possible to jointly manage spatial and temporal information by preserving in its internal state the memory of past inputs. In this case, the RNN network makes it possible to analyze the temporal dependence between radar data frames.

[0058] The encoder-decoder architecture of an object detection system equipped with such a spatio-temporal encoder is schematically represented in Fig. 3.

[0059] The space-time encoder E comprises [Q > 2 cascaded stages, each stage consisting of a CNN network and an RNN network, arranged in series. The state of the space-time encoder at the given instant is then defined by the respective internal states, at that instant, of the RNN networks of the different stages, in other words:

[0060] [Math.4] where h k , h k , . . . , h are the respective internal states of the RNN networks belonging to the Q stages of the space-time encoder.

[0061] The set of spatial characteristics F k and the state of the space-time encoder H k (in other words, states of the RNN networks that it includes, its memory) are provided to the decoder D.

[0062] CNN neural networks can advantageously be separable convolutional networks.

[0063] In particular, CNN neural networks can be advantageously chosen of the Inverted Residual Bottleneck type, hereinafter referred to as IR. A detailed description of such a neural network can be found in the article by M. Sandler et al. entitled "MobileNetV2: Inverted residuals and linear bottlenecks" published in Proc, of the IEEE Conf, on computer vision and pattern recognition, 2018, pp. 4510-4520. The use of IR networks makes it possible to reduce the number of parameters compared to a classic CNN network as well as the number of MAC (Multiple-ACcumulate) operations since an IT network uses a separable convolution, i.e. a convolution that is decomposed separately into a depthwise convolution, i.e. on a single channel at a time, and a pointwise convolution, i.e. on all channels at once.

[0064] Similarly, RNN neural networks can advantageously be separable convolutional networks.

[0065] In particular, RNN networks can be chosen of the LSTM (Long Short Term Memory) type and advantageously of the Bottleneck LSTM type. A detailed description of such a neural network can be found in the article by M. Liu et al. entitled “Mobile video object detection with temporally-aware feature maps” published in Proc, of the IEEE Conf, on computer vision and pattern recognition, 2018, pp. 5686-5695.

[0066] Advantageously, a normalization layer can be provided before activation (sigmoid function) at the input, forget and output gates of each LSTM network, the normalization being carried out on all spatial characteristics.

[0067] Finally, the first layer of the space-time encoder comprises, as shown in Fig. 3, a classic convolutional neural network to increase the number of channels, each channel corresponding to a block convolution kernel. The CNN-RNN network stages operate on the radar data frames that have previously undergone this block convolution step.

[0068] During training, the object detection system is sequentially presented with N data frames as input to the spatio-temporal encoder and associated prediction results, i.e., mappings of objects actually present in the scene. The model is thus trained to encode a sequence of N data frames to predict at each time t the corresponding mapping, P t , either for the decoder:

[0069] [Math.5] where F t and H tare respectively the set of characteristics provided by the space-time encoder and its internal state after updating at time t.

[0070] Fig. 4A shows a step in the training phase of an example object detection system according to Fig. 3. In the example shown, the space-time encoder comprises Q = 2 cascaded stages, each stage comprising a CNN network (IR network) and an RNN network, arranged in series. The state of the space-time encoder at time k, after updating, is therefore given by H k = (h k , h k ~) . A block convolutional network is provided upstream of the CNN-RNN stages, and an IR network is arranged downstream of the CNN-RNN cascade. The space-time encoder sequentially receives radar data frames as input, and the decoder performs a prediction P kbased on the last N = 3 frames received, this prediction being compared to a mapping of the existing objects in the scene. The parameters of the different IR and RNN neural networks are updated after calculation of the loss function and a backpropagation of the gradient aimed at minimizing the prediction error.

[0071] Once trained, the detection system can be used in an inference phase to predict information relating to the objects present in the scene.

[0072] Fig. 4B represents a step of the inference phase of the previously trained object detection system. At each time k, the encoder provides from its state before update, and the radar data frame it receives, r k , a set of spatial characteristics as well as the updated state H k = (Ji k , h k~) , to the decoder D which deduces a map of the objects detected in the scene.

[0073] Fig. 5 schematically represents the encoder-decoder architecture of an object detection system according to an example of the first embodiment of the present invention.

[0074] The architecture of this example implementation is of the U-net type, a description of which can be found in the article by O. Ronneberger et al. entitled “U-Net: convolutional networks for biomedical image segmentation” published in Proc, of Medical Image Computing and Computer-Assisted Intervention, MICCAI 2015, pp. 234-241,

[0075] This architecture includes a contraction branch, formed by the space-time encoder E, and an expansion branch, formed by the decoder D.

[0076] The space-time encoder E comprises two stages 520, 530 in cascade (in other words Q = 2), each stage comprising a CNN network (of the Inverted Residual Bottleneck type) and an RNN network (of the Bottleneck LSTM type), in series. The RNN network of the first stage, 520, provides, at each instant k, its internal state h k to the decoder, via a first temporal skip connection and, similarly, the RNN network of the second pair, 530, provides its internal state h k to the decoder, via a second skip-layer temporal connection. The space-time encoder E comprises upstream of the first stage at least one block convolutional neural network, 510, providing as output blocks of size C x H x W where C is the number of channels (i.e. the number of convolution kernels and therefore of space-time characteristics), H and W are respectively the height and width of the radar data frames.

[0077] Finally, the space-time encoder includes, downstream of stage 530, a third and final stage, a CNN network (of the Inverted Residual Bottleneck type), 540.

[0078] The number of channels is doubled between CNN networks of successive stages and the width as well as the width of the blocks are divided by 2. In other words, C = 2C ± = 4C2= 8C3.

[0079] The decoder comprises three transposed 2D convolution stages, 550, 560, 570, followed by a convolution stage, 580, to obtain the mapping of the objects which is the output information (for example confidence map or semantic segmentation of the scene image). Each stage of the encoder corresponds to a transposed convolution stage of the decoder: stage 550 corresponds to the CNN network 540, stage 560 corresponds to the CNN-RNN stage, 530, and stage 560 corresponds to the CNN-RNN stage, 520.

[0080] Each decoder stage receives the spatial characteristics from the corresponding encoder stage (520,530,540) via a skip connection and, if applicable, the internal state of the RNN belonging to this stage (520,530) via a temporal skip connection or TSC. This TSC connection avoids the loss of temporal information along the decoding path. The internal state (h^,h^) of the RNN belonging to a stage of the encoder is concatenated to the result of the transposed convolution provided by the stage preceding the corresponding stage of the decoder.

[0081] For example, the internal state of the second stage RNN network, 530, of the encoder is concatenated or added to the result of the transposed convolution of the first stage, 550, of the decoder. Similarly, the internal state of the first stage CNN network, 520, of the encoder is concatenated or added to the result of the transposed convolution of the second stage, 560 of the decoder.

[0082] Fig. 6 schematically represents the architecture of an object detection system according to a second embodiment of the present invention.

[0083] This second embodiment differs from the first in that it allows the detection of objects present in the scene using the different dimensions of distance, speed and azimuth / angle). This embodiment therefore allows the exploitation of all the information present in the radar signal.

[0084] According to a first variant (not illustrated by the figures), successive radar frames rk are received at each time interval by a space-time encoder ERAD.

[0085] In this variant, the radar frames rk, integrate three pieces of information: distance, speed and angle (R,D,A). These radar frames can for example take the form of a tensor, in which two of the three pieces of information: distance, speed and angle (R,D,A) are coded according to the lateral dimensions (height and width) of the tensor, and the third of these pieces of information is coded according to the depth dimension (the 'channels') of the tensor.

[0086] A second variant is illustrated in Figure 6. This variant differs from the first variant in that different views (or projections) of the radar measurement tensor, for example RA, RD and AD frames in the sense defined previously, are used to detect the objects present in the scene. This variant therefore exploits the dimensions of distance, speed and azimuth / angle separately in pairs, using a plurality of spatio-temporal encoders.

[0087] The object detection system according to this variant comprises as many encoders as there are views of the measurement tensor, namely, in the illustrated example, three encoders E RA ,E RD ,E AD each having the structure of the first embodiment. More precisely, each of these coders has the same number Q of stages, each stage consisting of a CNN network in series with an RNN network. At each time k, the spatial characteristics, F^ A , F^ D ,F AD from the different encoders are concatenated in a multi-view latent variable space to form a composite spatial feature F^ AD , supplied to one or more decoders, here the D decoders RA and D RD . Similarly, the internal states of the RNN networks of the different stages q = 1, , . , Q of the encoders E RA ,E RD ,E AD , are concatenated to form a composite internal state transmitted to the corresponding stage of each of the decoders, as in Fig. 5. Thus, in the example shown, each of the decoders D RA ,D RD can benefit from temporal information coming not only from the corresponding encoder (E RA ,E RD ) but of the three encoders associated with the 3 views of the measurement tensor. The internal states q = 1, . . , Q are transmitted to the decoders by a temporal skip-layer connection common to all views, designated in the figure by TMVSC (Temporal Multi-View Skip Connection).

[0088] In the example shown, the decoder allows D RA to predict the position of objects in the scene in an RA space (distance, azimuth) while the D decoder RD allows us to predict the positions of these objects, otherwise to represent them in an RD space (distance, speed).

[0089] Fig. 7 shows in more detail the concatenation operation of composite internal states performed by the TMVSC connection of Fig. 6.

[0090] Internal states h^ D,q ,h AD,q , h R A,q RNN networks forming part of the q-rank stages of the respective encoders E RD ,E AD ,E RA are concatenated, where appropriate after a sub-sampling or over-sampling step to harmonize their dimensions to C x H x W, resulting in a concatenation result of size 3C x H x V / which is then subject to a 2D lxl convolution (channelwise pooling) to reduce the number of channels from 3C to C and therefore to use them within the decoder stages as in the first embodiment, as a composite internal state.

Claims

CLAIMS 1. System for detecting objects in a scene from radar data frames of this scene, said frames (r fe ) following each other temporally, said frames (r fe ) of radar data giving two pieces of information (RA, RD, AD), or three pieces of information (RAD), among information on distance (R), angle of arrival (A) and Doppler speed (D), characterized in that said system comprises at least one space-time encoder (E , E RD ,E AD ,E RA ) and at least one decoder ( , D RA ,D RD ), the space-time encoder being intended to receive said frames and comprising a (Q) plurality of cascaded stages (520,530), each stage comprising a convolutional neural network, CNN, and a recurrent neural network, RNN, arranged in series, the encoder being configured to provide, depending on its internal state, and at each new data frame r kreceived at a time k, a set E k of spatial characteristics and to update its internal state according to this new frame, said internal state being formed by the internal states of the RNN networks of the stages of the encoder, the decoder ( , D RA ,D RD ) using the set F fe spatial characteristics at time k, as well as optionally the internal state updated by the encoder, H k , to provide output information (Pk) relating to the objects present in the scene.

2. System for detecting objects in a scene according to claim 1, characterized in that the frames (r fe ) of radar data are obtained by projection of a radar measurement tensor into a 2-dimensional space or constitute 3-dimensional tensors.

3. System for detecting objects in a scene according to claim 1 or 2, characterized in that said output information (Pk) comprises at least one map, in particular a map giving a probability of presence of the objects at each point of the scene.

4. System for detecting objects in a scene according to any one of claims 1 to 3, characterized in that said output information comprises a mapping constituting a semantic segmentation of the scene in the form of an image and giving for each pixel of the image a class to which it belongs, from among a plurality of possible classes.

5. System for detecting objects in a scene according to one of the preceding claims, characterized in that the convolutional network of a stage or of each stage of the cascade of stages of the encoder is a separable convolution network, for example an Inverted Residual Bottleneck type network.

6. System for detecting objects in a scene according to one of the preceding claims, characterized in that the recurrent network of a stage or of each stage of the cascade of stages of the encoder is a separable convolution network, for example a ConvLSTM network, in particular a Bottleneck LSTM type network.

7. System for detecting objects in a scene according to claim 5 or 6, characterized in that the encoder comprises downstream of the cascade of stages a last stage (540) consisting of a convolutional network with separable convolutions, for example a network of the Inverted Residual Bottleneck type.

8. System for detecting objects in a scene according to one of the preceding claims, characterized in that the decoder comprises a plurality of cascaded stages (550,560,570), each stage of the decoder being configured to perform a transposed convolution.

9. System for detecting objects in a scene according to any one of the preceding claims, characterized in that at least one stage of the decoder (550,560,570) receives, via a skip-layer connection, a spatial characteristic provided by a corresponding stage of the encoder as well as optionally the internal state of the recurrent neural network belonging to this stage.

10. System for detecting objects in a scene according to one of claims 1 to 9, characterized in that it comprises a plurality of coders (E RD ,E AD ,E RA ), each encoder being intended to receive radar data frames obtained by projection of a radar measurement tensor (T RAD ) on a distinct space of dimension 2, each encoder being configured to provide, according to its internal state, and at each new data frame r kreceived at a time k, a set (Fk,RA; Fk,RD; Fk,AD) of spatial characteristics and to update its internal state according to this new frame; and said at least one decoder being configured to use said set of spatial characteristics (Fk RA ;Fk RD ;Fk AD ) at time k, as well as optionally the internal states updated by said encoders to provide the output information (Pk).

11. System for detecting objects in a scene according to claim 10, characterized in that at least one stage of the decoder receives, via a skip-layer connection, a composite spatial characteristic (Fk RAD ) resulting from the concatenation of spatial characteristics (Fk RA ;Fk RD ;Fk AD ) provided by corresponding stages of the same rank of the different encoders as well as optionally a composite internal state obtained from internal states (hk , h k f h k ) recurrent neural networks belonging to the corresponding stages of the same rank (q) of the different coders.

Images