Computerized management of the monitoring of a traffic environment so that a vehicle can circulate there autonomously

Abstract

The invention relates to a method for managing, by a computer device (100) integrated into observation equipment (1), the monitoring of a traffic environment so that a vehicle (2) can circulate therein autonomously. The invention also relates to a device (100) implementing such a method as well as observation equipment (1) comprising such a device. Figure for the abstract: 1

Description

Title of the invention: Computerized management of the monitoring of a traffic environment so that a vehicle can circulate there autonomously Technical field of the invention

[0001] The present invention relates to the field of systems that enable the autonomous circulation of vehicles. The invention relates in particular to a method for managing, by a computer device integrated into observation equipment, the monitoring of a traffic environment so that a vehicle can circulate therein autonomously. The invention also relates to a device implementing such a method. The invention applies to land motor vehicles, in particular robots or other transport shuttles intended to take charge of transport tasks within industrial or logistics environments. State of the prior art

[0002] It is known that obstacle detection is an essential element of autonomous traffic. Indeed, the implementation of autonomous traffic requires a precise understanding of the traffic environment in order to be able to determine a traffic zone in which a vehicle can circulate safely without colliding with an obstacle. To enable this understanding, the driver assistance systems that equip the most modern motor vehicles use on-board detection devices, in particular cameras or radars. In other words, the on-board perception of the traffic environment requires the integration of a plethora of detection devices to achieve performance levels that can guarantee the safety of the various road users.However, while the integration of such detection devices is justified for cars or other vehicles intended to circulate in a conventional road environment, the cost of these devices and their integration proves impossible when it comes to vehicles whose cost must remain minimal, such as those intended to take charge of transport tasks within logistics environments. In fact, such vehicles are generally devoid of any on-board detection device and, for these vehicles, autonomous circulation therefore remains today too expensive and too complex to implement. Furthermore, regardless of the vehicle in which detection devices are integrated, autonomous circulation is inevitably limited by physical constraints, the first of which is the perception capacity of the detection devices.Naturally, obstacles located outside the field of vision of the on-board detection devices cannot be detected and, in fact, their presence does not . can be deduced, which requires the use of methodologies in which sensing data generated at various times must be used and a good understanding of traffic flows must be achieved. And in this respect too, the known vehicles used to support transport tasks within logistics environments are not suitable to be able to assume such constraints. Summary of the invention

[0003] The invention aims to solve these problems. In particular, it aims to provide a solution to help enable a vehicle without any on-board detection device to nevertheless be able to circulate autonomously within a traffic environment. By this means, the invention aims to minimize the cost and complexity of implementing autonomous vehicle circulation, in particular for less advanced vehicles that are intended to take on transport tasks within industrial or logistics environments.

[0004] To achieve this objective, the invention relates, according to a first aspect, to a method for managing, by a computer device integrated into observation equipment, the monitoring of a traffic environment so that a vehicle can circulate therein autonomously, said equipment comprising a detection assembly consisting of a plurality of detection devices, including a lidar capable of scanning over 360° as well as several cameras, and a trigger element controlling the carrying out of detections by said detection devices, said device comprising an information processing unit, provided with one or more graphics processors, and a data storage medium configured to execute a synchronization module and a data processing module which operates at least one artificial neural network, the method comprising the steps of: i. obtaining detection data generated by the detection assembly based on a signal transmitted periodically by said trigger element, the internal clocks of all the detection devices having been previously synchronized by the synchronization module; ii. process said detection data by means of the data processing module in order to determine data characterizing at least one bounding box for each obstacle present in said traffic environment; and iii. cause a transmission of the data determined during step ii) to said vehicle.

[0005] According to a variant, step ii) may comprise a step consisting of determining data characterizing a semantic segmentation of an image generated by a camera so as to determine data characterizing at least one mask for each obstacle present in said traffic environment.

[0006] According to another variant, step ii) may comprise a step consisting of associating a point cloud generated by the lidar and said semantic segmentation in order to determine data characterizing a fragment of the point cloud for each obstacle present in said traffic environment.

[0007] According to yet another variant, step ii) may comprise the steps of: • filter said fragment based on a confidence value; • determine data characterizing an estimate of a difference between the real center of an obstacle and the center of the corresponding mask; and • determine data characterizing at least one parameter of a bounding box as a function of said deviation.

[0008] According to yet another variant, step ii) may comprise a step consisting of comparing a first bounding box determined using detection data generated by a first camera and a second bounding box determined using detection data generated by a second camera.

[0009] According to yet another variant, step ii) may comprise a step consisting of determining data characterizing at least one movement dynamics value for each obstacle present in said traffic environment by comparing bounding boxes determined using detection data generated at distinct times.

[0010] According to a second aspect, the invention relates to a device for managing the monitoring of a traffic environment so that a vehicle can circulate there autonomously, the device comprising at least one information processing unit, provided with one or more graphics processors, and a data storage medium, which are configured to execute a synchronization module and a data processing module in order to implement a method as described above.

[0011] According to a third aspect, the invention relates to a computer program comprising program code instructions for executing the steps of a method as described above when said program is executed by at least one processor.

[0012] According to a fourth aspect, the invention relates to a medium usable in a computer on which a program as described above is recorded.

[0013] According to a fifth aspect, the invention relates to observation equipment comprising a detection assembly consisting of a plurality of detection devices, including a lidar capable of scanning 360° and several cameras, and a trigger element controlling the carrying out of detections by said detection devices, said equipment further incorporating a device as described above. Brief description of the figures

[0014] Other characteristics and advantages of the invention will appear on examining the detailed description below, and the appended figures, in which:

[0015] [Fig-1] is a schematic illustration of the invention;

[0016] [Fig.2] is a functional diagram of a device according to the invention; and

[0017] [Fig.3] is a flowchart of the steps of a method according to the invention. Detailed description of the invention

[0018] [Fig.l] schematically illustrates the context of the invention. It shows an observation equipment 1 which performs the detection of obstacles located in a traffic environment in which it is installed, and which can communicate by means of conventional wireless communication networks and protocols with a vehicle 2 in order to transmit to the vehicle 2 data concerning the detection of obstacles in the traffic environment in order to enable the vehicle 2 to be able to circulate autonomously within the traffic environment. For this, the observation equipment 1 comprises a detection assembly 3 consisting of a plurality of detection devices, including a lidar 4 capable of scanning over 360° and several cameras 5. Depending on requirements, the cameras are for example arranged on the observation equipment 1 so that their cumulative fields of vision cover the space all around the observation equipment 1.The observation equipment 1 further comprises a trigger element 6, which controls the simultaneous performance of detections by the detection devices of the detection assembly 3, a receiver 7 which interacts with a satellite positioning system or a remote reference clock, and a radiofrequency signal communication system 8, which makes it possible to communicate with remote communicating entities, in particular the vehicle 2, by means of conventional communication networks and protocols (eg 3 / 4 / 5G). The trigger element 6 is preferably an electronic device which is capable of periodically producing detection trigger signals for each of the detection devices of the detection assembly 3.Advantageously, the observation equipment 1 also comprises a device for managing the monitoring of a traffic environment 100 within the meaning of the present invention, as described below, which implements a method for managing the monitoring of a traffic environment within the meaning of the present invention, as described below.

[0019] When implementing the method, the detection devices of the detection assembly 3 carry out detections on the basis of a signal transmitted by the triggering element 6 in order to generate detection data which are acquired by the device 100 according to the invention. These detection data are then processed by the device 100 according to the invention to determine at least one bounding box (i.e. in three dimensions) for each obstacle present in the traffic environment covered by the observation equipment 1. These obstacle identification and positioning information are finally transmitted to the vehicle 2, on board which a navigation system capable of managing the guidance and propulsion of the vehicle can use them to implement the autonomous circulation of the vehicle 2. As will be seen below, this is how the invention contributes to allowing a vehicle devoid of any detection device allowing it to perceive its environment to nevertheless be able to circulate autonomously without risking colliding with an obstacle present in the traffic environment located near the observation equipment 1.

[0020] The device 100 for managing the monitoring of a traffic environment according to the invention is illustrated in [Fig. 2]. It is basically a computer device which comprises at least one information processing unit 101, with one or more graphics processors, a data storage medium 102, on which is recorded in particular a program which comprises program code instructions for executing the steps of the method according to the invention described later, and an input and output interface 103 allowing the reception and transmission of data. Advantageously, the device 100 according to the invention executes, by means of its information processing unit 101 and its data storage medium 102, at least one synchronization module 104 and one data processing module 105, the latter operating at least one artificial neural network.

[0021] Preferably, the device 100 according to the invention is an independent computer device, for example a computer, and it interacts via its input and output interface 103 and by means of a wired communication network (e.g. Ethernet) - shown in [Fig.l] by the two-way arrows - with the detection assembly 3, with the triggering element 6, with the receiver 7 and with the radiofrequency signal communication system 8. As a result, the device 100 according to the invention can, in particular, obtain detection data generated by the detection assembly 3, obtain location data generated by the location apparatus 3 and it can cause a transmission of data to the vehicle 2.

[0022] According to the invention, all the elements described above contribute to enabling the implementation of a method for managing the monitoring of a traffic environment so that a vehicle can circulate there autonomously, as described below in connection with [Fig.3].

[0023] [Fig. 3] illustrates by means of a flowchart the steps of the process according to the invention. According to a first step 201 of the method according to the invention, the device 100 according to the invention obtains detection data generated by the detection assembly 3 as a function of a signal transmitted periodically by said triggering element 6.In order for these detection data to be able to be subsequently processed appropriately, in particular to ensure that the detection devices of the detection set 3 carry out detections simultaneously, the device 100 according to the invention interacts with the triggering element 6, which ensures that all the detections by each of the detection devices of the detection set 3 are carried out simultaneously, while it has previously used its synchronization module to allow the synchronization of the internal clocks of all the detection devices of the detection set 3, for example by using the receiver 7 so as to obtain a reference time from a satellite positioning system or a remote reference clock.Thus, during this first step 201 of the method, the device 100 according to the invention obtains detection data which are generated simultaneously in relation to a reference schedule common to all the detection devices of the detection assembly 3. As will be seen below, this will advantageously enable it to identify and locate obstacles in the traffic environment located near the observation equipment 1, but also to be able to determine parameters relating to the movement dynamics for each obstacle identified.

[0024] According to a second step 202 of the method according to the invention, the device 100 according to the invention processes the detection data obtained during the previous step by means of its data processing module 105 in order to determine data characterizing at least one bounding box for each obstacle present in said traffic environment. To do this, the device 100 according to the invention proceeds in the following manner.

[0025] First, the data processing module 105 determines data characterizing a semantic segmentation of an image generated by a camera 5 so as to determine data characterizing at least one mask for each obstacle present in said traffic environment. For this, the image generated by the camera feeds a convolutional neural network trained to detect different types of obstacles and provide their semantic masks in the image, i.e. identify the pixels of the image belonging to each obstacle. This step can be carried out using a two-dimensional semantic segmentation framework, for example “Mask R-CNN”.

[0026] Secondly, the data processing module 105 associates a point cloud generated by the lidar 4 and the semantic segmentation previously determined during the previous step 201 in order to determine data characterizing a fragment of the point cloud for each obstacle present in said traffic environment. In other words, once the obstacles have been detected and segmented in an image generated by a camera 5, the points of the global point cloud generated by the lidar 4 which belong to each obstacle (i.e. the points which belong to a portion of the image corresponding to an obstacle identified by the semantic segmentation) are identified and segmented into fragments of the global point cloud. In this way, the detection information coming from a camera 5 and that coming from the lidar 4 are advantageously associated, this association requiring that the position of the camera with respect to the lidar be predefined.

[0027] Thirdly, the data processing module 105 proceeds, for each fragment of point cloud corresponding to each identified obstacle, by first filtering the fragment of point cloud according to a confidence level (i.e. a pre-established confidence value), then it determines data characterizing an estimate of a difference between the real center of an obstacle and the center of its corresponding mask, and finally it determines data characterizing at least one parameter of a bounding box according to this difference.

[0028] More specifically, during filtering, the points that correspond to a mask established during the semantic segmentation are used as input to a three-dimensional segmentation network, whose function is to refine the representation of the obstacles in the point cloud by filtering out all the outliers that could have been classified as obstacle points during the two-dimensional semantic segmentation. Thus, for each point of the masked obstacles, a confidence level is estimated, indicating whether the point belongs to the corresponding obstacle or, on the contrary, whether it should be deleted. This network therefore performs a binary classification to differentiate foreground points and background points. Then, an approximate estimation of the center of each obstacle is carried out via the so-called T-Net network.This network, based on the PointNet architecture, can calculate an estimate of the distance between the center of the points corresponding to a mask and the real center of the corresponding obstacle. Once this distance is determined, the masked points are expressed in a new reference system established according to the distance, and this data is used to feed another artificial neural network which determines bounding boxes (i.e. in three dimensions) oriented for all obstacles. Like the previous ones, this last network is based on a PointNet type architecture. The output of the fully convolutional layers located after the feature coding block correspond to the parameters of the bounding boxes of the obstacles, including their dimensions, differences between real centers and more refined estimated centers as well as their orientations.

[0029] Fourth, the data processing module 105 compares a first box bounding box determined using detection data generated by one camera and a second bounding box determined using detection data generated by a second camera. More specifically, the detections from each camera-lidar pair are aggregated and undergo a non-maximum suppression procedure to avoid duplicates and preserve the most reliable estimate for each obstacle. Given a correct extrinsic calibration of all sensors involved, a complete map representing the instantaneous detections of all obstacles present in the traffic environment located near observation equipment 1 is available at this point.

[0030] Alternatively, such a step of comparing the detections carried out using different cameras can also take place at another time, for example following the semantic segmentation described above. In this case, it is carried out directly from the obstacle masks established using different cameras.

[0031] Fifth, the data processing module 105 determines data characterizing at least one movement dynamics value for each obstacle present in the traffic environment by comparing bounding boxes determined using detection data generated at distinct times. In this way, consistency of detections is ensured, in particular by associating earlier detections with more recent detections, so that the movement of a specific obstacle can be estimated on the basis of a history of detections. Furthermore, in the event of erroneous detection, it is possible to maintain consistency of tracking, i.e. to provide an output for an obstacle on the basis of a prediction of its movement.To do this, the processing module 105 uses at least one unscented Kalman filter (UKF) and a data association technique that uses the Mahalanobis distance to establish a correspondence between detections made at distinct times.

[0032] Finally, according to a third step 203 of the method according to the invention, the device 100 according to the invention causes a transmission of the data characterizing at least one bounding box for each obstacle present in said traffic environment, those determined during the previous step 202, to the vehicle 2. In this way, a navigation system that the latter comprises and which is able to manage the guidance and propulsion of the vehicle according to received detection data can allow the vehicle 2 which is devoid of any detection means to nevertheless be able to circulate within the traffic environment located near the observation equipment 1. To implement this final step of the method, the device 100 according to the invention conventionally interacts with the radiofrequency communication system 8 of the observation equipment 1, which ensures the transmission data to vehicle 2.

[0033] Thus, thanks to the method and the device according to the invention described above, a solution is provided to enable at low cost robust obstacle detection which can allow the safe autonomous circulation of vehicles lacking means of detecting their environment, in particular low-tech vehicles such as robots or other autonomous shuttles intended to take charge of transport tasks within industrial or logistics environments.

Claims

Claims

1. Method for managing, by a computer device (100) integrated into observation equipment (1), the monitoring of a traffic environment so that a vehicle (2) can circulate therein autonomously, said equipment comprising a detection assembly (3) consisting of a plurality of detection devices, including a lidar (4) capable of scanning over 360° as well as several cameras (5), and a trigger element (6) controlling the carrying out of detections by said detection devices, said device comprising an information processing unit (101), provided with one or more graphics processors, and a data storage medium (102) configured to execute a synchronization module (104) and a data processing module (105) which operates at least one artificial neural network, characterized in that the method comprises the steps of: i.obtaining detection data generated by the detection assembly (3) as a function of a signal transmitted periodically by said trigger element (6), the internal clocks of all the detection devices having been previously synchronized by the synchronization module (104); ii. processing said detection data by means of the data processing module (105) in order to determine data characterizing at least one bounding box for each obstacle present in said traffic environment; and iii. causing a transmission of the data determined during step ii) to said vehicle (2).

2. Method according to claim 1, characterized in that step ii) comprises a step consisting of determining data characterizing a semantic segmentation of an image generated by a camera so as to determine data characterizing at least one mask for each obstacle present in said traffic environment.

3. Method according to claim 2, characterized in that step ii) comprises a step of associating a point cloud generated by the lidar and said semantic segmentation in order to determine data characterizing a fragment of the point cloud for each obstacle present in said traffic environment.

4. Method according to claim 3, characterized in that step ii) comprises the steps of: • filtering said fragment according to a confidence value; • determining data characterizing an estimate of a difference between the real center of an obstacle and the center of the corresponding mask; and • determining data characterizing at least one parameter of a bounding box according to said difference.

5. Method according to one of the preceding claims, characterized in that step ii) comprises a step consisting of comparing a first bounding box determined using detection data generated by a first camera and a second bounding box determined using detection data generated by a second camera.

6. Method according to one of the preceding claims, characterized in that step ii) comprises a step consisting of determining data characterizing at least one movement dynamics value for each obstacle present in said traffic environment by comparing bounding boxes determined using detection data generated at distinct times.

7. Device (100) for managing the monitoring of a traffic environment so that a vehicle can circulate therein autonomously, characterized in that the device comprises at least one information processing unit (101), provided with one or more graphics processors, and a data storage medium (102), which are configured to execute a synchronization module and a data processing module, which operates at least one artificial neural network, in order to implement a method according to any one of the preceding claims.

8. A computer program comprising program code instructions for executing the steps of a method according to any one of claims 1 to 6 when said program is executed by at least one processor.

9. Support usable in a computer, characterized in that a program according to claim 8 is recorded therein.

10. Observation equipment (1) comprising a detection assembly (3) consisting of a plurality of detection devices, including a lidar (4) capable of scanning 360° and several cameras (5), and a trigger element (6) controlling the carrying out of detections by said detection devices, characterized in that said equipment integrates a device (100) according to claim 7.

Images