Method and system for the remote supervision of a fleet of self-driving vehicles

Abstract

The invention relates to a method for remotely supervising a fleet of self-driving vehicles (10, 20, 30, 40), wherein each vehicle comprises sensors, a storage unit and a self-driving unit that is able to detect an obstacle and to stop the vehicle if an obstacle is detected. The method comprises: subsequent to the vehicle stopping, establishing a wireless communication session between the vehicle and a remote supervision system (50); during the communication session: transmitting, from the vehicle to the supervision system: information currently acquired by the sensors; and information previously acquired by the sensors stored in a storage unit of the vehicle; transmitting, from the supervision system to the vehicle, a restart command or a restart prevention command; and subsequently closing the communication session.

Description

[0001] Method and system for remote supervision of a fleet of autonomous vehicles

[0002] TECHNICAL FIELD

[0003] The field of the invention is that of vehicles with delegated driving intended for the transport of goods and / or people, also called autonomous vehicles. The invention relates more particularly to securing the operation of a fleet of vehicles with delegated driving during operation by means of remote supervision and assistance of the vehicles.

[0004] PRIOR ART

[0005] A vehicle with delegated driving is equipped with a battery of sensors including in particular exteroceptive sensors (cameras, Lidars for example) each capable of providing information on the environment in which the vehicle is located. Such a vehicle also includes an automated driving unit configured to develop, on the basis of the information obtained by the battery of sensors, a spatio-temporal trajectory of the vehicle in line with the operational missions entrusted to it.

[0006] Unlike an automated metro, for example, a vehicle with delegated driving operates in an open environment. The environment in which the vehicle operates can therefore change very quickly and place the vehicle in a transiently disturbed situation. A change in the environment can induce an exit from the functional design domain of the vehicle. This exit must be detected in order to be able to fold the vehicle into a stable state of safe stopping. Thus, the vehicle's automated driving unit is particularly capable of detecting a surrounding obstacle and bringing the vehicle to a stop if a surrounding obstacle is detected.

[0007] Currently operated driver-assisted vehicles request a takeover or assistance from a human driver present on board for safety reasons to respond to an exit from their functional design domain. Thus, it is the human driver who, following a safety stop, makes the decision whether or not to restart the vehicle in automated driving mode. Future driver-assisted vehicles, particularly those at level 4 according to the SAE (Society of Automotive Engineers) standard, will be operated without a human driver on board. For these vehicles, there is a need for a solution to ensure a resumption of automated driving following a safety stop automatically initiated by a vehicle as a result of the vehicle detecting an exit from its functional design domain.

[0008] STATEMENT OF THE INVENTION

[0009] The invention aims to meet this need and proposes for this purpose a method for remote supervision of a fleet of vehicles with delegated driving where each vehicle in the fleet comprises exteroceptive sensors each capable of providing information on the environment of the vehicle, a recording unit capable of storing the information obtained by the exteroceptive sensors and an automated driving unit capable of detecting a surrounding obstacle on the basis of the information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle.

[0010] The method comprising the following steps:

[0011] - following the stopping of a vehicle by its automated driving unit, establishment of a wireless communication session between the vehicle and a remote vehicle supervision system;

[0012] - during the wireless communication session:

[0013] - transmission from the vehicle to the supervision system:

[0014] ■ information currently acquired by the vehicle's exteroceptive sensors constituting information on the vehicle's real-time environment; and

[0015] ■ information stored in the vehicle recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors and constituting information on the vehicle's past environment; transmission from the supervision system to the vehicle of a restart command or a restart prohibition command; then

[0016] - closing the wireless communication session.

[0017] Some preferred but non-limiting aspects of this method are as follows: it further comprises the selection of a time window for which the information on the past environment of the vehicle is transmitted to the supervision system; the time window includes the date of stopping of the vehicle; the establishment of the wireless communication session comprises the transmission by the vehicle to the supervision system of a stopping signal; the stopping signal is transmitted when a duration of the stopping of the vehicle exceeds a threshold; the stopping signal carries information relating to a cause of the stopping of the vehicle; the stopping signal carries information relating to a category of the surrounding obstacle;the exteroceptive sensors comprise a camera, the transmission of information on the real-time environment of the vehicle comprises the transmission of a video stream currently captured by the camera and the transmission of information on the past environment of the vehicle comprises the transmission of a video extract previously captured by the camera and stored in the recording unit of the vehicle; it further comprises the display, on a display device of the supervision system, of information on the real-time environment of the vehicle and information on the past environment of the vehicle, the information on the real-time environment of the vehicle being displayed on the right of the display device and the information on the past environment of the vehicle being displayed on the left of the display device;it further includes the transmission by the supervision system to the vehicle of a horn command or a command to flash the headlights; it further includes the transmission by the supervision system of an intervention request to a local operator.;

[0018] BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other aspects, aims, advantages and characteristics of the invention will appear better on reading the following detailed description of preferred embodiments thereof, given by way of non-limiting example, and made with reference to the appended drawings in which:

[0020] - Figure 1 is a diagram illustrating a fleet of vehicles with driving delegation and a remote fleet supervision system;

[0021] - Figure 2 is a diagram illustrating a possible distribution of information of different types on a display device of the supervision system.

[0022] DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

[0023] With reference to Figure 1, the invention relates to a method for remote supervision of a fleet of vehicles with driving delegation 10, 20, 30, 40. The vehicles in the fleet can be operated on the same site or, on the contrary, be deployed on different sites.

[0024] Each vehicle in the fleet includes exteroceptive sensors, each capable of providing information about the vehicle's environment. Each vehicle in the fleet may also include proprioceptive sensors, each capable of providing information about the vehicle's state.

[0025] Each vehicle in the fleet also has a recording unit capable of storing the information obtained by the various sensors. The recording unit can thus store: raw obstacle detection measurements, position information from the fusion of different data streams from, for example, Lidars, an inertial unit and a satellite positioning system, video streams captured by cameras, for example four exterior cameras (one camera imaging the front of the vehicle, one camera imaging the rear of the vehicle, one camera imaging the right of the vehicle and one camera imaging the left of the vehicle) and an interior camera imaging the vehicle's passenger compartment.

[0026] The recording unit may further store: vehicle commands, such as propulsion, braking, steering, emergency braking, lighting or door position commands, vehicle status information, such as its speed or engine temperature; events, such as an emergency stop of the vehicle or a sensor failure.

[0027] Each vehicle in the fleet also has an automated driving unit capable of detecting a surrounding obstacle based on information obtained by exteroceptive sensors and of stopping the vehicle if a surrounding obstacle is detected.

[0028] In order to ensure the resumption of service in automated driving mode while ensuring safety, the method according to the invention comprises, following the stopping of a vehicle in the fleet by its automated driving unit, the establishment of a wireless communication session between the stopped vehicle and a supervision system 50 remote from the vehicle. This communication session can in particular be carried out via a cloud computing service 60. This communication session is therefore not established continuously between the vehicle and the remote supervision system, but as a consequence of stopping the vehicle, which makes it possible to limit the volume of data exchanged between the vehicle and the supervision system.

[0029] During the wireless communication session, the vehicle transmits to the remote monitoring system both information currently acquired by the vehicle's exteroceptive sensors (constituting information on the vehicle's real-time environment) and information stored in the vehicle's recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors (constituting information on the vehicle's past environment).

[0030] Then, the remote monitoring system transmits a restart command or a restart prohibition command to the vehicle. The wireless communication session is then interrupted.

[0031] As previously indicated, the exteroceptive sensors comprise cameras and in one possible embodiment, the transmission of information on the real-time environment of the vehicle comprises the transmission of a video stream currently captured by one or more of the cameras and the transmission of information on the past environment of the vehicle comprises the transmission of a video extract previously captured by the camera(s) and stored in the recording unit of the vehicle.

[0032] In another embodiment, which can be implemented independently or in addition to the previous one, the transmission of information on the real-time environment of the vehicle and on the past environment of the vehicle comprises the transmission of data from one or more exteroceptive sensors other than the cameras, for example information from a Lidar or a radar.

[0033] The monitoring system typically includes a display device on which information on the vehicle's real-time environment and information on the vehicle's past environment are displayed. This typically involves displaying one or more real-time video streams as well as one or more past video clips. When information from other types of sensors is fed back, this information is formatted to provide a human representation of the vehicle's perception.

[0034] An operator of the supervision system is thus able to view information on the vehicle's past environment, which allows him to contextualize what happened before the stop and thus understand the reasons for the stop. This same operator is also able to view the vehicle's real-time environment, which allows him to check the presence or absence of obstacles. Observing information on the past environment also allows him to resolve ambiguities such as, for example, the non-visualization of a pedestrian in the real-time information while the analysis of past information makes it possible to understand that the pedestrian is under the vehicle's chassis. Depending on his analysis of the displayed information, the operator authorizes or prohibits the restart of the vehicle in automated driving mode.

[0035] For example, the operator can thus verify that the obstacle detected by the vehicle is a false positive (case of a cloud of midges, projection of grass by a mower operating at the edge of a traffic lane, a cloud of dust or bags and cardboard boxes on the lane) and that it is thus possible to ignore it. The operator can also ensure that a disturbing obstacle has been removed from the lane or decide that a detected obstacle is not disturbing in that it is possible to go around it (for example, a vehicle parked on the hard shoulder). The operator can also ensure that a moving obstacle (a person, an animal) no longer presents a danger.

[0036] As shown in Figure 2, the display device can be subdivided into several display members or sections, for example by means of several screens. In one possible embodiment, four screens are used: three screens arranged horizontally (a left screen EG, a central screen EC and a right screen ED) and an upper screen ES placed above the central screen EC.

[0037] The information about the real-time environment of the vehicle can be displayed on the right of the display device (e.g. on the right screen ED) and the information about the past environment of the vehicle can be displayed on the left of the display device (e.g. on the left screen ED). This presentation mode, with the past on the left and the real time on the right, helps the operator to carry out his task consisting in deciding whether or not to allow the restart of the vehicle in automated driving mode.

[0038] In a possible embodiment using four exterior cameras and one interior camera, each of the left and right screens is subdivided into five zones: a central zone displaying the information captured by the interior camera, an upper zone displaying the information captured by the exterior camera imaging the front of the vehicle, a lower zone displaying the information captured by the exterior camera imaging the rear of the vehicle, a left side zone displaying the information captured by the exterior camera imaging the left of the vehicle and a right side zone displaying the information captured by the exterior camera imaging the right of the vehicle.

[0039] The EC central screen can be used to display and / or record information on the management and planning of the fleet of vehicles under supervision. Commands, instructions or intervention requests can be placed via it.

[0040] The upper ES screen placed above the central screen can be used to display a map of a fleet vehicle deployment site, for example with a representation of the route followed by each of the vehicles.

[0041] The display device thus allows the operator to view the current situation (access to vehicle videos, to a representation for humans of the vehicle's perception), to review the scene before stopping the vehicle (in the form of videos, of the representation of the perception), to pass commands to the vehicle (restart command in automated driving mode, restart prohibition command, horn command, headlight flash command), to pass instructions (such as limiting the speed on the route) or to configure the operation of the vehicle (times for reporting a stop alert that can be configured according to the location or service, definition of zones where any stop must generate a high safety alert, definition of specific danger zones).

[0042] In one possible embodiment, the supervision method comprises the selection, for example by the aforementioned operator, of a time window for which the information on the past environment of the vehicle is transmitted to the supervision system. This time window preferably includes the date of stopping the vehicle. This time window typically covers a period before the date of stopping (for example of a duration of 10 seconds) and a period after the date of stopping (for example of a duration of 20 seconds).

[0043] In one possible embodiment, establishing the wireless communication session comprises the transmission by the vehicle to the supervision system of a shutdown signal. This shutdown signal may in particular be transmitted when a duration of the vehicle shutdown exceeds a threshold. This threshold is preferably configurable, for example depending on the areas of the site (case of an area near a stop sign for example where the threshold will typically be higher).

[0044] The stopping signal preferably carries information relating to a cause of the vehicle stopping (for example the presence of a fixed obstacle completely obstructing the lane or the presence of a moving obstacle), which cause is determined by the automated driving unit of the vehicle. The stopping signal also preferably carries information relating to a category of the surrounding obstacle (for example from an object categorization algorithm and of the type the obstacle is a person with a probability of 90%), which category is determined by the automated driving unit of the vehicle.

[0045] A first use case of the monitoring system concerns a badly parked car parked in a double lane on the path of an autonomous vehicle. The vehicle automatically stops in front of this car and informs the monitoring system. The monitoring system operator views real-time videos and past video clips. The monitoring system operator can order the vehicle to honk or flash its headlights and use the real-time videos to verify that the initially badly parked car is no longer interfering with the operation of the vehicle.

[0046] Another use case involves an object falling from a car. The vehicle automatically stops in front of this object and informs the monitoring system. The monitoring system operator views real-time videos and past video clips. The monitoring system operator can request the intervention of a local operator to remove the object from the traffic lane. The monitoring system operator can verify the actual removal of the object in the real-time videos and initiate an automated restart of the vehicle.

[0047] Yet another use case concerns the passage of a pedestrian or an animal near the vehicle. The vehicle stops automatically in front of this moving object and informs the monitoring system. The monitoring system operator views real-time videos and past video clips. He can order the vehicle to honk. Once the traffic lane is clear and after consulting past video clips to understand where the pedestrian or animal has gone, the operator can authorize an automated restart of the vehicle.

[0048] Another use case is the loss of a sensor, for example due to deliberate damage. The vehicle stops automatically following this loss and informs the monitoring system. The monitoring system operator views real-time videos and past video clips. He can request the intervention of a local operator.

[0049] In a possible embodiment, each vehicle is further provided with an accident situation detection unit configured to detect, on the basis of the information provided by the sensors, warning signs of incidents. This unit is for example capable of identifying a repetition of behaviors not foreseen in the conditions of deployment of the vehicle, such as a repetition of sudden braking of the vehicle in the same zone. These sudden brakings are for example the consequence of the installation of a new pedestrian crossing on the path of the vehicle, of the recurring presence of vehicles regularly parked just before a pedestrian crossing inducing occlusion and sudden braking or even of the recurring presence of badly parked vehicles on the path of the vehicle inducing slowdowns.These warning signs thus detected can be transmitted to the supervision unit which, for example by means of an analysis of the videos captured by the vehicle's cameras, can understand their origins. An action to reduce the occurrences of the risk can then be launched, for example by raising awareness among certain recurring delivery drivers, positioning posts to reduce the occlusion on the pedestrian crossing, requesting the installation of speed bumps, requesting the rectification of the layout of the white stripes to influence the trajectories or even updating the conditions of deployment of the vehicle with, for example, the inclusion in the route of a new pedestrian crossing.

[0050] The invention is not limited to the method as previously described but also extends to the remote supervision system previously described and in particular to a system comprising a processor configured, during a wireless communication session established with one of the vehicles in the fleet following the stopping of the vehicle by its automated driving unit, to: control the reception from the vehicle of information on the real-time environment of the vehicle and information on the past environment of the vehicle; transmit to the vehicle a restart command or a restart prohibition command.

[0051] Such a supervision system is advantageously equipped with the display device described above.

[0052] Thus, one possibility would be the implementation of a method for remotely supervising a fleet of vehicles with driving delegation 10, 20, 30, 40, the vehicles in the fleet being able to be operated on the same site or be deployed on different sites, where each vehicle in the fleet comprises exteroceptive sensors. In addition, each vehicle may comprise proprioceptive sensors, each capable of providing information on the environment and / or the state of the vehicle, a recording unit capable of storing the information obtained by the sensors, by data fusion, the vehicle commands, the status of the vehicle, events, and an automated driving unit capable of detecting a surrounding obstacle on the basis of the information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle. The method comprises the following steps:

[0053] - following the stopping of a vehicle by its automated driving unit, establishment of a wireless communication session between the vehicle and a remote supervision system 50 of the vehicle;

[0054] - during the wireless communication session:

[0055] - transmission from the vehicle to the supervision system:

[0056] ■ information currently acquired on the vehicle's real-time environment; and

[0057] ■ information stored in the vehicle recording unit corresponding to previously acquired information and constituting information on the vehicle's past environment; transmission from the supervision system to the vehicle of a restart command or a restart prohibition command; then

[0058] - closing the wireless communication session.

[0059] A system for supervising a fleet of vehicles with driving delegation may also be proposed. The vehicles in the fleet may be operated on the same site or deployed on different sites, where each vehicle in the fleet includes exteroceptive sensors and may include proprioceptive sensors, each capable of providing information on the environment and / or the state of the vehicle, a recording unit capable of storing the information obtained by the sensors, by data fusion, the vehicle commands, the status of the vehicle, events, and an automated driving unit capable of detecting a surrounding obstacle on the basis of the information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle.The system further comprises a processor configured, during a wireless communication session established with one of the vehicles in the fleet following the stopping of the vehicle by its automated driving unit, to: control the reception from the vehicle of information currently acquired by the vehicle's sensors constituting information on the real-time environment of the vehicle and of information stored in the vehicle's recording unit corresponding to information previously acquired by the vehicle's sensors and constituting information on the vehicle's past environment; transmit to the vehicle a restart command or a restart prohibition command.

[0060] This possible method and system can be advantageously supplemented by the following features taken alone or in any of the technically feasible combinations. A use case of the supervision system concerns the loss of a sensor causing the vehicle to stop automatically following this loss and to inform the supervision system.

[0061] Each vehicle is also equipped with an accident detection unit configured to detect warning signs of incidents. These warning signs are transmitted to the monitoring unit, which can understand their origins.

[0062] The unit is able to identify a repetition of behaviors not anticipated in the vehicle deployment conditions.

[0063] The unit is able to identify a repetition of sudden braking of the vehicle in the same area.

[0064] An action to reduce the occurrence of the risk can be launched.

[0065] The action to reduce risk occurrences consists of one or more of the following actions:

[0066] - raise awareness among certain recurring delivery people;

[0067] - position posts to reduce occlusion on the pedestrian crossing;

[0068] - request the installation of speed bumps;

[0069] - request the rectification of the white stripes;

[0070] - update the vehicle deployment conditions.

[0071] The recording unit stores video streams of the vehicle's interior.

[0072] When information from other types of sensors than those providing video streams is fed back, it is formatted to provide a human representation of the vehicle's perception.

[0073] The display device is subdivided into several display organs or sections, for example by means of several screens.

[0074] The display device of the supervision system is used to display and / or record information for controlling and planning the fleet of vehicles under supervision, display a map of a fleet vehicle deployment site, and a representation of the route followed by each of the vehicles.

[0075] The method may further comprise the transmission by the supervision system to the vehicle of directives or the parameterization of the operation of the vehicle.

Claims

DEMANDS 1. A method for remotely supervising a fleet of autonomously driven vehicles (10, 20, 30, 40) where each vehicle in the fleet includes exteroceptive sensors, each capable of providing information on the vehicle's environment, a recording unit capable of storing the information obtained by the exteroceptive sensors, and an automated driving unit capable of detecting a surrounding obstacle based on the information obtained by the exteroceptive sensors and bringing the vehicle to a stop if a surrounding obstacle is detected, the method comprising the following steps: - following the stopping of a vehicle by its automated driving unit, establishment of a wireless communication session between the vehicle and a remote vehicle supervision system (50); - during the wireless communication session: - transmission from the vehicle to the monitoring system: ■ of information currently acquired by the vehicle's exteroceptive sensors, constituting information about the vehicle's real-time environment; and ■ information stored in the vehicle's recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors and constituting information about the vehicle's past environment; transmission from the supervisory system to the vehicle of a restart command or a restart prohibition command; then - closing the wireless communication session.

2. A method according to claim 1, further comprising the selection of a time window for which information on the past environment of the vehicle is transmitted to the monitoring system.

3. Method according to claim 2, wherein the time window includes the date of stopping the vehicle.

4. A method according to any one of claims 1 to 3, wherein the establishment of the wireless communication session includes the transmission by the vehicle to the supervisory system of a stop signal.

5. Method according to claim 4, wherein the stop signal is transmitted when a duration of the vehicle being stopped exceeds a threshold.

6. A method according to any one of claims 4 and 5, wherein the stop signal carries information relating to a cause of the vehicle stopping.

7. A method according to any one of claims 4 to 6, wherein the stop signal carries information relating to a category of the surrounding obstacle.

8. A method according to any one of claims 1 to 7, wherein the exteroceptive sensors comprise a camera, the transmission of information on the real-time environment of the vehicle comprises the transmission of a video stream currently captured by the camera and the transmission of information on the past environment of the vehicle comprises the transmission of a video extract previously captured by the camera and stored in the vehicle's recording unit.

9. A method according to any one of claims 1 to 8, further comprising displaying, on a display device of the supervisory system, information on the real-time environment of the vehicle and information on the past environment of the vehicle, the information on the real-time environment of the vehicle being displayed on the right of the display device and the information on the past environment of the vehicle being displayed on the left of the display device.

10. A method according to any one of claims 1 to 9, further comprising the transmission by the supervisory system to the vehicle of a horn command or a command to perform a headlight flash.

11. A method according to any one of claims 1 to 10, further comprising the transmission by the supervisory system of a request for intervention to a local operator.

12. Supervision system for a fleet of autonomously driven vehicles where each vehicle in the fleet includes exteroceptive sensors, each capable of providing information on the vehicle's environment, a recording unit capable of storing the information obtained by the exteroceptive sensors, and an automated driving unit capable of detecting a surrounding obstacle based on the information obtained by the exteroceptive sensors and of bringing the vehicle to a stop if a surrounding obstacle is detected, the system including a configured processor, during a wireless communication session established with one of the vehicles in the fleet following the vehicle being brought to a stop by its automated driving unit,to: control the reception from the vehicle of information currently acquired by the vehicle's exteroceptive sensors constituting information on the vehicle's real-time environment, and of information stored in the vehicle's recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors and constituting information on the vehicle's past environment; transmit to the vehicle a restart command or a restart prevention command.

Images