Method and system for remote monitoring of a fleet of autonomous vehicles

Abstract

The invention relates to a method for remotely supervising a fleet of autonomously driven vehicles (10, 20, 30, 40) where each vehicle includes sensors, a recording unit, and an automated driving unit capable of detecting an obstacle and bringing the vehicle to a stop upon detection of an obstacle. The method comprises: following the stopping of a vehicle, establishing a wireless communication session between the vehicle and a remote supervision system (50); during the communication session: transmitting from the vehicle to the supervision system: information currently acquired by the sensors; and information previously acquired by the sensors stored in the vehicle's recording unit; transmitting from the supervision system to the vehicle a restart command or a restart prohibition command; and then closing the communication session. Figure 1

Description

Title of the invention: Method and system for remote supervision of a fleet of autonomous vehicles Technical field

[0001] The field of the invention is that of vehicles with driving delegation intended for the transport of goods and / or people, also called autonomous vehicles. The invention relates more particularly to securing the operation of a fleet of vehicles with driving delegation during operation by means of remote supervision and assistance of the vehicles. Prior art

[0002] A vehicle with delegated driving is equipped with a battery of sensors including in particular exteroceptive sensors (cameras, Lidars for example) each capable of providing information on the environment in which the vehicle is located. Such a vehicle also comprises an automated driving unit configured to develop, on the basis of the information obtained by the battery of sensors, a spatio-temporal trajectory of the vehicle in line with the operational missions entrusted to it.

[0003] Unlike an automated metro, for example, a vehicle with delegated driving operates in an open environment. The environment in which the vehicle operates can thus change very quickly and place the vehicle in a transiently disturbed situation. A change in the environment can induce an exit from the functional design domain of the vehicle. This exit must be detected in order to be able to fold the vehicle into a stable state of safe stopping. Thus, the automated driving unit of the vehicle is in particular capable of detecting a surrounding obstacle and of bringing the vehicle to a stop in the event of detection of a surrounding obstacle.

[0004] The vehicles with driving delegation that are currently in operation make a request for takeover or assistance by a human driver present on board for safety reasons to respond to an exit from their functional design domain. It is thus the human driver who, following a safety stop, makes the decision whether or not to restart the vehicle in automated driving mode.

[0005] Future vehicles with delegated driving, particularly those at level 4 according to the SAE (Society of Automotive Engineers) standard, will be operated without a human driver on board. For these vehicles, there is a need for a solution to ensure the resumption of automated driving following a safety stop. automatically engaged by a vehicle as a result of the vehicle detecting an exit from its functional design range. Statement of the invention

[0006] The invention aims to meet this need and proposes for this purpose a method for remote supervision of a fleet of vehicles with delegated driving where each vehicle in the fleet comprises exteroceptive sensors each capable of providing information on the environment of the vehicle, a recording unit capable of storing the information obtained by the exteroceptive sensors and an automated driving unit capable of detecting a surrounding obstacle on the basis of the information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle.

[0007] The method comprising the following steps: - following the stopping of a vehicle by its automated driving unit, establishment of a wireless communication session between the vehicle and a remote vehicle supervision system; - during the wireless communication session: - transmission from the vehicle to the supervision system: • information currently acquired by the vehicle's exteroceptive sensors constituting information on the vehicle's real-time environment; and • information stored in the vehicle's recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors and constituting information about the vehicle's past environment; - transmission from the supervision system to the vehicle of a restart command or a restart prohibition command; then - closing the wireless communication session.

[0008] Some preferred but non-limiting aspects of this method are as follows: - it further comprises the selection of a time window for which information on the vehicle's past environment is transmitted to the supervision system; - the time window includes the date the vehicle was stopped; - the establishment of the wireless communication session includes the transmission by the vehicle to the supervision system of a shutdown signal; - the stop signal is transmitted when the duration of the vehicle stop exceeds a threshold; - the stopping signal carries information relating to a cause of the vehicle stopping; - the stop signal carries information relating to a category of the surrounding obstacle; - the exteroceptive sensors comprise a camera, the transmission of information on the real-time environment of the vehicle comprises the transmission of a video stream currently captured by the camera and the transmission of information on the past environment of the vehicle comprises the transmission of a video extract previously captured by the camera and stored in the vehicle's recording unit; - it further comprises the display, on a display device of the supervision system, of information on the real-time environment of the vehicle and information on the past environment of the vehicle, the information on the real-time environment of the vehicle being displayed on the right of the display device and the information on the past environment of the vehicle being displayed on the left of the display device; - it also includes the transmission by the supervision system to the vehicle of a horn command or a command to flash the headlights; - it also includes the transmission by the supervision system of an intervention request to a local operator. Brief description of the drawings

[0009] Other aspects, aims, advantages and characteristics of the invention will appear better on reading the following detailed description of preferred embodiments thereof, given by way of non-limiting example, and made with reference to the appended drawings in which:

[0010] - [Fig.l] is a diagram illustrating a fleet of vehicles with driving delegation and a remote fleet monitoring system;

[0011] - [Fig.2] is a diagram illustrating a possible distribution of information of different natures on a display device of the supervision system.

[0012] DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS

[0013] With reference to [Fig.l], the invention relates to a method for remote supervision of a fleet of vehicles with driving delegation 10, 20, 30, 40. The vehicles of the fleet can be operated on the same site or, on the contrary, be deployed on different sites.

[0014] Each vehicle in the fleet includes exteroceptive sensors, each capable of providing information about the vehicle's environment. Each vehicle in the fleet may also include proprioceptive sensors, each capable of providing information about the vehicle's state.

[0015] Each vehicle in the fleet also has a recording unit capable of storing the information obtained by the various sensors. The recording unit can thus store: - raw obstacle detection measurements, - position information from the fusion of different data streams from, for example, Lidars, an inertial unit and a satellite positioning system, - video streams captured by cameras, for example four exterior cameras (one camera imaging the front of the vehicle, one camera imaging the rear of the vehicle, one camera imaging the right of the vehicle and one camera imaging the left of the vehicle) and one interior camera imaging the passenger compartment of the vehicle.

[0016] The recording unit can further store: - vehicle controls, such as propulsion, braking, steering, emergency braking, lighting or door position controls, - vehicle status information, such as its speed or engine temperature; - events, such as an emergency stop of the vehicle or a sensor failure.

[0017] Each vehicle in the fleet also has an automated driving unit capable in particular of detecting a surrounding obstacle on the basis of information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle.

[0018] In order to ensure the resumption of service in automated driving mode while ensuring safety, the method according to the invention comprises, following the stopping of a vehicle in the fleet by its automated driving unit, the establishment of a wireless communication session between the stopped vehicle and a supervision system 50 remote from the vehicle. This communication session can in particular be carried out via a cloud computing service 60. This communication session is therefore not established continuously between the vehicle and the remote supervision system, but as a consequence of stopping the vehicle, which makes it possible to limit the volume of data exchanged between the vehicle and the supervision system.

[0019] During the wireless communication session, the vehicle transmits to the remote supervision system both information currently acquired by the exteroceptive sensors of the vehicle (constituting information on the real-time environment of the vehicle) and information stored in the vehicle recording unit corresponding to information previously acquired by the exteroceptive sensors of the vehicle (constituting information on the past environment of the vehicle).

[0020] Then, the remote supervision system transmits a restart command or a restart prohibition command to the vehicle. The wireless communication session is then interrupted.

[0021] As indicated previously, the exteroceptive sensors comprise cameras and in one possible embodiment, the transmission of information on the real-time environment of the vehicle comprises the transmission of a video stream currently captured by one or more of the cameras and the transmission of information on the past environment of the vehicle comprises the transmission of a video extract previously captured by the camera(s) and stored in the recording unit of the vehicle.

[0022] In another embodiment, which can be implemented independently or in addition to the previous one, the transmission of information on the real-time environment of the vehicle and on the past environment of the vehicle comprises the transmission of data from one or more exteroceptive sensors other than the cameras, for example information from a Lidar or a radar.

[0023] The supervision system typically comprises a display device on which information on the real-time environment of the vehicle and information on the past environment of the vehicle are displayed. This typically involves displaying one or more real-time video streams as well as one or more past video extracts. When information from other types of sensors is fed back, this information is formatted to provide a representation for humans of the perception of the vehicle.

[0024] An operator of the supervision system is thus able to view information on the past environment of the vehicle, which allows him to contextualize what happened before the stop and thus to understand the reasons for the stop. This same operator is also able to view the real-time environment of the vehicle, which allows him in particular to check the presence or absence of an obstacle. Observing information on the past environment also allows him to resolve ambiguities such as, for example, the non-visualization of a pedestrian in the real-time information while the analysis of the past information makes it possible to understand that the pedestrian is under the chassis of the vehicle. Depending on its analysis of the information displayed, the operator authorizes or prohibits the restart of the vehicle in automated driving mode.

[0025] For example, the operator can in this way check that the obstacle detected by the vehicle is a false positive (case of a cloud of midges, projection of grass by a mower operating at the edge of a traffic lane, a cloud of dust or bags and cardboard boxes on the lane) and that it is thus possible to ignore it. The operator can also ensure that a disturbing obstacle has been removed from the lane or decide that a detected obstacle is not disturbing in that it is possible to go around it (for example, it is a vehicle parked on the hard shoulder). The operator can also ensure that a moving obstacle (a person, an animal) no longer presents a danger.

[0026] As shown in [Fig. 2], the display device can be subdivided into several display members or sections, for example by means of several screens. In one possible embodiment, four screens are used: three screens arranged horizontally (a left screen EG, a central screen EC and a right screen ED) and an upper screen ES placed above the central screen EC.

[0027] The information on the real-time environment of the vehicle can be displayed on the right of the display device (for example on the right screen ED) and the information on the past environment of the vehicle can be displayed on the left of the display device (for example on the left screen ED). This presentation mode, with the past on the left and the real time on the right, helps the operator to carry out his task consisting in deciding whether or not to authorize the restart of the vehicle in automated driving mode.

[0028] In a possible embodiment using four exterior cameras and one interior camera, each of the left and right screens is subdivided into five zones: a central zone displaying the information captured by the interior camera, an upper zone displaying the information captured by the exterior camera imaging the front of the vehicle, a lower zone displaying the information captured by the exterior camera imaging the rear of the vehicle, a left lateral zone displaying the information captured by the exterior camera imaging the left of the vehicle and a right lateral zone displaying the information captured by the exterior camera imaging the right of the vehicle.

[0029] The EC central screen can be used to display and / or record information for controlling and planning the fleet of vehicles under supervision. Commands, instructions or intervention requests can be placed via it.

[0030] The upper screen ES placed above the central screen can be used to display a map of a fleet vehicle deployment site, for example with a representation of the route followed by each of the vehicles.

[0031] The display device thus allows the operator to view the current situation (access to vehicle videos, to a representation for humans of the vehicle's perception), to review the scene before stopping the vehicle (in the form of videos, of the representation of the perception), to pass commands to the vehicle (restart command in automated driving mode, restart prohibition command, horn command, headlight flash command), to pass instructions (such as limiting the speed on the route) or to configure the operation of the vehicle (times for raising a stop alert that can be configured according to the location or service, definition of zones where any stop must generate a high safety alert, definition of specific danger zones).

[0032] In one possible embodiment, the supervision method comprises the selection, for example by the aforementioned operator, of a time window for which the information on the past environment of the vehicle is transmitted to the supervision system. This time window preferably includes the date of stopping the vehicle. This time window typically covers a period before the date of stopping (for example of a duration of 10 seconds) and a period after the date of stopping (for example of a duration of 20 seconds).

[0033] In one possible embodiment, establishing the wireless communication session comprises the transmission by the vehicle to the supervision system of a stopping signal. This stopping signal may in particular be transmitted when a duration of the stopping of the vehicle exceeds a threshold. This threshold is preferably configurable, for example depending on the areas of the site (case of an area near a stop sign for example where the threshold will typically be higher).

[0034] The stopping signal preferably carries information relating to a cause of the vehicle stopping (for example the presence of a fixed obstacle completely obstructing the lane or the presence of a moving obstacle), which cause is determined by the automated driving unit of the vehicle. The stopping signal also preferably carries information relating to a category of the surrounding obstacle (for example from an object categorization algorithm and of the type the obstacle is a person with a probability of 90%), which category is determined by the automated driving unit of the vehicle.

[0035] A first use case of the supervision system concerns a badly parked car parked in a double line on the path of an autonomous vehicle. The vehicle stops automatically in front of this car and informs the monitoring system. The monitoring system operator views real-time videos and past video clips. The monitoring system operator can order the vehicle to honk or flash its headlights and use the real-time videos to verify that the initially badly parked car is no longer interfering with the operation of the vehicle.

[0036] Another use case concerns an object that falls from a car. The vehicle automatically stops in front of this object and informs the supervision system. The operator of the supervision system views the real-time videos and past video clips. The operator of the supervision system can request the intervention of a local operator to remove the object from the traffic lane. The operator of the supervision system can verify the effective removal of the object in the real-time videos and initiate an automated restart of the vehicle.

[0037] Yet another use case concerns the passage of a pedestrian or an animal near the vehicle. The vehicle stops automatically in front of this moving object and informs the supervision system. The operator of the supervision system views the real-time videos and past video clips. He can order the vehicle to honk. Once the traffic lane is empty and after consulting the past video clips to understand where the pedestrian or animal has gone, the operator can authorize an automated restart of the vehicle.

[0038] Another use case is that of the loss of a sensor, for example due to deliberate damage. The vehicle stops automatically following this loss and informs the supervision system. The operator of the supervision system views real-time videos and past video clips. He can request the intervention of a local operator.

[0039] In a possible embodiment, each vehicle is further provided with an accident situation detection unit configured to detect, on the basis of the information provided by the sensors, warning signs of incidents. This unit is for example capable of identifying a repetition of behaviors not foreseen in the conditions of deployment of the vehicle, such as a repetition of sudden braking of the vehicle in the same zone. These sudden brakings are for example the consequence of the installation of a new pedestrian crossing on the path of the vehicle, of the recurring presence of vehicles regularly parked just before a pedestrian crossing inducing occlusion and sudden braking or even of the recurring presence of badly parked vehicles on the path of the vehicle inducing slowdowns.These warning signs thus detected can be transmitted to the supervision unit which, for example by means of an analysis of the videos captured by the vehicle's cameras, can understand their origins. An action to reduce the occurrences of the risk. can then be launched, for example by raising awareness among certain recurring delivery drivers, positioning posts to reduce occlusion on the pedestrian crossing, requesting the installation of speed bumps, requesting the rectification of the layout of the white stripes to influence the trajectories or even updating the conditions for deploying the vehicle with, for example, the inclusion of a new pedestrian crossing in the route.

[0040] The invention is not limited to the method as previously described but also extends to the remote supervision system previously described and in particular to a system comprising a processor configured, during a wireless communication session established with one of the vehicles in the fleet following the stopping of the vehicle by its automated driving unit, to:

[0041] controlling the reception from the vehicle of information on the real-time environment of the vehicle and information on the past environment of the vehicle;

[0042] transmit to the vehicle a restart command or a restart prohibition command.

[0043] Such a supervision system is advantageously equipped with the display device described above.

Claims

1. Claims Method for remote supervision of a fleet of vehicles with driving delegation (10, 20, 30, 40) where each vehicle of the fleet comprises exteroceptive sensors each capable of providing information on the environment of the vehicle, a recording unit capable of storing the information obtained by the exteroceptive sensors and an automated driving unit capable of detecting a surrounding obstacle on the basis of the information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle, the method comprising the following steps: - following the stopping of a vehicle by its automated driving unit, establishment of a wireless communication session between the vehicle and a remote supervision system (50) of the vehicle; - during the wireless communication session: - transmission from the vehicle to the supervision system: • information currently acquired by the vehicle's exteroceptive sensors constituting information on the vehicle's real-time environment; and • information stored in the vehicle recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors and constituting information on the vehicle's past environment; - transmission from the supervision system to the vehicle of a restart command or a restart prohibition command; then - configure the vehicle's operation. - closing the wireless communication session; - transmission by the supervision system of an intervention request to a local operator.

2. Method according to claim 1, in which each vehicle is further provided with a unit for detecting accident-prone situations, this unit detecting warning signs of incidents, these warning signs being transmitted to the supervision unit.

3. Method according to claim 2, said accident situation detection unit being able to identify a repetition of behaviors not foreseen in the conditions of deployment of the vehicle.

4. Method according to one of claims 2 or 3, said unit for detecting accident-prone situations being able to identify a repetition of sudden braking of the vehicle in the same zone.

5. A method according to any preceding claim, wherein the recording unit stores video streams imaging the interior of the vehicle.

6. The method of claim 1, further comprising selecting a time window for which information about the vehicle's past environment is transmitted to the monitoring system.

7. The method of claim 6, wherein the time window includes the date the vehicle was stopped.

8. Method according to one of claims 1 to 7, in which the establishment of the wireless communication session comprises the transmission by the vehicle to the supervision system of a shutdown signal.

9. The method of claim 8, wherein the shutdown signal is transmitted when a duration of the vehicle shutdown exceeds a threshold.

10. Method according to one of claims 8 and 9, in which the stopping signal carries information relating to a cause of stopping the vehicle.

11. Method according to one of claims 8 to 10, in which the stopping signal carries information relating to a category of the surrounding obstacle.

12. Method according to one of claims 1 to 11, in which the exteroceptive sensors comprise a camera, the transmission of information on the real-time environment of the vehicle comprises the transmission of a video stream currently captured by the camera and the transmission of information on the past environment of the vehicle includes the transmission of a video extract previously captured by the camera and stored in the vehicle's recording unit.

13. Method according to one of claims 1 to 12, further comprising displaying, on a display device of the supervision system, information on the real-time environment of the vehicle and information on the past environment of the vehicle, the information on the real-time environment of the vehicle being displayed on the right of the display device and the information on the past environment of the vehicle being displayed on the left of the display device.

14. Method according to one of claims 1 to 13, further comprising the transmission by the supervision system to the vehicle of a horn command or a command to flash the headlights.

15. System for supervising a fleet of vehicles with delegated driving where each vehicle in the fleet comprises exteroceptive sensors each capable of providing information on the environment of the vehicle, a recording unit capable of storing the information obtained by the exteroceptive sensors and an automated driving unit capable of detecting a surrounding obstacle on the basis of the information obtained by the exteroceptive sensors and of stopping the vehicle in the event of detection of a surrounding obstacle, the system comprising a processor configured, during a wireless communication session established with one of the vehicles in the fleet following the stopping of the vehicle by its automated driving unit,to: control the reception from the vehicle of information currently acquired by the vehicle's exteroceptive sensors constituting information on the vehicle's real-time environment and of information stored in the vehicle's recording unit corresponding to information previously acquired by the vehicle's exteroceptive sensors and constituting information on the vehicle's past environment; transmit to the vehicle a restart command or a restart prohibition command;, - configure the vehicle's operation. - transmit an intervention request to a local operator via the supervision system.

Images