System for safely and remotely controlled driving

Abstract

A system (10) for remotely controlled driving, characterized in that the system has the following features: the system (10) includes a vehicle (20), a backend (80), a remote operation device (90), and the vehicle (20), the backend (80) and the remote operation device (90) are arranged to communicate via a mobile wireless network (60).

Description

Technical Field

[0001] The present invention relates to a system for teleoperated driving (ToD). Background Art

[0002] The premise of a partially autonomous vehicle according to the prior art is a vehicle guidance interface ("driver's workplace") and a person as a vehicle occupant who is capable of driving and authorized to guide the vehicle, and who can take over the guidance when needed. The so-called teleoperated driving has become the subject of many research projects, in which the vehicle can be assisted in coping with challenging scenarios (such as detouring on rural dirt roads, alternative and unconventional routes, etc.) through remote control or the driving task can be temporarily completely taken over by an external operator at the dispatching center, that is, the said operator. For this purpose, the vehicle and the dispatching center or its operator are connected to each other through a mobile wireless network with low latency and high data rate.

[0003] US 9494935 B2 discloses computer devices, systems and methods for remotely operating (Fernbedingung) autonomous passenger vehicles. When an autonomous vehicle encounters an unexpected surrounding environment that is not suitable for autonomous operation (such as a road construction site or an obstacle), vehicle sensors can sense data about the vehicle and the unexpected surrounding environment, including pictures, radar data, lidar data, etc. The sensed data can be sent to a remote operator. The remote operator can manually remotely operate the vehicle or issue instructions to be executed by various vehicle systems to the autonomous vehicle. The sensed data sent to the remote operator can be optimized to save bandwidth, for example, by sending a limited subset of the sensed data.

[0004] The vehicle according to US 9767369 B2 can receive one or more pictures of the vehicle's surrounding environment. The vehicle can also obtain a surrounding environment map. The vehicle can also compare at least one feature in the picture with one or more features in the map. The vehicle can also identify a specific area in the one or more pictures corresponding to a part of the map, and the part is at a threshold distance from the one or more features. The vehicle can also compress the one or more pictures in order to record a smaller amount of details in an area of the picture as a given area. The vehicle can also provide the compressed pictures to a remote system and receive operating instructions from the remote system in response thereto.

[0005] The systems and methods according to US 9465388 B1 implement that when the trust level of the vehicle in its operation is low, an autonomous vehicle can request the assistance of a remote operator. Exemplary methods include operating the autonomous vehicle in a first autonomous mode. The method can also include identifying a situation in which the trust level of autonomous operation in the first autonomous mode is lower than a threshold level. The method can further include sending a request for assistance to a remote assistant, where the request includes sensor data representing a portion of the surrounding environment of the autonomous vehicle. Additionally, the method can include receiving a response from the remote assistant, where the response describes a second autonomous operation mode. The method can also cause the autonomous vehicle to operate in the second autonomous operation type according to the response from the remote assistant.

[0006] US 9720410 B2 discloses another method for remotely assisting an autonomous vehicle in a predetermined situation. SUMMARY OF THE INVENTION

[0007] The present invention provides a system for safely remotely-operated driving according to claim 1.

[0008] The solution according to the present invention is based on the following recognition: There are situations that automated vehicles cannot solve independently and human intervention is required to overcome these situations or system deficiencies and put the entire system into a safe state. According to the present invention, this intervention is performed remotely so that a driver does not necessarily have to be in the vehicle.

[0009] The advantage of the proposed solution is to implement the architecture and integration of the components of a system for (functionally) safely remotely controlling a partially or fully automated vehicle by an operator in a control center. This is achieved by determining the safety-related system components for remotely-operated driving and by describing a system integration that is operationally safe and information-secure for implementing the corresponding system behavior.

[0010] Advantageous extensions and improvements of the basic concept described in the independent claims can be achieved by the measures listed in the dependent claims. Thus, additional optional components can be provided to implement or improve the tasks of remote detection and remote control. In this way, a system for remotely-operated driving and the associated system architecture are achieved, which integrate all relevant system components to perform remote sensing and remote control of the driving operation in a functionally safe manner considering the characteristics of the mobile communication connection and different operation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Embodiments of the present invention are shown in the drawings and described in more detail below.

[0012] The drawings show a block diagram of a system according to one embodiment. Detailed Description

[0013] The drawings show, at a highly abstract level, a ToD vehicle (20), a mobile radio network (60), such as a fifth generation (5G) network, a backend (80), a remote operation device (90), infrastructure components (70), and the most important components included therein. Accordingly, components (21) for remote sensing collect all information related to the surroundings of the ToD vehicle (20), for example, by means of radar sensors, camera sensors, ultrasonic sensors, lidar sensors, tachometers, inertial measurement units (IMUs), and collision detectors. Components (22) for vehicle interior sensing use all sensors in the vehicle (20) for monitoring the driver and passengers, such as driver activity sensors and seat occupation information. Components (23) for vehicle motion control are responsible for vehicle motion and vehicle stability.

[0014] Autonomous driving (AD) and advanced driver assistance systems (ADAS) functions are also mentioned. The corresponding components (24) are involved, for example, in perception, situation analysis, function behavior, reaction manager, and prediction.

[0015] All system states are processed in the system state manager (25). Two operating modes are considered here:

[0016] 1. A remote operation, in which an operator guides or drives an automated vehicle (20) without direct line-of-sight contact, such that vehicle information and the vehicle surroundings must be transmitted and presented to the operator, and

[0017] 2. A remote operation, in which an operator uses direct line-of-sight contact to guide or drive an automated vehicle (20), making it possible for the operator to directly control the vehicle state and the surrounding environment.

[0018] The telematics unit (Connectivity control unit, CCU26, connection control unit) forms an interface of the system (10) for communication via a 5G mobile wireless network (60). The component (27) for diagnostic information management is responsible for general system diagnosis; the human-machine interface (HMI28) inside the vehicle constitutes an interface for the driver or co-driver of the vehicle (20).

[0019] The device (29) for passive safety includes, for example, airbags, so-called pre-crash recognizers, and an event data recorder. The component (30) for body control is responsible for power supply and communication in the vehicle (20), the vehicle access system, and the lighting system. Other safety-related components (40) are responsible for all safety-related objectives of remotely operated driving.

[0020] The system (10) operates with the following safety objectives in mind:

[0021] 1. Identify communication errors between both parties (transmitter, receiver) in order to place the system (10) in a safe state within a pre-given tolerance time t C

[0022] 2. Identify the compatibility of all system components in order to place the system (10) in a safe state within the tolerance time t O

[0023] 3. Identify unauthorized access to the system (10) in order to place the system (10) in a safe state within the tolerance time t S

[0024] 4. Identify collision data, pre-crash data, or other relevant data for the safety ToD function and send it to the control room upon request,

[0025] 5. Sense objects near the vehicle (20) - for example, at any angle up to 50 cm away from the vehicle - and objects under the vehicle (20) in order to report these objects to the operator, and

[0026] 6. Sense the system boundary and react within a pre-given time period t b when the system boundary is breached. ​​​

[0027] Different safety components are used to achieve these safety goals. To achieve Safety Goal 1, for example, the communication protocol monitor (41) monitors the 5G communication line in all of the above communication errors (see ISO 26262-6, D.2.4) and reports the error to the system status manager (25) if necessary.

[0028] The component (44) for system inspection before handover to the remote operation device (90) and the operator is used to achieve Safety Goal 6. Handover is not performed when the situation is not defined. The system boundary inspection performed after handover from the operator to the automated vehicle (20) is implemented by the corresponding component (47) for clarifying the question of "whether the automated vehicle (20) can perform its normal driving tasks".

[0029] The following diagnostic management (50) is also used to achieve Safety Goal 6: The ToD diagnosis (for automation levels 2 to 5 according to SAE J3016) is triggered before activating the ToD function. In addition to the inspection of the ToD function in the narrow sense (sensor availability, brakes, etc.), this diagnosis also includes determining possible ToD controls (maneuvering, path planning, behavior planning, speed, steering, reversing, etc.). If the ToD function cannot be activated, the vehicle repair shop or the vehicle manufacturer should be contacted.

[0030] Finally, the activation manager (42) is also set up to achieve Safety Goal 6. Here, all important and available safety-related parameters such as the perceived quality of Service (pQoS) and path complexity perceived by the user should be used for activation in order to reduce the complexity of the safety components in the vehicle (20).

[0031] The authentication manager (45) is used to achieve Safety Goal 3. The authentication of the complete safety chain takes the following points into consideration:

[0032] · The list of operators authorized to access the vehicle (20),

[0033] · The availability of the correct software and hardware,

[0034] · Operator authorization,

[0035] · The control room,

[0036] · The backend (80), and

[0037] · The communication channel and the server (switching to other servers or channels should be avoided).

[0038] The start command transmitter (48) is used to achieve safety goal 5. In this regard, the drive away of the automated vehicle (20) must be checked and the operator informed, because the vehicle (20) is not allowed to move in case of violations. In particular, bottom vehicle monitoring, all-round vehicle monitoring within a free space of 50 cm, checking of local weather conditions (in terms of temperature, road icing, etc.) and available sensor power (visibility of sensors, blindness, etc.) are considered.

[0039] The ToD data recorder (51) is used to achieve safety goal 4: All ToD-related data - such as handover timestamps, operator IDs, operator driving styles, communication channels used, authorization information, and possible collisions - are locally recorded by these components and transmitted to the server on request.

[0040] To achieve safety goal 1, the received network QoS values should be checked by the quality of service calculator (43) and forwarded to the corresponding safety components. The driving task checker (46) is responsible for checking: whether the driving task requested by the operator can be performed, and whether the safety goals regarding the ToD function and the AD function are not violated while the driving task execution control unit (49) ensures the monitoring of the driving task and the operator is updated as the process progresses. The operator can control the system (10) in case of errors.

[0041] Finally, the system compatibility checker (52) is used to achieve safety goal 2. Here, it should be considered that the compatibility of the hardware and software in the automated vehicle (20), the hardware and software in the backend (80), the control room, and the protocols executed on the communication channels are checked before and during the activation of the ToD function.

[0042] According to an alternative approach, the safety goals mentioned can be evaluated differently based on a hazard analysis and risk assessment (HARA) according to ISO 26262, ISO 25119, or DIN EN 16590. Therefore, for the system (10) according to the invention, the automotive safety integrity level (ASIL), which is defined for the entire functionality, is decisive.

Claims

1. A system (10) for remotely controlled driving, It is characterized in that The system has the following characteristics: - The system (10) includes a vehicle (20), a backend (80), a remote operation device (90), and - The vehicle (20), the backend (80) and the remote operation device (90) are arranged to communicate with each other via a mobile wireless network (60), wherein the vehicle (20) includes safety-related components (40), and wherein the components (40) include a communication protocol monitor (41), a system compatibility checker (52) and a driving task checker (46), wherein the communication protocol monitor (41) is arranged to identify communication errors during communication via the mobile wireless network (60), and the communication protocol monitor (41) is also arranged to report the identified communication errors to the system status manager (25) of the vehicle (20), wherein the system compatibility checker (52) is arranged to check the compatibility of the hardware and software in the vehicle (20), the hardware and software in the backend (80), the control room and the protocols executed on the communication channels before activating the remotely controlled driving function and during the execution of the remotely controlled driving function, wherein the driving task checker (46) is arranged to check whether the driving tasks requested by the operator can be executed, wherein the safety-related components (40) further include a first device (44) for performing a system boundary check before handing over to the remote operation device (90), and no handover is performed when the situation is not defined, wherein the safety-related components (40) further include a second device (47) for performing a system boundary check after handing over from the operator to the vehicle (20), wherein the safety-related components (40) further include an authentication manager (45) for identifying unauthorized access to the system (10) at least through a list of operators authorized to access the vehicle (20) and operator authorizations.

2. The system (10) according to claim 1, characterized in that The components (40) further include at least one of the following: - A device (50) for diagnostic management, - An activation manager (42), - A start instruction transmitter (48), - A data recorder (51), - A quality of service calculator (43), or - A driving task execution control unit (49).

3. The system (10) according to claim 2, It is characterized in that The system has the following characteristics: - The quality of service calculator (43) is arranged to check the quality of service of the mobile wireless network (60), and, - The quality of service calculator (43) is also arranged to inform the safety-related components (40) of the checked quality of service.

4. The system (10) according to any one of claims 1 to 3, characterized in that The vehicle (20) has at least one of the following: - A device (21) for sensing the surrounding environment, - A device (22) for sensing the interior space of a vehicle, - A device (23) for controlling the movement of a vehicle, - Autonomous driving and driver assistance system functions (24), - A system state manager (25), - A telematics unit (26) for connecting to the mobile radio network (60), - A device (27) for managing diagnostic information, - A human-machine interface (28) inside the vehicle, - A passive safety device (29), or - A device (30) for body control.

5. The system (10) according to any one of claims 1 to 3, characterized in that the communication error is identified based on at least one of the following reasons or effects: - Duplication of information, - Loss of information, - Delay of information, - Insertion of information, - Unauthorized or incorrect information addressing, - Incorrect order of information, - Tampering with information, - Asymmetric information sent from one transmitter to multiple receivers, - Information from one transmitter received by only a subset of the configured receivers, or - Blocking access to the communication channel.

6. The system (10) according to any one of claims 1 to 3, characterized in that the backend (80) includes at least one of the following: - An authorization control program (81), - A data memory (82), - A map server (83), or - A device (84) for route planning.

7. The system (10) according to any one of claims 1 to 3, characterized in that the remote operation device (90) includes at least one of the following: - A first operator interface (91) for operating the vehicle (20) out of sight, and - A second operator interface (92) for operating the vehicle (20) in sight.

8. The system (10) according to any one of claims 1 to 3, It is characterized in that the system has the following characteristics: - The system (10) further includes infrastructure components (70), and - The infrastructure components (70) include intelligent parking infrastructure (71).

Images