Driver change while a teleoperated vehicle is in motion

Abstract

The invention relates to a system for transferring control of a teleoperated vehicle (1) from an initial driver to a subsequent driver while the vehicle (1) is in motion, comprising a computing unit (3) and at least one stationary control station (5) for teleoperation of the vehicle (1), wherein the computing unit (3) is configured to perform a transfer control mechanism for the transfer at the at least one stationary control station (5), wherein the transfer control mechanism includes at least a safety query regarding the readiness of the subsequent driver, and machine-controlled vehicle guidance during the transfer and / or feedback to the subsequent driver as to whether the latter is assuming a sufficiently adequate operating position at an input element for vehicle guidance of the vehicle (1) of the at least one stationary control station (5).

Description

[0001] The invention relates to a system for transferring control of a teleoperated vehicle from an initial driver to a subsequent driver while the vehicle is in motion.

[0002] Typical conventional motor vehicles, such as passenger cars, are designed to be driven by a person inside the vehicle. With increasing levels of automation, this person's role shifts from being a driver to actively controlling the vehicle, and increasingly to the sole task of monitoring the driving maneuvers and control interventions performed autonomously by the vehicle. At the forefront of this development are fully automated vehicles, which can autonomously execute not only individual maneuvers but also complete an entire planned route. While a human driver in a non-automated vehicle is the sole decision-making body, they also assume, at least in part, the role of actuators when their movements mechanically (and perhaps with direct assistance) determine control parameters.However, with increasing levels of vehicle automation, actuators become necessary, such as electric or hydraulic actuators, all of which are controlled by electronic control units. This allows the vehicle's computer to send commands to the actuators, which then execute them mechanically. This enables the vehicle to be controlled from an external control center via an interface on a digital control unit or directly on the vehicle's actuators. This opens up the possibility of teleoperation of the automated vehicle, where a human driver does not need to be in the vehicle itself, but can sit outside at a stationary workstation and issue commands to this interface. These commands are then transmitted to the vehicle for execution via data transfer, particularly wirelessly.Furthermore, relevant vehicle information (for example, a video stream with images from the driver's seat) can be transmitted to the remotely controlled driver. If the teleoperation takes place in an environment with other road users, the human driver can, by perceiving the vehicle's local surroundings and the other road users within them, initiate necessary vehicle movements (through lateral / longitudinal control) to avoid traffic obstructions and hazardous situations in order to reach a destination.

[0003] In this context, DE 10 2021 123 234 A1 relates to a teleoperator workstation for a teleoperated motor vehicle, wherein the motor vehicle has a front camera, a rear camera, a left side camera which is optionally directed towards a rear left side of the motor vehicle, and a right side camera which is optionally directed towards a rear right side of the motor vehicle, and wherein the motor vehicle is designed to send images captured by the front camera, the rear camera, and the left and right side cameras to the teleoperator workstation, wherein the teleoperator workstation is designed to change the display of the images received from the motor vehicle depending on the orientation of a teleoperator's head and / or depending on the teleoperator's viewing direction.

[0004] DE 10 2021 117 448 A1 also relates to a tax procedure for a teleoperated motor vehicle, wherein the tax procedure includes: the setting of a corridor to a predetermined destination by an operator, and the automated steering of the motor vehicle to the predefined destination within the set corridor.

[0005] DE 10 2019 204 942 A1 also relates to a procedure for transferring a teleoperable vehicle from an initial operating mode to a target operating mode, comprising the following features: Upon request, a transfer of driving responsibility to a designated operator is planned and prepared in the initial operating mode, areas of driving responsibility and operating limits of the vehicle in the target operating mode are negotiated, the negotiated operating limits are set, and the transfer is initiated and brought to a safe conclusion in the target operating mode.

[0006] DE 10 2015 118 489 B4 further relates to a computer device for an autonomous passenger vehicle, comprising: one or more processors for controlling the operation of the computer device; a communication interface configured to communicate with a remote server via a network; and a memory unit for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory in order to: identify an unexpected environment;Based on the unexpected driving environment, information received from one or more sensors mounted on the vehicle is sent to a remote operator using a remote server, with the information received from the one or more sensors being selectively sent based on the remote operator's need for information for remote operation of the vehicle; instruction sent by the remote operator concerning one or more vehicle systems; and the instruction being sent to one or more vehicle systems for execution.

[0007] EP 3 893 073 A1 relates to a method for invoking a remote driving session for a vehicle equipped with an automated driving function, wherein the method comprises: identifying an impending situation that might require a remote driving session; predicting a quality of service for communication between the vehicle and a control center for a location where the remote driving session is to be carried out; and reducing the vehicle's speed to a maximum drivable speed for a remote driving session with the predicted quality of service.

[0008] Finally, DE 10 2019 208 403 A1 relates to a method for operating a teleoperated vehicle, comprising the following features: In normal operation, the vehicle is in active exchange with a first active operator in a first control center via a first passive operator on an active transmission channel, while a standby transmission channel to a second control center, mediated by a second passive operator, is in operational readiness, and, if the standby transmission channel fails, a third passive operator is sought and, if this is found within a first time period, normal operation is maintained.

[0009] Teleoperation is usually offered as a service, with professional drivers, for example, driving vehicles to a destination where a local driver takes over (also known as "valet service"). Commercial vehicles can also be driven by remotely operating drivers to transport goods or people. However, teleoperation drivers are generally bound by rest periods and working hours. This creates the following problem: If a vehicle cannot be driven due to a teleoperating driver's mandatory rest period, delays occur, increasing logistical and financial costs. If a vehicle needs to be parked for the handover from the initial teleoperator to the replacing teleoperator, this must be factored into route planning.This also causes delays, and planning for suitable handover situations further reduces efficiency, as longer distances and detours have to be accepted – especially in cities with tight parking conditions, but also on long-distance routes with busy or limited parking areas. Vehicle usage must be planned according to the availability of individual teleoperating drivers. When a driver's shift ends, a secure parking space for the vehicle must be found and the vehicle parked by that time.

[0010] The object of the invention is to simplify the handover of a teleoperated vehicle from an initial driver to a subsequent driver.

[0011] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0012] A first aspect of the invention relates to a system for transferring a teleoperated vehicle from an initial driver to a subsequent driver while the vehicle is in motion, comprising a computing unit and at least one stationary control station for teleoperating the vehicle, wherein the computing unit is configured to perform a transfer control mechanism for the transfer at the at least one stationary control station, the transfer control mechanism comprising at least a safety query regarding the readiness of the subsequent driver, and machine-controlled vehicle guidance during the transfer and / or feedback to the subsequent driver as to whether the latter is assuming a sufficiently adequate operating position at an input element (in particular a steering wheel or an accelerator pedal) for guiding the vehicle of the at least one stationary control station.

[0013] The transition control mechanism thus advantageously ensures that the following driver is ready to take over control of the teleoperated vehicle. For this purpose, the stationary control station provided for the following driver advantageously has a corresponding switch that the following driver can operate to signal their readiness to take over. This can also be done automatically by monitoring and verifying the following driver's actions at their stationary control station. As soon as their actions are sufficiently plausible and / or sufficiently continuous and / or meet another criterion, the following driver can take over active teleoperated control from their stationary control station.

[0014] A sufficiently adequate operating position is, in particular, one that is sufficiently similar (at the stationary control station of the following driver) to the respective operating position of the stationary control station of the initial driver, if a first and a second stationary control station are provided, so that the initial driver can use a first stationary control station and the following driver a second control station. If only a single stationary control station is provided, and the handover takes place by the initial driver exiting and the following driver entering, the sufficiently adequate operating position is one that is as consistent as possible with the previously executed driverless-machine-controlled vehicle operation.

[0015] Advantageous effects of the invention include the fact that a driver change from an initial driver to a subsequent driver for the same teleoperated vehicle no longer necessitates bringing the vehicle to a standstill in a safe location such as a parking lot. This leads to increased efficiency and cost reductions, as the planning of detours to find such a parking lot and the time required for parking are eliminated. Instead, the teleoperated vehicle can undergo a driver change while driving and can thus remain on its predetermined route to the original destination. Mandatory rest breaks for an initial driver are therefore less critical, as they are not dependent on successfully finding a parking space.The system also simplifies the handover to a subsequent driver at a predetermined location along a route, for example, when a driver specializing in urban areas leaves an urban traffic zone and wants to hand over to a long-haul driver for the highway. This is possible in a seamless transition between teleoperating drivers. "Wide transition" here means that the teleoperated vehicle does not need to be brought to a standstill for the driver change; the handover can take place seamlessly during operation.

[0016] According to an advantageous embodiment, the system has exactly one stationary control station, wherein the computing unit is configured to take over the teleoperated vehicle by means of driverless-machine controlled vehicle operation after confirmation of the planned handover by the initial driver, while the initial driver remains responsible for the teleoperated vehicle until the subsequent driver has taken over the stationary control station, wherein the computing unit is configured to deactivate the driverless-machine controlled vehicle operation only upon a corresponding input by the subsequent driver or after vehicle operation with control inputs at the stationary control station of the subsequent driver that meet a predefined operating metric, and to allow a manual takeover by the subsequent driver.

[0017] According to a further advantageous embodiment, the system has a first control station for the initial driver and a second control station for the subsequent driver of a common teleoperated vehicle, wherein the computing unit is configured to set an accelerator pedal position and / or a steering wheel position of the first control station by means of corresponding actuators at the second control station for the subsequent driver during the handover and / or to provide the second driver with feedback during the handover on approaching the accelerator pedal position and / or steering wheel position of the first control station.

[0018] According to a further advantageous embodiment, the computing unit is designed to provide haptic feedback at the accelerator pedal of the second stationary control station as to whether the position of this accelerator pedal is sufficiently similar to the position of the accelerator pedal of the first stationary control station.

[0019] According to a further advantageous embodiment, the computing unit is designed to perform speed control of the teleoperated vehicle for at least the time of the handover, so that neither the initial driver nor the subsequent driver has to operate the accelerator pedal of their respective stationary control station.

[0020] According to a further advantageous embodiment, the computing unit is designed to check a current driving state of the teleoperated vehicle against at least one predetermined condition, and to carry out the transfer only if the condition is met.

[0021] According to a further advantageous embodiment, the at least one predetermined condition comprises at least one of the following: The teleoperated vehicle falls below a predetermined speed limit; the teleoperated vehicle has a lateral acceleration below a predetermined limit; the teleoperated vehicle has a steering angle below a predetermined limit; the longitudinal acceleration of the teleoperated vehicle is below a predetermined limit; the torque at the steering wheel of the first stationary control station is below a predetermined limit;

[0022] According to a further advantageous embodiment, the computing unit is designed to synchronize the steering wheel position in the second stationary control station with the steering wheel position of the first stationary control station by means of an actuator, to check for falling below a limit torque on the steering wheel of the second control station while the following driver operates the steering wheel of the second control station, and only if the limit torque is not exceeded during a specified period of time, to allow the transfer of teleoperated vehicle control to the following driver.

[0023] According to a further advantageous embodiment, the computing unit is designed to transmit a data stream with video images to the first stationary control station and, synchronized to this, to the second stationary control station, and to monitor the synchronization for temporal and content equality.

[0024] Further advantages, features, and details will become apparent from the following description, in which – possibly with reference to the drawing – at least one embodiment is described in detail. Identical, similar, and / or functionally equivalent parts are identified by the same reference numerals.

[0025] They show: Fig. 1: A stationary control station of a system according to an embodiment of the invention. Fig. 2: A system with a single stationary control station according to an embodiment of the invention. Fig. 3: A system with a first and a second stationary control station according to an embodiment of the invention. Fig. 4: A possible pedal force curve at the second stationary control station during driver change according to an embodiment of the invention.

[0026] The representations in the figures are schematic and not to scale.

[0027] Fig. Figure 1 shows a stationary control station 5 with three screens. The middle screen displays the image from a front camera of the real, teleoperated vehicle 1, while the left and right screens display the images from the left and right side cameras of the vehicle, respectively. Due to its special arrangement, the front of the vehicle 1 remains constantly visible in the field of view of the front camera.

[0028] In the stationary control station 5, a driver sits just like in the actual vehicle 1. However, control station 5 is stationary and wirelessly transmits commands received from the steering wheel, accelerator pedal, etc., at control station 5 to vehicle 1. Similarly, video images from the aforementioned cameras of vehicle 1 are continuously and wirelessly transmitted to the screens of the stationary control station 5 in near real-time. Control station 5 is also connected to a processing unit 3, which supports the handover to a subsequent driver, as described in the following examples. The processing unit 3 can be located at the relevant control station 5, or it can be located any distance away and, for example, connected to control station 5 via the internet.

[0029] Fig. Figure 2 shows the process in a system for transferring control of a teleoperated vehicle 1 from an initial driver to a subsequent driver while the vehicle 1 is in motion. The system includes a computing unit 3 and exactly one stationary control station 5 for teleoperating the vehicle 1. The initial driver and the subsequent driver are located at the same stationary control station 5. The initial driver is responsible for the driving task and is therefore located at the stationary control station 5. The initial driver signals, for example via a button on the steering wheel, voice input, or another interaction method at the stationary control station 5, that a driver change is necessary. A prerequisite for the subsequent change is that the vehicle 1 or the stationary control station 5 can be controlled by an automated vehicle device of at least SAE Level 3, implemented in the stationary control station 5 or in the vehicle 1.This can also mean that the initial driver must first manually move vehicle 1 into a Level 3 environment. As soon as it is possible to drive vehicle 1 autonomously (at least Level 3), this is displayed to the initial driver at their stationary control station 5. The initial driver can then activate the autonomous driving mode, for example, via a button or another interaction option. The subsequent driver also indicates (e.g., by holding down a button) that they are ready to take over when they are in close proximity to the stationary control station 5 with a view of the driving process. Subsequently, a driverless, machine-controlled vehicle system in the form of a Level 3 ADS (Automated Driving System) takes over vehicle control. The initial driver is then shown at the stationary control station 5 that a changeover is possible.The initial driver now leaves stationary control station 5, whereupon the subsequent driver enters stationary control station 5 and prepares to take over driving duties. As soon as the subsequent driver is ready to take over, they interact with stationary control station 5, for example, by taking control of vehicle 1 or by signaling to the ADS (Automatic Driving System) via a button, for example, that they are ready to take over. Through this interaction, the driving task is transferred to the subsequent driver. It is important to consider that sufficient time is required for the driver changeover, during which the ADS controls vehicle 1. The required time for the changeover is an empirical value that must be determined according to the specific circumstances.

[0030] Fig. Figure 3 shows the process in a system for handing over control of a teleoperated vehicle 1 from an initial driver to a subsequent driver while the vehicle 1 is in motion. The system includes a computing unit 3 and a first and a second stationary control station 5 for teleoperating the vehicle 1 for the initial driver and the subsequent driver, respectively. Thus, each driver has their own stationary control station 5. For improved data transmission, these stations could be located in the same office or building. Alternatively, they could be located at different locations. In this case, sufficient data transmission is essential. Sufficient data transmission can be achieved, for example, by a dedicated line between two stationary control stations 5. Initially, the initial driver is responsible for driving. They are therefore located at a stationary control station 5 to control the vehicle 1.Similarly, the following driver is located at a stationary control station 5. As soon as a changeover is scheduled, the sensor data of vehicle 1 is displayed at both teleoperation stations. The following driver thus has the same information as the initial driver, who is performing the driving task. In addition to duplicating the sensor information, the current control information from the stationary control station 5 of the initial driver can also be transmitted to the stationary control station 5 of the following driver. In this case, the pedal position and the steering wheel position are transmitted to the stationary control station 5 of the following driver, for example by servomotors, in such a way that they correspond to those at the control station 5 of the initial driver. The following driver now holds the steering wheel at their stationary control station 5.The following driver indicates their readiness to take over via an interaction method, such as a steering wheel button. The steering wheel position and the current torque applied to the steering wheel, resulting from the following driver's input, are continuously measured. The initial driver indicates their readiness to take over, for example, by pressing a button on the steering wheel of their stationary control station 5. A suitable time for the handover must then be determined. This time is calculated by monitoring the current steering torque and the steering wheel position of the following driver. If these values ​​are below a specific threshold (position deviation and steering torque must be minimized), the control unit 3 initiates the handover. The driving task is then transferred to the following driver.The initial driver is now relieved of the driving task, which can be indicated at their stationary control station 5. The transfer of the driving task to the subsequent driver is also indicated to the subsequent driver. This can be represented by a color change of a display at this stationary control station 5. The synchronization phase ensures a safe, low-acceleration transition from the initial driver to the subsequent driver. However, the described phase only refers to the steering wheel position and steering torque. Longitudinal steering can also be synchronized. Preferably, this synchronization takes place in a range without brake pedal use or, as is the case with electric vehicles, in a one-pedal control mode. As already described, the pedal position is transmitted from the first stationary control station 5 to the second stationary control station 5.The following driver manually moves the pedal to a position corresponding to that of stationary control station 5. This is supported, for example, by visual cues. It is conceivable that longitudinal control is supported by adaptive cruise control, so that neither the following nor the initial driver needs to perform longitudinal control. It is also possible that, instead of a manual takeover, at least a Level 3 ADS (short for "automated driving system"), analogous to the first configuration, assumes responsibility for the driving task during synchronization. Preferably, the changeover occurs at a constant speed, such as when parking or stopping, at a traffic light, on a straight road, in traffic jams, slow-moving traffic, or stop-and-go traffic. The processing unit 3 preferably ensures that a handover does not occur during moments of high deceleration or tight cornering.For this purpose, the current acceleration of vehicle 1 is monitored. Additionally, the processing unit 3 performs steering torque monitoring to check the prerequisites for the handover of vehicle 1: Initially, the first driver steers the vehicle, while the subsequent driver does not use the steering wheel. The steering wheel of the subsequent driver is synchronized according to its position. After a certain period, during which the subsequent driver occupies the second control station 5, the subsequent driver actively uses the steering wheel and generates a steering torque. If the steering torque of the subsequent driver falls below a predefined threshold for a sufficiently long period, the driving task is transferred to the subsequent driver, and the handover takes place.

[0031] Fig.Figure 4 shows a coordinate system whose vertical axis represents a force F on the pedal of the following driver, and whose horizontal axis represents a pedal travel x_ped on the following driver. A possible force curve for synchronizing the pedal at the second control station 5 for the following driver is shown: If the following driver's foot is not on the pedal, the force F acting on the driver's foot is zero; this is represented by the lower horizontal dashed line. The uppermost wavy curve describes the force F acting on the following driver's foot as soon as the driver places their foot on the pedal, where the force applied by the following driver is greater than the required force. In this case, the system generates a counterforce that is greater than the required pedal force, which is represented by the ascending dash-dot line.An oscillation of the counterforce can indicate to the driver that he needs to reduce the force.

[0032] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description. Reference symbol list 1 vehicle 3 Calculation unit 5 stationary control station QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2021 123 234 A1

[0003] DE 10 2021 117 448 A1

[0004] DE 10 2019 204 942 A1

[0005] DE 10 2015 118 489 B4

[0006] EP 3 893 073 A1

[0007] DE 10 2019 208 403 A1

[0008]

Claims

[1] System for handing over a teleoperated vehicle (1) from an initial driver to a subsequent driver while the vehicle (1) is traveling, comprising a computing unit (3) and at least one stationary control station (5) for teleoperating the vehicle (1), wherein the computing unit (3) is designed to execute a transition control mechanism for the handover at the at least one stationary control station (5), wherein the transition control mechanism comprises at least one safety query regarding the readiness of the subsequent driver, and driverless-machine-controlled vehicle guidance during the handover and / or feedback to the subsequent driver as to whether the latter is adopting a sufficiently adequate operating position on an input element for vehicle guidance of the vehicle (1) of the at least one stationary control station (5). [2] System according to claim 1, wherein the system has exactly one stationary control station (5), wherein the computing unit (3) is designed to take over the teleoperated vehicle (1) by means of driverless-machine-controlled vehicle guidance after confirmation of the planned handover by the initial driver, while the initial driver continues to be responsible for the teleoperated vehicle (1) until the subsequent driver has taken over the stationary control station (5), wherein the computing unit (3) is designed to switch off the driverless-machine-controlled vehicle guidance only upon a corresponding input by the subsequent driver or after vehicle guidance with control inputs at the stationary control station (5) of the subsequent driver that meet a predetermined operating metric, and to allow a manual takeover by the subsequent driver. [3] System according to claim 1, wherein the system comprises a first control station (5) for the initial driver and a second control station (5) for the following driver of a common teleoperated vehicle (1), wherein the computing unit (3) is designed to set an accelerator pedal position and / or a steering wheel position of the first control station (5) by means of corresponding actuators on the second control station (5) for the following driver during the handover and / or to output feedback to the second driver during the handover for approaching the accelerator pedal position and / or steering wheel position of the first control station (5). [4] System according to claim 3, wherein the computing unit (3) is designed to output haptic feedback on the accelerator pedal of the second stationary control station (5) as to whether the position of this accelerator pedal is sufficiently similar to the position of the accelerator pedal of the first stationary control station (5). [5] System according to claim 3, wherein the computing unit (3) is designed to carry out a speed control of the teleoperated vehicle (1) for at least the time of the handover, so that neither the initial driver nor the following driver has to operate the accelerator pedal of their respective stationary control station (5). [6] System according to one of the preceding claims, wherein the computing unit (3) is designed to check a current driving state of the teleoperated vehicle (1) for at least one predetermined condition and to carry out the handover only if the condition is met. [7] System according to claim 6, wherein the at least one predetermined condition comprises at least one of the following: the teleoperated vehicle (1) falls below a predetermined limit speed; the teleoperated vehicle (1) has a lateral acceleration below a predetermined limit; the teleoperated vehicle (1) has a steering angle below a predetermined limit; a longitudinal acceleration of the teleoperated vehicle (1) is below a predetermined limit; a torque on the steering wheel of the first stationary control station (5) is below a predetermined limit; [8] System according to one of claims 3 to 7, wherein the computing unit (3) is designed to synchronize the steering wheel position in the second stationary control station (5) with the steering wheel position of the first stationary control station (5) by means of an actuator, to check whether a limit torque on the steering wheel of the second control station (5) is undershot while the following driver operates the steering wheel of the second control station (5), and to allow the handover of the teleoperated vehicle control to the following driver only if the limit torque is undershot during a predetermined period of time. [9] System according to one of claims 3 to 8, wherein the computing unit (9) is designed to transmit a data stream with video images to the first stationary control station and synchronized therewith to the second stationary control station and to monitor the synchronization for temporal and content equality.

Images