Method for remotely controlling a vehicle, and remote control system

Abstract

The invention relates to vehicle remote control by means of a remote control system (10), comprising a remote control centre (16) having multiple control stations (18), a vehicle-based control apparatus (14) and an organization device (20). A primary control station (18) generates a control data stream for remotely controlling a vehicle (12) and transmits said stream to the control apparatus (14), which converts control commands (24) contained therein into driving actions of the vehicle (12). The control apparatus (14) generates a driving data stream (26) describing the driving actions of the vehicle (12) and transmits said stream to the organization device (20), which replicates said stream and provides it for the primary control station and at least one secondary control station (18). The driving actions are then observed by the at least one secondary control station (18) on the basis of the driving data stream (26), and / or the control commands (24) of the primary control station (18) are checked in at least one of the secondary control stations (18), and if they initiate unwanted driving actions of the vehicle (12), a takeover command (30) to take over remote control of the vehicle (12) is generated and transmitted to the organization device (20).

Description

[0001] Method for remotely controlling a vehicle and remote control system

[0002] The invention relates to a method for remotely controlling or remotely operating a vehicle, in particular a motor vehicle. Another aspect of the invention relates to a remote control system comprising a vehicle-external remote control center with a plurality of control stations from which the vehicle can be remotely controlled. Further aspects of the invention relate to a computing device configured to perform method steps according to the invention, as well as a corresponding computer program and a computer-readable storage medium with such a computer program.

[0003] Methods are known in which a vehicle, in particular a motor vehicle, is controlled by a human remote controller, in particular a teleoperator, positioned externally to the vehicle. The teleoperator is typically located in a stationary remote control center and thus in a building in which one or more display devices, in particular screens, are also arranged. On these screens, the remote controller can identify the vehicle to be remotely controlled and / or the surroundings of this remotely controlled vehicle, or this is displayed on the screens. In other words, the teleoperator is located in a control center that can be operated geographically independently of the location of the vehicle.

[0004] Such procedures differ significantly from those in which a driver is positioned outside of their own vehicle in close proximity and must be there in order to, for example, perform driving maneuvers with the vehicle they have left using a portable remote control held in one hand. For example, parking maneuvers are performed here. However, such procedures require, and are only permitted if, the driver is positioned with such a portable remote control in close proximity to the vehicle and must also be able to see the vehicle in order to control such maneuvers.

[0005] This is not the case with significantly different procedures, in which the above-mentioned scenario is carried out with a central remote controller located remotely from the vehicle. The remote controller in the remote control center is positioned in such a way that it cannot directly see the vehicle or the vehicle's surroundings. Therefore, with such a procedure, it is not necessary and may not even be possible for the remote controller to be positioned within a specified close range, for example, a maximum of 5 meters, of the vehicle to be remotely controlled.

[0006] Therefore, with such procedures it is more difficult for a remote control operator in a remote control center to be able to maneuver the vehicle safely during corresponding movements.

[0007] During teleoperated driving or the remote-controlled operation of a motor vehicle, the teleoperator typically controls the motor vehicle at least in part based on an image of the motor vehicle's surroundings, for example based on camera images or based on an image data stream that is transmitted or sent to them by an environmental sensor system or a sensor device of the motor vehicle, in particular by camera sensors of the motor vehicle. Typically, one vehicle at a time is remotely controlled by exactly one teleoperator by operating exactly one control station. In other words, in the known methods for remotely operating a vehicle from a control station, there is a 1:1 mapping between the control station and the vehicle. This gives rise to the disadvantage that, for example, during training of teleoperators, the teaching, experienced teleoperator and the novice have to share a control station.While the novice can observe the instructor operating the remote control, they lack direct control experience, including the haptic feedback from the control center's controls. However, if the instructor leaves the operation to the novice, they no longer have the opportunity to intervene directly, for example, to prevent the novice from maneuvering the remote-controlled vehicle into a critical or undesirable position.

[0008] The described 1:1 assignment also has the disadvantage that when there is a change of teleoperators, for example, during a shift change, the control center assigned to the vehicle must be handed over. The taking over teleoperator must undergo a complex training process on the previous route, with no or only limited remote control of the vehicle possible during the training. The invention is based on the object of improving the remote-controlled operation of vehicles, in particular remote-controlled motor vehicles, compared to the known procedure.

[0009] The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description, and the figures.

[0010] The invention is based on the finding that the remote-controlled operation of a vehicle can be improved by supplementing the described 1:1 assignment of exactly one control station to exactly one vehicle by allowing, in addition to the controlling or primary control station, other or secondary control stations to monitor the remote-controlled operation in real time and / or, under certain circumstances, even to take over.

[0011] The invention thus provides a method for remotely controlling a vehicle using a remote control system, wherein the remote control system comprises a vehicle-external remote control center with a predetermined number of control stations. The remote control system according to the invention also includes a vehicle-mounted control device and an organizational device.

[0012] The remote control center can comprise 10, 20, or even 50 or more control stations, which can be combined, for example, in a pool of control stations. Each control station can contain control elements for remotely controlling or operating the vehicle. In particular, the control stations can be identically designed. The control stations can be located in close proximity to one another. However, it is also conceivable and possible for the control stations to be distributed over a larger area. For example, the control stations can be distributed across several countries or even continents, enabling collaboration between international teams of teleoperators.

[0013] According to the invention, the control stations have different current authorizations. In other words, the control stations are not all permitted and / or able to perform the same remote control actions at the same time. Rather, according to the invention, a primary control station generates a control data stream for remotely controlling the vehicle and transmits it to the vehicle-side control device. In other words, the primary control station takes over the remote control of the vehicle. In particular, the control data stream can contain control commands that describe remote-controlled driving actions of the vehicle. For example, the control data stream can contain a control command that, when executed by the vehicle-side control device, causes it to control one or more actuators of the vehicle in order to adjust the current travel speed of the vehicle and / or to adjust a currently selected steering angle such that the vehicle negotiates a curve.

[0014] In other words, the vehicle-side control device receives the control data stream, in particular directly, from the primary control station and converts control commands contained in the control data stream into remote-controlled driving actions of the vehicle.

[0015] The control device not only executes the control commands of the control data stream, but also generates a driving data stream describing the remotely controlled driving actions of the vehicle. This can, for example, include sensor data acquired by sensors of an environmental sensor system of the vehicle while the driving actions are being executed. This can include, among other things, camera sensors that capture images of the vehicle's current surroundings. These images can be generated by the control device as an image data stream describing the driving actions as part of the driving data stream. The control device then transmits the driving data stream to the primary control center and / or to the organizational facility.The driving data stream may additionally or alternatively include driving parameters such as the current driving speed and / or information about a currently set steering angle and / or a current direction of travel (forward, reverse) and / or a currently engaged gear. The driving data stream may alternatively or additionally include data from acceleration sensors and / or any other vehicle sensors.

[0016] As described, the driving data stream is preferably transmitted from the vehicle-side control device to the organizational device, which may be a data processing device. The transmission can take place via a mobile radio connection that has the necessary bandwidth or transmission capacity for the transmission. The driving data stream can also be compressed as needed to enable transmission even with reduced transmission capacity. For transmission, the vehicle-side control device, the organizational device, and the control centers can have appropriately designed, known communication interfaces.

[0017] According to the invention, the organizational device receives the driving data stream from the control device, duplicates it, and then makes it available not only to the primary control center, but also to at least one additional, secondary control center of the predetermined number of control centers interconnected in the remote control center. A true end-to-end connection preferably exists only between the primary control center and the vehicle-side control device. The secondary control centers preferably receive duplicates of the driving data stream from the organizational device, which can be referred to as a "teleoperation data stream manager" (TDSM), and which handles the duplication and provision of the driving data stream. This advantageously protects the data connection between the vehicle and the remote control center.In addition, the provision of the driving data stream for the secondary control stations is advantageously not dependent on or limited by the existing transmission capacity of the data connection, which increases the reliability of the described method.

[0018] Based on the driving data stream or its duplicate, the remote-controlled driving actions are observed by at least one secondary control station. In other words, the operator of the secondary control station experiences which remote-controlled driving actions the vehicle performs based on the control data stream transmitted by the primary control station. In other words, they can view the remote-controlled driving actions in their control station or can follow them. Furthermore, by observing or evaluating the duplicate of the driving data stream, the operator of the secondary control station can precisely understand which actions are performed in the vehicle as a result of executing the control commands of the control data stream. The experience of the remote-controlled driving actions for the operator of the secondary control station is no different from that of the operator of the primary control station.In particular, a steering device simulation in the secondary control station can perform the same movements, especially rotations, as a steering device simulation in the primary control station. This advantageously allows the operator of the secondary control station to be trained in a realistic manner in remote control of the vehicle by the operator of the primary control station, thereby avoiding the disadvantages described above. Alternatively or additionally, the control commands of the primary control station are checked in at least one of the secondary control stations based on the driving data stream. If the control commands trigger undesired driving actions of the vehicle, a takeover command to overwrite the control commands of the primary control station is generated and transmitted to the organizational facility to take over remote control of the vehicle.In other words, alternatively or in addition to the observation described above, at least one of the secondary control stations can override the control commands from the primary control station and thus take over remote control of the vehicle. In one application situation, for example, a training teleoperator could take a seat in the secondary control station and, based on the driving data stream, understand the control commands of the learner driver in the primary control station. If the analysis of the driving data stream shows that the control commands from the primary control station generate undesirable driving actions, for example because they steer the vehicle away from a previously agreed route or because they lead to a vehicle speed that is higher than a previously agreed maximum value, the driving instructor in the secondary control station can generate the takeover command, which is then transmitted to the organizational facility.The organizational unit can then interrupt the connection between the primary control center and the vehicle-side control device and transfer it to the secondary control center. This allows the teleoperator of the primary control center to be effectively trained in remote vehicle control, while simultaneously reliably preventing critical or undesirable vehicle actions.

[0019] The invention also includes embodiments which provide additional advantages.

[0020] One embodiment provides that the organizational unit receives an authorization request from a respective control center and, based on the authorization request, assigns the respective control center authorization as the primary or secondary control center. In other words, all control centers that wish to participate in the described process must first obtain or request the corresponding authorization from the organizational unit. This can be done via an automated request, which can, for example, always be triggered at fixed and known shift change times in the operation of the remote control center. Alternatively or additionally, it can be provided that a new, previously unknown control center wishes to participate in an ongoing training session. To this end, the control center can submit a corresponding authorization request to the organizational unit, which can then admit or deny the request.Authorizations can, for example, be restricted to different vehicle types. For example, a control center may be authorized to remotely control a vehicle of one type, but not a vehicle of another type. The organizational unit can review and take this into account when granting authorizations.

[0021] According to a further development, the organizational device has a storage unit in which a predetermined number of authorization tokens is stored. In response to the authorization request, the organizational device issues the authorization tokens to the control centers in order to assign the respective authorizations to the control centers. In other words, electronic identification tags, preferably in the form of a sequence of related characters or bits, can be stored in the storage unit. These identification tags can be issued by the organizational device to the control centers to answer the authorization requests. The character sequence of a token can thus comprise, for example, the authorization to generate and transmit a control data stream with one or more control commands relating to a predetermined vehicle.The character string of another token can include the authorization to generate and transmit a takeover command to take over remote control of the vehicle. It can be provided that a respective control center returns the token assigned to it to the organizational unit at the end of a remote control session, for example, at the shift change described above, so that the organizational unit can transfer the token to another control center for further operation. For example, a flying change between two control centers can occur, whereby the taking over control center has been able to monitor the remote-controlled operation for some time prior to the takeover using the duplicate driving data stream. In particular, it can be provided that no separate token is necessary for simply observing the remote-controlled driving actions.

[0022] Further training requires the organizational unit to issue authorization tokens based on the qualification level of the requesting control center. In other words, a driving instructor token can only be issued to a control center where a teleoperator qualified as a driving instructor is currently logged on. Allocation can also be restricted to specific vehicle classes. In other words, the tokens can be assigned to specific, predetermined vehicles. For example, a situation may arise where a control center where a teleoperator qualified as a learner driver in a predetermined vehicle class is currently logged on requests a takeover token or a driving instructor token for a vehicle in the predetermined vehicle class. This cannot be granted, even though the teleoperator may be qualified as a driving instructor in a different vehicle class.

[0023] This ensures that only authorized groups of people receive extensive authorizations. It should be noted that the qualification level of a control center is preferably defined by its current user. Alternatively or additionally, a specific control center can also be assigned a qualification level as such, regardless of the user currently operating it. For the sake of simplicity, here and in the following, we will generally refer to the qualification level of the control center, although both alternatives can be intended.

[0024] In this context, further training requires the organizational unit to issue a first-level authorization token to a control center with a first-level qualification level and a second-level authorization token to a control center with a second-level qualification level. Holders of the first-level authorization tokens can be authorized to overwrite the control commands of holders of the second-level authorization tokens. Whether a token is first-level or second-level can be reflected in its predetermined character sequence. Based on the number of issued tokens, it is possible, for example, for quality assurance purposes, to reliably track which teleoperators have participated in which training courses on which vehicle classes and, if applicable, how and / or how often their control commands were overwritten in order to take corrective action.

[0025] A further development stipulates that the organizational unit only accepts the takeover command from a control center with the first-level qualification level and only forwards the takeover command to the vehicle's control device upon receipt of proof of the first-level qualification level. Preferably, the organizational unit requests proof of the first-level qualification level from the receiving control center. Alternatively, the overriding control center can also transmit the proof of the first-level qualification level to the organizational unit along with the takeover command.

[0026] One embodiment provides that the organizational facility, in particular based on the current authorizations of the control centers, deregisters a current primary control center and subsequently registers a formerly secondary control center as the new primary control center. For example, the current primary control center can have a current authorization which defines that it is only valid until a specific secondary control center registers with the organizational facility. In this case, the specific secondary control center can be registered directly as the primary control center and assume remote control. Alternatively or additionally, it can be provided that the current authorization of the current primary control center is only valid until a shift change and that the formerly secondary control center is automatically registered as the new primary control center when the shift of the current primary control center ends.At the end of the current primary control center's shift, the organizational unit can also automatically maneuver the vehicle into a predetermined parking position, from which the former secondary control center can take over. This advantageously keeps the vehicle under continuous control.

[0027] One embodiment provides that the vehicle to be remotely controlled is selected from a list of available vehicles by the primary control center. In other words, the list can be displayed on a display device of the primary control center. The operator or teleoperator of the primary control center can select the vehicle, for example, using a touch command on the display device. This selection can then be transmitted as an authorization request to remotely control this vehicle to the organizational facility, where the authorization can be assigned as described.

[0028] Preferably, in the event of a communication interruption between at least one of the control centers and the organizational facility, the current authorizations of the control centers are retained for a predetermined period of time. In other words, the assignment of control center - token - vehicle should not be canceled for a predetermined period of time in the event of a connection loss. This allows remote control to be resumed seamlessly once the connection is restored.

[0029] A further aspect of the invention relates to a remote control system for remotely controlling a vehicle, comprising a vehicle-external remote control center, a vehicle-mounted control device and an intermediate organizational device, which can be designed as a data processing device or computing device, wherein - the vehicle-external remote control center comprises a predetermined number of control stations with different authorizations, wherein a primary control station of the predetermined number of control stations is authorized to generate a control data stream for remotely controlling the vehicle and to transmit it to the vehicle-mounted control device,

[0030] - the vehicle-side control device is designed to receive the control data stream and to convert the control commands contained therein into remote-controlled driving actions of the vehicle, and to generate a driving data stream describing the remote-controlled driving actions and to transmit it to the organizational device,

[0031] - the organizational device is designed to receive or retrieve the driving data stream describing the remote-controlled driving actions from the vehicle-side control device, to reproduce it and to make it available to the primary control station and at least one further, secondary control station of the predetermined number of control stations.

[0032] At least one secondary control center is authorized to

[0033] - to observe the remote-controlled driving actions based on the driving data stream and / or

[0034] - to check the control commands of the primary control station on the basis of the driving data stream and, if the control commands trigger undesired driving actions of the vehicle, to overwrite the control commands with a takeover command in order to take over the remote control of the vehicle from the primary control station.

[0035] A further development of the remote control system provides that a respective control station comprises a passenger compartment simulation with multiple control elements, wherein the control data stream can be generated by operating the control elements. The passenger compartment simulation can, for example, comprise foot pedals that are reminiscent of or modeled on real brake, clutch, and / or accelerator pedals. The passenger compartment simulation can also contain a steering device simulation, for example in the form of a steering wheel. Similar to a drive-by-wire or conventional vehicle, the steering wheel can be configured to generate a restoring force upon a steering movement to return the steering wheel to a neutral starting position.

[0036] In this context, a further development provides for the steering wheel to move in accordance with the remote-controlled driving actions in the control center. The steering wheel therefore preferably rotates in all control centers of the remote control system according to the control commands of the control data stream. The takeover command can preferably be generated by gripping the steering wheel of at least one secondary control center. In other words, the takeover command can be generated by operating one or more control elements, in particular by operating the steering wheel. This enables a particularly intuitive takeover by the authorized teleoperator.

[0037] For applications or application situations that may arise in a method according to the invention and which are not explicitly described herein, it may be provided that, according to the method, an error message and / or a request to enter user feedback is output and / or a standard setting and / or a predetermined initial state is set.

[0038] A further aspect of the invention relates to a computing device configured to execute method steps of the method according to the invention as an organizational device for a remote control system according to the invention. For this purpose, the computing device can have multiple computing units.

[0039] In the present disclosure, a computing unit can be understood, for example, as a data processing device with processing circuits. A computing unit can therefore perform computing operations to process data. The computing operations can also include indexed accesses to a data structure, for example, a look-up table (LUT).

[0040] A computing unit can in particular comprise one or more computers, one or more microcontrollers and / or one or more integrated circuits, for example one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more single-chip systems (SoCs). The computing unit can also contain one or more processors, for example one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit can also comprise a physical or virtual cluster of computers or other of the aforementioned units.

[0041] A computing unit can also comprise one or more hardware and / or software

[0042] Interfaces and / or one or more memory units. A memory unit can be designed as a volatile data memory, for example as a dynamic random access memory (DRAM) or a static random access memory (SRAM), or as a non-volatile data memory, for example as a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or flash EEPROM, a ferroelectric random access memory (FRAM), a magnetoresistive random access memory,MRAM (magnetoresistive random access memory), or phase-change random access memory, PCRAM (phase-change random access memory).

[0043] According to a further aspect of the invention, a computer program with instructions is provided. When the instructions are executed by a computing device or at least one computing unit of the computing device, the instructions cause the computing device or the at least one computing unit to perform a method according to the invention.

[0044] The instructions can be provided, for example, as program code. The program code can be provided, for example, as binary code or assembly code and / or as source code of a programming language, for example, C, and / or as a program script, for example, Python.

[0045] According to a further aspect of the invention, a computer-readable storage medium is provided which stores a computer program according to the invention.

[0046] The computer program and the computer-readable storage medium are each computer program products containing the instructions.

[0047] Further embodiments of the remote control system according to the invention follow directly from the various embodiments of the method according to the invention for remotely controlling a vehicle using the remote control system, and vice versa. In particular, individual features and corresponding explanations, as well as advantages relating to the various embodiments of the method according to the invention, can be transferred analogously to corresponding embodiments of the further aspects of the invention. In particular, the computing device according to the invention is designed or programmed to carry out a method according to the invention.

[0048] Further features of the invention emerge from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and / or shown in the figures can be encompassed by the invention not only in the respectively specified combination, but also in other combinations. In particular, the invention can also encompass embodiments and combinations of features that do not have all the features of an originally formulated claim. Furthermore, the invention can encompass embodiments and combinations of features that go beyond the combinations of features set out in the backreferences to the claims or deviate from them.

[0049] The figures show:

[0050] Fig. 1 is a schematic representation of a remote control system according to one aspect of the present invention;

[0051] Fig. 2 is a schematic representation of the duplication of a driving data stream in an organizational device which is connected between a vehicle-side control device and a remote control center with several control stations;

[0052] Fig. 3 shows a schematic representation of process steps for issuing an authorization token to a control center based on an authorization request; and

[0053] Fig. 4 is a schematic representation of a method for remotely controlling a vehicle by means of a remote control system according to one aspect of the present invention.

[0054] Fig. 1 shows a schematic representation of a remote control system 10 for remotely controlling a vehicle 12. The vehicle 12 can be, for example, a remote-controlled passenger car, passenger bus, or delivery van. The vehicle 12 can also be a remote-controlled logistics vehicle used, for example, in the fully or partially automated operation of a logistics company, such as a remote-controlled forklift.

[0055] The remote control system 10 comprises, in addition to a control device 14 of the vehicle 12, a vehicle-external remote control center 16 with a predetermined number n of control stations 18. At the center of the remote control system 10 shown is an organizational device 20 which is interposed for communication purposes between the control device 14 of the vehicle 12 and the remote control center 16.

[0056] The organizational device 20, also called the "teleoperation data stream manager," or TDSM for short, operates a communication link 22 to the control device 14 of the vehicle 12, via which control commands 24 can be transmitted from a primary control center 18 to the control device 14. In the opposite direction, a driving data stream 26 can be transmitted from the control device 14 to the organizational device 20 via the data link 22, wherein the driving data stream 26 describes remotely controlled driving actions of the vehicle 12, which the vehicle performs based on the transmitted control commands 24.

[0057] The data connection 22 can be a cellular connection and, as such, have a limited data capacity. In order to keep the data volume transmitted via the data connection 22 low and simultaneously make the information from the driving data stream 26 available to all control centers 18, the driving data stream 26 is duplicated or multiplied in the organization device 20 and made available to the control centers 18. This is illustrated in Fig. 1 by the transmitted duplicates 28. In addition to duplicating and providing the driving data stream 26, the organization device 20 can also receive the control commands 24 from the primary control center 18 and forward them to the control device 14 of the vehicle 12. The organization device 20 thus bundles and organizes, in a sense, the communication between the remote control center 16 and the vehicle 12.

[0058] With reference to the components identified and described in connection with Fig. 1, Fig. 2 shows a schematic detailed view of the described communication paths and the duplication of the driving data stream 26 in the organizational device 20. In the exemplary embodiment shown here, the control station 18.1 is a primary control station 18, which generates the control data stream with the control commands 24 and transmits it to the control device 14 of the vehicle 12. The control data stream can be transmitted to the control device 14 via the organizational device 20, as shown here. Alternatively, the primary control station 18.1 can also operate a separate data connection to the control device 14, whereby a redundantly secured connection can be provided.

[0059] The control device 14 transmits the driving data stream 26 to the organizational device 20. In other words, the organizational device 20 is the only communication partner for the control device 14. This makes it easy to implement a standardized communication interface in the vehicle 12.

[0060] The driving data stream 26 is received and copied or duplicated in the organizational facility 20. The duplicates 28 are made available to all control centers 18, i.e., both the primary control center 18.1 and the secondary control centers 18.2 to 18.n. This allows all control centers 18 to monitor the driving actions of the vehicle 12 without overloading the data connection 22.

[0061] By generating and transmitting a takeover command 30, one of the secondary control stations 18.2 to 18.n (in the example in Fig. 2, this is control station 18.n) can interrupt and take over the control data stream of the primary control station 18.1, provided it is authorized to do so. The organizational unit 20 can determine whether the corresponding authorization exists in the manner described above, for example, by checking whether the secondary control station 18.n has a corresponding authorization token.

[0062] Fig. 3 shows, with reference to the components shown and described in connection with the previously described figures, a schematic representation of method steps for issuing an authorization token to a control center 18 based on an authorization request. The method shown here begins with a step S3.1, which comprises starting the described remote control system 10. In a step S3.2, the organizational device 20 waits for an authorization request from one of the control centers 18. In a step S3.3, such an authorization request is received by the organizational device 20 from a control center 18. Here, the control center 18 asks for a token, for example, which would authorize it to remotely control a very specific vehicle 12 from a list of available vehicles 12. In a step S3.4, the organizational facility 20 checks whether the requested token is available and preferably also whether the requesting control center 18 is authorized to remotely control the specific vehicle 12. If the organizational facility 20 must deny one or both of these requirements, it rejects the authorization request in a step S3.5 and preferably returns to the wait state of step S3.2. However, if the organizational facility 20 can affirm that both requirements are met, it allocates the requested token to the control center 18 in a step S3.6. The decision made by the organizational facility 20 is stored in a step S3.7, so that even in the event of a brief interruption of the data connection 22 or other communication connections in the remote control system 10, the allocation of the token to the control center 18 remains intact. In a step S3.8, it is checked whether the remote control system 10 is functioning properly.If this is affirmative, the organizational device 20 returns to the waiting state of step S3.2. Otherwise, the described method ends in step S3.9.

[0063] Fig. 4 shows, with reference to the components identified and described in connection with the preceding figures, a schematic representation of a method for remotely controlling a vehicle 12 by means of a remote control system 10 according to one aspect of the present invention.

[0064] The remote control system 10 comprises a vehicle-external remote control center 16 with a predetermined number of control stations 18 with different authorizations, a vehicle-side control device 14 and an organizational device 20.

[0065] In a step S4.1, a primary control station 18 of the predetermined number of control stations 18 generates a control data stream for remotely controlling the vehicle 12 and transmits it to the vehicle-side control device 14. In a step S4.2, the vehicle-side control device 14 receives the control data stream and converts control commands 24 contained in the control data stream into remote-controlled driving actions of the vehicle 12. In a step S4.3, the vehicle-side control device 14 generates a driving data stream 26 describing the remote-controlled driving actions of the vehicle 12 and transmits it to the organizational device 20. In a step S4.4, the organizational device 20 receives the driving data stream 26, duplicates it, and makes it available to the primary and at least one further, secondary control station 18 of the predetermined number of control stations 18. In a step S4.5, the remote-controlled driving actions are monitored by the at least one secondary control station 18 based on the driving data stream 16. Alternatively or additionally, in a step S4.6, the control commands 24 of the primary control station 18 are checked in at least one of the secondary control stations 18 based on the driving data stream 26. If the control commands 24 trigger undesired driving actions of the vehicle 12, a takeover command 30 for overwriting the control commands 24 of the primary control station 18 is generated in the secondary control station 18 and transmitted to the organizational device 20 in order to take over the remote control of the vehicle 12.

[0066] One aspect of the invention relates to the described organizational device 20. This device can connect an active vehicle 12 to an active teleoperation cockpit or control center 18 and any number of passive control centers 18. To this end, the device preferably uses a token system with automatic detection of connection losses and control recovery to ensure smooth operation.

[0067] Although a conventional computer network may be used to interconnect the individual elements of the teleoperation system or remote control system 10, the device's data flow is preferably organized in a star topology to connect all cockpits and the teleoperated vehicle(s) 12 (see Fig. 1).

[0068] The device, the Teleoperation Data Stream Manager (TDSM), contains a token for each vehicle 12. The cockpits 18 can request the token of any vehicle 12 to take control of that vehicle 12. If the device still has the requested token, it is passed on to the cockpit 18. This cockpit 18 is then authorized to control the vehicle 12. All other cockpits 18 can receive the vehicle data or the driving data stream 26 as passive participants.

[0069] To minimize data traffic, the TDSM supports stream splitting (see Fig. 2). The data from the vehicle 12 (e.g., video) can be transmitted in a single stream or driving data stream 26 from the vehicle 12 to the device, i.e., to the organizational facility 20. The device then makes the necessary data available to all connected cockpits 18 to minimize data traffic on the mobile data connection. In the event of a connection interruption of the remote-controlled vehicle 12, the device waits a defined time before invalidating the token assigned to the cockpit 18 with the connection interruption. It can then generate a new, valid token that can be passed on to another cockpit 18. Since the old token is no longer valid, the first cockpit 18 to experience the connection loss cannot control the vehicle 12 in parallel if it later resolves the connection loss.

[0070] An embodiment of the invention allows two control stations 18 to control a vehicle 12 simultaneously, with either one control station 18 overriding the other or interrupting the simultaneous control with force feedback.

[0071] The TDSM 20 is a practical tool for training new teleoperators. Instead of interfering with the day-to-day operations of the control center or remote control center 16, future teleoperators can be trained in real cockpits 18 while observing normal operations. Supervisors can use each cockpit 18 to monitor their team members' performance live and in real time.

[0072] For this purpose, the new teleoperators can be located in teleoperation cockpits 18 operating in passive mode and observe the active display on which an experienced teleoperator is controlling a vehicle 12. An instructor teaches the trainees, and the experienced teleoperator is not disturbed in their work. Another application example concerns the monitoring of team members. For this purpose, the team leader of a teleoperation team can monitor and evaluate the performance of each team member. The team leader can do this by observing the vehicles 12 controlled by the team using a teleoperation cockpit 18 operating in passive mode.

[0073] Overall, the examples show how the remote-controlled operation of a vehicle can be improved by integrating multiple control centers.

Claims

Patent claims 1. A method for remotely controlling a vehicle (12) by means of a remote control system (10), comprising a vehicle-external remote control center (16) with a predetermined number of control stations (18) with different authorizations, a vehicle-side control device (14) and an organization device (20), wherein - a primary control station (18) of the predetermined number of control stations (18) generates a control data stream for remotely controlling the vehicle (12) and transmits it to the vehicle-side control device (14), - the vehicle-side control device (14) receives the control data stream and converts control commands (24) contained in the control data stream into remote-controlled driving actions of the vehicle (12), - the vehicle-side control device (14) generates a driving data stream (26) describing the remote-controlled driving actions of the vehicle (12) and transmits it to the organizational device (20), - the organization device (20) receives the driving data stream (26), duplicates it and makes it available to the primary and at least one further secondary control station (18) of the predetermined number of control stations (18), and wherein - the remote-controlled driving actions are observed by the at least one secondary control station (18) based on the driving data stream (26), and / or wherein - in at least one of the secondary control stations (18), the control commands (24) of the primary control station (18) are checked on the basis of the driving data stream (26) and, if the control commands (24) trigger undesired driving actions of the vehicle (12), a takeover command (30) for overwriting the control commands (24) of the primary control station (18) is generated and transmitted to the organizational device (20) in order to take over the remote control of the vehicle (12).

2. The method according to claim 1, wherein the organizational device (20) receives an authorization request from a respective control center (18) and based on the authorization request, assigns the respective control station (18) the authorization as primary or secondary control station (18).

3. The method according to claim 2, wherein a predetermined number of authorization tokens is stored in a memory unit of the organizational device (20), wherein the organizational device (20) issues the authorization tokens to the control centers (18) in response to the authorization request in order to assign the respective authorization to the control centers (18).

4. The method according to claim 3, wherein the organizational device (20) assigns the authorization tokens taking into account a qualification level of the requesting control center (18).

5. The method according to claim 4, wherein the organizational device (20) assigns a first-ranking authorization token to a control center (18) with a first-ranking qualification level, and assigns a second-ranking authorization token to a control center (18) with a second-ranking qualification level.

6. The method according to claim 5, wherein the organizational device (20) accepts the takeover command (30) only from a control center (18) with the first-ranking qualification level and forwards the takeover command (30) to the control device (14) of the vehicle (12) only upon the presence of proof of the presence of the first-ranking qualification level.

7. The method according to claim 6, wherein the organizational unit (20) requests proof of the existence of the first-ranking qualification level from the receiving control center (18).

8. The method according to claim 6, wherein the receiving control center (18) transmits the proof of the existence of the first-ranking qualification level together with the takeover command (30) to the organizational facility (20).

9. Method according to one of the preceding claims, wherein the organizational device (20), in particular based on the current authorizations of the control centers (18), deregisters a current primary control center (18) and a subsequent registration of a former secondary control center (18) as the new primary control center (18).

10. The method according to any one of the preceding claims, wherein the vehicle (12) to be remotely controlled is selected from a list of available vehicles (12) by the primary control center (18).

11. Method according to one of the preceding claims, wherein in the event of a communication interruption between at least one of the control stations (18) and the organizational device (20), the current authorizations of the control stations (18) are retained for a predetermined time.

12. Remote control system (10) for remotely controlling a vehicle (12), comprising a vehicle-external remote control center (16), a vehicle-side control device (14) and an intermediate organization device (20), wherein - the vehicle-external remote control center (16) comprises a predetermined number of control stations (18) with different authorizations, wherein a primary control station (18) of the predetermined number of control stations (18) is authorized to generate a control data stream for remotely controlling the vehicle (12) and to transmit it to the vehicle-side control device (14), - the vehicle-side control device (14) is designed to receive the control data stream and to convert control commands (24) contained therein into remote-controlled driving actions of the vehicle (12), and to generate a driving data stream (26) describing the remote-controlled driving actions and to transmit it to the organizational device (20), - the organization device (20) is designed to receive the driving data stream (26) describing the remote-controlled driving actions from the vehicle-side control device (14), to duplicate it and to provide it to the primary control station (18) and at least one further, secondary control station (18) of the predetermined number of control stations (18), wherein the at least one secondary control station (18) is authorized to do so, - to observe the remote-controlled driving actions based on the driving data stream (26) and / or - to check the control commands (24) of the primary control station (18) on the basis of the driving data stream (26) and, if the control commands (24) trigger undesired driving actions of the vehicle (12), to overwrite the control commands (24) by a takeover command (30) in order to take over the remote control of the vehicle (12) from the primary control station (18).

13. Remote control system (10) according to claim 12, wherein a respective control station (18) comprises a passenger compartment simulation with a plurality of control elements, wherein the control data stream can be generated by operating the control elements.

14. Remote control system (10) according to claim 13, wherein the passenger compartment simulation comprises a steering device simulation with a steering wheel which is designed to move in accordance with the remote-controlled driving actions in the control station (18).

15. Remote control system (10) according to one of claims 13 or 14, wherein the takeover command (30) can be generated by operating one or more control elements, in particular by operating the steering wheel.

16. Computing device designed to carry out method steps of a method according to one of claims 1 to 11 as an organizational device (20) for a remote control system (10) according to one of claims 12 to 15.

17. A computer program comprising instructions which, when executed by a computing device according to claim 16, cause the computing device to perform method steps of a method according to any one of claims 1 to 11.

18. A computer-readable storage medium comprising a computer program according to claim 17.

Images