Method and system for remotely controlling a vehicle detected by an infrastructure facility by a teleoperator and computer program product for a remote control station

Abstract

The invention relates to a method and a system (10) for remotely controlling a vehicle (14) by a teleoperator (12), in which at least one image (20) is output to the teleoperator (12) on a display device (18) of a remote control station (16) external to the vehicle for the remote control. The teleoperator (12) causes at least one control command (26) to be output by the remote control station (16). A receiving device of the vehicle (14) receives the at least one control command (26) wirelessly, wherein the at least one control command (26) is implemented by a device internal to the vehicle. To create the at least one image (20), the remote control station (16) uses sensor signals which are transmitted to the remote control station (16) by at least one infrastructure device (42, 44, 46, 48) external to the vehicle. The invention further relates to a computer program product for a remote control station (16) external to the vehicle.

Description

[0001] The invention relates to a method for remotely controlling a vehicle by a teleoperator, in which at least one image is output to the teleoperator on a display device of a remote control station external to the vehicle for the remote control. The teleoperator causes at least one control command to be output by the remote control station, with a receiving device of the vehicle receiving the at least one control command wirelessly. The at least one control command is implemented by a device internal to the vehicle. Furthermore, the invention relates to a system for remotely controlling a vehicle by a teleoperator and a computer program product for a remote control station external to the vehicle.

[0002] DE 10 2020 112 468 A1 describes the monitoring of an automated parking process by a teleoperator. A vehicle user initiates the automated parking process, the execution of which is then monitored by the teleoperator. Using on-board environmental sensors, monitoring information, for example in the form of a video stream, is transmitted via a mobile network to a teleoperator station during the parking process. The monitoring information is displayed to the teleoperator via monitors at a teleoperator workstation.

[0003] A disadvantage here is the fact that very powerful environmental sensors must be provided on the vehicle to ensure that sufficiently meaningful monitoring information can be provided to the teleoperator via the monitors. This entails a correspondingly high level of effort in terms of the vehicle's equipment.

[0004] Furthermore, a high-performance mobile network is required to ensure that the surveillance information, particularly in the form of a video stream, can be transmitted to the teleoperator station via the mobile network. In this regard, it should be noted that data packet loss may occur, especially when attempting to transmit large amounts of data to the teleoperator station over a mobile network with insufficient performance.

[0005] Furthermore, it can result in a high load or even an overload of the mobile network if monitoring information is transmitted from a large number of vehicles to the teleoperator station via the mobile network.

[0006] Ultimately, it is difficult to provide a mobile network with high availability and, at the same time, high bandwidth or high performance everywhere where this would be desirable. Firstly, coverage by a sufficiently powerful mobile network is not available everywhere. Secondly, spatial conditions can make it difficult to wirelessly transmit the data collected by the vehicle's environmental sensors to the teleoperator station. This is especially true if the vehicle is located in a building such as a parking garage or similar.

[0007] US 2022 / 0139222 A1 describes a method in which a vehicle is not controlled by a teleoperator, but automatically by a device external to the vehicle.

[0008] The disadvantage here is that, due to the lack of a teleoperator, the teleoperator cannot intervene if problems arise during automated remote control.

[0009] The object of the present invention is to provide a method of the type mentioned at the outset which is particularly low-complexity, as well as to create a corresponding system and a computer program product.

[0010] This object is achieved by a method having the features of patent claim 1, a system having the features of patent claim 13, and a computer program product having the features of patent claim 14. Advantageous embodiments with expedient further developments of the invention are specified in the dependent patent claims and in the following description.

[0011] In the method according to the invention for remotely controlling a vehicle by a teleoperator, at least one image is output to the teleoperator for remote control on a display device of a remote control station external to the vehicle. The teleoperator causes at least one control command to be output by the remote control station, with a receiving device of the vehicle receiving the at least one control command wirelessly. The at least one control command is implemented by an internal vehicle device. In the method, the remote control station uses sensor signals, which are transmitted to the remote control station by at least one infrastructure device external to the vehicle, to create the at least one image.

[0012] This eliminates the need to provide very complex equipment on the vehicle side to provide sensor signals that can be used to create the at least one image. Rather, the infrastructure facility can be used to capture at least a portion of the vehicle and / or at least part of the surroundings of the vehicle being driven by the teleoperator. This is advantageous.

[0013] If, on the other hand, at least one image that can be displayed to the teleoperator on the display device of the remote control station is to be output based on environmental sensor signals provided by an environmental sensor system of the vehicle itself, a correspondingly complex environmental sensor system must be provided on the vehicle side.

[0014] In this case, the remote control station accesses the sensor signals transmitted to the remote control station by the infrastructure facility. This allows the teleoperator to remotely control even vehicles that have no or only very low-performance environmental sensors. This allows the teleoperator to remotely control a wide variety of different vehicles by causing the remote control station to issue at least one control command to the respective receiving device of the vehicle.

[0015] Furthermore, it is no longer necessary to provide a very powerful transmission device on the vehicle side, which is designed to transmit large amounts of data in the form of environmental sensor signals. Furthermore, the bandwidth requirements of a vehicle-side transmission device are reduced.

[0016] Furthermore, receiving the at least one control command by the vehicle's receiving device requires significantly less effort in terms of the amount of data to be transmitted than transmitting, for example, a video stream to the remote control station. This contributes to the low effort required to implement the method.

[0017] Furthermore, a high degree of robustness in the transmission of sensor signals from the off-board infrastructure to the remote control station can be ensured much more easily than would be the case with wireless transmission of large amounts of data from the vehicle to the remote control station. This is advantageous with regard to high reliability or dependability in the transmission of sensor signals to the remote control station.

[0018] In particular, if video data is compressed to provide the at least one image outputtable on the display device, the computational effort required to compress video data does not need to be performed on the vehicle side. Rather, such compression of video data can be accomplished very easily on the infrastructure device side and / or on the remote control station side. The remote control station and / or the infrastructure device can advantageously provide a correspondingly high computing capacity particularly easily.

[0019] The remote control station, from which the teleoperator can issue the control commands to the vehicle, is preferably a stationary station, located, for example, in a building. Such a remote control station is therefore designed differently from a portable remote control or a remote control device by means of which an operator can control a vehicle while maintaining direct visibility of the vehicle. When controlling the vehicle remotely using the remote control, it is imperative that the operator is in close proximity to the vehicle and constantly has the vehicle in view.Furthermore, due to relevant regulations, remote control or maneuvering of the vehicle using the remote control device is only permitted if the operator is in close proximity to the vehicle and the vehicle is within the operator's direct field of vision. Furthermore, such a remote control device is a portable device, whereas the fixed remote control station is at least essentially immobile.

[0020] For the teleoperator, who can make operating inputs at the remote control station in order to cause the remote control station to issue at least one control command, it is not necessary, however, to have the vehicle directly in view. Rather, the teleoperator can view the vehicle on the display device, which can be designed as a screen or the like, for example, and which is associated with the remote control station. Additionally or alternatively, at least part of the surroundings of the vehicle driven by the teleoperator can be displayed on such a screen. However, this is not permissible or sufficient if driving maneuvers of the vehicle are to be effected using the portable remote control device or the portable remote control. Rather, when using the remote control or the remote control device, direct visual contact of the operator with the vehicle is required and prescribed.

[0021] The vehicle-external infrastructure device is preferably a stationary device which, on the one hand, has at least one sensor device which generates the sensor signals during operation and, on the other hand, comprises a unit for forwarding or transmitting the sensor signals.

[0022] The at least one sensor device of the infrastructure facility can be designed, in particular, as a camera and / or a lidar device. Such sensor devices can be used to very reliably create a model, particularly a three-dimensional one, of the vehicle's surroundings in which the vehicle is located. This is advantageous for outputting an image that is easily usable by the teleoperator on the display device of the remote control station, which can be designed, for example, as a screen.

[0023] Preferably, the sensor signals are transmitted from the at least one infrastructure device to the remote control station via a wired connection. This is advantageous with regard to highly reliable and efficient transmission of the sensor signals to the remote control station. In particular, the difficulties and problems associated with transmitting data in the form of sensor signals to the remote control station via a mobile radio network are advantageously eliminated.

[0024] Preferably, a sensor device of the at least one infrastructure facility detects the vehicle within a detection range of the sensor device, wherein the sensor signals of the sensor device are transmitted to the remote control station depending on the presence of the vehicle within the detection range. This ensures that only the infrastructure facility whose sensor device actually detects the vehicle within its detection range needs to transmit sensor signals to the remote control station. This contributes to the low-complexity implementation of the method.

[0025] In particular, the detection of the vehicle or at least a portion of the vehicle within the detection range of the sensor device on the infrastructure facility side can be used as a trigger for transmitting the sensor signals to the remote control station. This can be implemented very easily in terms of control technology.

[0026] Preferably, the at least one infrastructure facility tracks a trajectory of the vehicle. When the vehicle leaves the detection range of a sensor device of a first infrastructure facility and enters the detection range of a sensor device of a second infrastructure facility, the transmission of sensor signals from the sensor device of the first infrastructure facility to the remote control station is prevented. In contrast, the sensor signals detected by the sensor device of the second infrastructure facility are transmitted to the remote control station. On the one hand, this advantageously minimizes the volume of sensor signals to be transmitted.On the other hand, this ensures that the teleoperator can use at least one image very effectively to remotely control the vehicle by issuing the control commands, which are carried out by the remote control station at the request of the teleoperator.

[0027] Switching from the sensor device of the first infrastructure facility to the sensor device of the second infrastructure facility can be effected by at least one of the two infrastructure facilities. This makes the method particularly simple. Additionally or alternatively, switching from the sensor device of the first infrastructure facility to the sensor device of the second infrastructure facility can be effected by the teleoperator. For example, the respective infrastructure facilities can inform and / or indicate to the teleoperator whether the vehicle is within the detection range of the sensor device of the respective infrastructure facility. In this case, the infrastructure facilities can suggest to the teleoperator that the switch should be made. The teleoperator can then confirm the switch.

[0028] Additionally or alternatively, the teleoperator can, on its own initiative or without such a suggestion, ensure that the transmission of sensor signals from the sensor device of the first infrastructure facility to the remote control station is prevented when the vehicle is no longer within the detection range of the sensor device of the first infrastructure facility. In this way, the teleoperator can replace an evaluation device that would otherwise be provided on at least one of the infrastructure facilities, or at least reduce the load on such an evaluation device.

[0029] A sensor device of the at least one infrastructure facility can be designed as a camera, with a camera image captured by the camera and showing at least a partial area of the vehicle being output as the at least one image on the display device of the remote control station. This allows the teleoperator to have a very realistic impression of the vehicle's driving situation on the display device. This is advantageous. Furthermore, no complex processing of the sensor signals transmitted from the camera to the remote control station is required. This contributes to the low-effort implementation of the method.

[0030] In an advantageous embodiment, the remote control station creates the at least one image from sensor signals transmitted to the remote control station by a plurality of infrastructure facilities. To create the at least one image, a plurality of sub-elements are combined, with each sub-element being formed using the sensor signals from one of the infrastructure facilities. The use of the at least one image combined or composed from multiple sub-elements makes it particularly easy for the teleoperator to trigger the output of suitable control commands by the remote control station based on the image and thus to remotely control the vehicle. This is because the teleoperator can be given a very comprehensive impression of the vehicle's current driving situation on the display device of the remote control station by outputting the at least one image.

[0031] This is particularly advantageous because, when remotely controlling the vehicle, the teleoperator must ensure that the vehicle does not collide with stationary or mobile objects in the vehicle's vicinity. The teleoperator can be effectively supported in this task if at least one image, composed of several sub-elements, is displayed on the remote control station's display.

[0032] The combining of the sub-elements to create the at least one image can be performed by a corresponding processing unit of at least one of the infrastructure devices. Additionally or alternatively, the remote control station can create the at least one image by combining or assembling the sub-elements to form the image. The computing capacity required for this can be provided particularly easily both on the side of the at least one infrastructure device and on the side of the remote control station.

[0033] Preferably, the infrastructure devices that transmit the sensor signals usable for forming the respective sub-elements to the remote control station determine a respective position of the vehicle. The position of the vehicle is taken into account for creating the at least one image from the sub-elements. This is advantageous for outputting an image that is particularly well-suited for remote control to the teleoperator on the display device of the remote control station.

[0034] Preferably, the at least one image assembled from the subelements is a bird's-eye view of the vehicle displayed on the remote control station's display device. This makes it particularly easy for the teleoperator to remotely control the vehicle safely using the control commands issued by the remote control station. This is especially true if the vehicle's position is used as the center of the bird's-eye view of the vehicle.

[0035] Additionally or alternatively, the at least one image assembled from the subelements can be a view from the vehicle into the vehicle's surroundings displayed on the remote control station's display device. This provides the teleoperator with a view similar to what a driver of the vehicle would see if the driver were driving the vehicle themselves. This is advantageous for easy remote control of the vehicle by the teleoperator.

[0036] The at least one infrastructure device can transmit the sensor signals from a first transmission node of a transmission link to a second transmission node of the transmission link via a wired connection, wherein the second transmission node is associated with the remote control station. This is particularly advantageous when the first transmission node, which is associated with the at least one infrastructure device, is located relatively far away from the second transmission node. This is because the infrastructure device then does not need to transmit the sensor signals directly to the transmission node of the remote control station. Instead, the first transmission node can advantageously be used as an infrastructure-side collection point.

[0037] Such a modular structure is particularly advantageous when additional infrastructure facilities are to be integrated and / or when the same remote control station is to be used to receive sensor signals from different application areas, such as operating sites, for example, sensor signals from a parking garage on the one hand and sensor signals from a factory site or the like on the other. Particularly in such situations, it is advantageous if the respective operating site has a transmission node at which the sensor signals delivered by the infrastructure facilities of the respective operating site arrive, in order to then transmit them via cable to the transmission node of the remote control station.

[0038] Preferably, at least a partial area of the vehicle's surroundings is detected by means of at least one environmental sensor of the vehicle, wherein a transmission device of the vehicle wirelessly transmits environmental sensor signals from the at least one environmental sensor to the remote control station. The remote control station uses the environmental sensor signals to create the at least one image. The use of the environmental sensor signals contributed by the at least one environmental sensor of the vehicle to create the at least one image is particularly advantageous in order to output a highly informative image to the teleoperator on the display device. This is also useful if the vehicle already has at least one environmental sensor, such as a camera or the like, by means of which the at least one partial area of the vehicle's surroundings can be detected.

[0039] Preferably, the vehicle is moved remotely by the teleoperator due to the implementation of the at least one control command by the in-vehicle device. For example, the in-vehicle device, for example in the form of an in-vehicle control unit, can control at least one longitudinal guidance unit of the vehicle and / or at least one lateral guidance unit of the vehicle in order to effect the remote-controlled movement of the vehicle by the teleoperator. In addition, at least one braking device of the vehicle can preferably be actuated at the instigation of the in-vehicle device, provided this should be expedient or necessary for the remote-controlled movement of the vehicle. In this way, very efficient movement of the vehicle can be achieved when the vehicle is remotely controlled by the teleoperator.

[0040] In particular, due to the implementation of at least one control command by the in-vehicle device, the vehicle can be moved remotely by the teleoperator from a parking zone to a parking space. In other words, the teleoperator can ensure that the vehicle parked in the parking zone is remotely driven to a parking space or parking space. This advantageously makes the vehicle available for further use at the parking space or parking space. Furthermore, the then-released parking zone can be used by another vehicle.

[0041] Business premises or company grounds that are well suited for advantageous implementation of the method can include, for example, a factory site and / or a parking garage and / or an uncovered parking area with a large number of parking spaces or parking lots. Particularly if such business premises or company grounds already have infrastructure facilities that can be used, for example, for automated valet parking, the remote control of the vehicle by the teleoperator described here is advantageous. This is because, particularly if problems arise during automated or autonomous valet parking, i.e., the automated transfer of the vehicle from the parking zone to the parking space, the teleoperator can ensure reliable, remote-controlled guidance of the vehicle to the parking space.

[0042] In the system according to the invention for remotely controlling a vehicle by a teleoperator, at least one image can be output to the teleoperator for remote control on a display device of a vehicle-external remote control station of the system. The teleoperator can cause at least one control command to be output via the remote control station, wherein a receiving device of the vehicle is designed to receive the at least one control command wirelessly. The at least one control command can be implemented by a vehicle-internal device. The remote control station is designed to use sensor signals, which can be transmitted to the remote control station from at least one vehicle-external infrastructure device of the system, to create the at least one image. Using such a system, the remote control of the vehicle by the teleoperator can be effected with particularly little effort.

[0043] A further aspect relates to a computer program product for a vehicle-external remote control station. The computer program product is, in particular, a computer program. The computer program product comprises instructions which, when the program is executed by the remote control station, cause the station to execute at least the method according to the invention and preferably at least one embodiment of the method according to the invention.

[0044] The advantages and preferred embodiments described for the method according to the invention also apply to the system according to the invention and to the computer program product and vice versa.

[0045] The features and feature combinations mentioned above in the description, as well as the features and feature combinations mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective combination specified, but also in other combinations without departing from the scope of the invention. Thus, embodiments are also to be considered encompassed and disclosed by the invention that are not explicitly shown and explained in the figures, but which emerge and can be produced by separate feature combinations from the explained embodiments. Embodiments and feature combinations are also to be considered disclosed that therefore do not have all the features of an originally formulated independent claim.Furthermore, embodiments and combinations of features are to be regarded as disclosed, in particular by the embodiments set out above, which go beyond or deviate from the combinations of features set out in the references to the claims.

[0046] Further features of the invention emerge from the claims, the figures, and the description of the figures. These show: Fig. 1 schematically shows a system in which a teleoperator uses images from cameras associated with infrastructure facilities to remotely move a vehicle from a parking zone to a parking space; Fig. 2 schematic components of the system according to Fig. 1 as well as possibilities for signal transmission between the components of the system; and Fig. 3 a variant of the system in which the teleoperator is shown a bird's eye view of the vehicle on a display device of a remote control station external to the vehicle.

[0047] In the figures, identical or functionally identical elements are provided with identical reference symbols.

[0048] In Fig. 1 schematically shows components of a system 10 that enables a teleoperator 12 to remotely move a vehicle 14. The teleoperator 12 is located at a remote control station 16, which is arranged externally to the vehicle 14, i.e., outside the vehicle 14. The remote control station 16 comprises a display device 18, for example in the form of at least one screen. At least one image 20 can be output on the display device 18, as shown in Fig. 1 is shown very schematically. Images 20 displayed on the display device 18 can in particular be video images.

[0049] The teleoperator 12, who sees the image 20 displayed on the display device 18, can make operating inputs on an operating device 22 of the remote control station 16. This making of operating inputs causes control commands 26 to be sent from the remote control station 16 to a receiving device 24 (see Fig. 2) of the vehicle 14. For reasons of clarity, the receiving device 24 of the vehicle 14 is shown in Fig. 1 is not shown in detail. However, the wireless transmission of the control commands 26 to the receiving device 24 of the vehicle 14 is Fig. 1 and in Fig. 2 is illustrated by a respective arrow.

[0050] An in-vehicle device, for example in the form of an in-vehicle control unit 28, implements the control commands 26 and controls corresponding units 30 of the vehicle 14, which ensure longitudinal and transverse guidance of the vehicle 14. For example, the Fig. 2, the units 30 of the vehicle 14, which are only shown very schematically and are controlled by the control unit 28 of the vehicle 14, comprise at least one drive motor, a braking device, and a steering device. Accordingly, it is possible for the teleoperator 12, which causes the output of the control commands 26 by the remote control station 16, to move the vehicle 14 by remote control.

[0051] Applications for the use of the system 10 can be, for example, when on a company premises such as a vehicle manufacturing site or in a parking garage 32, the vehicle 14 is to be driven remotely by the teleoperator 12 to a parking space 34. For example, in Fig. 1 the parking garage 32 is schematically shown as the business premises or premises for the use of the system 10.

[0052] The preferably stationary remote control station 16 can be located at a location remote from the parking garage 32, as shown schematically and by way of example in Fig. 1. When using the remote control station 16 to move the vehicle 14, it is not necessary and preferably not intended that the teleoperator 12 has the vehicle 14 directly in view. Rather, for the remote control of the vehicle 14 by the teleoperator 12, it is sufficient if the teleoperator 12 sees at least a part of the vehicle 14 and / or at least a part of the surroundings of the vehicle 14 on the at least one image 20, which is output to the teleoperator 12 by means of the display device 18. The fact that the teleoperator 12 does not have the vehicle 14 directly in view or cannot see it is in Fig. 2 is illustrated by a symbol 36.

[0053] In principle, it is possible for at least one environment sensor 38, which the vehicle 14 may have, to transmit environment sensor signals to the remote control station 16 so that the image 20 can be generated from these environment sensor signals. However, such a procedure requires that the wireless transmission, particularly of environment sensor signals in the form of video data, to the remote control station 16 be very efficient with regard to the availability and bandwidth of a corresponding mobile network. This is associated with considerable effort, which can be avoided in the present case.

[0054] This is because the remote control station 16 uses sensor signals 40 to create the at least one image 20, which are transmitted to the remote control station 16 from at least one vehicle-external infrastructure device 42, 44, 46, 48. Examples are shown in Fig. 1 shows several stationary infrastructure facilities 42, 44, 46, 48, which can be arranged at different locations in the parking garage 32.

[0055] In Fig. For reasons of clarity, only one of the infrastructure devices 42, 44, 46, 48 is shown in Fig. 2, for example the first infrastructure device 42. The sensor signals 40 used to create the at least one image 20 are shown in Fig. 2 is shown by an arrow. Furthermore, a Fig. 2, that the infrastructure facilities 42, 44, 46, 48 are capable of detecting the vehicle 14. For this purpose, the stationary infrastructure facilities 42, 44, 46, 48 have respective sensor devices 52, which are Fig. 1 and in Fig. 2 are shown schematically.

[0056] For example, the sensor devices 52 can be designed as cameras and / or as lidar devices, wherein the lidar devices use laser beams to scan at least a partial area of the vehicle 14 and / or at least a partial area of an environment of the vehicle 14. For reasons of simplicity, it should be assumed here by way of example that the sensor devices 52, which are Fig. 1 and in Fig. 2 are shown schematically, are respective cameras.

[0057] According to Fig. 2, the sensor signals 40 of the remote control station 16 can be transmitted via a cable. A corresponding transmission line 54 is provided in Fig. 2. As a result, the sensor signals 40 used to create the at least one image 20 do not need to be transmitted wirelessly to the remote control station 16. Accordingly, the difficulties or problems associated with transmitting data via mobile communications are eliminated. In contrast, the transmission of data in the form of the sensor signals 40 via the transmission line 54, for example, in the form of a cable, is particularly robust, efficient, and reliable.

[0058] It can be provided that the sensor signals 40 are transmitted from a first transmission node 68 of a preferably entirely wired transmission path to a second transmission node 70 of the transmission path, as shown schematically in Fig. 2. Here, the first transmission node 68 can be associated with the at least one infrastructure device 42, 44, 46, 48. And the second transmission node 70 can be associated with the remote control station 16. The Fig. The transmission nodes 68, 70 shown schematically in Figure 2 can, for example, be designed as base stations, which can be placed at different locations of the system 10.

[0059] To create the image 20, in principle, additional environmental sensor signals can be used, which the at least one environmental sensor 38 of the vehicle 14 (cf. Fig. 2). In other words, the at least one environment sensor 38 of the vehicle 14 can detect environment sensor signals, which can be used by the remote control station 16 to create the image 20 in addition to the sensor signals 40. When the system 10 is used in this way, the receiving device 24 of the vehicle 14 can be used as a transmission device, which wirelessly transmits the environment sensor signals to the remote control station 16.

[0060] However, in the present case, the detection of at least a partial area of the vehicle 14 and at least a partial area of the surroundings of the vehicle 14 is carried out at best with the support of the at least one surroundings sensor 38 of the vehicle 14 and preferably without the at least one surroundings sensor 38 of the vehicle 14. In the present case, the at least one partial area of the vehicle 14 and / or the at least one partial area of the surroundings of the vehicle 14 is detected by means of the sensor devices 52 which are associated with the infrastructure devices 42, 44, 46, 48.

[0061] On the one hand, this significantly reduces the requirements for the system 10 with regard to the performance of a mobile network. This is because the transmission of the control commands 26 to the receiving device 24 of the vehicle 14 (see Fig. 2) involves a significantly smaller amount of data to be transmitted than would be the case if sensor signals 40, for example in the form of lidar data or video data, were to be transmitted wirelessly from the vehicle 14 to the remote control station 16. And in the present case, the sensor signals 40, for example in the form of lidar data and / or video data, are transmitted from the at least one infrastructure device 42, 44, 46, 48, preferably via a cable, to the remote control station 16. Therefore, the system 10 is particularly low-complexity.

[0062] In Fig. 1 schematically shows that the vehicle 14 is located at a parking zone 56, from which the vehicle 14 is to be moved to the parking space 34. The vehicle 14 can be brought to the parking zone 56 by a driver and / or autonomously. If the vehicle is remotely controlled from the parking zone 56 to the parking space 34 by the teleoperator 12, no driver of the vehicle 14 needs to take over this transport. This is advantageous.

[0063] According to Fig. 1, the vehicle 14 is located at the parking zone 56 in a detection range 58 of the sensor device 52, which is associated with the first infrastructure device 42. The first infrastructure device 42 can thus detect the vehicle 14 and transmit a camera image as the image 20 to the display device 18 of the remote control station 16. For example, in Fig. 1 schematically shows that the vehicle 14 has been parked laterally next to an outer wall 60 of the parking garage 32 when the vehicle 14 is located in the parking zone 56.

[0064] The sensor device 52 of the first infrastructure device 42 is according to Fig. 1 is capable of detecting the vehicle 14 and at least a portion of the outer wall 60. At least a portion of the parking zone 56 and at least a portion of the outer wall 60 are located within the detection range 58 associated with the sensor device 52 of the first infrastructure device 42. The image 20 detected by the sensor device 52 of the first infrastructure device 42 can be displayed or output on the display device 18 of the remote control station 16. And the teleoperator 12 viewing the image 20 can, by making operating inputs at the remote control station 16, cause the remote control station 16 to transmit the control commands 26 to the receiving device 24 (cf. Fig. 2) of the vehicle 14.

[0065] The infrastructure facilities 42, 44, 46, 48 can track a trajectory of the vehicle 14. For example, the vehicle 14 can drive into the parking garage 32 remotely controlled by the teleoperator 12 and enter a detection range 62 of the sensor device 52 of the second infrastructure facility 44. Switching from the sensor device 52 of the first infrastructure facility 42 to the sensor device 52 of the second infrastructure facility 44 can occur automatically, for example, caused by one of the infrastructure facilities 42, 44 as soon as the vehicle 14 leaves the detection range 58 of the first infrastructure facility 42 and enters the detection range 62 of the second infrastructure facility 44. Additionally or alternatively, it is possible for the teleoperator 12 to cause the switching from the first infrastructure facility 42 to the second infrastructure facility 44.

[0066] Subsequently, the vehicle 14 can enter detection zones 64, 66, which are associated with the sensor devices 52 of the third infrastructure device 46 and the fourth infrastructure device 48, respectively. Thus, the vehicle 14 can be effectively monitored by means of the camera images and tracked by the teleoperator 12 until it reaches the parking space 34, with the camera images being output to the teleoperator 12 on the display device 18, for example in the form of at least one screen.

[0067] In Fig. 3 shows a variant of the system 10 which differs from the one shown in FIG. 3 in particular with regard to the at least one image 20 which is output on the display device 18 of the remote control station 16. Fig. 1 shown variant of the system 10.

[0068] For example, the sensor devices 52 of several of the infrastructure devices 42, 44, 46, 48 can detect at least a partial area of the vehicle 14 as well as at least a partial area of the surroundings of the vehicle 14. With a corresponding size of the detection area 58, which is associated with the sensor device 52 of the first infrastructure device 42, and the detection area 62, which is associated with the sensor device 52 of the second infrastructure device 44, the sensor devices 52 of these two infrastructure devices 42, 44 can, for example, at least partially detect the vehicle 14. From corresponding partial elements, for example in the form of partial images, a view of the vehicle 14 from a bird's eye view can then be output on the screen or the display device 18 as the at least one image 20, as shown in Fig. 3 is shown schematically.

[0069] In this case, the infrastructure facilities 42, 44, 46, 48 can determine the respective position of the vehicle 14 and track the position of the vehicle 14. The respective position of the vehicle 14 in the parking garage 32 can be selected as the center of the bird's-eye view. The bird's-eye view can then be output to the teleoperator 12, which is located at the remote control station 16.

[0070] The creation of the bird's-eye view of the vehicle 14 can be carried out by at least one of the infrastructure devices 42, 44, 46, 48 and / or by a corresponding computing device (not shown) of the remote control station 16.

[0071] For example, taking into account images 20 in the form of video data, which show the vehicle 14 in a bird's eye view, the teleoperator 12 can then issue the control commands 26 (cf. Fig. 3) to the receiving device 24 of the vehicle 14 (see Fig. 2) output.

[0072] With reference to Fig. 1 and Fig. 2, the situation was explained as an example in which the vehicle 14 is remotely controlled by the teleoperator 12 from the parking zone 56 to the parking space 34. In particular, if the infrastructure facilities 42, 44, 46, 48 of the parking garage 32 are designed to carry out automated valet parking, i.e., an automated movement of the vehicle 14 from the parking zone 56 to the parking space 34, the stationary infrastructure facilities 42, 44, 46, 48 can be used effectively for the remote control effected by the teleoperator 12.

[0073] For example, the teleoperator 12 may intervene if a control device (not shown) associated with the parking garage 32 is unable to move the vehicle 14 from the parking zone 56 to the parking space 34 by means of the automated valet parking due to unforeseeable problems.

[0074] In an analogous manner, such a situation can arise when the vehicle 14 is finished following production at a production site where vehicle production takes place, and is to be driven, for example, on the company premises from the parking zone 56 to the parking space 34. For this purpose, one can in principle also use automated valet parking, which makes use of the infrastructure facilities 42, 44, 46, 48 that are available on the company premises.

[0075] Even in such a case, if there are problems with at least one infrastructure facility 42, 44, 46, 48 used for valet parking, the teleoperator 12 can intervene and cause the remote-controlled movement of the vehicle 14 to the parking space 34.

[0076] Overall, the examples show how infrastructure facilities 42, 44, 46, 48, which are particularly suitable for automated valet parking, can be used or reused in the context of an extended use. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2020 112 468 A1

[0002] US 2022 / 0139222 A1

[0007]

Claims

[1] Method for remotely controlling a vehicle (14) by a teleoperator (12), in which for the remote control, at least one image (20) is output to the teleoperator (12) on a display device (18) of a vehicle-external remote control station (16), wherein the teleoperator (12) causes at least one control command (26) to be output by the remote control station (16), and in which a receiving device (24) of the vehicle (14) receives the at least one control command (26) wirelessly, wherein the at least one control command (26) is implemented by a vehicle-internal device (28), characterized by that the remote control station (16) uses sensor signals (40) to create the at least one image (20), which are transmitted to the remote control station (16) from at least one vehicle-external infrastructure device (42, 44, 46, 48). [2] Method according to claim 1, characterized bythat the sensor signals (40) are transmitted from the at least one infrastructure device (42, 44, 46, 48) to the remote control station (16) via a cable. [3] Method according to one of the preceding claims, characterized by that a sensor device (52) of the at least one infrastructure device (42, 44, 46, 48) detects the vehicle (14) in a detection range (58, 62, 64, 66) of the sensor device (52), wherein the sensor signals (40) of the sensor device (52) are transmitted to the remote control station (16) depending on the presence of the vehicle (14) in the detection range (58, 62, 64, 66). [4] Method according to one of the preceding claims, characterized bythat the at least one infrastructure device (42, 44, 46, 48) tracks a trajectory of the vehicle (14), wherein when the vehicle (14) leaves a detection range (58) of a sensor device (52) of a first infrastructure device (42) and enters a detection range (62) of a sensor device (52) of a second infrastructure device (44), the transmission of sensor signals (40) of the sensor device (52) of the first infrastructure device (42) to the remote control station (16) is prevented and the sensor signals (40) detected by means of the sensor device (52) of the second infrastructure device (44) are transmitted to the remote control station (16). [5] Method according to claim 4, characterized bythat a switchover from the sensor device (52) of the first infrastructure device (42) to the sensor device (52) of the second infrastructure device (44) is effected by at least one of the two infrastructure devices (42, 44) and / or by the teleoperator (12). [6] Method according to one of the preceding claims, characterized by that a sensor device (52) of the at least one infrastructure device (42, 44, 46, 48) is designed as a camera, wherein a camera image captured by the camera and showing at least a partial area of the vehicle (14) is output as the at least one image (20) on the display device (18) of the remote control station (16). [7] Method according to one of the preceding claims, characterized bythat the remote control station (16) creates the at least one image (20) from sensor signals (40) which are transmitted to the remote control station (16) from a plurality of infrastructure devices (42, 44, 46, 48), wherein a plurality of sub-elements are combined to create the at least one image (20), and wherein the respective sub-element is formed using the sensor signals (40) of one of the infrastructure devices (42, 44, 46, 48). [8] Method according to claim 7, characterized by that the infrastructure devices (42, 44, 46, 48), which transmit the sensor signals (40) usable for forming the respective sub-elements to the remote control station (16), determine a respective position of the vehicle (14), wherein the position of the vehicle (14) is taken into account for creating the at least one image (20) from the sub-elements. [9] Method according to claim 7 or 8, characterized bythat as the at least one image (20) assembled from the sub-elements, a view of the vehicle (14) from a bird's eye view and / or a view from the vehicle (14) into an environment of the vehicle (14) is output on the display device (18) of the remote control station (16). [10] Method according to one of the preceding claims, characterized by that the at least one infrastructure device (42, 44, 46, 48) transmits the sensor signals (40) from a first transmission node (68) of a transmission link to a second transmission node (70) of the transmission link, the second transmission node (70) being associated with the remote control station (16). [11] Method according to one of the preceding claims, characterized bythat at least a partial area of an environment of the vehicle (14) is detected by means of at least one environment sensor (38) of the vehicle (14), wherein a transmission device of the vehicle (16) wirelessly transmits environment sensor signals of the at least one environment sensor (38) to the remote control station (16), and wherein the remote control station (16) uses the environment sensor signals to create the at least one image (20). [12] Method according to one of the preceding claims, characterized by that due to the implementation of the at least one control command (26) effected by the vehicle-internal device (28), the vehicle (14) is moved by the teleoperator (12) in a remote-controlled manner, in particular is moved by a remote-controlled manner from a parking zone (56) to a parking space (34). [13] System (10) for remotely controlling a vehicle (14) by a teleoperator (12), wherein for the remote control, at least one image (20) can be output to the teleoperator (12) on a display device (18) of a vehicle-external remote control station (16) of the system (10), wherein the teleoperator (12) can cause at least one control command (26) to be output by the remote control station (16), wherein a receiving device (24) of the vehicle (14) is designed to receive the at least one control command (26) wirelessly, wherein the at least one control command (26) can be implemented by a vehicle-internal device (28), characterized by that the remote control station (16) is designed to use sensor signals (40) for creating the at least one image (20), which can be transmitted to the remote control station (16) from at least one vehicle-external infrastructure device (42, 44, 46, 48) of the system (10). [14] Computer program product for a vehicle-external remote control station (16), comprising instructions which, when the program is executed by the remote control station (16), cause it to carry out the method according to one of claims 1 to 12.

Images