Method for transmitting at least one data packet between a vehicle and a remote operation device for remote operation of the vehicle, and remote operation system

Abstract

The invention relates to a method for transmitting a data packet (9) between a vehicle (1) and a remote operation device (5), wherein at least two different transmission paths (7, 8) are usable between the vehicle (1) and the remote operation device (5) at a current time. There is provision (a) for the data packet (9) to be transmitted redundantly via at least two of the usable transmission paths (7, 8), and / or (b) for a single transmission path (7, 8) from the usable transmission paths (7, 8) to be selected and for the data packet (9) to be transmitted via the selected transmission path (7, 8). Attribute information (20) is determined for the respective transmitted data packet (9) (S1) and a check is performed (S2) to ascertain whether the determined attribute information (20) is stored in a storage unit (13), wherein if this is the case then the data packet (9) is rejected (S3), and / or if this is not the case then the attribute information (20) is at least temporarily stored in the storage unit (13) and the data packet (9) is considered further (S4).

Description

[0001] Method for transmitting at least one data packet between a vehicle and a teleoperation device for a teleoperation of the vehicle and teleoperation system

[0002] The invention relates to a method for transmitting at least one data packet between a vehicle and a teleoperation device for teleoperation of the vehicle. Furthermore, the invention relates to a teleoperation system for implementing such a method.

[0003] A vehicle can be remotely controlled using teleoperation. For the vehicle to be teleoperated, a stable communication connection is required between the vehicle and a teleoperation device, for example, to reliably transmit control commands for the vehicle. Without a reliable transmission of data packets between the vehicle and the teleoperation device, the teleoperation cannot be reliably guaranteed.

[0004] US 11,223,556 B2 shows a communication system that simplifies transmissions between mobile devices and a server with low latency and high availability.

[0005] Furthermore, US 2003 / 0060808 A1 shows a transmission of data in connection with a mobile medical device in which control signals and feedback signals are transmitted redundantly via several communication channels.

[0006] It is the object of the invention to provide a solution by means of which the transmission of a data packet between a vehicle and a teleoperation device for teleoperation of the vehicle is particularly reliable.

[0007] The problem is solved by the subject matter of the independent patent claims.

[0008] A first aspect of the invention relates to a method for transmitting at least one data packet between a vehicle and a teleoperation device for teleoperation of the vehicle. Teleoperation within the meaning of the invention means that the vehicle is controlled remotely, in particular longitudinal and / or lateral guidance of the vehicle. There is a spatial decoupling between a driver of the vehicle and the actual vehicle. Teleoperation takes place from any location, which may be outside a predetermined area around the vehicle. However, communication with the vehicle is possible at that location, i.e., at least one data packet can be transmitted to and / or received from the vehicle, for example.Teleoperation within the meaning of the invention does not involve remote control of the vehicle, where the driver must be located within a fixed, predefined area around the vehicle and remotely controls the vehicle from there, for example, using a mobile device. The fixed, predefined area may, for example, be limited to a radius of a few meters.

[0009] In teleoperation, a teleoperator is provided which specifies control of the vehicle by means of an operating device, such as a steering wheel and / or at least one further operating element. The teleoperation device is, for example, a control device, i.e. it can be a computer or a computing device. Furthermore, the teleoperation device has at least one communication interface. The teleoperation device can, for example, translate an operating input from the teleoperator, which is detected by the operating device, into a control command for the vehicle and / or receive sensor data from the vehicle, in particular evaluate it, and display and / or output it for the teleoperator. The teleoperation device can transmit, in particular send, the control command to the vehicle by means of the communication interface.It is assumed that the vehicle has at least one communication interface via which a transmission path for transmitting data to the teleoperation device can be implemented. The communication interface can alternatively be referred to as a transmission interface. The transmission path can alternatively be referred to as a communication connection. The data transmitted between the vehicle and the teleoperation device are referred to below as data packets. The at least one data packet can therefore, for example, include at least one control command for the vehicle.

[0010] The vehicle has a control device designed for fully automatic control of the longitudinal and / or lateral guidance of the vehicle. The control device can therefore control or operate a steering system, a braking system, and / or a drive device. In particular, it controls the steering system, the braking system, and / or the drive device according to the control command of the teleoperation device. Furthermore, other components of the vehicle, for example, at least one display device in the vehicle, can be controlled by the control device.

[0011] The situation exists that at least two different transmission paths are usable between the vehicle and the teleoperation device at a current point in time. The two transmission paths can, for example, be provided by two separate mobile radio networks, which are operated by different mobile radio network operators, for example. Other transmission paths are possible. In the following, a mobile radio network is assumed as an example as the respective transmission path. The respective transmission path is considered "usable" if it can be used, i.e. if it is available. The respective transmission path is also considered usable within the meaning of the invention if its quality is such that temporary interruptions in the transmission of the data packet can occur and / or a loss of data packets during transmission is possible, for example due to instability of the transmission path.

[0012] To transmit the at least one data packet between the vehicle and the teleoperation device, a transmission type a and / or a transmission type b can be selected. Transmission type a provides that the at least one data packet is transmitted redundantly via at least two of the usable transmission paths. If a first transmission path and a second transmission path different from the first transmission path are usable, i.e. available, the data packet is generated twice. The two data packets are identical, in particular completely identical, at least in terms of their content. One of the two identical data packets is then transmitted via the first transmission path and the other of the two identical data packets via the second transmission path. This occurs simultaneously and / or concurrently or at least essentially simultaneously and / or concurrently.Essentially, this includes deviations from greater than 0 to, for example, 50 milliseconds, 100 milliseconds, 200 milliseconds, or in particular 300 milliseconds. The described redundant transmission of the data packet applies both to the transmission of any data packet from the vehicle to the teleoperation device and to the transmission of any data packet from the teleoperation device to the vehicle. If more than two usable transmission paths are available, the data packet can be more than doubled, i.e., it can be multiplied such that the data packet can be transmitted via three or more transmission paths. In other words, transmission type a can be described as redundant transmission via several currently usable transmission paths.

[0013] Transmission type b provides that a single transmission path is selected from the transmission paths used by applying a predetermined selection criterion, and that at least one data packet is transmitted via the selected transmission path. If only transmission type b is selected, the data packet may be transmitted only via the selected transmission path, i.e., via a single transmission path of the usable transmission paths. Therefore, with transmission type b, no redundant transmission takes place. The selection criterion may, for example, comprise at least one algorithm and / or a rule, the implementation of which at least determines which of the plurality of usable transmission paths is most suitable for the transmission, in particular at the current time, of all usable transmission paths.For example, the selection criterion can provide for determining a transmission quality value for each of the usable transmission paths, which describes the quality of the transmission using the respective transmission path. By comparing the determined transmission quality values, the transmission path with the highest transmission quality value and thus the highest quality can be determined. This transmission path is then selected. If transmission via transmission type a is also desired, the selected transmission path is, in a preferred example, one of the two redundantly used transmission paths.

[0014] The method provides for a piece of identifier information to be determined for each transmitted data packet. The identifier information describes the data packet regardless of the transmission path used. For example, the identifier information can be used to recognize and, in particular, uniquely identify the respective transmitted data packet. This means that, for example, with transmission type a, the two redundant data packets each have the same identifier information. This is because the identifier information is independent of whether the data packet is transmitted via the first transmission path or the second transmission path. In other words, the identifier information describes the data packet in such a way that the description depends only on the data packet, in particular on the content of the data packet, and is therefore independent of the transmission path used, via which the data packet is to be transmitted, is transmitted and / or was transmitted.The license plate information is determined, for example, by means of the vehicle's control device and / or the teleoperation device.

[0015] A check is carried out to determine whether the license plate information determined is stored in a storage unit. The storage unit is arranged in the vehicle, for example. It can be comprised of the control device or arranged spatially separate from it in the vehicle. Alternatively or additionally, the storage unit can be comprised of the teleoperation device. The method determines whether the license plate information determined is present for the first time, for example because it is being sent and / or received for the first time, or whether it is already present in the storage unit, i.e. is already known. In other words, a comparison is made as to whether the license plate information has been determined for the first time or whether it was already determined in the past, which is recognized here by the fact that it is stored in the storage unit.

[0016] In a preferred embodiment, the storage unit is designed as a ring memory or ring buffer. In this case, data can be stored continuously in the storage unit, which has a fixed size. When the storage unit is full, i.e., there is no more free space, the oldest data is overwritten. This results in the data being available only for a limited period of time. The stored data here is at least identification information.

[0017] If it is determined that the identified identifier information is already stored in the storage unit, the data packet is discarded. For example, it is deleted or overwritten. In any case, the data packet is not reused, i.e., it is not considered in the teleoperation. The reason for this is that the identifier information already present in the storage unit at least indicates that a data packet has already been received that contains the same identifier information, so that the data packet was received at least twice, for example, via two different transmission paths according to transmission type a.

[0018] If it is determined that the determined license plate information is not stored in the storage unit, the license plate information is at least temporarily stored in the storage unit and the data packet is further considered. If further considered, the data packet is evaluated, for example, for teleoperation and / or used in the vehicle to control the vehicle. This means that when the license plate information is first determined, it is always stored in the storage unit for at least a predetermined period of time greater than 0, so that a duplicate data packet can be recognized as such during the described check and a response can be made. This ensures that only previously unknown and therefore new data packets that are to be assessed as relevant are considered for teleoperation and that redundant data packets are recognized and sorted out.

[0019] Redundant transmission of the data packet according to transmission type a, for example, reduces the probability that a data packet is lost if one of the transmission paths is unreliable, since it is always sent via several different transmission paths. However, if only one transmission path according to transmission type b is selected and used, it can still be ensured that data packets are not used twice, for example if they are sent multiple times due to an error, since the identification information is also determined and checked in this case. Furthermore, applying the selection criterion ensures that the only transmission path is specifically selected, since even with transmission type b, several transmission paths are usable, from which exactly one is selected.It can therefore be assumed that the data packet will be transmitted reliably, even if this transmission is not redundant. This method thus makes the transmission of at least one data packet particularly reliable.

[0020] In a preferred example, either transmission type a or transmission type b is currently being used. It may be provided that both transmission types can only be used simultaneously if three or more transmission paths are available, so that at least two transmission paths are available for transmission type a and one different transmission path for transmission type b.

[0021] One embodiment provides that the identifier information is determined by applying a hash function to the respective transmitted data packet. The hash function can alternatively be referred to as a hash value function. The hash function can be understood as a mapping that maps an input set to a comparatively small target set, the so-called hash value. The input set can comprise elements of different lengths and / or sizes, i.e., any data packet. The elements of the target set typically have a fixed length and / or size. Typically, the hash value that is generated is a scalar value selected from a limited subset of natural numbers. Alternative or additional embodiments of the hash value are possible. The hash value is referred to as identifier information within the scope of the invention.By applying the hash function, the identifier information is a kind of name for the respective data packet, which describes precisely this data packet and is thus the same for the redundantly transmitted data packets, but always different for data packets that are at least partially different. The hash function can alternatively be referred to as a hash algorithm. The hash function comprises at least one rule and / or regulation that is applied to the data packet to determine the identifier information. Thus, a unique assignment of identifier information to data packets can be realized in a simple and reliable manner.

[0022] Another exemplary embodiment provides that the vehicle and / or the teleoperation device comprise a license plate information determination unit. The license plate information determination unit can, for example, be comprised by the control device of the vehicle and / or the teleoperation device. The license plate information determination unit determines the license plate information for the at least one data packet before it is sent and / or after it is received. For example, the license plate information is transmitted with the data packet and can, for example, be included therein. In a preferred example, however, the license plate information is determined on the side of a receiver of the data packet, so that the data packet is transmitted without license plate information. The component that receives the data packet, i.e., either the vehicle or the teleoperation device, then determines the license plate information for the received data packet.This is particularly useful in order to keep the transmitted data packet small, since the license plate information does not have to be transmitted without this having an impact on the reliability of the process.

[0023] A further exemplary embodiment provides that the stored license plate information is deleted and / or overwritten after a predetermined time interval. Thus, it is not intended that the license plate information, once determined and stored, is stored in the storage device forever; rather, it is deleted and / or overwritten again, particularly promptly, particularly with different license plate information. This is particularly suitable for using the ring buffer as a storage unit, wherein the ring buffer can be dimensioned such that it can store license plate information for the predetermined time interval and then begins to overwrite the stored license plate information. This minimizes the amount of data required to store the license plate information, which makes the method particularly useful for application in the automotive context.

[0024] According to an additional exemplary embodiment, the predetermined time interval corresponds to at least one transmission duration predetermined for the respective transmission path and / or determined for the respective transmission path. The transmission duration is, for example, a maximum duration that a transmission via this transmission path can last or lasts. The transmission duration can be an average value over recorded durations for the transmission via this transmission path, in particular over recorded maximum durations for the transmission. For example, it can be determined by means of at least one test measurement, in particular by means of several test measurements, how long the transmission duration is, currently or generally considered, for a respective transmission path, in particular for all transmission paths, and the time interval can then be determined and predetermined.The time interval can be stored, for example, in the storage unit, the control device and / or the teleoperation device.

[0025] For example, in the case of the ring buffer, its design, in particular storage capacity, can be dependent on the specified time interval, so that after the specified time interval has elapsed, the respective license plate information is overwritten.

[0026] For example, if it is known that at least one of the usable transmission paths has a transmission duration of 100 milliseconds or 200 milliseconds, the specified time interval can be limited to 100 milliseconds or 200 milliseconds, respectively. Alternatively or additionally, a time interval longer than the transmission duration can be selected, for example, to specify a specified time buffer. In the above example, the time interval can then be 200 milliseconds or 300 milliseconds or longer. If the usable transmission paths have different transmission durations, the specified time interval can, for example, correspond to at least the longest transmission duration.

[0027] Another exemplary embodiment provides for the determination of a current position of the vehicle. For this purpose, the vehicle may, for example, have a positioning device based on data from a global navigation satellite system (GNSS). The global positioning system (GPS), for example, may be used as the GNSS. The current position may be described by position information. The current position may alternatively be referred to as position information.

[0028] A check is carried out to determine whether the current position determined is located in a predefined area for which at least one of the transmission paths is predefined. The predefined area is fixed and stored, for example, in the memory unit of the vehicle and / or the teleoperation device. If the current position is located in the predefined area, the at least one data packet is always transmitted via at least this transmission path that is assigned to the predefined area. A geofence is therefore predefined for the at least one transmission path, and upon reaching this geofence, this at least one transmission path is automatically used for the transmission. It is therefore known whether and, if so, which area is intended for a specific transmission path. This location-dependent specification of the transmission path is suitable, for example, for transmission paths that can be used for a limited local area, such as a private mobile phone network.With transmission type b, for example, the transmission path specified for this area can be automatically selected and used as the sole transmission path as soon as the specified area is reached. With transmission type a, for example, the transmission path for the specified area can be permanently designated, i.e., used, as one of at least two transmission paths used. Thus, depending on the location, a specific transmission path can be known to be particularly suitable and always used.

[0029] The specified area for which the transmission path is specified can be specified manually by a vehicle user, for example, via an operator input in the vehicle and / or via a mobile device assigned to the user. The specified area is then stored, for example, in the vehicle.

[0030] In a preferred example, the position is determined and verified in the vehicle, wherein, for example, the information about the transmission path for the specified area is transmitted to the teleoperation device, for example as one of the data packets, and the latter can then be known. According to a further exemplary embodiment, a transmission quality value is determined for the respective usable transmission path. This describes the quality of the transmission via the transmission path, as already described above. For example, a transmission quality determination device can be provided in the vehicle and / or in the teleoperation device, which determines the transmission quality value. For example, a transmission quality determination criterion can be provided, upon application of which the transmission quality value is determined.In application type b, the transmission path with the highest transmission quality value of all available transmission paths is selected. The transmission quality value can be checked in addition to checking whether the current position is within the specified area, with the transmission quality value then only being determined, for example, if it has already been determined that the vehicle is already within the area for which the transmission path is specified. In this example, an additional check can be provided, for example, to ensure that the data packet is only transmitted via the transmission path for which the specified area is specified if the transmission quality value for this transmission path is the highest compared to the transmission quality values ​​of other available transmission paths.This ensures that only particularly useful, high-quality transmission paths are used and not, for example, transmission paths that are already known to be subject to interference and / or inaccuracies, which is why they have a comparatively low transmission quality value.

[0031] A further exemplary embodiment provides for a change from transmission type a to transmission type b if a transmission quality value for one of the transmission paths is greater than or equal to a predefined transmission quality limit. Alternatively or additionally, the change takes place if a determined current position of the vehicle is located in a predefined area for which at least one of the transmission paths is predefined. Alternatively or additionally, the change takes place when a user specifies the change by means of an operating input. The user can be the teleoperator or a user of the vehicle. The operating input can be made using a mobile device of the user. The mobile device can be a smartphone and / or tablet, for example. Alternatively or additionally, the operating input can be made using an operating device in the vehicle.The transmission quality value may correspond to the transmission quality value specified above. The determined current position of the vehicle and the specified area may correspond to the above-mentioned position or area.

[0032] Depending on the transmission quality, the location within the specified area, and / or manual user input, a decision can be made, for example, that instead of transmitting the redundant data packets via two transmission paths, at least one data packet is transmitted via a single selected transmission path. However, this switch is only desired if the transmission path meets a minimum quality requirement, i.e., its transmission quality value is greater than or equal to the transmission quality threshold, if the vehicle is actually located within the specified area, and / or if this is manually requested. Numerous scenarios are therefore possible in which switching between transmission types a and b should be made, as this is particularly useful.

[0033] In another exemplary embodiment, it is provided that a switch is made from transmission type b to transmission type a if it is determined that a transmission quality value for the transmission path used is lower than a transmission quality threshold. In this case, the transmission path used is the only transmission path selected according to transmission type b. The transmission quality value can correspond to the aforementioned transmission quality value. Therefore, if a sufficiently high transmission quality is no longer available, a switch can be made back to redundant transmission via at least two transmission paths.

[0034] Alternatively or additionally, the switch from transmission type b to transmission type a occurs if the determined current position of the vehicle is no longer within the specified area for which the transmission path used is specified. The determined current position of the vehicle and the specified area can correspond to the aforementioned position or area. If, for example, the vehicle leaves the specified area, the system automatically switches back to transmission type a. Alternatively or additionally, the aforementioned switch occurs if the user specifies the switch via an operator input. The operator input can correspond to the operator input described above.Various options are therefore provided for switching from transmission type b to transmission type a, so that this switch can take place automatically in situations where this is appropriate. A preferred exemplary embodiment provides that when switching between transmission types a and b, the previous transmission of at least one data packet is terminated immediately. For example, if transmission has previously taken place redundantly via at least two transmission paths (transmission type a) and a switch is made to transmission via only one selected transmission path (transmission type b), the redundant transmission is stopped immediately and the data packet is transmitted via the single selected transmission path. The same can happen in the opposite case when switching over, so that, for example, the redundant transmission takes place immediately after the switch.

[0035] Alternatively, when switching between transmission types a and b, the transmission of at least one data packet can occur simultaneously, at least temporarily, according to both transmission type a and transmission type b. For example, it can be provided that, at least for a predetermined period of time, both the redundant transmission via the at least two transmission paths and the transmission of the data packet via the one selected transmission path, which, for example, has the maximum overall transmission quality value, occur. This is suitable, for example, when there are more than two possible transmission paths, so that, for example, a first and second transmission path are used for a procedure according to transmission type a and a third transmission path for transmission according to transmission type b.By continuing to compare the identifier value of the storage unit, the receiver can prevent data packets from being counted twice. Therefore, both transmission methods can be used simultaneously if desired.

[0036] In another exemplary embodiment, if at least three transmission paths are usable, it is possible to switch between the individual transmission paths of the at least three transmission paths without switching between transmission type a and transmission type b by changing the respective transmission path via which the at least one data packet is transmitted. Thus, if three transmission paths are present, it is possible, for example, to switch to transmission via the second and third transmission paths instead of, for example, the first and second transmission paths without changing the transmission type for transmission type a. Other combinations of transmission paths and / or more transmission paths are possible.Alternatively, when using, for example, the first transmission path in transmission type b, it can be determined that a second transmission path or a third transmission path, for example, has a higher transmission quality value or that, for example, due to a change in position of the vehicle, the vehicle is now located in a predetermined area with a specific transmission path, which is why the previous transmission path is changed, for example from the first transmission path to the second transmission path.

[0037] It can be provided that repeated checks, especially continuous checks, are performed to determine whether the currently used at least one transmission path is still suitable. Based on this check, a decision is made as to whether or not a change in the at least one transmission path or paths used within the transmission type should be made. Thus, the transmission of the data packet is continuously optimized.

[0038] Furthermore, one embodiment provides for the respective transmission path to be a mobile radio network. Alternatively or additionally, the respective transmission path is a wireless local area network (WLAN). This means that there is a cable-free, in particular wireless, transmission that can be based on various transmission technologies. Due to the typically large distance between the teleoperation device and the vehicle, methods that are not locally limited but rather exist over a wide area, such as mobile radio networks, are particularly suitable. The respective mobile radio network or wireless local area network can be public or private. This enables a method that can be used in many locations.

[0039] An additional exemplary embodiment comprises that the at least one data packet comprises at least sensor data from a sensor device of the vehicle and / or the teleoperation device. For example, the data packet can comprise image data from a front camera of the vehicle, which describes at least part of the vehicle's surroundings and which may be required for the teleoperation. Alternatively or additionally, it can be provided that the teleoperator is detected, for example, using a camera as the sensor device, so that the teleoperator can be displayed in the vehicle by means of a display device using the sensor data detected and transmitted to the vehicle. The sensor data from the sensor device of the vehicle can alternatively or additionally be, for example, radar data, lidar data, ultrasonic sensor data, odometry data and / or other data describing the vehicle.In the case of image data, this can alternatively or additionally be captured by a rear camera, a side camera and / or a surround-view camera.

[0040] Alternatively or additionally, the at least one data packet comprises control command data for controlling the vehicle by the teleoperator. The teleoperator can be a person who specifies the control of the vehicle, whereupon the control command data is determined and transmitted to the vehicle via the teleoperation device. The control command data preferably relates to the longitudinal and / or lateral guidance of the vehicle, i.e., it describes at least one control command for the braking system, the steering system, and / or the drive system of the vehicle. Other or alternative data that can be included in the data packet are possible.

[0041] A further aspect of the invention relates to a teleoperation system. The teleoperation system comprises a vehicle, a teleoperation device, and at least two transmission paths between the vehicle and the teleoperation device. The teleoperation system can further comprise the teleoperator. The teleoperation system is designed to carry out the method described above. The teleoperation system is designed to transmit at least one data packet between the vehicle and the teleoperation device. At least the two mutually different transmission paths included in the teleoperation system can be used between the vehicle and the teleoperation device at a given time.The teleoperation system is designed to transmit the at least one data packet redundantly via at least two of the usable transmission paths and / or to select a single transmission path from the usable transmission paths by applying a predetermined selection criterion and to transmit the at least one data packet via the selected transmission path. Furthermore, the teleoperation system is designed to determine, for each transmitted data packet, a piece of identifier information that describes the data packet independently of the transmission path used and to check whether the determined identifier information is stored in a storage unit. If this is the case, the teleoperation system is designed to discard the data packet and / or, if this is not the case, to store the identifier information at least temporarily in the storage unit and to continue to process the data packet.The embodiments described in connection with the method according to the invention apply analogously, where applicable, to the teleoperation system according to the invention.

[0042] One embodiment of the teleoperation system provides that the vehicle and / or the teleoperation device have at least two different communication interfaces. These are each designed to provide a transmission path between the vehicle and the teleoperation device. The respective communication interface can, for example, be designed as a modem that is configured for a specific mobile network provider and thus enables transmission via the mobile network operator's mobile network. In this example, the vehicle and / or the teleoperation device each have at least two different modems.

[0043] Alternatively or additionally, the vehicle and / or teleoperation device can have precisely one communication interface configured, for example, to transmit the data packet via a dual SIM card over at least two different transmission paths, such as over at least two different mobile radio networks. Thus, versatile hardware configurations for the teleoperation system are possible.

[0044] The vehicle is, in particular, a motor vehicle. The motor vehicle is, for example, a passenger car, a truck, a bus, a motorcycle, and / or a moped.

[0045] The invention includes combinations of the described embodiments.

[0046] The figures show:

[0047] Fig. 1 is a schematic representation of a teleoperation system;

[0048] Fig. 2 shows a schematic representation of a signal flow graph of a method for transmitting at least one data packet between a vehicle and a teleoperation device;

[0049] Fig. 3 shows a schematic representation of a signal flow graph of process steps for switching between transmission types; and Fig. 4 shows a schematic representation of a signal flow graph with individual process steps for a transmission type a.

[0050] In the figures, functionally identical components are provided with the same reference numerals.

[0051] Fig. 1 shows a vehicle 1 having a control device 2. The control device 2 is designed to fully automatically perform longitudinal and / or transverse guidance of the vehicle 1. For this purpose, at least one control command for the vehicle 1 is transmitted to the control device 2 from a teleoperation device 5, for example, which the control device 2 executes.

[0052] In the example shown, vehicle 1 has two communication interfaces 3, 4. These can each be configured as a modem, for example. In the example shown, teleoperation device 5 also has two communication interfaces 3, 4. The communication interfaces 3, 4 can be referred to as the first communication interface 3 and the second communication interface 4.

[0053] A teleoperator 6 is assigned to the teleoperation device 5. This person is a person who, for example, can specify or generate at least one control command for the vehicle 1 using a steering wheel and / or at least one additional or different operating device. The control command can be transmitted to the vehicle 1. To determine the control command, the teleoperation device 5 can, for example, comprise a control command determination device 12. The teleoperator 6 can be understood as part of the teleoperation device 5.

[0054] Two different transmission paths 7, 8 are outlined here, which can be referred to as a first transmission path 7 and a second transmission path 8. The respective transmission path 7, 8 can be implemented via a mobile network.

[0055] Alternatively or additionally, the respective transmission path 7, 8 can be provided via a wireless local area network (WLAN).

[0056] At least one data packet 9 can be sent between the vehicle 1 and the

[0057] Teleoperation device 5. This occurs via at least one of the two transmission paths 7, 8. Here, the transmission of the data packet 9 via both transmission paths 7, 8 is sketched redundantly, with the transmission via the first transmission path 7 taking place via the first communication interfaces 3 and the transmission via the second transmission path 8 taking place via the second communication interfaces 4.

[0058] A teleoperation system 10 can comprise at least the vehicle 1, the teleoperation device 5 and the at least two transmission paths 7, 8.

[0059] The vehicle 1 and / or the teleoperation device 5 can have a sensor device 11. This can, for example, be designed as a camera or at least comprise a camera. In particular, in connection with the vehicle 1, alternative or additional sensor devices 11 can be provided, such as a radar device, a lidar device, an ultrasonic sensor and / or a sensor for detecting odometry data as sensor data. Here, purely as an example, a front camera is sketched as the sensor device 11 for the vehicle 1. Alternatively or additionally, the sensor device 11 can be a rear camera and / or a side camera. The sensor device 11 of the teleoperation device 5 can, for example, detect the teleoperator 6 and transmit a live stream from the teleoperator to the vehicle 1 (as at least one data packet).Alternatively or additionally, the sensor device 11 may comprise at least one microphone in order to detect and transmit, for example, speech from the teleoperator 6.

[0060] The vehicle 1 and / or the teleoperation device 5 each have a storage unit 13, which is designed, for example, as a ring memory or ring buffer. The vehicle 1 can include a position determination device 14, by means of which a current position 23 (see reference numeral 23 in Fig. 3) can be determined. The position determination device 14 uses, for example, data from a global navigation satellite system for this purpose.

[0061] Fig. 2 shows steps of a method for transmitting the at least one data packet 9 between the vehicle 1 and the teleoperation device 5 for the teleoperation of the vehicle 1. For simplicity, a single data packet 9 is described below. However, multiple data packets 9 can be transmitted using the method. It is assumed that at least two different transmission paths 7, 8 can be used between the vehicle 1 and the teleoperation device 5 at a current time, i.e., are available for the transmission of the data packet 9.

[0062] With a transmission type a, the data packet 9 is transmitted redundantly via at least two of the usable transmission paths 7, 8. Here, the transmission of the data packet 9 takes place both via the first transmission path 7 and via the second transmission path 8. With a transmission type b, a single transmission path 7, 8 is selected from the usable transmission paths 7, 8 by applying a predetermined selection criterion. The data packet 9 is then transmitted only via the selected transmission path 7, 8, i.e., not redundantly via two different transmission paths 7, 8. The transmission types a, b can be used consecutively, i.e., independently of one another, or at least temporarily simultaneously. In a preferred example, the simultaneous use is limited to a predetermined period of time.

[0063] In a method step S1, a piece of identifier information 20 is determined for each transmitted data packet 9. This describes the data packet 9 regardless of the transmission path 7, 8 used. Individual identifier information 20 is determined for each data packet 9, so that at least partially different data packets 9 can be distinguished from one another based on their identifier information 20. However, for data packets 9 with identical content, such as the data packets 9 transmitted redundantly according to transmission type a, the same identifier information 20 is determined.

[0064] The license plate information 20 can be determined by applying a license plate information determination criterion. The license plate information determination criterion is applied, for example, by a license plate information determination unit 22, which can be arranged in the vehicle 1 and / or in the teleoperation device 5. The license plate information determination criterion is, in particular, a hash function 21. In other words, the license plate information 20 can be determined by applying the hash function 21 to the respective transmitted data packet 9. The license plate information 20 can be determined before the data packet 9 is sent, and thus in the sending component. Alternatively or additionally, the license plate information 20 is determined after the data packet 9 is received, and thus on the receiver side.

[0065] In a method step S2, a check is carried out to determine whether the determined license plate information 20 is stored in a storage unit 13. If, for example, the teleoperation device 5 receives the data packet 9, the storage unit 13 of the teleoperation device 5 is considered. However, if the data packet 9 is transmitted to the vehicle 1, the storage unit 13 of the vehicle 1 is considered. A comparison is thus made with the storage unit 13 of the respective recipient. If it is determined that the determined license plate information 20 is already stored in the storage unit 13, a duplicate data packet 9 is detected and, in a method step S3, the data packet 9 is then discarded because it was received twice.

[0066] Alternatively or additionally, if it is determined that the determined license plate information 20 is not yet stored in the storage unit 13, a method step S4 is performed. In this step, the license plate information 20 is stored at least temporarily in the storage unit 13, and the data packet 9 is further considered, so that it is used, for example, to determine the control command and / or the control command contained in the data packet 9 is implemented in the vehicle 1.

[0067] It is possible for data packet 9 to include sensor data from sensor device 11 of vehicle 1 and / or teleoperation device 5. Alternatively or additionally, it may include control command data for controlling vehicle 1 by teleoperator 6, i.e., the at least one control command. In this case, the control command data is transmitted in vehicle 1 to control device 2, so that the latter can, for example, perform longitudinal and / or lateral guidance according to the control command data.

[0068] It can be provided that the identification information 20 is stored only temporarily, i.e., only for a predetermined time interval, and is deleted and / or overwritten after the expiration of the time interval. The predetermined time interval can correspond at least to the transmission duration that is predetermined for the respective transmission path 7, 8, i.e., is known, and / or is determined for the respective transmission path 7, 8. For example, if the transmission duration via the first transmission path 7 is 200 milliseconds, the predetermined time interval can be 200 milliseconds or more. For example, it can be 300 milliseconds long and thus include a time buffer. Time intervals other than the examples mentioned are possible.

[0069] Fig. 3 shows situations in which it is possible to switch between transmission types a, b. In a method step S5, a current position 23 of the vehicle 1 can be determined by means of the position determination device 14. In a method step S6, it can be checked whether the determined current position 23 is located in a predetermined area 24 for which at least one of the transmission paths 7, 8 is predetermined. There is then geofencing for the respective transmission path 7, 8, by which relevant data for the area 24 is stored, for example, in the memory unit 13. If it is determined that the current position 23 is in the predetermined area 24, it can be provided in a method step S7 that the data packet 9 is always transmitted via the transmission path 7, 8 for which the predetermined area is predetermined. This transmission path 7, 8 is referred to here as the third transmission path 25.The third transmission path 25 can correspond to the first transmission path 7 or the second transmission path 8 or can be different from them. In method step S7, transmission type a and / or transmission type b can be provided. Thus, for example, only the transmission path 7, 8 provided for the specified area 24 can be selected, or it can be provided as one of the multiple transmission paths 7, 8 for redundant data transmission.

[0070] Alternatively or additionally, in a method step S8, a transmission quality value 26 can be determined, which describes a quality of a transmission via the respective transmission path 7, 8. This is done by applying a transmission quality determination criterion 27, in particular by means of a transmission quality determination device.

[0071] In a method step S9, it can be specified that, of all usable transmission paths 7, 8, only the transmission path 7, 8 having the highest transmission quality value 26 is selected. This transmission path 7, 8 is identified here as the fourth transmission path 28. The fourth transmission path 28 can correspond to the first transmission path 7 or the second transmission path 8 or can be different from them. After the transmission quality value 26 has been determined, a method step S10 can check whether the determined transmission quality value 26 is greater than or equal to a transmission quality limit value 29. If this is the case, a change can be made, for example, from a previously used transmission type a to transmission type b in a method step S11.If this is not the case, for example, in a method step S12, transmission type a can be maintained or a switch can be made from transmission type b to transmission type a. If the transmission quality value 26 is at least as high as the transmission quality limit value 29, only a single transmission path 7, 8 is always selected, since a switch is made to transmission type b. Remaining in transmission type a and / or switching from transmission type b to transmission type a can occur alternatively or additionally if it is determined in method step S6 that position 23 is not located in the specified area 24.

[0072] It can be provided that, although the position 23 lies in the specified area 24, the change from transmission type a to transmission type b takes place without, for example, the transmission quality value 26 being determined and checked.

[0073] In method steps S13, S14, it can be provided that, in response to an operator input 30 from a user of the vehicle 1, the change between transmission types a, b occurs, either from transmission type b to transmission type a and, alternatively or additionally, from transmission type a to transmission type b. The change between the two transmission types a, b can thus be triggered by the position 23 within or outside the specified area 24, the comparison between the transmission quality value 26 and the transmission quality limit value 29, and / or initiated by the operator input 30.

[0074] When switching between the two transmission types a and b, the previous transmission of data packet 9 can be terminated immediately. A direct switchover can thus occur. Alternatively or additionally, the transmission of data packet 9 can be performed simultaneously using both transmission types a and b, at least temporarily. The duration of this can be individually determined.

[0075] If at least three transmission paths 7, 8 are usable, switching between the individual transmission paths 7, 8 of the at least three transmission paths 7, 8 is possible without switching between transmission type a and transmission type b by changing the respective transmission path 7, 8 via which the at least one data packet 9 is transmitted. Thus, for example, with transmission type a, the first transmission path 7 can be replaced by the third transmission path 25 and / or the second transmission path 8 can be replaced by the fourth transmission path 28, or vice versa.

[0076] Fig. 4 outlines a sequence for transmission type a. First, the data packet 9 is transferred to an internal receiver 40 of the unit transmitting the data packet 9. The transmitting unit here is the vehicle 1 and / or the teleoperation device 5. The internal receiver 40 forwards the data packet 9 to a duplication unit 41, which can generate two redundant data packets 9 to be transmitted. These can each be transferred to a transmitting unit 42, which can then transmit its data packet 9 via the respective transmission path 7 to a respective receiving unit 43. The respective receiving unit 43 for the transmission paths 7, 8 can transfer its respective data packet 9 to a data combining unit 44, which, for example, carries out the comparison with the storage unit 13.At the end, the result of the data comparison with the storage unit 13 is forwarded to an internal data transmitter 47, which receives the data packet 9 and makes it available for further processing in the vehicle 1 and / or in the teleoperation device 5.

[0077] Overall, the examples demonstrate hash-based data packet selection for multipath communication in the field of teleoperation. For the teleoperation of vehicles 1, a reliable connection between the vehicle 1 and the teleoperator 6, i.e., the teleoperation device 5, is a fundamental prerequisite. Common problems with the mobile networks used are interruptions and massive jitter in the throughput of individual data packets 9.

[0078] These issues result in poor system performance and can cause system failures affecting the transmission over the respective transmission path 7, 8. For example, video artifacts or a frozen screen can be caused by such issues.

[0079] The use of two parallel transmission paths 7, 8 according to transmission type a can solve the aforementioned problems. However, receiving redundant data packets 9 and processing them properly can be computationally intensive. The invention significantly reduces the aforementioned problems by using multiple independent transmission paths 7, 8 for mobile data communication. The invention supports transmission types a, b (and combinations of both):

[0080] 1) Redundant data transmission (transmission type a)

[0081] 2) Switching the transmission path 7, 8 (transmission type b)

[0082] 3) Combinations of 1 ) and 2)

[0083] Regarding transmission type a: This requires the sending device to duplicate the data packets 9 and transmit them redundantly over the network's transmission paths 7 and 8. There is actually no theoretical limit to the number of transmission paths 7 and 8, but in practice, two transmission paths 7 and 8 are likely to be the most common configuration.

[0084] The invention generates a unified data stream from at least two incoming data streams in the receiving device by selecting and storing suitable data packets 9. This is done using the following mechanism: For each incoming data packet 9 from the various reception paths, a sufficiently individual, yet short identification representation string is calculated. This is referred to here as the identification information 20. Hash functions 21 are used for this purpose.

[0085] In this example, the invention comprises a ring buffer as storage unit 13, in which these identification character strings (identification information 20) are stored if they are unknown to the receiving device. If the identification character string (identification information 20) is not found in the storage unit 13, it is assumed that the data packet 9 is new and is forwarded to the next module or processing step, and the identification character string (identification information 20) is written into the ring buffer as storage unit 13. If an identical identification character string (identification information 20) is found in the ring buffer, it is assumed that this data packet 9 is a duplicate and is discarded. The size of the ring buffer depends, for example, on the estimated maximum delay between the data packets 9 involved.A data packet 9 should be stored in the ring buffer for as long or longer than the time it takes for its delayed copy to arrive on another transmission path 7, 8. If this criterion is met, repeated data packets 9 can be detected and discarded. Regarding transmission type b: Another option for utilizing multiple transmission paths 7, 8 in the network is switching between them. Instead of transmitting identical data duplicates on different transmission paths 7, 8, in this use case the most suitable transmission path 7, 8 is selected depending on the current situation. One use case for this could be switching between a general public mobile network and a special private high-speed network that is restricted to certain areas as soon as the teleoperated vehicle 1 logs on to this private high-speed network or leaves the private high-speed network.

[0086] Switching between transmission paths 7 and 8 is supported, and there are several ways to trigger the switching process:

[0087] 1) User input (operating input 30),

[0088] 2) Geographical location (position 23 in area 24),

[0089] 3) Mobile network quality (comparison of transmission quality value 26 with transmission quality limit value 29).

[0090] In detail, this transmission type b, for example, is implemented as follows: On the sender side, the data path can be switched. Two types of transmission type switching are supported:

[0091] 1) Spontaneously without notice (that is, immediately),

[0092] 2) Overlapping, so that both transmission types are used during the switching process.

[0093] On the receiving side, our device uses the data packet selection mechanism already described.

[0094] The following combinations are also possible:

[0095] 1) Two transmission paths 7, 8: If two transmission paths 7, 8 are used, it is possible to switch between the two transmission types a, b also on the transmitter side.

[0096] 2) At least when more than two transmission paths 7, 8 are used, it is possible to use combinations of both transmission types a, b. Switching between different pairs of redundant transmission paths 7, 8 or a single transmission path 7, 8 and a pair of redundant transmission paths 7, 8 can be provided. Any combination is possible and can be applied. With hash-based data packet selection, the computational requirements in the receiver are significantly reduced, since only short hashes need to be compared instead of large data packets 9, thus avoiding data packet duplicates.

[0097] The following is a concrete embodiment: The vehicle 1, which is to be remotely controlled, is used in an area 24 with poor mobile network coverage. In this case, several mobile network operators with suboptimal connection quality could be available. To improve the performance of the overall connection, the present invention is applied, in which two modems from different mobile network operators are installed in the vehicle 1 and the data packets 9 are sent via both network operators (i.e., via both transmission paths 7, 8). On the receiving side (control station or teleoperator cockpit), the presented method of hash-based data packet selection is applied in the receiver to recover lost data packets 9 and thus reduce the data packet loss rate. This leads to a significantly better perceived mobile connection (transmission of the data packets 9) and thus increases the stability, reliability, and security of the application.Alternatively, the sender could use only one modem instead of two and send redundant data packets 9 via a mobile operator. The receiver ensures that the data packets 9 are not sent twice. Duplication of the data packets 9 is achieved here using hash-based data packet selection I.

Claims

Patent claims 1 . Method for transmitting at least one data packet (9) between a vehicle (1) and a teleoperation device (5) for a teleoperation of the vehicle (1), wherein at least two mutually different transmission paths (7, 8) can be used between the vehicle (1) and the teleoperation device (5) at a current time, wherein (a) the at least one data packet (9) is transmitted redundantly via at least two of the usable transmission paths (7, 8), and / or (b) a single transmission path (7, 8) is selected from the usable transmission paths (7, 8) by applying a predetermined selection criterion and the at least one data packet (9) is transmitted via the selected transmission path (7, 8), wherein for the respective transmitted data packet (9) an identifier information (20) which describes the data packet (9) independently of the transmission path (7, 8) used is determined (S1) and it is checked (S2) whether the determined identifier information (20) is stored in a memory unit (13), wherein - if this is the case, the data packet (9) is discarded (S3), and / or - if this is not the case, the identification information (20) is stored at least temporarily in the memory unit (13) and the data packet (9) is further taken into account (S4).

2. Method according to claim 1, wherein the identification information (20) is determined by applying a hash function (21) to the respective transmitted data packet (9).

3. Method according to one of the preceding claims, wherein the vehicle (1) and / or the teleoperation device (5) comprises a license plate information determination unit (22) which determines the license plate information (20) for the at least one data packet (9) before it is sent and / or after it is received.

4. Method according to one of the preceding claims, wherein the stored identification information (20) is deleted and / or overwritten after a predetermined time interval.

5. The method according to claim 4, wherein the predetermined time interval corresponds to at least one transmission duration predetermined for the respective transmission path (7, 8) and / or determined for the respective transmission path (7, 8).

6. Method according to one of the preceding claims, wherein a current position (23) of the vehicle (1) is determined (S5) and it is checked (S6) whether the determined current position (23) is located in a predetermined area (24) for which at least one of the transmission paths (7, 8) is predetermined, wherein if this is the case, the at least one data packet (9) is always transmitted at least via this transmission path (7, 8) (S7).

7. Method according to one of the preceding claims, wherein a transmission quality value (26) is determined for the respective usable transmission path (7, 8) (S8) and in (b) the transmission path (7, 8) is selected which has the highest transmission quality value (26) of the usable transmission paths (7, 8) (S9).

8. Method according to one of the preceding claims, wherein the change from (a) to (b) is made if a transmission quality value (26) for one of the transmission paths (7, 8) is greater than or equal to a transmission quality limit value (29) (S10, S11), and / or if a determined current position (23) of the vehicle (1) is in a predetermined area (24) for which at least one of the transmission paths (7, 8) is specified (S11) and / or if a user specifies the change by means of an operating input (30) (S6, S14).

9. Method according to one of the preceding claims, wherein a change is made from (b) to (a) if it is determined that a transmission quality value (26) for the transmission path (7, 8) used is smaller than a transmission quality limit value (29) (S10, S12), and / or if a determined current position (23) of the vehicle (1) is no longer in a predetermined area (24) for which the transmission path (7, 8) used is predetermined, (S6, S12) and / or if a user specifies the change by means of an operating input (30) (S13).

10. Method according to one of the preceding claims, wherein, upon a change between (a) and (b), the previous transmission of the at least one data packet (9) is terminated immediately or, at least temporarily, the transmission of the at least one data packet (9) takes place simultaneously both according to (a) and according to (b).

11. Method according to one of the preceding claims, wherein if at least three transmission paths (7, 8) are usable, it is possible to switch between the individual transmission paths (7, 8) of the at least three transmission paths (7, 8) without switching between (a) and (b) by changing the respective transmission path (7, 8) via which the at least one data packet (9) is transmitted.

12. Method according to one of the preceding claims, wherein the respective transmission path (7, 8) comprises a mobile radio network and / or a wireless local area network (WLAN).

13. Method according to one of the preceding claims, wherein the at least one data packet (9) comprises at least sensor data of a sensor device (11) of the vehicle (1) and / or the teleoperation device (5) on the one hand and / or control command data for controlling the vehicle (1) by a teleoperator (6) on the other hand.

14. Teleoperation system (10), comprising a vehicle (1), a teleoperation device (5) and at least two transmission paths (7, 8) between the vehicle (1) and the teleoperation device (5), wherein the teleoperation system (10) is designed to carry out a method according to one of the preceding claims.

15. Teleoperation system (10) according to claim 14, wherein the vehicle (1) and / or the teleoperation device (5) has at least two different communication interfaces (3, 4), each of which is designed to a transmission path (7, 8) between the vehicle (1) and the Teleoperation facility (5) to be provided.

Images