Method for displaying the environment of a remotely controlled motor vehicle to a teleoperator, computer program product, computer-readable storage medium and teleoperation device

Abstract

The present invention relates to a method for displaying the environment (4) of a remotely controlled motor vehicle (1), comprising the steps of: receiving environmental information (20) of the motor vehicle (1); dividing the environmental information (20) into at least a first view part (22), which is assigned to a main viewing area in the motor vehicle (1), and into at least a second view part (23), which is assigned to at least a side viewing area in the motor vehicle (1); generating a main view (18) for the teleoperator (9) depending on the first view part (22) with a first scaling factor; generating at least one side view (16, 17) for the teleoperator (9) depending on at least the second view part (23) with at least a second scaling factor different from the first scaling factor;and displays of the main view (18) with the first scaling factor and displays of at least one side view (16, 17) with the second scaling factor. Furthermore, the invention relates to a computer program product, a computer-readable storage medium, and a teleoperation device (2).

Description

The following invention relates to a method for displaying the environment of a remotely controlled motor vehicle to a teleoperator on a display device of a teleoperation device. The invention further relates to a corresponding computer program product, a computer-readable storage medium, and a teleoperation device. Teleoperation services for motor vehicles refer specifically to the remote control of cars using a remote control system, particularly a teleoperation device, or to autonomous driving functions where a human operator assumes control. This technology is used in various applications, such as in the mining industry, in hazardous environments, for overcoming obstacles, or for autonomous valet parking. Furthermore, this process can also be carried out in at least partially or fully autonomous operation of the vehicle, for example, when in a corresponding operating mode the driving task would normally have to be handed over to a driver, but can now be performed by the teleoperator. For this purpose, the vehicle's surroundings are captured by at least one camera or camera system, and the corresponding camera data is then transmitted to the teleoperation device for remote control. This can present particular challenges for the display to the teleoperator. For example, the delay time, which describes the latency between the camera recording and the display on the screen, can lead to a delayed reaction by the teleoperator, potentially compromising safety. It is therefore crucial to keep the delay time as short as possible to enable near real-time control. The camera must also be capable of capturing high-quality, sharp images under varying lighting conditions. Poor image quality can hinder the teleoperator's decision-making and lead to errors.The camera must also cover a sufficiently large field of view to give the teleoperator an adequate overview of the surroundings. A limited field of view can impair the perception of obstacles and potential risks. In some applications, multiple cameras may be required to cover different perspectives and angles. Integrating and synchronizing these cameras can be challenging. The teleoperator must have access to displays of appropriate size, resolution, and refresh rate to view the captured images clearly and without delay or distortion. An inadequate display can lead to misinterpretation of the environment and compromise safety. The teleoperation system or device must also be easy and intuitive to use to avoid placing undue strain on the teleoperator.A complex or confusing user interface can negatively impact the efficiency and security of the teleoperation service. Furthermore, to compensate for camera or display failures, it is essential to implement redundant systems. These should be seamlessly integrated into the teleoperation setup to ensure uninterrupted operation. Teleoperation services are also vulnerable to hacking attacks and cybersecurity threats. It is crucial to implement appropriate security measures to guarantee system integrity and user safety. As previously mentioned, teleoperation involves the remote control of motor vehicles. It is a process in which an operator controls a vehicle from a remote location, and in particular, without direct visual contact with the vehicle. To perceive the surroundings of the remotely controlled vehicle, the teleoperation device, which can also be called a teleoperation cockpit, receives various pieces of information about the vehicle. A key component of this information is the video streams showing the vehicle's environment. These camera images are displayed in the user interface (UI) of the teleoperation device. Several problems and challenges can arise when setting up the layout and arranging these images on the display device. One of the problems is the teleoperator's limited field of vision. The more live images that need to be displayed on the screen, the more difficult it is for the teleoperator to see them all at once. More images generally require more width, and at some point, the teleoperator is no longer able to perceive all the image information simultaneously or without moving their head. This can be a safety risk, for example, if a pedestrian on one side is not detected because the teleoperator is currently focused on something else, or on the other side of the monitor or display device. Another problem is that, at first glance, a vehicle's cameras may appear to have seamless transitions in the user interface. However, while driving, the teleoperator will often find that the cameras don't align perfectly; for example, some information from the center image might also be displayed in the left image, and vice versa. This can lead to misjudgments of the direction or distance of objects, such as when an object is displayed twice from different camera angles. A further issue is that different vehicles may have different cameras with varying intrinsic values / aspect ratios. The changing number of cameras and the changing attributes of the incoming camera images complicate the design of a consistent display device.Different camera configurations require a new, adapted layout to meet the teleoperator's needs and ensure that all essential information is captured in the videos. For example, different camera aspect ratios result in different cockpit space utilization. Sometimes, cropping images is necessary to fit the limitations of the monitor being used. Future teleoperations with different motor vehicles will require a lot of time and resources to constantly adjust the display device accordingly or to change the monitors used in order to adapt to the required aspect ratios of the videos. Additionally, user interfaces will vary significantly between different vehicles, meaning a teleoperator may need an adjustment period to a new vehicle with a new cockpit after becoming accustomed to a different vehicle in the final hours of their shift. This is a safety issue because a teleoperator should be exposed to as little additional stress as possible. Any parallel task or challenge can impair their attention. The object of the present invention is to provide a method, a computer program product, a computer-readable storage medium and a teleoperation device by means of which an improved teleoperation service can be provided for a teleoperator. This problem is solved by a method, a corresponding computer program product, a corresponding computer-readable storage medium, and a teleoperation device according to the independent claims. Advantageous embodiments are specified in the dependent claims. One aspect of the invention relates to a method for displaying the environment of a remotely controlled motor vehicle to a teleoperator on a display unit of a teleoperation device. Environmental information from the motor vehicle is received by means of an electronic computing unit of the teleoperation device. The environmental information is divided into at least a first view, which corresponds to a main viewing area in the motor vehicle, and at least a second view, which corresponds to at least a lateral viewing area in the motor vehicle, for this purpose in particular using an electronic computing unit of the teleoperation device. A main view for the teleoperator is generated by the electronic computing unit, depending on the first view, with a horizontal first scaling factor.A side view for the teleoperator is generated by the electronic computing device, depending on at least the second view section with at least one horizontal second scaling factor that differs from the horizontal first scaling factor. The main view is displayed with the horizontal first scaling factor, and at least one side view with the horizontal second scaling factor is displayed by the display device. This allows the teleoperator's field of view at the teleoperation device to be reduced accordingly, enabling them to see more information on, for example, a single screen. Specifically, the side view is displayed in a reduced size, particularly with horizontal compression, so that the operator can immediately distinguish that it is an image from the side. Simultaneously, a front view, such as that seen through a car windshield, can still be displayed on the main area of ​​the display, for example, in the center. Furthermore, this makes it possible, for example, to display both a left side area and a right side area with a horizontally reduced scaling factor, while, for example, the front area can be displayed larger compared to the side area(s). In other words, there can be areas in the camera images that are not compressed, and there can be areas that are compressed, particularly horizontally. For example, video streams from side cameras are displayed with horizontal compression. Furthermore, this allows, for instance, both the left and right sides to be displayed with width compression, thus saving space on the available monitor width and enabling a larger display of the front area. The fields of view are defined in this context specifically as being described from the perspective of a virtual driver. In other words, a front view is a view looking forward from inside the vehicle, a side view is a view to the right or left from inside the vehicle, and a rear view is a view looking backward from inside the vehicle. Depending on the position of the respective cameras, the area "from inside the vehicle" may vary. To solve the aforementioned problems, a deforming scaling module for teleoperation camera images is proposed. Specifically, to reduce the display space required for camera images without losing important and safety-relevant information for the teleoperation service, a corresponding horizontal scaling, particularly horizontal compression, is intended. The teleoperation device supports various functions that can scale incoming input images into an output image without maintaining the aspect ratio. By using the so-called deforming scaling mode, more image information can be displayed within the teleoperator's field of view. This deforming scaling mode does not result in any loss of semantic information about the image, unlike cropping or cutting out parts of the image. This reduces the number of monitors required for the teleoperating device and minimizes the number of cropped sections needed. In this way, the teleoperator can more easily access important parallel image information without, for example, having to turn their head. A horizontally scaled-down image also has a lower resolution than the original image. Therefore, less computing power is required on the part of the teleoperation device. In particular, a preferred embodiment provides for the downscaling to be performed in the vehicle itself, thus compressing the data at the sender's end of the camera information. This saves bandwidth during transmission, reducing the risk of delays and packet loss, and thereby improving the visual quality of the transmitted information and reducing latency. Another advantage from a user interface perspective is that the distortion of the side images disrupts the teleoperator's expectation of a seamless transition between, for example, the front camera and the side cameras. This alerts the teleoperator that the side images are horizontally compressed, unlike the central image (main view). This is primarily because the side images often contain redundant information compared to the main view. Since the camera images appear similar, the teleoperator expects a seamless continuation between them. In reality, however, they will see that the edges of the different cameras do not align perfectly, which can cause critical security issues.By compressing the side images, the teleoperator's expectations are directly disrupted, as he sees that the side images shrink in width, unlike the main view. In this way, the teleoperator will use the side views differently than he would have previously. As mentioned previously, the cameras can include, for example, a front camera, at least two side cameras, and a rear camera. Depending on the driving situation, the front camera can be used to generate the main view when driving forward, while the rear camera can be used for the main view when reversing. The side cameras are then required to generate the side view. In other words, it's possible to generate both a left and a right side view. It's also possible for the vehicle to have additional cameras, the images from which can be displayed, for example, on mirrors or in other locations on the vehicle that are not normally visible. In order for a vehicle to be remotely controlled, particularly by a teleoperator, the vehicle must be equipped with a system that allows the teleoperator to control it remotely. This includes control elements such as the steering wheel, accelerator and brake pedals, as well as brake and acceleration sensors. The vehicle must also have communication systems that allow the teleoperator to communicate with the vehicle and send commands to it. These include radio links or mobile networks. The vehicle must also have sensors that allow the teleoperator to perceive its surroundings; cameras, radar, and lidar systems can be used for this purpose. The vehicle also has appropriate actuators that allow the teleoperator to control the vehicle remotely. Electric motors or hydraulic systems, for example, can be used for this purpose.The vehicle is equipped with appropriate safety systems to protect the teleoperator or other road users in an emergency. These systems may include airbags, seatbelt pretensioners, and emergency braking systems. Furthermore, the vehicle must have software that allows the teleoperator to control it remotely. This software may offer a user-friendly interface and ensure secure and reliable communication with the vehicle. The teleoperator must receive appropriate training to safely operate the vehicle remotely. This training may include familiarizing themselves with the vehicle's controls and systems, as well as its communication and safety features. An independent aspect of the invention can also be that the main view and the side view are scaled / compressed identically, but exhibit a different compression than the original image, which was captured, for example, by the scanning device. "Compressed identically" here can be understood as compression to the same target aspect ratio and / or compression by the same factor. In one advantageous implementation, the side view is displayed horizontally compressed compared to the main view. This provides the teleoperator with an improved overview. The teleoperator can better see the side views and the information they contain without having to move their head. Horizontally compressing the side images disrupts the teleoperator's expectation of a seamless transition between the different camera images. This prevents misinterpretations and can avert critical security problems. Horizontally downscaling the side images on the sender's end reduces the transmission bandwidth, thereby minimizing the risk of delays and packet loss and improving the visual quality of the transmitted information.Furthermore, displaying narrowed side views reduces the number of monitors required in the teleoperation device and minimizes the need for cutouts, resulting in a more space-saving and organized work environment. These side views also help the teleoperator better perceive speed and distance, which is crucial for driving and navigating in confined or tight spaces. Furthermore, it has proven advantageous to display the side view separately from the main view. Displaying the side views separately allows for a clear distinction between the different camera images, helping the teleoperator to better understand the spatial relationship between objects. This is particularly beneficial when, for example, the environment is captured by a front camera and two side cameras. Alternatively, the environment can also be captured by a 180° camera. Additionally, displaying the side views separately can reduce the potential for confusion by preventing the teleoperator from mistaking the side images for the central / main view. The teleoperator can better view the side views and the information they contain without having to move their head.By compressing the side views, the number of monitors in the teleoperations cockpit can be reduced, and the need for cutouts decreased, resulting in a space-saving and uncluttered work environment. Furthermore, the separate display allows for flexible monitor placement and thus an optimal arrangement for the teleoperator, facilitating their work. For example, in this embodiment, the main view can be displayed on a central display element of the display unit, while at least one side view is displayed on a separate display element, such as a left or right one. This allows for a corresponding arrangement of these display elements, enabling more realistic use. The individual display elements can, in particular, be separate monitors. Another advantageous implementation involves statically generating a ratio between the first and second scaling factors. In particular, this allows for a first approach to solving the problem of inconsistent camera images and limited fields of view by statically reducing the width of the side images without reducing the aspect ratio of the main view. For side images, it may be less critical to display all pixels horizontally. A compressed video stream, compared to a cropped one, has the advantage that the information of the entire original width is retained, whereas cropping removes entire portions of the image. The key advantage, therefore, is that it enables consistent and predictable scaling of the camera images.The ratio between the scaling factors remains unchanged, which simplifies the integration and display of the images in the teleoperations user interface. This method can also help reduce the potential for confusion for the teleoperator, as the side images always maintain a consistent size. Furthermore, this approach allows for better control over the quality of the displayed images and helps ensure that the teleoperator's field of view is not unnecessarily restricted. Furthermore, it has proven advantageous to dynamically generate the ratio between the first and second scaling factors. The benefit of dynamically generating this ratio is that it allows for flexible and adaptable scaling of the camera images. The ratio between the scaling factors is dynamically adjusted to ensure that the most important objects or areas in the image are always visible and not cropped. This method can help reduce the potential for confusion for the teleoperator, as the side images are automatically adjusted to the specific situation, thus achieving optimal scaling. Moreover, this approach allows for better control over the quality of the displayed images and helps prevent the teleoperator's field of view from being unnecessarily restricted.Overall, the dynamic generation of the ratio between the scaling factors ensures a flexible, adaptable, and user-friendly display of the camera images in the teleoperation display, thus simplifying the teleoperator's work and increasing safety. This method is particularly advantageous when the object being controlled or the environment varies significantly and statistical scaling is insufficient to display all the important information in the image. It has also proven advantageous to display the second part of the side view with different, particularly horizontal, scaling. The benefit of displaying the second information section in the side view with varying scaling factors during dynamic scaling is that it allows for optimal presentation of the relevant information. Specifically, a scaling factor can be provided that, for example, incorporates additional vehicle information to adjust the scaling accordingly. This scaling factor can also be adjusted within a single side view image. For instance, outer edges in the side view can then experience greater horizontal compression than images closer to the center, i.e., image information adjacent to the main view in the side area.This allows for highly dynamic display. It has also proven advantageous to display a central view element in the second view element at a lower scale than a side view element. The central view element of the second view element connects to the main view, particularly in the side area. In other words, the central view element of the second view element forms the transition between the main view and the side view. For example, in the side view, the central part, or the part facing the main view, can then be displayed at a correspondingly lower scale than the side view element. This allows the second view element to be generated and transmitted to the teleoperation device at a significantly lower information rate, resulting in even lower latency.Furthermore, there is hardly any loss of information, as the information necessary for current driving operations can still be displayed in a low-scale format. Another advantageous design incorporates the consideration of a current vehicle dynamics parameter of the vehicle and / or the teleoperation device when generating the side view. This allows the display to respond appropriately to the current situation and thus be highly dynamic. This provides an even better overview of the driving or control conditions. By considering vehicle dynamics parameters such as lateral and / or longitudinal acceleration, the side view can be dynamically adapted to the current driving situation. This provides the teleoperator with a more informative picture of the current situation, facilitating decision-making and increasing safety, as all relevant information is presented to the teleoperator in the most concise way possible.Furthermore, considering vehicle dynamics parameters can contribute to better control of the vehicle or teleoperation device. Dynamically adjusting the side view provides the teleoperator with crucial information about the driving or control status, enabling faster and more precise reactions. Overall, incorporating vehicle dynamics parameters when generating the side view offers numerous advantages for the teleoperator. This method can improve vehicle or teleoperation device control, allowing the teleoperator to react more quickly and precisely to changes in the driving situation, thereby increasing safety and facilitating their work.The vehicle dynamics parameter of the teleoperation device is, in particular, a virtual vehicle dynamics parameter that is set, for example, by the teleoperation device on the motor vehicle. In a further advantageous embodiment, the current steering angle and / or speed are taken into account when generating the side view, particularly as vehicle dynamics parameters. This allows for a better overview of the driving or control conditions. By considering the steering angle and / or speed, the side view can be dynamically adapted to the current driving situation, providing the teleoperator with a more realistic representation of the situation. For example, if the vehicle is braking sharply or turning into a curve, the side view can be enlarged accordingly to alert the teleoperator to the current situation. Furthermore, considering the steering angle and speed can contribute to better control of the vehicle or the teleoperation device.Dynamic adjustment of the side view provides the teleoperator with crucial information about the driving or control status, enabling faster and more precise reactions. Considering steering angle and / or speed when generating the side view offers numerous advantages for the teleoperator. This method can improve control of the vehicle or teleoperating device, allowing the teleoperator to react more quickly and effectively to changes in the driving situation, thus enhancing safety and simplifying the operator's work. Therefore, a non-static reduction in size, combined with the use of the current steering angle of, for example, the teleoperator or the vehicle, is proposed.The steering angle can define the midpoint of the scaling function, which means that information about the steering track is scaled down less than information that is further away from the steering or driving track. According to a further advantageous embodiment, a central view area is determined based on the current steering angle and displayed with a lower scaling factor than a lateral view area. This has the particular advantage that the central information area can be determined based on the current steering angle and thus displayed with a lower scaling factor, i.e., with a greater width compared to the lateral areas. The dynamic adjustment of the lateral view provides the teleoperator with a more informative representation of the driving or control status, which facilitates decision-making and increases safety. The central information area with its lower scaling allows the teleoperator to obtain important information around the vehicle's driving path.The central information area could also be on the right edge of the monitor if the teleoperator is steering to the right. Furthermore, it has proven advantageous to suppress a portion of the second view depending on the current steering angle. This dynamic suppression of information allows the teleoperator to grasp the most important data at a glance and focus on the relevant information, thus reducing reaction time and improving the accuracy of their actions. This is particularly helpful in situations where information overload threatens to distract or overwhelm the teleoperator. Suppressing irrelevant information helps to focus the teleoperator's attention on what is essential, thereby increasing safety.Furthermore, it can be implemented, for example, that when steering to the right, the left-side view is correspondingly shortened or cropped, so that more information is available to the right than to the left. Thus, a combination of horizontal compression and cropping is proposed. In this case, the image from the front camera, for example, is neither reduced in size nor cropped. Only the side images are manipulated based on the steering angle. This function defines the section of the side images based on the steering angle. If the vehicle, for example, steers to the left and reverses, the left image can show more information from the blind spot. The width of the image can remain unchanged. This means that when steering to the left, the left image displays more side information and is therefore reduced in size more significantly.When returning to the direction of travel, less lateral information is displayed and the reduction in size is correspondingly less. Another advantageous design provides that a frontal or rearward view is displayed as the main view. In particular, this can be automated depending on whether, for example, the vehicle's forward or reverse driving function is activated. Thus, a corresponding main view can be adjusted for both forward and reverse driving. The presented method is, in particular, a computer-implemented method. Therefore, a further aspect of the invention relates to a computer program product with program code means which, when the program code means are executed by the electronic computing device, cause it to carry out a method according to the preceding aspect. A further aspect of the invention also relates to a computer-readable storage medium containing at least the computer program according to the previous aspect. Furthermore, the invention relates to a teleoperation device for displaying the environment of a remotely controlled motor vehicle to a teleoperator, comprising at least one electronic computing unit and a display unit, wherein the teleoperation device is configured to carry out a method according to the preceding aspect. In particular, the method is carried out by means of the teleoperation device. A further aspect of the invention relates to a motor vehicle designed to be operated via the teleoperation device and by means of the corresponding method. For this purpose, the motor vehicle has, in particular, features enabling it to participate in the teleoperation service. Furthermore, the motor vehicle is equipped with at least one electronic computing device, which, in particular, reduces the information from the side cameras and thus transmits it to the electronic computing device of the teleoperation device. Advantageous embodiments of the process are to be regarded as advantageous embodiments of the computer program product, the computer-readable storage medium, the teleoperation device, and the motor vehicle. The teleoperation device and the motor vehicle, in particular, possess tangible features to enable the execution of the corresponding process steps. In the present disclosure, a computing unit / electronic computing device can be understood, for example, as a data processing device with processing circuits. A computing unit can thus perform arithmetic operations to process data. These arithmetic operations can also include indexed access to a data structure, such as a lookup table (LUT). A computing unit may, in particular, comprise one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems on a chip (SoCs). The computing unit may also include one or more processors, for example, one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit may also comprise a physical or virtual cluster of computers or other units of the aforementioned type. A processing unit can also include one or more hardware and / or software interfaces and / or one or more memory units. A memory unit can be implemented as volatile data storage, for example as dynamic random access memory (DRAM) or static random access memory (SRAM), or as non-volatile data storage, for example as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or flash EEPROM, or ferromagnetic random access memory (FRAM).a magnetoresistive random access memory, MRAM (magnetoresistive random access memory), or a phase-change random access memory, PCRAM (phase-change random access memory). For use cases or application situations that may arise in a method according to the invention and that are not explicitly described herein, it may be provided that, according to the method, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set. Further features of the invention are evident from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as those subsequently mentioned in the description of the figures and / or shown in the figures alone, are not only usable in the combinations specified, but also in other combinations without departing from the scope of the invention. Thus, embodiments that are not explicitly shown and explained in the figures, but which can be derived and generated from the explained embodiments by separate combinations of features, are also to be considered as encompassed and disclosed by the invention. Embodiments and combinations of features that do not exhibit all the features of an originally formulated independent claim are also to be considered disclosed.Furthermore, embodiments and combinations of features, in particular those set out above, are to be considered disclosed which go beyond or deviate from the combinations of features set out in the cross-references of the claims. Figure 1 shows a schematic top view of an embodiment of a motor vehicle with an embodiment of a teleoperation device; Figure 2 shows a schematic block diagram according to an embodiment of a motor vehicle with an embodiment of a teleoperation device; Figure 3 shows a schematic view of a display device of an embodiment of a teleoperation device; Figure 4 shows a schematic top view of an embodiment of a display device of an embodiment of a teleoperation device; Figure 5 shows a schematic top view of an embodiment of a display device of an embodiment of a teleoperation device; Figure 6 shows another schematic top view of an embodiment of a display device of an embodiment of a teleoperation device; and Figure 6 shows...7. A further schematic top view of an embodiment of a display device of an embodiment of a teleoperation device. In the figures, identical or functionally equivalent elements are provided with the same reference symbols. Fig. 1 shows a schematic top view of an embodiment of a motor vehicle 1, wherein the motor vehicle 1 is configured for a teleoperation service, in other words, for remote operation by means of a teleoperation device 2. In the present embodiment, the motor vehicle 1 has at least one in-vehicle electronic computing device 3, which can, for example, receive environmental information 20 from the environment 4 of the motor vehicle 1 via cameras 5, 6, 7, 8. Furthermore, the electronic computing device 3 is configured to communicate with the teleoperation device 2. For this purpose, a suitable mobile network or the like can be used, for example. In particular, the electronic computing device 3 is also configured to compress at least parts of the environmental information 20 and then transmit the compressed information to the teleoperation device 2. In the present embodiment, the motor vehicle 1 has a first camera 5, a second camera 6, a third camera 7, and a fourth camera 8. The first camera 5 is configured as a front camera, the second camera 6 as a side camera (for example, the left side camera), the third camera 7 as a side camera (for example, the right side camera), and the fourth camera 8 as a rear camera. Thus, the surroundings 4 can be captured in a virtually 360° angle, and this environmental data can then be transmitted to the teleoperation device 2. Figure 2 shows a schematic block diagram according to an embodiment of the teleoperation device 2 with the motor vehicle 1. In particular, a teleoperator 9 is shown, who can perform the teleoperated driving service of the motor vehicle 1. For this purpose, it is specifically provided that the teleoperator 9 and the motor vehicle 1 are not within line of sight of each other, or at least not in the same position or the like. In particular, at least the teleoperator 9 cannot see the motor vehicle 1 from his position. On the side of the teleoperation device 2, the so-called cockpit 10 is provided, which in particular includes at least one display device 11 (Fig. 3). A corresponding landline 12 is provided between the cockpit 10 and the vehicle 1. A mobile communication link 13 can then be provided between the landline 12 and the vehicle 1. On the side of the vehicle 1, the electronic computing device 3 is provided, which is designed in particular for executing commands of the teleoperation service. Furthermore, the vehicle 1 has an environmental sensing system 14 at this location, which in particular includes at least the cameras 5, 6, 7, 8. Of course, other sensors such as a radar sensor, a lidar sensor, or an ultrasonic sensor are also possible. Furthermore, a corresponding functional unit 15 is provided on the motor vehicle side, which is designed in particular for the operation of a longitudinal control and / or a lateral control of the motor vehicle 1. Figure 3 again shows an embodiment of the display device 11 of the teleoperation device 2. In the present embodiment, a first side view 16, in particular a left side, a second side view 17, in particular a right side, and a main view 18 are shown. Furthermore, information can be displayed on an information panel 19. According to one embodiment of the method, it is provided in particular that environmental information 20 of the motor vehicle 1 is received by means of an electronic computing device 21 (Fig. 2) of the teleoperation device 2. The environmental information 20 is then divided into at least a first view part 22, which can be assigned to the main viewing area of ​​the motor vehicle 1. A second view part 23 is provided, which is assigned to at least one lateral viewing area in the motor vehicle 1. The main view 18 for the teleoperator 9 is then generated by the electronic computing device 21, depending on the first view part 22 with a first scaling factor. At least one side view 16, 17 for the teleoperator 9 is generated by the electronic computing device 21, depending on at least the second view part 23 with at least a second scaling factor different from the first. The main view 18 with the first scaling factor is then displayed, and at least one side view 16, 17 with the second scaling factor is displayed by the display device 11. Fig. 4 shows a schematic top view of an embodiment of the display device 11 with the teleoperator 9. It is shown in particular that the main view 18 is depicted larger than the side views 16, 17. Furthermore, it is shown in particular that both the side views 16, 17 and the main view 18 can be displayed on a single display element, in particular a single screen. Fig. 5 again shows a schematic top view of a display element 11. It is shown in particular that, in the present embodiment, the side view, the second side view 17, is displayed compressed compared to the main view 18. Furthermore, it is shown that the side views 16 and 17 are displayed separately from the main view 18. As already mentioned, only the second side view 17 is shown in the present embodiment. Furthermore, the term "separate display" can also be understood to mean that, for example, the side views 16 and 17 can be displayed on separate monitors. In particular, it may be provided that a ratio between the first scaling factor and the second scaling factor is generated statically. Furthermore, Fig. 5 shows in particular that a frontal view of the motor vehicle 1 or a rearward view is displayed as the main view 18. In particular, the appropriate display can be performed depending on the driving mode of the motor vehicle 1, for example, driving forward or in reverse. In particular, Figure 5 shows that a distortion-scaling module is provided for the teleoperation service. The aim is to reduce the space required for the corresponding camera images on the cockpit side for the teleoperator 9 without compromising important and safety-relevant information for the remote control process. The corresponding module supports various functions for resizing incoming input images into an output image without maintaining the aspect ratio. One way to solve the problem of inconsistent camera intrinsics and a limited field of view is to statically reduce the width of the side images—in other words, side areas 16, 17, for example—without changing the aspect ratio of the original image, i.e., the main view 18. For side images, displaying the entire pixel width may be less critical. Compared to a cropped video, a compressed video can have the advantage of retaining information from the entire original width, whereas cropping removes entire portions of an image. Figure 5 illustrates how the side-view front images can be reduced in size within a remote control cockpit user interface. This allows all important information to be retained across the entire width and condensed onto a smaller screen. Different camera resolutions, i.e., image widths, can thus be easily integrated into the user interface by simply reducing the image to the required width. Figure 5 specifically demonstrates how the aspect ratio of different camera images can be displayed within the user interface. Fig. 6 shows another schematic top view of an embodiment of the display element 11 with side views 16, 17 and the main view 18. Fig. 6 shows in particular that the ratio between the first scaling factor and the second scaling factor remains constant, especially as shown in Fig. 5, where both side views 16, 17 are shown in the present embodiment. Fig. 7 shows another schematic top view of the display element 11, in particular showing a steering angle 24. In the present embodiment, the steering angle 24 points, for example, to the left. It is shown in particular that a ratio between the first scaling factor and the second scaling factor can also be generated dynamically. In this case, the second view section can be displayed with different scalings in the side view 16, 17, and in particular in the second side view 17. A central view section 25 can be displayed with a lower scaling than a side view section 26. Furthermore, a current driving dynamics parameter of the motor vehicle 1 and / or the teleoperation device 2 can also be taken into account when generating the side view.In particular, as already mentioned, the steering angle 24 and / or a current speed can be taken into account as the vehicle dynamics parameter when generating the side view 16, 17. Furthermore, as shown in Fig. 7, it can be provided that, depending on the current steering angle 24, a central view element 25 is determined and displayed with a lower scaling factor than a side view element 26. Furthermore, it may be provided that, depending on the current steering angle 24, a part of the second view section 23 is suppressed. For example, in this case, the second view section 23 with respect to the left side view is suppressed and not displayed. Figure 7 thus shows in particular how, beyond static reduction, more complex scaling functions can be performed that use other vehicle information. The reduction factor can, for example, vary within an image. Thus, the information content of the center of an image can be reduced less than the information content at the sides of the same camera image. This non-static reduction can be used in combination with the current steering angle 24 of the teleoperator 9 or the vehicle 1. The steering angle 24 can define the center point of the reduction function, meaning that image information around the steering path is reduced less than information located further away from the steering or driving trajectory. Another possible function is the combination of resizing and cropping. In this case, the front image, specifically the main view 18, is not resizing or cropped. However, the side images, in other words, the side views 16 and 17, can be manipulated based on the steering angle 24. The function defines the cropping area of ​​the side images based on the steering angle 24. For example, if the vehicle 1 steers to the left and reverses, the left image may show more information from the blind spot. However, the width of the image is maintained, meaning that when steering to the left, the left image displays more lateral information and is therefore resizing more. If the steering angle is applied to the front of the vehicle again, less lateral information is displayed, resulting in less resizing. As an example, the teleoperator 9 remotely controls the vehicle 1 and searches for a parking space. The three front cameras are seamlessly connected for the teleoperator. While searching for a parking space, the teleoperator focuses primarily on the right-hand camera image. Once a parking space is found, the teleoperator steers to the right to drive towards it. With the proposed solution, the side images are reduced in size, which means that Teleoperator 9 is no longer so reliant on the right camera image, as it can see that the image is distorted. Teleoperator 9 primarily uses the right image for general orientation, carefully checking spatial distances by referring to the central screen. Now, it has started parking early enough because the differing appearance of the camera images has forced it to verify its impressions. Another example shows Teleoperator 9 using a cockpit with three monitors, each displaying a front camera image, specifically left, center, and right. While driving straight ahead, he focuses on the center monitor. He then turns right at an intersection. He remains focused on the center monitor because he can see everything at the intersection. Suddenly, a cyclist approaches from the rear right and has to brake hard because Teleoperator 9 didn't see the cyclist in time due to his focus on the front camera image. This situation could have been prevented with the proposed distorted scaling approach, where the right camera image is displayed smaller and therefore positioned closer to Teleoperator 9's focus. Now, when he turns right at an intersection, he sees the cyclist on the right camera earlier because it is closer to his focus.

Claims

Method for displaying an environment (4) of a remotely controlled motor vehicle (1) to a teleoperator (9) on a display device (11) of a teleoperation device (2), comprising the steps of: - Receiving environment information (20) of the motor vehicle (1) by means of an electronic computing device (21) of the teleoperation device (2); - Dividing the environment information (20) into at least a first view part (22), which is assigned to a main viewing area in the motor vehicle (1), and into at least a second view part (23), which is assigned to at least a side viewing area in the motor vehicle (1), by means of the electronic computing device (21); - Generating a main view (18) for the teleoperator (9) depending on the first view part (22) with a horizontal first scaling factor by means of the electronic computing device (21);-Generating at least one side view (16, 17) for the teleoperator (9) depending on at least the second view part (23) with at least one horizontal second scaling factor different from the horizontal first scaling factor by means of the electronic computing device (21); and- Displaying the main view (18) with the horizontal first scaling factor and displaying at least one side view (16, 17) with the horizontal second scaling factor by means of the display device (11).; Method according to claim 1, characterized in that the side view (16, 17) is displayed horizontally compressed compared to the main view (18). Method according to claim 1 or 2, characterized in that the side view (16, 17) is displayed separately from the main view. Method according to one of the preceding claims, characterized in that a ratio between the first scaling factor and the second scaling factor is generated statically. Method according to one of claims 1 to 3, characterized in that a ratio between the first scaling factor and the second scaling factor is dynamically generated. Method according to claim 5, characterized in that in the side view (16, 17) the second view part (23) is displayed with different scales. Method according to claim 6, characterized in that a central view part (25) is displayed in the second view part (23) with a lower scale than a side view part (26). Method according to one of claims 5 to 7, characterized in that, in addition, a current vehicle dynamics parameter of the motor vehicle (1) and / or the teleoperation device (2) is taken into account when generating the side view (16, 17). Method according to claim 8, characterized in that a current steering angle (24) and / or a current speed are taken into account as the vehicle dynamics parameter when generating the side view (16, 17). Method according to claim 9, characterized in that a central view part (25) is determined depending on the current steering angle (24) and is displayed with a lower scaling factor than a lateral view part (26). Method according to claim 9 or 10, characterized in that, in addition, depending on the current steering angle (24), a part of the second view part (23) is suppressed during the display. Method according to one of the preceding claims, characterized in that a frontal viewing area or a rearward viewing area is displayed as the main view (18). Computer program product with program code means which cause an electronic computing device (21) to perform a method according to one of claims 1 to 12 when the program code means are processed by the electronic computing device (21). Computer-readable storage medium comprising at least the computer program product according to claim 13. Teleoperation device (2) for displaying an environment (4) of a remotely controlled motor vehicle (1) to a teleoperator (9), comprising at least one electronic computing device (21) and a display device (11), wherein the teleoperation device (2) is configured for carrying out a method according to one of claims 1 to 12.

Images