Method for inspecting image material for an image analysis system, device and vehicle used in the method, and computer-readable storage medium

Abstract

The invention relates to a method for checking image material for checking an image analysis system, wherein the image analysis system is checked for correct object recognition with respect to time or position. The detected images are recorded in a memory. The novelty lies in determining which images or image parts (14) should be archived for a more accurate check if errors outside a permissible range are determined relative to a temporal or positional reference during object recognition, and archiving these determined images or image parts (14).

Description

Technical Field

[0001] The present invention relates to the technical field ranging from driver assistance systems to autonomously driven vehicles. In particular, the present invention relates to a method for detecting image material for use in an image analysis system. The present invention also relates to a device and a vehicle used in the method, as well as a computer-readable storage medium. Background Art

[0002] A significant amount of work is currently being invested in the technologies that will eventually enable automated driving. A first approach involves the introduction of various driver assistance systems to offload certain tasks from the driver. Examples of driver assistance systems include blind spot warning assistance, emergency braking assistance, parking assistance, cornering assistance, lane keeping assistance, and speed control assistance. Further development steps could involve combining multiple assistance systems. This does not completely relieve the driver of their duties, but rather allows them to assume control of the vehicle at all times. Furthermore, the driver also assumes monitoring functions.

[0003] Therefore, it is expected that in the near future, systems will be able to provide comprehensive information about objects (especially vehicles) in the vehicle's visible and obscured / invisible surroundings using newer technologies (vehicle-to-vehicle communication, database utilization, backend connectivity, cloud services, server use, vehicle sensors, etc.). Vehicle sensors, in particular, include the following components that enable environmental observation: RADAR devices, which correspond to radio detection and ranging, LIDAR devices, which correspond to optical detection and ranging (primarily for distance detection / warning), and cameras with corresponding image processing for object recognition. Ultrasonic and infrared sensors are also mentioned. Therefore, data about the environment can be used as a basis for system-side driving recommendations, warnings, and the like. For example, it is conceivable to display / warn other nearby vehicles about the direction they intend to turn (possibly along their own trajectory). Traffic sign recognition is also considered an important application for determining legal framework conditions.

[0004] Vehicle-to-vehicle communication also plays a key role in autonomous driving. Mobile wireless communication systems such as Long Term Evolution (LTE) or 5G have also been developed, and they also support vehicle-to-vehicle communication. Alternatively, systems based on WLAN technology, particularly WLANp-based systems, can be used for direct communication between vehicles. The term "autonomous driving" is sometimes used differently in the literature.

[0005] Therefore, the following explanation is provided to clarify this terminology. Automated driving (sometimes also referred to as autonomous driving, automated driving, or driverless driving) refers to the largely autonomous movement of vehicles, mobile robots, and unmanned transport systems. The term "autonomous driving" has different levels. At a certain level, automated driving is also involved when a driver is still in the vehicle, who sometimes only supervises the automated driving process. In Europe, the various transport authorities (in Germany, the Federal Highway Research Institute is involved) have collaborated and defined the following levels of autonomy.

[0006] Level 0: "Driver only": The driver himself drives, steers, accelerates, brakes, etc.

[0007] Level 1: Certain assistance systems provide assistance in vehicle conditions (among them, distance control systems—Automatic Cruise Control (ACC)).

[0008] Level 2: Partial automation, in which automatic parking, lane keeping functions, general longitudinal guidance, acceleration, braking, etc. are performed by assistance systems (among them, the traffic jam assistant).

[0009] Level 3: High Automation. The driver does not need to constantly monitor the system. The vehicle autonomously performs functions such as triggering turn signals, changing lanes, and maintaining lanes. The driver can override other tasks, but is required to take control if necessary within the system's warning time. This form of autonomy is technically feasible on highways. Legislators are working towards achieving Level 3 vehicles, and the legal framework conditions for this have already been established.

[0010] Level 4: Full Automation. The system continuously controls the vehicle. If the system is no longer able to handle the driving task, the driver can be asked to take over control.

[0011] Level 5: No driver required. No human intervention is required other than determining the destination and activating the system.

[0012] Automated driving functions from level 3 onwards share responsibility for controlling the vehicle with the driver. Similarly different levels of autonomy have been published by the German Association of the Automotive Industry (VDA) and are also available.

[0013] Given the current trend towards higher levels of autonomy, whereby a large number of vehicles remain driver-controlled as before, it can be assumed that corresponding additional information will be available for manually guided vehicles in the medium term, rather than only in the long term for highly automated systems.

[0014] The challenge facing driver-vehicle interaction is how to display this information so that it provides real added value for the human driver and the human driver can quickly and relatively intuitively find the provided information. The following technical solutions in this area are known in the prior art.

[0015] In the automotive sector, the prospect is that the windshield of a vehicle can display virtual elements to achieve advantages for the driver. This is done using so-called "augmented reality" (AR) or "mixed reality" (MR) technology. The corresponding German term is "erweiterten ” or “gemischten " is less common. Here, the real environment is enriched with virtual elements. This has several advantages: Since a large amount of relevant information is displayed when looking through the windshield, there is no need to look at a display below, which is separate from the windshield. The driver therefore does not have to take his eyes off the road. In addition, the precise positioning of the virtual elements in the real environment can reduce the cognitive effort on the driver's part, as there is no need to interpret the images on a separate display. This can also generate added value in the field of automated driving. In this regard, reference is made to the article "3D-FRC: Depiction of the future road course in the Head-Up Display" by CA Wiesner, M. Ruf D. Sirim, and G. Klinker at the 2017 IEEE International Symposium on Mixed and Augmented Reality, which explains these advantages in detail.

[0016] Due to the limited current technical solutions, it is expected that fully displayable windshields will not be available in vehicles in the medium term. Head-up displays (HUDs) are currently used in vehicles. These have the advantage of displaying HUD images close to the real environment. These displays are actually projection units that project images onto the windshield. However, to the driver, the image appears several meters in front of the vehicle, up to 15 meters, depending on the module's design.

[0017] The "image" here is structured as follows: not so much a virtual display as a "keyhole" into a virtual world. This virtual environment is theoretically built on the real world and contains virtual objects that assist and inform the driver while driving. The limited display surface of the HUD means only a partial view is possible. The HUD's display surface allows the user to see a portion of the virtual world. Because this virtual environment complements the real environment, it is also referred to as "mixed reality."

[0018] The great advantage of the "augmented reality" displays (AR displays) known to date is that the corresponding display content is shown directly in the environment or as part of the environment. Most of the relatively easy-to-think-of examples relate to the field of navigation. Conventional navigation displays (in conventional HUDs) usually show schematic diagrams (for example, an arrow extending at a right angle to the right as a symbol to indicate that you should turn right at the next opportunity), while AR displays offer significantly more efficient possibilities. Since the display can be shown as "part of the environment", extremely fast and intuitive explanations can be achieved for the user. The driver and higher-level autonomous driving functions must be able to expect that the object is detected correctly in the first place. If additional information is derived from this, it should be displayed at the correct position in the image.

[0019] Therefore, it is essential to verify the correct function of the image detection system at least during test runs. However, this can also be provided for normal operation. Public transportation is extremely complex, involving so many moving road users that obstructions can occur. Other issues include limited visibility due to various weather conditions (e.g., fog, rain, snow, etc.), backlighting, darkness, and other factors.

[0020] Patent document US2016 / 0170414 A1 discloses a vehicle-based system for traffic sign recognition. This system uses environmental sensors such as LIDAR sensors, RADAR sensors, and cameras. The vehicle's position is also detected via GPS. Recognized traffic signs, along with their locations, are reported externally and entered into a database.

[0021] Patent document US2017 / 0069206 A1 discloses a vehicle-based system for checking traffic sign recognition. This system also uses environmental sensors such as LIDAR sensors, RADAR sensors, and cameras. Recognized traffic signs are also reported externally along with their positions, where they are entered into a database. To detect misidentifications, various checking measures are proposed, including manual visual inspection by trained personnel, visual inspection through crowdsourcing, computer-assisted analysis, and statistical analysis.

[0022] Patent document US 2018 / 0260639 A1 discloses a vehicle-based system and method for traffic sign recognition. The recognized traffic signs are stored together with their positions in an image processing unit.

[0023] Patent document US 2008 / 0239078 A1 discloses displaying recognized objects in a video image of a camera in a vehicle for observing the surroundings with additional information derived from other sensors in the vehicle.

[0024] Patent document US 2018 / 0012082 A1 discloses a method for image analysis, in which a vehicle-mounted camera captures a sequence of images. The image analysis system includes an object recognition module. A bounding box is defined as a boundary for highlighting the recognized objects.

[0025] Patent document US2018 / 0201227 A1 discloses a vehicle environment monitoring system in which a vehicle environment is monitored by a camera. The vehicle also includes a display unit for displaying video images captured by the camera. An image analysis unit is provided that monitors the image data for the presence of moving external objects in the environment. Furthermore, a control device is programmed to determine a threat assessment value based on conditions near the vehicle and, if the threat assessment value exceeds a first threshold, upload the image data to an external server.

[0026] Known solutions have various drawbacks, which are known within the scope of the present invention. A problem with currently known image acquisition methods used in the field of driver assistance is that their correct functioning is not checked, or only insufficiently, during operation. This can lead to erroneous recognition or even misclassification of objects.

[0027] Therefore, there is a need to further improve the inspection of vehicle-based image detection systems, especially to enable post-inspection inspection methods in the event of erroneous identification. Summary of the Invention

[0028] The technical problem addressed by the present invention is to find such a means. The outlay for archiving image data, a prerequisite for subsequent inspections, should also be kept low. This problem is solved by a method for detecting image material for inspecting an image analysis system, a device and a vehicle for use in the method according to the invention, and a computer-readable storage medium storing a computer program.

[0029] In another patent application of the applicant, it is shown how, using a ring memory for images captured by surrounding sensors and taking into account the odometer data of the vehicle, only specific images can be used locally for data recording / transmission to the backend in order to reduce the required data volume.

[0030] According to one aspect of the present invention, in a method for detecting image material for checking an image analysis system, it is checked whether the image analysis system has correctly identified an object in terms of time or position. If so, there is no need to retain the image data for subsequent analysis. However, if errors outside the permissible range are detected in the analysis in terms of time or position, it is determined in a step which images or image parts should be archived for more accurate inspection, and these determined images or image parts are archived. The method has the advantage that image analysis systems that provide safety-related data can be checked during operation, wherein the storage consumption for the inspection is low. The method can be used particularly advantageously for testing image analysis systems that are still in the development stage. However, the method can also be used advantageously in mass production. Advantageously, subsequent improvements can be made to the installed image analysis system.

[0031] This solution is particularly suitable for static objects such as traffic signs. However, autonomous driving also requires the identification of moving objects in recorded images. However, the corresponding image analysis algorithms are prone to errors. To improve the algorithms, post-mortems are also necessary in the event of false detections. Therefore, further improvements are needed in the inspection of vehicle-based image detection systems.

[0032] To enable this even for dynamic objects, one aspect of the present invention calculates or obtains trajectories for moving objects in which deviations are detected. In the future, vehicles equipped with automated driving functions will independently calculate trajectories for their own movement. These vehicles will then be able to transmit these calculated trajectories to surrounding vehicles using direct vehicle communication. This allows the observer vehicle to estimate the position of the object before it was detected by the object detection device. Images and image parts that are useful for subsequent analysis are then determined by reverse calculation. The present invention also provides that the image parts determined in this way are extracted from the recorded images and archived only to the extent necessary. This is achieved using the vehicle's odometer data and position data as well as the trajectories of the relevant dynamic objects. These dynamic objects are typically other road users. In order to still be able to retrieve images or image parts determined later, a ring-shaped memory for image data is required. The size of this ring-shaped memory (i.e., the maximum storage duration) is optionally designed according to the function and, if necessary, the driving speed.

[0033] The method offers the advantage of enabling the testing of image analysis systems that provide safety-critical data during operation, with low memory requirements. The method is particularly advantageous for testing image analysis systems that are still in the development phase. However, it can also be used advantageously in series production. This is advantageous for subsequent improvements to installed image analysis systems or for determining whether miscalibration exists in cameras or sensors, necessitating maintenance of the vehicle.

[0034] In this case, it is advantageous to perform a back calculation using the trajectories in order to calculate a plurality of images or image parts in which moving objects could be visible, although object recognition could not detect moving objects in these images or image parts.

[0035] Furthermore, it is advantageous to use an object recognition algorithm for image analysis, and in this case, to check whether the object recognition was correct in terms of time or position, the distance from the recognized object at which the object was recognized is determined, wherein a standard recognition distance is determined, which specifies the distance from which the object recognition algorithm should provide object recognition. If the distance at which the object is actually recognized deviates from the standard recognition distance, images or image sections recorded from the standard recognition distance to the actual object recognition distance are archived for a more precise check.

[0036] In an expanded embodiment, provision can be made for the initial analysis of problematic image data already within the vehicle. This can be done, for example, by determining whether occlusion of the object by other objects during this period could explain a misidentification. This can occur in many ways in public road traffic, for example, if a bus or truck obscures a traffic sign. In such cases, the misidentification can be explained and, if necessary, no data or only a reduced data set (e.g., just a few seconds before the first successful object recognition) can be archived.

[0037] The present invention can be used particularly advantageously for testing image analysis systems in vehicles. The vehicles are now equipped with imaging environment detection sensors, such as cameras, LIDAR or RADAR sensors. Traffic signs, vehicles traveling ahead and other traffic participants, intersections, turning positions, potholes, etc. are detected by the imaging environment detection sensors. Such vehicles are also equipped with a position detection system. The determined vehicle position can be used to check whether there are deviations in image recognition. In the simplest case, when the vehicle passes the position of an object, the vehicle detects the position of the identified object itself. This allows the distance between the vehicle and the object to be calculated. In another embodiment, the position of the object can be obtained from a highly accurate map. In another variant, the position can be estimated based on the position of the vehicle.

[0038] Satellite navigation and / or odometry are suitable for determining the position of vehicles and objects. The general term for such satellite navigation systems is GNSS, which stands for Global Navigation Satellite System. Existing satellite navigation systems include the Global Positioning System (GPS), Galileo, GLONASS (Globalnajanawigazionnaja sputnikowaja sistema), or BeiDou.

[0039] In one embodiment, the images or image parts determined by the back calculation are sent to an external archiving location. This has the advantage that only problematic images need to be archived. Archiving at an external location has the advantage that the images do not need to be archived in the vehicle first. The analysis should be performed by experts at the external location, who can then also improve the image analysis system.

[0040] In another variant, the images or image parts recorded from the standard recognition distance to the object recognition distance or determined by back calculation are archived in a storage unit arranged in the vehicle. The corresponding memory must be provided in the vehicle for this purpose. The archived images can later be read by a specialist. This can be done when the vehicle arrives at the repair shop. Another advantageous variant is that when the vehicle returns to the owner's residence, the images temporarily stored in the vehicle are transmitted to an external location. Modern vehicles are equipped with WLAN. If the vehicle is now logged into the owner's personal WLAN network, the archived image data can be transmitted at a high data rate. Compared with direct transmission via a mobile wireless communication system (the vehicle is logged into the mobile wireless communication system while driving), this has the advantage of lower costs for the owner and less load on the mobile wireless network. The capacity of the mobile wireless network is fully utilized by various other applications.

[0041] Alternatively, it can be provided that additional images or image parts are archived in order to expand the inspection possibilities. Advantageously, images or image parts taken from the standard detection range to the object detection range are archived at high quality, while the additional images or image parts are archived at lower quality. This way, the storage and transmission costs for archiving the additional images or image parts can be kept low.

[0042] Furthermore, an advantageous variant provides that the size of the image portion to be archived and / or the recording period of the image / image portion to be archived is determined as a function of one or more of the following environmental parameters:

[0043] - Accuracy of determination of the position of vehicles and / or objects

[0044] - Time of day, especially the distinction between day and night

[0045] -Weather conditions

[0046] - Traffic conditions

[0047] - Road conditions

[0048] This is always beneficial when environmental conditions are not always the same. In practice, environmental conditions are constantly changing. If you always archive the same number of images or image sections, it is easy to miss important scenes as environmental conditions change and fail to find the cause of the problem.

[0049] Advantageously, the device used in the method according to the present invention comprises an image generation device, a computing device, and a storage device. The computing device is designed to perform object recognition on the images provided by the image generation device. The computing device also has the task of determining whether object recognition has been performed correctly in terms of time or position. To this end, the computing device can be designed so that, when a deviation outside the permissible range is detected when performing object recognition at a standard recognition distance, it determines which images or image portions should be archived for more accurate inspection and issues instructions for archiving these determined images or image portions. Furthermore, it is advantageous if the computing device is designed to calculate the trajectory of a moving object and to determine the images or image portions to be archived based on the calculated trajectory. By using the trajectory, the images or image portions to be archived can be better defined, thereby reducing the cost of archiving.

[0050] In a particularly advantageous variant, the device also includes a communication module, and upon receiving an instruction to archive the determined images or image parts, the communication module is designed to send these determined images or image parts to an external archiving location. The external archiving location can be an image analysis system or a computer center of a vehicle manufacturer.

[0051] Furthermore, a variant is proposed in which the device includes a storage device, and the storage device is designed to store the determined images or image parts after receiving an instruction to archive these images or image parts. In this variant, archiving occurs in the vehicle until the data is retrieved. As described above, the retrieval can occur at the repair shop, or the data can be transferred to an external archiving location upon return via the home WLAN network.

[0052] For the vehicle-side temporary storage of data, it is advantageous if the memory device is designed as a ring memory, wherein, in the event of memory overflow, the oldest previously stored images or image parts are overwritten by new images or image parts.

[0053] A video camera or a LIDAR sensor or a RADAR sensor can advantageously be used as the image generating device. For the interface for wireless communication, it is advantageous to use at least one of the WLAN communication systems of the IEEE 802.11 family of standards or an interface of an LTE or 5G mobile radio communication system according to the 3GPP standard.

[0054] It is advantageous for the vehicle used in the method to be equipped with a device corresponding to the proposed device.

[0055] The corresponding advantages described for the method according to the invention also apply to a computer program for execution in a computing device in order to carry out the steps for detecting image material according to the method according to the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Exemplary embodiments of the invention are illustrated in the drawings and are explained in more detail with reference to the drawings.

[0057] In the attached figure:

[0058] Figure 1 A typical cockpit of a vehicle is shown;

[0059] Figure 2 A schematic diagram showing different communication options provided in a vehicle for external communication is shown;

[0060] Figure 3 A block diagram showing the onboard electronics of a vehicle;

[0061] Figure 4 A first view of a first driving situation is shown in order to illustrate the problems encountered when checking the functionality of an image analysis system of a vehicle;

[0062] Figure 5 shows a second view of the first driving situation, wherein the second view shows the driving situation at an earlier point in time, and

[0063] Figure 6 A flow chart showing a procedure for detecting image material for checking the functionality of an image analysis system in a first driving situation;

[0064] Figure 7 A first view of a second driving situation is shown in order to illustrate the problems encountered when checking the functionality of an image analysis system of a vehicle;

[0065] Figure 8 A second view of a second driving situation is shown, wherein the second view shows the second driving situation at an earlier point in time, and

[0066] Figure 9 A flow chart showing a procedure for detecting image material for checking the functionality of an image analysis system in a second driving situation;

[0067] Figure 10 A diagram illustrating changes in image quality for archival image material according to environmental conditions;

[0068] Figure 11 A diagram showing changes in recording duration for archiving image material according to environmental conditions; DETAILED DESCRIPTION

[0069] This description explains the principles of the disclosure of the present invention. It is self-evident that those skilled in the art can design different arrangements that, although not explicitly described herein, embody the principles of the disclosure of the present invention and should also be protected within the scope of the present disclosure.

[0070] Figure 1 A typical cabin of a vehicle 10 is shown. A sedan is shown. However, other vehicles are also contemplated as vehicle 10. Examples of such vehicles include buses, commercial vehicles, particularly trucks, agricultural machinery, construction machinery, rail vehicles, and the like. The present invention is generally applicable to land vehicles, rail vehicles, ships, and aircraft.

[0071] The cockpit shows two display units of the infotainment system. These are a touch-sensitive display screen 30 mounted in the center console and an instrument cluster 110 mounted in the instrument panel. The center console is not in the driver's field of view while driving. Therefore, no additional information is displayed on display unit 30 during driving.

[0072] The touch-sensitive display 30 is used, in particular, to operate functions of the vehicle 10 . For example, it can be used to control the vehicle 10 's radio, navigation system, playback of stored music, and / or air conditioning system, other electronic devices, or other comfort functions or applications. In general, the term "infotainment system" is often used. An infotainment system in a motor vehicle, particularly a passenger car, encompasses the entirety of the vehicle's radio, navigation system, hands-free system, driver assistance system, and other functions within the central operating unit. The term "infotainment" is a portmanteau word, combining the words "information" and "entertainment." To operate the infotainment system, a touch-sensitive display 30 ("touchscreen") is primarily used. The display 30 is clearly visible and operable, particularly by the driver of the vehicle 10, but also by the passenger. Furthermore, mechanical operating elements, such as buttons, rotary knobs, or combinations thereof, such as a push-button rotary knob, may be arranged below the display 30 in an input unit 50. Steering wheel operation is also typically possible via components of the infotainment system. This unit is not shown separately but is shown as part of the input unit 50.

[0073] Figure 1 Also shown is a head-up display 20. In vehicle 10, head-up display 20 is positioned behind instrument cluster 110 in the instrument panel area, in the driver's field of view. The head-up display is an image projection unit. Projecting information onto the windshield causes additional information to appear in the driver's field of view. This additional information is displayed as if it were projected onto a projection surface 21 located 7-15 meters in front of vehicle 10. However, the real world remains visible through projection surface 21. The displayed additional information creates a somewhat virtual environment. This virtual environment is theoretically based on the real world and contains virtual objects that assist and inform the driver while driving. However, the projection is only onto a portion of the windshield, so the additional information cannot be arranged arbitrarily in the driver's field of view. This type of display is also known as "augmented reality."

[0074] Figure 2 The system configuration for vehicle communication using mobile wireless communication technology is shown. A vehicle 10 is equipped with an onboard communication module 160 with corresponding antenna units, enabling the vehicle to participate in various types of vehicle-to-vehicle communication, such as V2V and V2X. Figure 1 It is shown that vehicle 10 is able to communicate with a mobile radio base station 210 of a mobile radio provider.

[0075] Such a base station 210 may be an eNodeB base station of an LTE (Long Term Evolution) mobile radio provider. The base station 210 and corresponding equipment are part of a radio communication network having a plurality of mobile radio cells, wherein each cell is served by the base station 210 .

[0076] The base station 210 is positioned near the main road on which the vehicle 10 travels. In LTE terminology, a mobile terminal device corresponds to a user equipment UE, which enables a user to access network services, wherein the user is connected to the UTRAN or evolved UTRAN via a radio interface. Such a user equipment typically corresponds to a smartphone. Such a mobile terminal device is used by passengers in the vehicle 10. The vehicles 10 are additionally equipped with an on-board communication module 160. The on-board communication module 160 corresponds to an LTE communication module, via which the vehicle 10 can receive mobile data (downlink) and send this data in the uplink direction (uplink). The on-board communication module 160 can also be equipped with a WLANp module so that it can participate in the ad-hoc-V2X communication mode. The new fifth-generation mobile radio system also supports V2V and V2X communications. The corresponding radio interface is referred to there as the PC5 interface. For the LTE mobile wireless communication system, the LTE Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) consists of multiple eNodeBs, which provide the E-UTRA user level (PDCP / RLC / MAC / PHY) and control level (RRC). The eNodeBs are connected to each other via the so-called X2 interface. The eNodeBs are also connected to the Evolved Packet Core (EPC) 200 via the so-called S1 interface.

[0077] Figure 2 Based on this general architecture, a base station 210 is shown connected to the EPC 200 via an S1 interface, and the EPC 200 is connected to the Internet 300. A backend server 320 is also connected to the Internet 300, to which the vehicle 10 can send and receive messages. In the scenario considered here, the backend server 320 may be located in the vehicle manufacturer's computer center. Finally, a road infrastructure station 310 is also shown. This road infrastructure station can be represented, for example, by a roadside unit, which is often referred to in technical terms as a roadside unit (RSU) 310. To simplify the implementation, it is assumed that all components are assigned Internet addresses, typically in the IPv6 format, to enable appropriate routing of packets and information transmitted between the components. The various interfaces mentioned are standardized. Reference is made to the corresponding published LTE technical specifications for this purpose.

[0078] Figure 3 A block diagram schematically illustrates an onboard electronic system 200, which also includes the infotainment system of vehicle 10. A touch-sensitive display unit 30, a computing device 40, an input unit 50, and a memory 60 are used to operate the infotainment system. Display unit 30 includes not only a display surface for displaying changing graphical information but also an operating surface (touch-sensitive layer) arranged on the display surface for user input of commands. This can be designed as an LCD touchscreen display.

[0079] The display unit 30 is connected to the computing device 40 via a data line 70. The data line can be designed according to the LVDS standard, which is equivalent to low voltage differential signaling (Low Voltage Differential Signaling). The display unit 30 receives control data for controlling the display surface of the touch screen 30 from the computing device 40 via the data line 70. In addition, the instructions input through the touch screen 30 are also transmitted to the computing device 40 via the data line 70. Reference numeral 50 represents an input unit. The input unit includes the operating elements already mentioned, such as buttons, rotary adjustment parts, sliding adjustment parts or rotary press adjustment parts, with the help of which the operator can complete the input through menu guidance. "Input" is generally understood to mean calling a selected menu option, and is also understood to mean changing parameters, turning functions on and off, etc.

[0080] The memory device 60 is connected to the computing device 40 via a data line 80. A pictogram directory and / or a symbol directory with a plurality of pictograms and / or symbols is stored in the memory 60 for the possible display of additional information.

[0081] The other components of the infotainment system: camera 150, radio 140, navigation system 130, telephone 120, and instrument cluster 110 are connected to the devices for operating the infotainment system via data bus 100. A high-speed variant of the CAN bus conforming to ISO standard 11898-2 is considered as data bus 100. Alternatively, a bus system based on Ethernet technology, such as BroadR-Reach (In-Vehicle Ethernet), can also be used. Bus systems that transmit data via optical waveguides are also possible. Examples include the MOST (Multimedia Oriented System Transport) bus or the D2B bus (Digital National Bus). A vehicle measurement unit 170 is also connected to data bus 100. This vehicle measurement unit 170 is used to detect vehicle movement, particularly vehicle acceleration. It can be designed as a conventional IMU unit (for Inertial Measurement Unit). An IMU unit typically contains an acceleration sensor and a rotation rate sensor, such as a laser gyroscope or a magnetometer gyroscope. Vehicle measurement unit 170 can be considered a component of the odometer of vehicle 10. However, this also includes wheel speed sensors.

[0082] It is also mentioned here that the camera 150 can be designed as a conventional video camera. In this case, it records 25 full images per second, which is equivalent to 50 half images per second in interlaced scanning mode. Alternatively, special cameras can be used that record more images per second to improve the accuracy of object recognition in the event of rapid object movement or to record light in a spectrum other than visible light. Multiple cameras can be used for environmental observation. In addition, the RADAR or LIDAR systems 152 and 154 already mentioned can be used in addition or alternatively to implement or expand environmental observation. For wireless communication inward and outward, the vehicle 10 is equipped with a communication module 160, as already mentioned.

[0083] Camera 150 is primarily used for object recognition. Typical objects to be recognized include traffic signs, vehicles traveling ahead, around, or parked, other traffic participants, intersections, turning points, potholes, and the like. When a potentially significant object is recognized, information can be output via the infotainment system. Typically, for example, a symbol for the recognized traffic sign is displayed. If an object poses a hazard, a warning can also be displayed. This information is projected directly into the driver's field of view via HUD 20. An object that poses a hazard is, for example, if image analysis of the image provided by camera 150 indicates that a vehicle is approaching an intersection from the right, while also moving toward the intersection. Image analysis is performed in computing unit 40. Known algorithms can be used for object recognition. The result is that a hazard symbol is displayed at the location of the vehicle. This display is achieved so that the hazard symbol does not obscure the vehicle, as otherwise the driver would not be able to accurately identify where the hazard exists.

[0084] The object recognition algorithm is executed by the computing unit 40. The number of images that can be analyzed per second depends on the efficiency of this computing unit. The functions for autonomous driving are usually implemented in different stages: In the perception stage, the data from different environmental sensors are processed and aggregated on a high-performance platform. In addition, the vehicle must be positioned on a high-precision digital map based on its GNSS position. The data is used to generate a three-dimensional model of the environment and provide information about dynamic and static objects and open areas around the vehicle (object list). Under certain conditions, the accuracy of the GNSS position is not sufficient. Therefore, the odometry data in the vehicle is also used to improve the accuracy of the position determination.

[0085] Reference numeral 181 denotes an engine control unit. Reference numeral 182 corresponds to the ESP control unit, and reference numeral 183 denotes a transmission control unit. Other control units may also be present in a vehicle, such as (for vehicles with electrically adjustable dampers) an additional driving dynamics control unit, an airbag control unit, and so on. These control units, all belonging to the drivetrain category, are typically networked using the CAN bus system (Controller Area Network) 104, which is standardized in ISO standards, primarily ISO 11898-1. For various sensors 171 to 173 in a motor vehicle that are no longer connected solely to individual control units, provision is also made for these sensors to be connected to the bus system 104 and for their sensor data to be transmitted via the bus to the individual control units. Examples of sensors in a motor vehicle include wheel speed sensors, steering angle sensors, acceleration sensors, rotation rate sensors, tire pressure sensors, distance sensors, knock sensors, air quality sensors, and so on. In particular, wheel speed sensors and steering angle sensors are part of the vehicle odometer. Acceleration sensors and rotation rate sensors may also be directly connected to the vehicle measurement unit 170.

[0086] Figure 4 and Figure 5 The following illustrates the principle operating principle of the image analysis system according to the first aspect when used in a vehicle. A camera image captured by a front camera 150 of the vehicle 10 is shown. The example selected is traffic sign recognition. Figure 4 A camera image is shown at a closer distance to the traffic sign 15 . Figure 5 The camera image shows a distance from traffic sign 15. This is a traffic sign 15 that indicates an absolute prohibition on overtaking. The design of the traffic sign recognition system stipulates that when vehicle 10 approaches traffic sign 15 under standard conditions, it should be recognized at a standard recognition distance of 80 meters from the traffic sign. Figure 4 The camera image is shown when the distance is close to 20 m. Figure 5 The situation is shown where the vehicle 10 is still 40 meters away from the traffic sign 15. Image analysis shows that the traffic sign is only recognized at a distance of 40 meters. The checking function of the image analysis system determines that the traffic sign is recognized too late under the given conditions. The checking function then performs a test, which pre-analyzes the captured image. As shown, it can be seen that the truck 13 traveling ahead is visible in the camera image. Figure 5As shown, traffic sign 15 can be seen immediately to the right of truck 13. The checking function therefore concludes that traffic sign 15 would be obscured by the truck at a greater distance. Therefore, the images captured from a distance of 80 m to the distance of 40 m, where the actual traffic sign recognition takes place, are archived with lower quality.

[0087] Figure 4 and Figure 5 This also serves to explain other cases of misrecognition. At a distance of 40 m, a traffic sign 15 indicating a restrictive prohibition of overtaking for trucks is first recognized. Only at a distance of 20 m does it recognize that the traffic sign 15 indicates an absolute prohibition of overtaking. In this case, the check function responds as follows: the area 17 of the captured camera image marked for storage is archived in the image to be archived at high quality. The remaining portion of the image is recorded at lower quality. This applies to all images captured from a distance of 40 m to 20 m.

[0088] Figure 6 A flow chart of a program for implementing the checking function is shown. This program variant is intended for the second variant, in which a traffic sign is initially incorrectly identified and is only correctly identified when the vehicle approaches closer. Reference numeral 310 indicates the start of the program. In program step 312, an algorithm for traffic sign recognition is executed. This algorithm is an object recognition algorithm that is capable of performing pattern recognition based on patterns stored in a table. All valid traffic signs are known, and their patterns can be stored in a table. Typically, image recognition is improved by a convolution operation, in which a captured image is convolved with a known pattern. Such algorithms are known to those skilled in the art and are available for use. When a traffic sign is recognized in this way, the recognition distance relative to the traffic sign is also determined and stored.

[0089] In program step 314, a check is performed to determine whether traffic sign recognition meets the standard. In the case under consideration, traffic sign recognition was incorrect for two reasons. First, recognition was not performed at the standard recognition distance of 80 m from traffic sign 15. Furthermore, the initial recognition was a restricted prohibition of overtaking, which was revised to an absolute prohibition of overtaking as the vehicle approached closer. If compliance with the standard traffic sign recognition is determined in this program step, there is no need to archive the image for review, and the program ends in step 322. However, in the case under consideration, an incorrect traffic sign recognition was detected. The program then continues with program step 316. In program step 316, calculations are performed on multiple problematic images. Due to deviations from the standard recognition distance, the calculations can be performed as follows. The first recognition was performed at a distance of 40 m. Correct recognition was only achieved at a distance of 20 m. Therefore, the images from the standard recognition distance of 80 m to a distance of 20 m are considered.

[0090] In program step 316, a check is also performed to determine whether the misrecognition can be explained. Image analysis confirms that truck 13 is traveling ahead and may have obscured traffic sign 15 until the first recognition. This leads to the conclusion that images in the distance range from 80 m to 40 m are less relevant for subsequent verification. Therefore, these images are stored at a lower quality.

[0091] In program step 318, the calculation of the problematic image part is performed. In the case of traffic sign recognition, the image around the traffic sign 15 is Figure 4 and Figure 5 The marked image local part is selected as important. Therefore, the image local part is selected to have high image quality in the distance range of 40m to 20m.

[0092] Then, in program step 320, the problematic image and the problematic image part are transferred from the memory 60 to the communication module 160, and the communication module 160 sends these image data to the back-end server 320 via mobile radio. The image data is archived there. The archived images are then analyzed by experts in a computing center or by machines using artificial intelligence. The purpose of the inspection is to identify possible problems with the image analysis system. The results of the inspection can be used to improve the analysis algorithm. Ideally, the improved analysis algorithm can be transferred back to the vehicle 10 via OTA download (Over the Air) and installed there. However, it can also be concluded that the inspection determines a systematic error caused by an incorrect calibration of the camera 150. In this case, a message can be sent to the vehicle 10 to inform the driver that he should go to a repair shop. The program terminates in program step 322.

[0093] exist Figure 7Figure 2 shows a situation where a vehicle is moving onto a priority road. Shortly before entering the priority road, an oncoming vehicle 12 with right of way is detected by analyzing camera images and / or data from radar and lidar sensors 154, 152. Autonomously driven vehicle 10 must therefore brake strongly. The need for particularly strong braking creates an uncomfortable situation for the vehicle occupants. Precisely for autonomously driven vehicles, such braking procedures should be avoided to provide a sense of security for the vehicle occupants. The onboard electronics classify this event as undesirable, at least during the testing phase. The same situation can occur during normal operation, i.e., after the testing phase. Several possibilities exist for this purpose. One possibility is to perform this classification when a specific acceleration value is exceeded. Another possibility is to perform this classification when a negative expression from the occupant or driver is observed in the driver / passenger state recognition. Another possibility is to perform the classification when a specific minimum distance is exceeded before an object is detected (in this case, a vehicle approaching from the right).

[0094] In order to reduce the data volume of the image data to be stored, only the partial portion of the image in which the object may have been present should be stored / transmitted from the ring buffer.

[0095] Figure 8 Shows the Figure 7 Camera image of the same driving situation at an earlier point in time. Vehicle 12 approaching from the right is still quite far from the intersection position. Figure 8 The camera image even shows a time at which vehicle 12 has not yet been identified as an approaching vehicle by object recognition. This can have various reasons. The vehicle is still relatively far away, and the object recognition algorithm has not yet identified it based on the available image information. Vehicle 12 is also obscured by other objects (e.g., trees or bushes).

[0096] The reason for the late detection of the approaching vehicle 12 should be determined by subsequent analysis. To this end, the recorded camera images and, if necessary, images recorded by other imaging sensors are determined and archived for subsequent analysis. The amount of data to be archived should be kept as low as possible.

[0097] The following Figure 9 Explain how this is done. Figure 9A flow chart of a program for implementing the inspection function according to the second aspect of the present invention is shown. Reference numeral 340 indicates the start of the program. The program is always initiated when a dangerous situation is detected, in which a dynamic object is detected too late. In program step 342, the trajectory of the late-detected dynamic object is estimated. This is done as follows: At the time of object detection, the current velocity and acceleration of the detected object are determined from a plurality of successive video images using various sensor data from the observer vehicle 10. This, and optionally, the road geometry known from the map of the positioning system 130, is used to estimate the trajectory of the detected object. Combined with an estimation of the vehicle's own motion, this allows the determination of the image portion 14 in the image data recorded at a past time, within which the object (not yet detected at that time) should be located. This image portion 14 is also referred to as a bounding box. The bounding box can be considered larger at an earlier time than at a more recent time. The size of the bounding box is influenced on the one hand by the object parameters (eg size) at the time of recognition, but also by the uncertainty of the recognition, possible alternatives in the object trajectory, known sensor errors / inaccuracies and empirically determined factors.

[0098] In program step 344, the estimated trajectory is used for back-calculation to identify multiple problematic images, using these images to determine why the approaching vehicle 12 could not be detected. The distance between observer vehicle 10 and approaching vehicle 12 is also taken into account. If this distance exceeds a preset limit, no further previous images need to be archived. Each of the various environmental monitoring sensors has a known distance up to which the desired object can still be detected. The program then continues with program step 346, where calculations are performed on the multiple problematic images.

[0099] In program step 348, corresponding bounding boxes are calculated for the plurality of problematic images. This is also done while taking into account the estimated trajectory.

[0100] In another variant, a determination is also made in steps 344 and / or 346 as to whether the object in question may not have been identified due to being obscured by other potentially correctly identified objects (e.g., other vehicles, houses / walls known from navigation data, noise barriers, trees, bushes, forest). In this case, the image may not be transmitted. Since the estimated object trajectory indicates that the object in question was not already in the camera's field of view at an earlier point in time, the time period of the problematic image can be limited.

[0101] Then, in program step 348, the problematic image part 14 is transferred from the ring memory 60 of the storage device 60 to the communication module 160, and the communication module 160 sends these image data to the back-end server 320 via mobile radio. The image data is archived there. The archived images are then analyzed by experts or by machines using artificial intelligence in a computing center. The purpose of the inspection is to identify possible problems with the image analysis system. The results of the inspection can be used to improve the analysis algorithm. Ideally, the improved analysis algorithm can be transferred back to the vehicle 10 via OTA download (Over the Air) and installed there. However, it can also be concluded that the inspection determines a systematic error caused by the incorrect calibration of the camera 150. In this case, a message can be sent to the vehicle 10 to inform the driver that he should go to a repair shop. The program ends in program step 350.

[0102] The number and weight of images to be archived can be influenced by various factors. In addition to the already mentioned factors, the following are also mentioned:

[0103] Weather conditions

[0104] In bad weather conditions, visibility is severely restricted. This can reach the point where object detection is no longer possible. However, other environmental detection sensors, such as RADAR and LIDAR sensors, can provide better results. In any case, it can be provided that the number of images or image sections to be archived is limited in bad weather conditions, since the corresponding objects cannot be detected from the standard detection distance.

[0105] Position accuracy

[0106] The accuracy of positioning based on GNSS and odometer signals can also depend on the weather. However, this positioning accuracy can also depend on other influencing factors. An example is the environment in which the vehicle is moving. In cities, the reception of satellite signals can be restricted by the large number of buildings. This can also occur when driving off-road. Satellite signals can be weakened in forests. In mountainous areas, the reception can also be poor due to geological reasons. Therefore, in such cases, it is recommended to record more images or image sections. This makes it more likely that relevant route segments will be recorded despite inaccurate positioning.

[0107] Road conditions

[0108] Road conditions can also have a similar influence. On rocky roads, there are strong vibrations, which can blur the recorded image. In the event of slippage due to ice, snow, or rain, the drive slip control system determines an estimated value for the coefficient of friction. If the slippage is sufficiently severe, the odometer data is no longer reliable and this influence must be taken into account, just like any influence caused by position inaccuracies.

[0109] time

[0110] Image quality obviously varies significantly depending on the time of day. A distinction should be made between day and night. During the night, the data from the RADAR or LIDAR sensor is recorded first, rather than the camera image data.

[0111] Traffic conditions

[0112] This allows for differentiation between vehicles traveling in city traffic, on the highway, or on state roads. Accurately detecting traffic signs is particularly important in city traffic. This can increase the number of images to be recorded. In contrast, in highway traffic jams, the number of images to be recorded can be reduced.

[0113] The images or image parts 14 to be archived are temporarily stored in the vehicle 10. A memory 60 can be used for this purpose. In a practical implementation, a ring memory is provided in the memory 60. This ring memory is managed so that newly captured images are written sequentially into a free storage area of the ring memory. When the end of the free storage area is reached, the already written portion of the ring memory is overwritten. By managing the allocated storage area as a ring memory, the oldest portion of the memory is always overwritten. The images or image parts 14 can be stored uncompressed or compressed. However, it is advantageous to use a lossless compression method so that no important image content is lost. The FFmpeg codec is cited as an example of a lossless compression method. The image data is stored in a corresponding file format. Various data storage formats are considered. Examples include the ADTF and TIFF formats developed for the automotive industry. Other storage formats for image and sound data include MPEG, Ogg, Audio Video Interleave, DIVX, Quicktime, Matroska, and others. If the images are transmitted to the backend server 320 during driving, a ring memory designed for a recording period of 20 seconds is sufficient. A separate memory can be provided in the vehicle in which the images are archived. For example, a USB hard drive can be provided that can be removed and connected to a computer for analyzing the archived images.

[0114] Figure 10The effect of being able to reduce the amount of data to be archived by reducing the image quality is also shown. From top to bottom, the amount of image data is shown for different image qualities, namely full HD with a resolution of 1920×1080 pixels, standard definition with a resolution of 640×480 pixels, and VCD with a resolution of 352×288 pixels.

[0115] Figure 11 The effect of the number of images or image portions 17 on variations in position accuracy, weather conditions, etc. is shown. If the recording duration is increased to a distance of 160 m due to position inaccuracies, then twice as much data is generated for archiving. If the distance is reduced to 40 m, only half the data needs to be stored.

[0116] The examples and conditional expressions mentioned herein should not be construed as limiting the scope of the present invention to the specifically cited examples. Those skilled in the art should note that the block diagrams described herein illustrate schematic diagrams of exemplary circuit arrangements. Similarly, it should be noted that the flowcharts, state transition diagrams, pseudocode, and similar variations are all used to illustrate processes that are primarily stored on a computer-readable medium and can thus be implemented by a computer or processor. In particular, the subject described herein may be a person.

[0117] It should be understood that the proposed method and the associated apparatus can be implemented in various ways using hardware, software, firmware, a dedicated processor, or a combination thereof. A dedicated processor may include an application-specific integrated circuit (ASIC), a reduced instruction set computer (RISC), and / or a field programmable gate array (FPGA). The proposed method and apparatus are preferably implemented as a combination of hardware and software. The software is preferably installed as an application program on a program storage device. Typically, the proposed method and apparatus are based on a computer platform having hardware, such as one or more central processing units (CPUs), a random access memory (RAM), and one or more input / output (I / O) interfaces. An operating system is also typically installed on the computer platform. The various processes and functions described herein may be part of an application program or may be implemented by an operating system.

[0118] The present disclosure is not limited to the above-mentioned embodiments. For those skilled in the art, there is room for various adaptations and modifications based on their professional knowledge and considerations within the scope of the present disclosure.

[0119] The present invention furthermore elaborates on examples of applications in vehicles in the exemplary embodiments. Application possibilities in airplanes or helicopters, such as landing strategies or search and rescue missions, can also be cited.

[0120] The invention can also be used for remotely controlled devices such as drones and robots, where image analysis is important. Other possible applications involve smartphones, tablets, personal assistants or data glasses.

[0121] Reference Signs List

[0122] 10 Observer Vehicles

[0123] 12 Approaching means of transportation

[0124] 13 Vehicles moving ahead

[0125] 14 Important first image parts

[0126] 15 Traffic Signs

[0127] 17 Important Second Image Part

[0128] 20 Heads-up display

[0129] 30 touch-sensitive display unit

[0130] 40 computing units

[0131] 50 input units

[0132] 60 storage units

[0133] 70 Data lines for display units

[0134] 80 data lines for storage cells

[0135] 90 Data lines for input units

[0136] 100 data bus

[0137] 110 Instrument Cluster

[0138] 120 phone number

[0139] 130 Navigation Equipment

[0140] 140 Radio

[0141] 142 Entrance

[0142] 144 On-board diagnostic interface

[0143] 150 cameras

[0144] 160 Communication Module

[0145] 170 Vehicle Measurement Unit

[0146] 181 Sensor 1

[0147] 172 Sensor 2

[0148] 173 Sensor 3

[0149] 180 Control unit for automated driving functions

[0150] 181 Engine Control Equipment

[0151] 182 ESP control equipment

[0152] 183 Transmission Controller

[0153] 200 Evolved Packet Core

[0154] 210 base stations

[0155] 300 Internet

[0156] 310 Roadside Unit

[0157] 320 Backend Server

[0158] 330 -

[0159] 336 Different program steps of the first computer program

[0160] 340 -

[0161] 350 Different program steps of the second computer program

[0162] Uu is the air interface used for communication between UE and eNodeB

[0163] PC5 Air interface for direct vehicle communications

Claims

1. A method for detecting image material for inspecting an image analysis system, wherein: A check is made as to whether an object has been correctly identified by an image analysis system relative to a reference in time or position, wherein images are recorded in a memory, characterized in that when a deviation outside a permissible range relative to the reference is determined, it is determined which images or image parts (14, 17) should be archived for a more accurate check, and these determined images or image parts (14, 17) are archived, wherein an object recognition algorithm is used for the image analysis, and wherein, in order to check whether the object recognition has been correctly performed with respect to time or position, it is checked at what distance from the identified object (12) the object recognition has been performed, wherein a standard recognition distance is determined, which indicates the distance from which the object recognition algorithm should provide object recognition.

2. The method according to claim 1, wherein The step of determining an image or an image part (14) for archiving comprises calculating a trajectory of the moving object (12) and determining an image or an image part (14) to be archived based on the calculated trajectory.

3. The method according to claim 2, wherein: Backward calculations are performed using the trajectories in order to calculate a plurality of images or image parts (14) in which the moving object (12) is possibly visible, even though the moving object (12) cannot be identified in the images or image parts using object recognition.

4. The method according to any one of the preceding claims 1 to 3, wherein: The image analysis system is an image analysis system for a vehicle (10) and, in order to detect deviations in image recognition, position data of the vehicle (10) and position data of detected objects (12, 15) are analyzed in order to determine the distance between the vehicle (10) and the objects (12, 15).

5. The method according to any one of the preceding claims 1 to 3, wherein: Images recorded in the memory (60), images or image parts (14) determined using the trajectory, or images or image parts (14, 17) recorded from a standard detection distance to an object detection distance are sent to an external archiving location (320).

6. The method according to any one of the preceding claims 1 to 3, wherein: The memory is organized as a ring memory and the images recorded in the ring memory, the images or image parts (14) determined by means of trajectories or the images or image parts (17) recorded from a standard detection distance to an object detection distance are archived in a memory unit (60) arranged in the vehicle (10).

7. The method according to any one of the preceding claims 1 to 3, wherein: Other images or image parts (14) are archived, wherein the images or image parts (14) determined by means of the trajectory or the images or image parts (14, 17) taken from the standard detection distance to the object detection distance are archived at high quality and the other images or further image parts are archived at lower quality.

8. A device for use in the method according to one of the preceding claims, comprising an image generating device, a computing device (40) and a storage device (60), wherein: The computing device (40) is designed to perform object recognition in an image provided by an image generating device and to determine whether the object recognition was performed correctly in terms of time or position, and is characterized in that the computing device is designed to determine which images or image parts (14) should be archived for a more precise check when a deviation from a permissible range relative to a reference is detected during object recognition, and to issue an instruction for archiving these determined images or image parts (14), wherein the device is designed to execute an object recognition algorithm for image analysis, and wherein the device is designed to check at what distance from the recognized object (12) the object recognition was performed in order to check whether the object recognition was performed correctly in terms of time or position, wherein a standard recognition distance is determined, which indicates the distance from which the object recognition algorithm should provide object recognition.

9. The device according to claim 8, wherein The calculation device (40) is designed to calculate the trajectory of the moving object (12) and to determine the image or image portion (14) to be archived based on the calculated trajectory.

10. The device according to claim 8 or 9, wherein The device has a communication module (160) and the communication module (160) is designed to send the determined images or image parts (17) to an external archiving location (320) after receiving the instruction for archiving these determined images or image parts (14).

11. The device according to claim 8 or 9, wherein The memory device (60) is designed as a ring memory, wherein, in the event of memory overflow, the oldest previously stored image or image portion (14, 17) is overwritten by a new image or image portion (14, 17).

12. The device according to claim 10, wherein The image generating device is a video camera (150) or a LIDAR sensor or a RADAR sensor, and wherein the communication module (160) is used for wireless communication according to at least one of a WLAN communication system corresponding to the IEEE 802.11 family of standards or an LTE or 5G mobile wireless communication system corresponding to the 3GPP standard.

13. A means of transport, characterized in that: The vehicle (10) is equipped with a device according to one of claims 8 to 12.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computing unit (40), the steps of the method according to one of claims 1 to 8 are performed.

Images