METHOD AND ASSISTANCE SYSTEM FOR USING A DIGITAL CARD IN A VEHICLE

Description

[0001] The present invention relates to a method for using a digital map in a vehicle, particularly when driving into a parking space. The present invention further relates to a corresponding assistance system.

[0002] Nowadays, a wide variety of driver assistance systems are used in motor vehicles. These systems include sensors for monitoring the vehicle's surroundings, such as one or more video cameras or sensors based on ultrasound, radar, or laser technology.

[0003] Various driver assistance systems are available to support the parking process. A common system is parking assist, also known as Park Distance Control (PDC), which monitors the area in front of and / or behind the vehicle and alerts the driver to potential obstacles or the remaining distance to them with visual or audible signals, thus making parking easier. Similarly, so-called parking steering assistants are used, which further relieve the driver during parking. These systems use side-mounted parking sensors, positioned perpendicular to the direction of travel, to measure potential parking spaces as the vehicle drives past. If a parking space is large enough, the vehicle can then automatically perform the necessary steering maneuvers; the driver only needs to operate the accelerator and brake pedals.Remote-controlled parking systems go a step further, allowing the driver to remain outside the vehicle during the parking process. This can be advantageous in tight parking spaces or narrow garages. In this case, the driver only monitors the parking process and can control or cancel it via a remote control, for example, integrated into the vehicle key.

[0004] Autonomous parking systems are still under development. For example, in so-called trained parking, after a training drive in which the driver manually drove the vehicle to the parking space, and characteristic features of the vehicle's surroundings were recorded by a camera and possibly other sensors, the vehicle can automatically navigate to the parking space.

[0005] So-called parking garage pilot systems are designed to operate without driver training. These systems allow the vehicle to autonomously enter a parking space, such as a parking garage, and find a parking space without a driver. The driver can leave the vehicle at a designated handover point before entering the garage. From there, the vehicle drives autonomously to its assigned parking position and later returns to the handover point to be retrieved by the driver. To determine the correct trajectory, the piloted vehicle requires sufficiently accurate information about the spatial characteristics of the parking garage, such as the driving lanes and the arrangement of individual parking spaces on one or more levels. Digital map data of the parking garage is very helpful for this purpose.

[0006] Even though most modern vehicles are equipped with navigation systems, these systems typically do not contain map information about the layout of parking garages. While there are considerations to provide digital maps of parking areas via a digital download through an IT interface between the vehicle and the parking garage, this requires suitable IT interfaces on the part of both the vehicle and the various parking garage operators.

[0007] Against this background, DE 10 2014 224 073 A1 describes how a section of a digital map of a parking lot, which the vehicle is to autonomously traverse on its journey to the destination position, is transmitted to a vehicle via a communication network. For this purpose, a server comprises a database in which a digital map of the parking lot is stored, a processor that determines the destination position in the parking lot and the section of the digital map, and a communication interface that sends the section of the digital map and the destination position to the vehicle via a communication network.

[0008] Furthermore, DE 10 2015 202 482 A1 discloses a vehicle that autonomously uses environmental sensors to search for a suitable free parking position in a parking lot and parks itself in that position. Signal transmitters at potential parking positions are detected, and the signal emitted by each transmitter is evaluated to determine whether it indicates a free or occupied parking position. This can be done without the use of a digital map or with the aid of a digital map that can be provided to the vehicle in advance by a map service.

[0009] Finally, US Patent 2016 / 265919 A1 discloses a method for determining the absolute position of a motor vehicle. Using a camera to capture the surroundings, markers, each assigned an absolute position retrievable from a map dataset, are detected and identified by an image processing algorithm. The vehicle's absolute position is then determined based on its relative position to a marker and the marker's absolute position. This allows vehicles to be directed to an available parking space in a parking garage. For this purpose, a sign at the entrance to the parking garage is equipped with a two-dimensional barcode encoded with the map dataset, enabling the map dataset to be transmitted wirelessly by the vehicle's camera.The map data set can also contain a complete map including walls and other obstructions to the view, so that it can be determined whether there is a line of sight from the last determined position to the marker or whether there is an optical obstacle in the way. US 2016 / 265919 A1 discloses, in particular, a method for using a digital map in a vehicle, wherein: - image data of the vehicle's surroundings are captured with one or more cameras of the vehicle; - the captured image data are processed to recognize a visually represented overview map, which contains an abstract representation of spatial features near the overview map; - the image data of a recognized overview map are transformed into a navigable digital map; and - using the digital map, guidance information for a driving route in the spatial area represented in the overview map is generated.

[0010] It is an object of the invention to provide a method for using a digital map in a vehicle, as well as a corresponding assistance system, which can be used in particular for automatically driving into a parking space.

[0011] This problem is solved by a method having the features of claim 1 and by an assistance system according to claim 9. Preferred embodiments of the invention are the subject of the dependent claims.

[0012] In the inventive method for using a digital map in a vehicle, image data of the vehicle's surroundings is captured using one or more of the vehicle's cameras. The captured image data is processed to recognize a visually represented overview map, which contains an abstract representation of spatial conditions in the vicinity of the overview map. The image data of a recognized overview map is transformed into a navigable digital map. Using the digital map, route guidance information for a driving route within the spatial area depicted in the overview map is generated.

[0013] This guidance information can be used, for example, to guide a vehicle autonomously within the area shown on the overview map, even if no digital map data was previously stored in the vehicle and transmission to the vehicle via a digital communication interface is neither possible nor intended. This also eliminates the need for otherwise required data management and updates of the digital map data. Furthermore, this method minimizes the amount of data stored in the vehicle, since instead of permanently maintaining digital map data for a multitude of arbitrary locations, some of which the vehicle may never visit, only the information currently needed is temporarily recorded and used.

[0014] Furthermore, in addition to the digital map generated from the captured image data, the vehicle also carries a second digital map for the same spatial area, and both digital maps are evaluated together. Areas with discrepancies between the two digital maps are identified. These areas are either avoided or approached and inspected using the vehicle's sensors. This redundant approach also offers the advantage of greater robustness, as one of the two map sources can fail without affecting the overall system. Moreover, this method allows for the early detection and, if necessary, resolution of discrepancies between the two digital maps and thus potentially problematic or dangerous situations.

[0015] Preferably, the overview map is identified in the captured image data by evaluating a marker contained within the overview map. This simplifies reliable identification of the overview map, for example, when different formats or designs are used for overview maps, or when part of an overview map is obscured and therefore cannot be captured by the camera.

[0016] The overview map also advantageously contains information about the location of the overview map and / or the vehicle at the time the overview map was captured. This information is recorded, analyzed, and used in the transformation of the image data into the navigable digital map. The vehicle can use its own map location to initialize its localization system, thus creating an entry point into the digitized map.

[0017] Furthermore, the overview map can advantageously contain information about the map scale, which is recorded and evaluated and used in the transformation of the image data into the navigable digital map. This is particularly beneficial when overview maps with different scales are used, for example, due to varying sizes of the spatially covered areas.

[0018] According to one embodiment of the invention, the overview map provides an overview of a parking area with a large number of parking spaces, in particular a parking garage with parking spaces on one or more parking levels. This allows, for example, the generation of guidance information even in parking garages where no digital map data is available in the vehicle. The guidance information can then, for example, specify a route within a parking garage, enabling a parking assistant to execute a fully automated parking process from the moment of entry. Similarly, based on the guidance information, a driving recommendation to an available parking space can be issued to the driver of the vehicle.

[0019] Preferably, the overview map is captured as the vehicle approaches or enters the parking area. This ensures that the guidance information is available in good time before entering the parking area.

[0020] Furthermore, the overview map can advantageously consist of several sub-maps, whereby A first partial map is recorded when the vehicle enters the parking area; the vehicle is guided to a sub-area of ​​the parking area using the first partial map; a second partial map is recorded when the vehicle enters the sub-area of ​​the parking area; and the vehicle is guided to a parking space in the sub-area of ​​the parking area or to another sub-area of ​​the parking area using the second partial map.

[0021] Especially in larger parking areas such as several interconnected parking lots or parking garages, or parking garages with multiple floors, the necessary information can be distributed across several maps.

[0022] According to a further embodiment of the invention, the additional digital map present in the vehicle has been received via a digital communication device or is already contained in the original digital map data of a navigation application.

[0023] The invention also relates to an assistance system for carrying out the method according to the invention.

[0024] Further features of the present invention will become apparent from the following description and the claims in conjunction with the figures. Fig. 1 shows a flowchart of a method according to the invention; Fig. 2 schematically shows a top view of a vehicle approaching a parking garage, which uses a vehicle camera to capture an overview map of the parking garage; and Fig. 3 schematically shows an example of an overview map for a parking garage.

[0025] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features can also be combined or modified without departing from the scope of protection of the invention as defined in the claims.

[0026] Figure 1Figure 1 shows a flowchart of a method according to the invention for using a digital map in a vehicle, as can be applied, for example, when driving into a parking space. Although the method is described below using the example of a parking garage, it is not limited to driving into parking spaces, but can also be used in other areas where digital map data is not available in the vehicle or is only incomplete.

[0027] In the first process step 1, the vehicle's surroundings are recorded using one or more of the vehicle's cameras. This can be done, for example, with a forward-facing video camera mounted in the area of ​​the rearview mirror.

[0028] In a second step of the process, the image data captured by the camera is fed into an image processing system, which interprets the image data using a suitable algorithm. This can involve, for example, automatic segmentation after digitization and, if necessary, further image preprocessing. If an image area is detected that corresponds to a visually represented overview map based on its shape, this area is further evaluated through additional analysis of the image data, for example, as the vehicle approaches a parking garage. This analysis continues until the area is either clearly recognized as a visually represented overview map or is rejected as a false positive.

[0029] For this purpose, the internal structure of the image area can be analyzed. For example, it can be checked whether the image area contains an abstract representation of spatial features typical of a parking garage and / or a predefined marker indicating that the display is a map of the parking garage. In this way, vehicles entering parking areas with an "analog" map, such as one displayed at the entrance, can capture and process this map upon entering the parking area. The capture and processing of the image data can be continuous or activated only when the vehicle's navigation system detects its approach to a parking garage.

[0030] In process step 3, the image data from the recognized overview map is transformed into a navigable digital map. This map can include both the spatial characteristics of the parking garage and the vehicle's own location. The spatial characteristics of the parking garage can, for example, reflect the entrance and exit, the ramps between the individual levels, the driving lanes and the arrangement of the individual parking bays and spaces, or hazards. It can also include information on special height and vehicle restrictions or designated parking spaces, such as disabled parking spaces, women's parking spaces, parent-child parking spaces, as well as exits, stairs, and elevators for vehicle occupants. The vehicle can use its own location on the map to initialize its localization system and thus create an entry point into the digitized map.

[0031] Furthermore, the overview map can contain information about the scale that the vehicle processes to transform the image map into a navigable map. The scale can be displayed in plain text, e.g., as "1:500," and then converted using an OCR plain text reading algorithm and used during the transformation, or it can be included in the overview map in encoded form, e.g., as predefined QR codes. However, specifying a scale is unnecessary if binding agreements between parking garage operators stipulate that the same scale is always used for the overview maps and this information is available to the vehicle.

[0032] The overview map can be designed, for example, as a metal sign with the aforementioned information permanently printed on it. Alternatively, an electronic display, such as an LCD screen, can be used. The advantage of an electronic display is that it can be easily read even in darkness without additional lighting, and the displayed information can be easily adjusted, for example, in the event of a temporary closure of sections of the parking garage.

[0033] The overview map can be a single map of the entire parking garage, providing information on the different levels, for example, at the entrance. Alternatively, the overview map can be divided into several sub-maps, each providing information on one of the parking levels. These sub-maps can be displayed together at the entrance and / or at the entrances to each level as a single floor map. In addition to a general overview map showing only the different parking aisles without details of individual parking spaces, separate parking aisle maps can also be provided for each aisle, showing the individual parking spaces within that aisle, possibly including their names.

[0034] In process step 4, a route guidance information for a journey within the spatial area shown in the overview map is generated using the navigateable digital map. The vehicle can then use this information to plan and execute its autonomous journey through the parking garage. This can also be done sequentially, with the vehicle first using a rough overview map, as described above, to navigate to a parking aisle and then evaluating the map of that aisle to find a designated parking space.

[0035] It may also be possible to signal the correct recognition of an overview map to the driver, for example by means of a corresponding visual display in the instrument cluster or on the screen of the navigation device / infotainment system, thus conveying to the driver that the information required for autonomous driving in the parking garage is available and that the route planning can be carried out.

[0036] Similarly, the generated guidance information can also be output to a driver searching for a parking space in manual driving mode. For example, the guidance information can be displayed on a display device connected to an assistance system, particularly as part of a navigation display, which can be shown on a screen in the dashboard or center console area, an instrument cluster, or a head-up display. Alternatively, or in addition, the guidance information can also be provided as a voice output.

[0037] If, in addition to the digital map generated from the captured image data, the vehicle already has access to another digital map for the same area via an IT interface, these two digital maps can be evaluated together. Discrepancies may arise between the two digital maps. The vehicle can then, where possible, avoid areas with discrepancies in the digital maps, or drive through the affected areas to inspect and measure them using its sensors, thus helping to resolve the discrepancy. The vehicle can also share its measurement results with other vehicles or an existing IT infrastructure, e.g., via Car2X, to resolve discrepancies. In this way, the vehicle can contribute to identifying errors in data transmission or detecting soiling or vandalism on the displayed maps.

[0038] Figure 2 Figure 1 schematically shows a top view of a vehicle 5 approaching a parking garage 6, in which some of the parking spaces and some of the other vehicles occupying these spaces are schematically indicated. The vehicle is equipped with a camera 7 that captures the area in front of the vehicle. The captured image or video data is transmitted, for example via a digital data bus in the vehicle, to an evaluation and control unit 8, which analyzes the image data to determine whether it represents an overview map 9 of the parking garage. Using suitable image processing methods, an image object can then be recognized as such an overview map, for example by evaluating markings shown on the overview map, and then used, for example, to generate guidance information.

[0039] In addition to the vehicle camera 7 and the evaluation and control unit 8, the vehicle may contain further components not shown. For example, a device for controlling autonomous or semi-autonomous driving operation can use the overview map to determine a trajectory for the vehicle within the parking garage and control the vehicle accordingly. This trajectory determination, particularly in the entrance area, can be carried out in conjunction with a navigation system and taking into account information provided by vehicle sensors. Furthermore, the vehicle 5 may have a memory for storing the digital map generated from the overview map and a data transmission unit with an IT interface.

[0040] The overview map 9 can also be located in positions other than those shown in the figure, such as above the entrance to the parking garage. It can also be oriented differently and captured by other cameras on the vehicle. For example, if the vehicle has to stop when entering the parking garage to purchase a parking ticket from a ticket machine located next to the vehicle, the overview map can be placed near the ticket machine and, if necessary, captured by a side-facing camera. This has the advantage that the distance between the camera and the overview map can be kept short, and no moving images need to be analyzed since the vehicle has come to a standstill. This allows even a particularly detailed overview map to be captured sharply by the camera.

[0041] Figure 3Figure 9 schematically shows an example of such an overview map for a parking garage. The overview map provides, in particular, an overview 10 of the spatial characteristics of the parking garage, as well as information 11, 12 about the location of the overview map and the vehicle at the time the overview map was scanned. However, information for only one of the two locations can also be provided. Furthermore, the location of the overview map or the vehicle location can be indicated not only by a correspondingly positioned symbol, but also, for example, by coordinates.

[0042] Furthermore, to facilitate the identification of the overview map in the captured image data, a marker 13 is displayed to indicate the overview map, and information 14 about the map scale is included. Finally, additional information, such as a figure 15 about the parking price or, if an electronic display is used, information about the occupancy level of the parking garage, can also be provided.

[0043] Instead of being displayed in plain text, the information shown on the overview map can also be presented in a coded form. While this has the disadvantage that the information cannot be readily understood by the vehicle occupants, the encoding allows for a significantly larger amount of data to be transmitted to the vehicle via camera capture using suitable compression methods, and then used there after appropriate decoding. Reference symbol list

[0044] 1. First process step 2. Second process step 3. Third process step 4. Fourth process step 5. Vehicle 6. Parking garage 7. Camera 8. Evaluation and control unit 9. Overview map 10. Overview of the parking garage's spatial features 11. Vehicle symbol 12. Overview map symbol 13. Parking garage marker 14. Map scale information 15. Parking price information

Claims

1. Method for using a digital map in a vehicle, wherein - image data of the vehicle surroundings are acquired (1) using one or more cameras (7) of the vehicle (5); - the acquired image data are processed (2) to identify a visually represented overview map (9) containing an abstracted representation of spatial conditions in the vicinity of the overview map; - the image data of an identified overview map (9) are transformed (3) into a navigable digital map; - guidance information for a travel route in the spatial region represented in the overview map is generated (4) using the digital map; and wherein - in addition to the digital map generated from the acquired image data, there is also an additional digital map in the vehicle for the same spatial region, and the two digital maps are evaluated together; - spatial regions with contradictions between the two digital maps are identified; and - the identified regions are avoided or are approached and inspected using vehicle sensors.

2. Method according to claim 1, wherein, in order to identify the overview map in the acquired image data, a marking (13) contained in the overview map is evaluated.

3. Method according to claim 1 or 2, wherein the overview map contains information (11, 12) about the location of the overview map and / or the location of the vehicle when the overview map is acquired, which information is acquired and evaluated and is used during the transformation of the image data into the navigable digital map.

4. Method according to any of the preceding claims, wherein the overview map contains information (14) about a map scale, which information is acquired and evaluated and is used during the transformation of the image data into the navigable digital map.

5. Method according to any of the preceding claims, wherein the overview map contains an overview (10) of a parking area comprising a plurality of parking spaces, in particular of a parking garage comprising parking spaces on one or more parking levels.

6. Method according to claim 5, wherein the overview map is acquired (1) when the vehicle approaches the parking area or when the vehicle enters the parking area.

7. Method according to claim 5, wherein the overview map consists of a plurality of partial maps and wherein - a first partial map is acquired when the vehicle enters the parking area; - the vehicle is guided to a portion of the parking area by means of the first partial map; - when the vehicle enters the portion of the parking area, a second partial map is acquired; and - the vehicle is guided to a parking space in the portion of the parking area or to another portion of the parking area by means of the second partial map.

8. Method according to any of the preceding claims, wherein - the additional digital map also present in the vehicle has been received via a digital communication device or is already contained in the original digital map data of a navigation application.

9. Assistance system for carrying out a method according to any of the preceding claims.