Methods for at least semi-automatic parking of motor vehicles, driver assistance systems, and motor vehicles.

Abstract

The present invention relates to a method for at least semi-automatic parking of a motor vehicle (10), providing a trajectory (24) along which the motor vehicle (10) travels at least semi-automatically, and the trajectory (24) is displayed on a display device (22, 38) via an overview map (28) of the environment. The corresponding environmental regions (U1, U2, U3, U4, U5, U6, U7, U8) of the environment (U) are detected by at least one camera as corresponding image data (B, B1, B2, B3, B4, B5, B6, B7, B8). These image data are stored, at least in part, in a manner that they are assigned to the environmental regions involved (U1, U2, U3, U4, U5, U6, U7, U8). At the time point when the overview map (28) is displayed, all the image data (B, B1, B2, B3, B4, B5, B6, B7, B8) stored up to that time point are displayed in the corresponding display areas (A1, A2, A3, A4, A5, A6, A7, A8) of the overview map (28).

Description

Technical Field

[0001] This invention relates to a method for at least semi-automatic parking of a motor vehicle, wherein a trajectory from a starting point to an end point is provided and stored, and when the motor vehicle is near a predetermined location on the provided trajectory, the motor vehicle travels along the trajectory at least semi-automatically to the end point, the trajectory is displayed on a display device, an overview map of the environment surrounding the trajectory is displayed, and the trajectory is displayed in the overview map. The invention also includes a driving assistance system and a motor vehicle. Background Technology

[0002] This invention primarily relates to the field of so-called trained parking. A trained parking function typically teaches the driver a specific lane for the parking process in such a way that the driver manually drives through that lane during a recorded drive and subsequently stores the lane via the function. The stored parking process can then be retrieved from any point in the record, thereby allowing the vehicle to traverse the path at least semi-automatically until reaching the destination. In this case, "at least semi-automatically" means that the vehicle can autonomously traverse the trajectory and can perform steering, acceleration, and braking. However, it may additionally be necessary for the driver to operate a safety switch during this period, or at least be prepared to intervene. In such at least semi-automatic parking, the vehicle can therefore drive fully automatically, i.e., automatically with respect to the automatic longitudinal and lateral guidance of the vehicle.

[0003] In this context, DE 10 2017 115 810 A1 describes an automated parking method based on a trajectory from a starting point to a target point. The trajectory is displayed in a user interface, along with the environmental area of ​​the trajectory. A drivable area within the environmental area is determined, and the trajectory is automatically adjusted from the starting point to the target point, taking into account the drivable area. Here, the user can perform corresponding inputs through the user interface to define impassable or drivable areas, such as flowerbeds or lawns. The trajectory can be described by calculation, or alternatively learned by the driver maneuvering the vehicle from the starting point to the target point. The vehicle records the trajectory traveled or the driving maneuver used to traverse the trajectory and subsequently reproduces the trajectory or the driving maneuver.

[0004] Furthermore, DE 2013 213 225A1 describes a parking assistance system in which stored parking or maneuvering maneuvers along a trajectory are provided, and the trajectory and environmental objects from the vehicle's starting position to the target position are displayed to the driver via a display device. Additionally, obstacles located in front of the vehicle are displayed to the driver, who instructs the parking assistance system via input through operating elements how the vehicle should maneuver around the obstacles. Such obstacles or general environmental objects can be briefly displayed to the driver on the display device. The driver can then decide how to maneuver around such objects.

[0005] In parking systems known to date, only obstacles currently detectable by the vehicle's corresponding sensors are displayed to the driver. If the intended path extends through a hidden observation area, such as around a building, these objects may only be shown to the driver for a very short time—that is, only when the vehicle has traveled far enough that these objects are also within the detection range of the vehicle's sensors. However, a clear illustration of the surrounding environment plays a crucial role not only in collision avoidance but also, for example, in the selection of the appropriate course of the path itself. Even when the driver teaches the vehicle the intended path by initially driving through it, the driver—especially when the parking path is long—is not always clear whether the chosen path is unfavorable, such as passing directly through a building entrance or other unfavorable location. The often only rough sketches of detected environmental objects in the overview map do not always clearly identify which objects are involved, as only rough outlines are shown. With such a rough overview map, the driver often finds it difficult, or even impossible, to understand, whether an object is actually correctly classified as either impassable or drivable. Summary of the Invention

[0006] Therefore, the object of the present invention is to provide a method, a driving assistance system, and a motor vehicle that can design at least a semi-automatic parking function for the motor vehicle in a way that is as simple, clear, and safe as possible for the user.

[0007] This objective is achieved by a method, a driving assistance system, and a motor vehicle having the features according to the respective independent claims. Advantageous embodiments of the invention are the subject of the dependent claims, the description, and the drawings.

[0008] In the method for at least semi-automatic parking of a motor vehicle according to the present invention, a trajectory from a starting point to an ending point is provided and stored. When the motor vehicle is located in a predetermined vicinity of the provided trajectory, the motor vehicle travels along the trajectory at least semi-automatically to the ending point. The trajectory is displayed on a display device, along with an overview map of the environment surrounding the trajectory, and the trajectory is displayed in the overview map. Here, the overview map has multiple different display areas corresponding to various environmental areas of the trajectory environment. Once image data relating to each environmental area is detected by at least one camera of the motor vehicle, at least a portion of the detected image data is stored as current image data portions (B, B1, B2, B3, B4, B5, B6, B7, B8) associated with the corresponding environmental area. At the time the overview map is displayed, all current image data portions stored up to that time are displayed in the display area of ​​the overview map corresponding to the respective environmental area.

[0009] Here, the present invention has two very significant advantages: Firstly, it is possible to show not only the vehicle's environment schematically in the overview map, but also, to a certain extent, as a real image captured / photographed of the environment, because the image data detected by at least one camera of the vehicle is displayed in the overview map only after necessary distortion or processing. Similar to a top-down camera image, the environment around the track can also be displayed to the user on the display device, provided that image data already exists for this purpose. Through such a real image of the environment, composed or potentially composed of camera images, the user can understand significantly more easily which objects are arranged in the vehicle's environment or near the track, or even on the track, and the exact direction of the track in the environment. However, another major advantage of the present invention is that it can display not only currently detectable image data to the user, but also image data detected in advance, for example, when the vehicle is still in a completely different position. While the vehicle is moving on or near the track, one or more cameras of the vehicle can, for example, capture images of the environment. Here, the vehicle does not necessarily have to be currently in recording driving or at least in semi-automatic parking. Typically, image data related to environmental areas in the trajectory environment, detected at any time by at least one camera on the vehicle, can be detected and stored for the corresponding environmental areas, and displayed when an overview map is shown, for example, for the purpose of automatic or semi-automatic parking. This overview map can advantageously be supplemented at any time, and environmental areas that cannot be detected by the vehicle's sensors from its current position can also be displayed in the overview map. Thus, for example, by means of a top-down camera on the vehicle, the route traveled, such as a courtyard entrance or garage, can be recorded and combined into an overview map in such a way that the traveled, stored lanes, currently referred to as the trajectory, are drawn and made visible to the user. Even if the overview map cannot be completely filled during the recording of the journey, and the corresponding area is displayed as a black rectangle, for example, because image data from the corresponding environmental area is not yet available, the overview map, for example, the black rectangle, can be further filled with camera images by traveling through the remaining, undetected environmental areas, regardless of the recording of the journey. Finally, a complete overview map can be displayed to the user, showing image information related to the complete environmental area surrounding the provided trajectory from its starting point to its end point. Therefore, the driver can not only see the stored trajectory against a black background but also directly reference the real-world environment. This allows for a clear understanding of which path the vehicle will select or has already selected when traversing the trajectory. Furthermore, it clarifies which areas are drivable and allows for configuration of this feature, which will be explained in more detail later.

[0010] Here, at least semi-automatic parking of a motor vehicle should also be understood as fully automatic or autonomous parking. During such semi-automatic or automatic parking, the motor vehicle should be able to independently travel along a provided trajectory from a starting point or other starting position near the trajectory to a predetermined endpoint. Here, at least the steering function is autonomously undertaken by the motor vehicle, but preferably, the acceleration and / or deceleration of the motor vehicle are also automatically undertaken by the motor vehicle. Optionally, the driver may undertake monitoring tasks during this period and press safety switches if necessary. Furthermore, the trajectory is understood as a trajectory curve from the starting point to the endpoint. The starting point and the endpoint represent two corresponding, distinct spatial locations. The endpoint simultaneously defines the parking position of the motor vehicle. Here, parking of a motor vehicle should at least be understood as the stopping of the motor vehicle. In other words, the motor vehicle can travel along the trajectory to the endpoint and stop there during at least a semi-automatic parking process. Preferably, at the endpoint of the motor vehicle, the parking brake or handbrake is also autonomously operated. To provide the trajectory, the driver can teach the trajectory. That is, within the recorded driving range, the driver drives along the trajectory from the starting point to the endpoint, thereby pre-setting the trajectory for the motor vehicle. During manual driving by the driver, the vehicle records and stores the trajectory curve, i.e., the trajectory from the starting point to the ending point. If the vehicle subsequently returns to a predetermined vicinity of the provided trajectory, the driver can trigger at least semi-automatic parking, and the vehicle will then automatically drive along the trajectory to the predetermined ending point. In this predetermined vicinity, it should be understood that the vehicle should not be too far from the provided trajectory. Ideally, the vehicle is already on the trajectory in the initial state and preferably, at least in most cases, is oriented towards the ending point in the predetermined direction of the trajectory. However, even when the vehicle is at a certain distance from the trajectory, for example, within a few meters, the vehicle can be designed to autonomously drive to the trajectory and from there drive along the trajectory to the ending point. If the vehicle is initially too far from the provided trajectory, for example, a prompt can be output to the driver, i.e., to move closer to the provided trajectory. The environment displayed in the overview map preferably represents the environment directly adjacent to the trajectory. For example, to display the overview map, an appropriate segment surrounding the trajectory, completely encompassing it, can be selected and displayed. Depending on the length and geometry of the trajectory, the displayed environmental area, i.e., the complete environmental area displayed, can be correspondingly changed to be larger or smaller. This environmental area can be further divided into sub-regions, which are currently referred to as the environmental regions of the trajectory's environment. In principle, these sub-regions can be divided arbitrarily small, for example, with pixel-precise division based on the pixels provided by the display device. The geographical environment of the trajectory and its corresponding environmental region are assigned to the corresponding display area in the overview map. This assignment preserves the spatial arrangement of the environmental regions relative to each other. In other words, there is a correctly positioned allocation relationship between the environmental regions and the display areas.Once image data of the corresponding environmental area for the trajectory environment is detected, this image data, or at least a portion thereof, is stored, processed as necessary, and displayed in the corresponding display area at the time when the overview map is displayed.

[0011] Preferably, the environmental area is shown in the overview map from a bird's-eye view. This significantly improves clarity for the user. Image data detection and overview map display can also be performed during driving recording and during automatic parking. In other words, the overview map can be populated in real-time or on-site using currently detected image data while the vehicle is moving in the environment around the track or on the track. However, the display time may be significantly later than the image data detection. If necessary, only a portion of the detected image data may be displayed; in particular, it is understood that specific image data from the selected display viewpoint may be invisible and therefore not part of the display. This results in the display viewpoint showing the overview map not necessarily corresponding to the viewpoint from which the vehicle camera captured the corresponding environmental area. Particularly preferably, for image data detection, not only one vehicle camera but multiple vehicle cameras are used, which can be arranged around the vehicle, for example, one vehicle camera at the front of the vehicle, one vehicle camera in the area at the rear of the vehicle, one vehicle camera on the corresponding exterior rearview mirror, and / or a forward-pointing camera on the interior rearview mirror. The image data recorded by these cameras can be merged and perspective-transformed, and then projected onto a desired projection plane to produce a bird's-eye view. Therefore, in order to present an overview map in a bird's-eye view, the environmental images are not taken from a bird's-eye view by cameras positioned above the vehicle, but rather by cameras in other locations. These images are then processed and merged to create the impression that the resulting composite image was taken by cameras positioned above the vehicle.

[0012] If a vehicle and, consequently, a camera, move through the environment, the newly detected image data can be simply added to the existing environmental image or supplemented as a new display area. This practice is also known as matching and stitching. Therefore, for example, filling in areas not directly detected during recording of movement can be subsequently done through image processing-based and / or GPS-based matching and image stitching. That is, by image analysis methods and / or based on detected location data, such as GPS, it can be determined at which location in the overview map the new image fragment obtained from the newly detected image data will be added to the existing image fragment. Here, location accuracy should be maintained, which can be achieved through the image analysis methods and / or location-based methods. Image stitching is understood here as providing the smoothest possible transition between display areas based on images detected in different ways. This allows for the final provision of an overview map that shows the environment of the trajectory from a bird's-eye view and displays individual image captures from a bird's-eye view. For example, it can be stipulated that when the motor vehicle is in a first position, first image data about a first environmental area of ​​the trajectory is captured by means of at least one camera of the motor vehicle, at least a portion of the first image data is stored and displayed on a display device in a first display area assigned to the first environmental area at a determinable later time point, at which time the motor vehicle is, for example, in a second position different from the first position.

[0013] The display device can also be a display device within the vehicle itself. This is particularly advantageous because an overview map can be simultaneously displayed to the user on the in-vehicle display device not only during driving recording but also when performing at least semi-automatic parking. However, it is also conceivable to preferably display the overview map on an external display device. This has the advantage that the user can also be provided with information about the overview map, its contents, and additional processing possibilities regarding the trajectory, which will be explained in detail later, and the user can perform these processing possibilities on a computer device external to the vehicle, such as their mobile phone or home computer.

[0014] In another advantageous embodiment of the invention, when at least one camera re-detects current image data related to the first environmental region, the first image data portion (B, B1, B2, B3, B4, B5, B6, B7, B8) in the stored current image data portion of the first environmental region within the environmental region is updated. This has the significant advantage of automatically detecting and displaying changes in the environment of the trajectory. Therefore, a current environmental image of the trajectory can always be provided to the user. For the user, temporary obstacles and movable objects are always visible in real time during the use of the function. Thus, the risk of accidents or collisions can be minimized.

[0015] In another highly advantageous embodiment of the invention, objects detected in a specific environmental area of ​​the environment by means of a vehicle's detection device are classified as drivable or indestructible objects, and objects with the assigned categories are displayed in an overview map in a display area allocated to the specific environmental area. To detect objects in the vehicle environment or trajectory environment, at least one of the aforementioned cameras and / or other sensors equipped on the vehicle can also be used, through which image data displayed in the overview map is also detected. Such sensors can be, for example, parking sensors or ultrasonic sensors, additional cameras or nanoradar. Laser scanners or lidar (light detection and ranging) sensors can also be used for environmental detection. By means of such onboard sensors, the vehicle can easily measure the direct vehicle environment and classify it accordingly in the map, i.e., the detected objects are shown in the map in the form of the aforementioned image data, and in addition to these shown objects, it is also shown whether the object is assigned a drivable or indestructible category. The classification of objects as drivable or indestructible can be achieved, for example, by marking objects classified as indestructible, indicating objects as obstacles. If objects are classified as drivable, they do not need to be explicitly marked on the overview map. The absence of markings can be used as an indication that these objects are classified as drivable. This object display and classification also does not necessarily have to be done in real time. When a vehicle is near the trajectory, it can detect the trajectory environment, for example, using a detection device. The detected objects and their classifications can then be stored accordingly. If the overview map is displayed at a later point in time, the detected objects and their classification results can also be displayed in connection with that. Therefore, it is advantageous to mark objects or obstacles on the overview map that are not detectable from the vehicle's current position using a detection device. This has particularly significant advantages in trajectory planning or trajectory replanning and allows the user to intervene, for example, in advance, and to change the parking trajectory.

[0016] In another highly advantageous embodiment of the invention, when a specific environmental area including an object is re-detected, an update and / or confidence test of the object's presence and / or classification is performed, wherein the results of the update and / or confidence test are output. Detected objects can also be updated during re-travel. If some objects are movable objects, such as flower pots, trash cans, parked bicycles, etc., which exist near the track at one point in time, or even on the track, but not at another point in time, then these can always be detected and displayed on the overview map. Therefore, the user is always notified whether any objects currently exist as obstacles on the track. Particularly advantageous is to perform confidence tests on detected objects or their classifications based on newly detected image data or sensor data. For example, it is possible that obstacles are detected at different locations in detections at different times, or that different lighting conditions exist in detections at different times, leading to deviations in object detection or classification results. If such inconsistencies arise in the re-detected sensor data, the user may, for example, participate in the update decision. This can further increase the reliability of the displayed obstacles.

[0017] In another advantageous embodiment of the invention, the correction of object classification is performed based on user input. As described above, the user can participate in the decision to update objects or their classifications, thereby ultimately achieving more reliable obstacle identification. However, the user can participate not only when, for example, discrepancies arise regarding object detection or classification after confidence testing, but also, for example, by specifying that the user can correct the object classification of objects that are, in principle, classified by the system, through user input. If an object is drivable but is shown as an obstacle in the overview map, for example, the user can inform the system of this through their user input. Conversely, if an object is shown in the overview map but not marked as an obstacle, the same applies to the aforementioned user input. The user can also inform the system that the object is an obstacle. For example, the user can mark areas in the overview map and classify these areas as obstacles, i.e., mark them as impassable objects. This can further improve the reliability of environmental detection.

[0018] In another highly advantageous embodiment of the invention, the starting point and / or ending point and / or direction of the trajectory within a predetermined drivable area are changed based on user input. In other words, the user can arbitrarily change the provided, stored trajectory, at least within a predetermined drivable area, through such user input.

[0019] This offers significant advantages, namely that users do not need to re-perform the recorded drive to make such changes to the provided trajectory. Advantageously, it provides users not only with the possibility of changing the trajectory between the initially determined start and end points, but also, for example, the possibility of changing the start and / or end points of the trajectory themselves. Therefore, users can, for example, change the final parking position by changing the end point without having to perform a new recorded drive. Adjustments to the trajectory can be made by the driver through the user interface, such as optimizing the recorded lane regarding lane length or desired end point location. Compared to purely schematic diagrams, an overview map can also utilize real-world imagery to identify obstacles not detected by the vehicle's sensors. Users can, for example, perform this user input using an overview map. For example, an overview map with the trajectory can be displayed on a touchscreen, and users can arbitrarily change the trajectory and its start or end point within a preset drivable area via touch input. Therefore, stored trajectories can be configured through a user interface, such as a touchscreen. Thus, trajectories can be configured, moved, and adapted to environmental changes. This can be done not only retrospectively but also in real-time based on the overview map, i.e., during the use of the function, specifically during at least the semi-automatic parking process. Alternatively, other input devices, such as a mouse or arrow keys, can be provided to enable pointing functionality on the overview map display device, allowing for user input, for example, to change the trajectory. This design is particularly advantageous when combined with a realistic representation of the environment in the overview map, as it visually conveys to the user which areas are drivable, which are not, and within which boundaries the trajectory is variable, and especially how the trajectory can be meaningfully modified. Conversely, in the case of a purely schematic environmental diagram with schematically drawn obstacles, the allocation of the displayed area to the actual environmental area becomes significantly difficult for the user, ultimately leading to frequent misplanning when the user modifies the trajectory.

[0020] In another highly advantageous embodiment of the invention, the overview map and / or trajectory are forwarded to external vehicle devices, particularly a central data processing device, such as an Internet server, and / or to other vehicles and / or mobile communication devices by means of wireless communication.

[0021] This has the significant advantage that the scanned environmental map can be shared with others in conjunction with the trajectory. This is achieved, on the one hand, by providing an overview map with the trajectory on other communication devices, such as home computers, smartphones, or similar devices. This allows users to perform, for example, adjustments to the trajectory and / or corrections regarding object classification at any arbitrary location. For this purpose, the user need not be inside a vehicle. To enable this, a suitable application (App) can be provided, for example, that runs on a computer device. Here, the overview map and / or trajectory can be transmitted via a central data processing device, such as an internet server. In other words, the detected image data and / or the currently stored overview map with the currently stored trajectory can be transmitted to such an internet server and retrieved by any other communication device. Here, data retrieval can be coupled with corresponding authorization, such as a password. This creates many other application possibilities. For example, individualized configuration possibilities can be envisioned for each record, such as public, commercial, or private sharing, and allowing changes to the stored route. Furthermore, it is particularly advantageous that the overview map and / or trajectory can be transmitted to another vehicle. Therefore, such overview maps and / or tracks can also be used by other residents of the same household. Through the possibility of sharing records, families with multiple vehicles can, for example, forward stored tracks to each other. For instance, a father can teach his son parking maneuvers and transfer the maneuvers to his vehicle. Thus, it is not necessary to drive through the same area multiple times. Furthermore, track adjustments can be shared. Thus, a newly placed flowerpot can be bypassed, for example, by a simple track adjustment via a touchscreen, and the track adjustment can then be shared with other vehicles that want to perform the same parking process. In principle, this also allows stored parking lines to react to new situations, such as flowerpots, and to be updated and shared graphically with other users. In the commercial field, it is conceivable that near-perfect maneuvers will now be sold or licensed, or that maneuvers will be coupled with entry mechanisms, such as parking lot entry mechanisms.

[0022] Therefore, another highly advantageous aspect of the invention is that, based on user input, a computing device external to the vehicle performs the processing of an overview map and / or trajectory, and transmits the processing results to the vehicle, wherein the display and / or at least semi-autonomous driving of the vehicle is implemented based on the transmitted processing results. Thus, the user can configure this from any arbitrary location outside the vehicle and transmit the processing results to their vehicle or other vehicles. This significantly improves operational comfort.

[0023] Furthermore, the present invention relates to a driving assistance system for at least semi-automatic parking of a motor vehicle, wherein the driving assistance system is designed to store a provided trajectory from a starting point to an end point, and, when the motor vehicle is located in a predetermined vicinity of the provided trajectory, drive along the trajectory at least semi-automatically to the end point, display the trajectory on a display device, display an overview map of the environment surrounding the trajectory, and display the trajectory in the overview map. Here, the overview map has multiple different display areas corresponding to various environmental areas of the trajectory environment, wherein the driving assistance system is designed to, once image data relating to each environmental area is detected by means of at least one camera of the motor vehicle, store at least a portion of the detected image data as current image data portions (B, B1, B2, B3, B4, B5, B6, B7, B8) associated with the corresponding environmental area, and, at the time of displaying the overview map, display all current image data portions stored up to that time in the display areas of the overview map corresponding to each environmental area. The advantages mentioned for the method and its design according to the invention are applied in the same manner to the driving assistance system according to the invention.

[0024] The present invention also includes an improvement to the driving assistance system according to the invention, which has features already described in conjunction with the improvement to the method according to the invention. For this reason, the corresponding improvement to the driving assistance system according to the invention will not be described here.

[0025] Furthermore, the present invention also relates to a motor vehicle having a driving assistance system or a design thereof according to the present invention.

[0026] The motor vehicle according to the invention is preferably designed as an automobile, especially a passenger car or truck, or a bus or motorcycle.

[0027] The present invention also includes a control device for a motor vehicle. The control device may have a data processing device or a processor device designed to perform embodiments of the method according to the invention. The processor device may for this purpose have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). Furthermore, the processor device may have program code designed to perform embodiments of the method according to the invention when implemented by the processor device. The program code may be stored in the data memory of the processor device.

[0028] The present invention also includes combinations of features of the described embodiments. Therefore, the present invention also includes implementations having combinations of features of multiple embodiments described herein, provided that these embodiments are not described as mutually exclusive. Attached Figure Description

[0029] The embodiments of the present invention will now be described. For this purpose:

[0030] Figure 1 A schematic diagram of a motor vehicle having a driving assistance system for training parking is shown according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of a motor vehicle during recording of a parking trajectory is shown according to an embodiment of the present invention.

[0032] Figure 3 A schematic top view of a motor vehicle at multiple different locations at different times in a trajectory environment is shown according to an embodiment of the present invention; and

[0033] Figure 4 A schematic diagram illustrating the display of an overview map with a trajectory on a display device according to an embodiment of the present invention. Detailed Implementation

[0034] The embodiments explained below are preferred embodiments of the present invention. In the embodiments, the components described in the embodiments represent individual, considered independent features of the invention, which also independently improve the invention. Therefore, this disclosure should also include combinations different from the combinations of features shown in the embodiments. Furthermore, the described embodiments may be supplemented by other features of the invention already described.

[0035] In the accompanying drawings, the same reference numerals denote elements that have the same function.

[0036] Figure 1 A schematic diagram of a motor vehicle 10 having a driving assistance system 12 for training parking, according to an embodiment of the present invention, is shown. The driving assistance system 12 has at least one camera 14, four cameras in this example, and optional other sensors 16 for detecting the environment U of the motor vehicle 10. These sensors may be designed as parking sensors, ultrasonic sensors, nanoradar, and / or laser scanners, etc. The cameras 14 are also designed to detect the environment U of the motor vehicle 10—particularly in the form of image captures of the environment U—and provide image captures in the form of image data B from the control unit 18 of the driving assistance system 12. Sensor data detected by the other sensors 16 (not explicitly shown herein) is also provided to the control unit 18. Furthermore, the driving assistance system 12 includes a location determination device 20, such as a GPS receiver. Additionally, the driving assistance system 12 includes a display device 22.

[0037] The trained parking function, which can be implemented by the driver assistance system 12, typically teaches the driver a specific lane for the parking process; this lane is currently also referred to as trajectory 24 (see, for example). Figure 2 The method involves the driver manually navigating the lane during the recorded drive and subsequently storing the lane using functions provided by the driver assistance system 12. For this purpose, the control unit 18 may, for example, have a storage device. During the recorded drive, the environmental sensors 16 and camera 14 detect the environment U, and the vehicle 10 or control unit 18 positions the vehicle 10 based on the position detected by the position determination device 20 during the recorded drive. Thus, the vehicle 10 or control direction 18 can position the vehicle 10 within its environment U, and therefore also the route traversed from the starting point 24a to the ending point 24b during the recorded drive. For example, in Figure 2 In this diagram, the recorded driving is shown as a top view of the vehicle 10 during the recorded driving period. If the vehicle 10 re-enters the vicinity of trajectory 24 at a subsequent point in time, the stored parking procedure can be retrieved from that point. In other words, the stored parking procedure can be retrieved from any point recorded in trajectory 24, or at least from the vicinity of the trajectory, thereby allowing the vehicle 10 to complete the journey at least semi-automatically until reaching the destination 24b. Here, the stored trajectory 24 can be shown on the overview map 28 on the display device 22 of the vehicle 10.

[0038] In this situation, it is common practice to date to provide such an overview map only schematically, showing the trajectory and obstacles detected in the environment by means of the vehicle's sensors. This makes it difficult for the user to orient themselves in the environment. Due to this schematic diagram, especially regarding obstacles, the user often struggles to understand whether these obstacles are actually impassable or whether they do not affect the desired trajectory at all. Instead, in this schematic diagram, only objects in the vehicular environment identified as obstacles by the vehicle are visible to the user. Other objects in the environment—which are, for example, classified as drivable by the vehicle or a corresponding driver assistance system—are usually not shown at all in this overview map. If, for example, there is a miscalculation due to the system, and an object is an impassable obstacle but is incorrectly classified as drivable, this is not directly identifiable to the user in the overview map. This can involve significant safety risks. Accordingly, obstacles are often overlooked. While it may also be known to show the environment by stitching together images into a top-down view, such vehicle-mounted top-down cameras typically only display real-time images. However, with the aid of a trained parking function, trajectories, such as those up to 50 meters long, can be planned and stored. Consequently, the large environmental area covered by such long trajectories cannot be detected by the overhead camera, which only shows the currently detectable environment. Consequently, it becomes difficult for the user to estimate whether the remaining distance from trajectory 24 to endpoint 24B is drivable as planned, or whether there are any obstacles to circumvent due to sudden changes in the environment. This requires a very high level of concentration and attention from the user or driver in each parking maneuver, significantly impacting the comfort of the automatic parking process. Furthermore, during such automatic parking, driver negligence or inattention often leads to collisions with overlooked, unavoidable objects.

[0039] The present invention or its embodiments are now advantageously capable of displaying an overview map 28 to a user on a display device, such as the display device 22 of the vehicle 10, the overview map being created based on image data B detected by the camera 14. Advantageously, not only is the detectable environmental area of ​​the environment U at the current point in time displayed on the display device 22 in the form of image data B, but also, for example, image data B detected in the environment U of the trajectory 24 at a previously completed drive or an earlier point in time is also displayed on the display device 22. Therefore, an overview map 28 can ultimately be provided that not only shows the environment as a top-down image near the current position of the vehicle 10, but also shows all environmental areas surrounding the complete trajectory 24. Now, by means of… Figure 3 and Figure 4 This point will be explained in detail.

[0040] Figure 3This diagram illustrates the vehicle 10 at multiple different locations within the environment U of trajectory 24 at different times t1, t2, and t3. However, due to clarity, the trajectory is not shown in the diagram. Figure 3 As shown in the figure. Here, the vehicle 10 may also be in the process of recording travel, meaning that the trajectory 24 may not be fully recorded at the indicated time point. Therefore, during the movement of the vehicle 10 in the environment U, the camera 14 detects the environment U of the vehicle 10 in the form of image data at these different times t1, t2, t3. Here, the environment U is divided into various environment regions U1, U2, U3, U4, U5, U6, U7, U8 for better illustration. Here, the image data B detected by the camera 14 corresponding to the corresponding environment regions U1 to U8 is indicated by the reference numerals B1, B2, B3, B4, B5, B6, B7, B8 respectively. Therefore, in a given time step, it is not necessary for the camera 14 of the vehicle 10 to detect all these individual environment regions U1 to U8. These environment regions are only detected and stored during the movement of the vehicle 10 through the environment region U. In this example, the vehicle 10 detects environment regions U8 and U7 in the form of corresponding image data B8 and B7 in the first time step t1. In a later time step t2, vehicle 10 detects environmental region U7 in the form of image data B7 corresponding to that region. It should be noted that the re-detected image data B7 may differ from the image data B7 previously detected in the first time step t1. Furthermore, in the second time step t2, vehicle 10 still detects environmental regions U4 and U2 in the form of corresponding image data B4 and B2. In the final time step t3 shown, vehicle 10 detects environmental regions U1, U3, U5, and U6 in the form of corresponding image data B1, B3, B5, and B6. Therefore, during the movement of vehicle 10 through environment U, all shown environmental regions U1 to U8 are detected. This involves image technology detection implemented using camera 14. However, additionally, objects in environment U can also be detected using other sensors 16, which will be explained in more detail later.

[0041] Figure 4 A schematic diagram of the overview map 28 displayed on the display device 22 is now shown. In this overview map 28, the provided trajectory 24, the current position of the vehicle 10 (e.g., in the form of symbols or icons 10'), and all environmental areas U1 to U8 detected at the display time are shown in the form of corresponding image data B1 to B8 displayed in the display areas A1, A2, A3, A4, A5, A6, A7, A8 allocated to the corresponding environmental areas U1 to U8. Figure 4In the current example shown, the overview map 28 is displayed at a point in time when all environmental areas U1 to U8 have already been detected by vehicle 10 using image technology. Therefore, the display time is later than [previous time]. Figure 3 The third time point t3 shown is either the same as or coincides with this third time point. If the display is provided earlier, for example at time point t2, then... Figure 4 In the illustration, the corresponding images in display areas A1, A3, A5, and A6 are missing from the overview map 28. These display areas can be shown as black rectangles, with the provided trajectory 24 superimposed on these display areas. Through manual driving of the vehicle 10, such as in a recorded drive for parking training, the route traveled, for example, a courtyard entrance or garage, can be recorded in the form of trajectory 24 using a top-down camera 14, and stitched together to form the overview map 28. The driven, stored lanes, i.e., trajectory 24, are plotted in the overview map, and these lanes are visible to the user on the display device 22. If the map is not filled or not fully filled during the recorded drive, for example, including only the black rectangles containing images of the driven route 24, then the overview map 28 can now advantageously be further filled with camera images of the remaining undetected areas of the environment U, independent of the recorded drive. Thus, advantageously, the driver can not only see the stored trajectory 24 against a black background but also directly reference the real environment U. Therefore, it is clear which path vehicle 10 will select or has selected when traversing trajectory 24. It can then be seen which areas are drivable and can be configured for this function. The techniques used to illustrate the environment U and to stitch together images or image data B are already partially present in the current top-down view implementation. Once vehicle 10 moves and the corresponding area can be detected by external camera 14, the black areas around or below vehicle 10 are, for example, filled in the image that was initially not recognized by camera 14. This technique can be extended to fully present the overview display 28 used for training parking. This display 28 can be overlaid with the recorded lane 24. Filling in areas not directly detected in the recording can then be done via image processing-based and / or GPS-based matching and image stitching. Here, the following display error may occur: for example, objects detected in the environment appear higher than they actually are because the real camera is viewing the object from the side rather than from above. However, this does not affect the functionality of the overview map 28.

[0042] As mentioned earlier, in addition, other onboard sensors 16, and additional or camera-only devices 14, can measure the surrounding vehicle environment U in real time and classify it accordingly in map 28. In other words, objects in the environment U of the vehicle 10 can also be detected and, for example, classified as drivable or indestructible. In the current... Figure 3In the example shown, flower bed 30 is detected as a drivable object, for example. This flower bed can be correspondingly... Figure 4 As shown in the overview map 28, it is indicated by corresponding markers 32. For example, flower beds 30 can be highlighted in the overview map 28 with color or with borders. In this way, objects 30 that are not drivable as detected by vehicle 10 can also be additionally displayed to the driver. The map 28 can also display to the user several areas that are classified as drivable or indestructible by means of onboard sensors 14, 16, for example, because obstacles such as walls or other objects have been identified. Furthermore, the overview map 28 can be updated when the recorded location in the environment U is revisited. Updating the map 28 can be done by overlaying an older map 28. For example, in cases of different brightness or different objects, the unreliability can be detected by sensors 14, 16. In cases of unreliability, such as when obstacles are in different locations at different times or under different lighting conditions, the user can participate in the update decision. To update the map 28, data from other users who have driven through or have driven through the location, or data from other interfaces or sensors, can also be used. Data from other users or sensors used to update map 28 can be exchanged via vehicle-to-vehicle communication, radio communication, or vehicle-to-infrastructure communication. Server optimization is also possible. Accordingly, such as Figure 1The control unit 18 of the illustrated driver assistance system 12 can also be designed to communicate with other devices or communication devices, such as other vehicles, an internet server, or, in this example, with the mobile communication device 34 of the user 36. Here, the control unit 18 can not only receive data from other devices, such as the described overview map 28 and / or trajectory 24, but also transmit this data to external devices of the vehicle. In this example, the mobile communication device 34 associated with the user 36 can also have a display device 38 on which the overview map 28 can be displayed. That is, the overview map 28 with trajectory 24 does not necessarily have to be displayed on the vehicle's own display device 22, but can also be displayed, for example, additionally or alternatively, on the display device 38 of the external device 34 of the vehicle. It is particularly advantageous if, as preferred, the trajectory 24 can be adjusted according to user input. For example, the trajectory can be adjusted by the driver or user 36 via an operating interface. The stored trajectory 24 can be configured accordingly via an operating interface, such as a touchscreen, such as the display device 22, or the mobile communication device 34. Therefore, trajectory 24 can be corrected and moved, and for example, the starting point 24a and / or the ending point 24b can also be changed. Thus, trajectory 24, or its course, can be advantageously adapted to possible changes in the environment U. This can be done not only retrospectively, for example in a particularly comfortable manner, by means of communication device 34 or other external computing devices of the vehicle, but also in real time during functional use by means of the overview display 28 in the vehicle 10.

[0043] Furthermore, it is conceivable that the scanned environmental map 28 could be shared with others in conjunction with the trajectory 24. Individualized configuration possibilities are envisioned for each record, such as public, commercial, or private sharing, and the stored routes could be altered.

[0044] The configuration of the recorded lane 24 can be performed via any operating interface and display interface, such as directly in the vehicle 10 via a touchscreen or other control elements, such as a display and rotary buttons, via a data-accessible mobile device 34, or via a website using a mouse. Map 28 can be shared, trusted, and optimized on a server. Direct V2V (vehicle-to-vehicle) transmission, such as via Bluetooth, especially BTLE (Bluetooth Low Energy) or WLAN (Wireless Local Area Network), is also possible. Figure 1 In the current example, this wireless communication is represented by the dashed line 40.

[0045] In summary, these examples demonstrate how the present invention can provide a lane view and configuration menu for training parking, which can greatly improve clarity and predictability for the user.

Claims

1. A method for at least semi-automatic parking of a motor vehicle (10), wherein, - Provide and store the trajectory (24) from the starting point (24A) to the ending point (24B); - When the motor vehicle (10) is located at a predetermined location near the provided trajectory (24), the motor vehicle (10) is driven at least semi-automatically along the trajectory (24) to the destination (24B). - Display the trajectory (24) on the display devices (22, 38); - Displays an overview map (28) of the environment (U) surrounding the trajectory (24); and - The trajectory (24) is shown in the overview map (28); The overview map (28) is characterized by having multiple different display areas (A1, A2, A3, A4, A5, A6, A7, A8), which correspond to the corresponding environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) of the environment (U) of the trajectory (24). Once image data (B, B1, B2, B3, B4, B5, B6, B7, B8) involving the corresponding environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) is detected by at least one camera of the motor vehicle (10), the corresponding detected image data (B, B1, B2, B3, B4, B5, B6, B7, B8) will be displayed. At least a portion of 8) is stored as the current image data portion (B, B1, B2, B3, B4, B5, B6, B7, B8) and associated with the corresponding environmental area (U1, U2, U3, U4, U5, U6, U7, U8), and at the time point when the overview map (28) is displayed, all the current image data portions (B, B1, B2, B3, B4, B5, B6, B7, B8) stored up to that time point are displayed in the display area (A1, A2, A3, A4, A5, A6, A7, A8) of the overview map (28) corresponding to the corresponding environmental area (U1, U2, U3, U4, U5, U6, U7, U8). Using the detection device of the motor vehicle (10), objects (30) detected in specific environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) within environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) are classified as drivable or indestructible. The objects (30) and the categories (32) assigned to the objects (30) are displayed in the overview map (28) in the display areas (A1, A2, A3, A4, A5, A6, A7, A8) assigned to the specific environmental areas (U1, U2, U3, U4, U5, U6, U7, U8). The correction of the classification of object (30) is performed based on user input.

2. The method according to claim 1, characterized in that, When at least one camera re-detects the current image data (B, B1, B2, B3, B4, B5, B6, B7, B8) involving the first environmental region (U1, U2, U3, U4, U5, U6, U7, U8), the first image data portion (B, B1, B2, B3, B4, B5, B6, B7, B8) of the stored current image data portion (B, B1, B2, B3, B4, B5, B6, B7, B8) of the first environmental region (U1, U2, U3, U4, U5, U6, U7, U8) is updated.

3. The method according to claim 1 or 2, characterized in that, When re-inspecting a specific environmental region (U1, U2, U3, U4, U5, U6, U7, U8) including object (30): - Update the existence of object (30) and / or perform confidence checks; and / or - Update the classification of objects (30) and / or perform confidence checks. The output includes the results of updates and / or confidence checks.

4. The method according to claim 1 or 2, characterized in that, The trajectory (24) is changed according to user input within a pre-determined drivable area, starting point (24A) and / or ending point (24B) and / or direction.

5. The method according to claim 1 or 2, characterized in that, The overview map (28) and / or trajectory (24) are transmitted to an external vehicle device (34) and / or to other motor vehicles (10) and / or mobile communication devices via wireless communication (40), the external vehicle device being a central data processing device.

6. The method according to claim 1 or 2, characterized in that, Based on user input, a computing device (34) outside the vehicle performs the processing of an overview map (28) and / or a trajectory (24), and transmits the processing results to the vehicle (10), wherein the transmitted processing results are used for display and / or at least semi-autonomous driving of the vehicle (10).

7. A driver assistance system (12) for at least semi-automatic parking of a motor vehicle (10), wherein, The driver assistance system (12) is designed to be used for, - Store the trajectory (24) provided from the starting point (24A) to the ending point (24B); - When the motor vehicle (10) is located at a predetermined location near the provided trajectory (24), the motor vehicle (10) is driven at least semi-automatically along the trajectory (24) to the destination (24B). - Display the trajectory (24) on the display devices (22, 38); - Displays an overview map (28) of the environment (U) surrounding the trajectory (24); and - The trajectory (24) is shown in the overview map (28). The overview map (28) is characterized by having multiple different display areas (A1, A2, A3, A4, A5, A6, A7, A8), which correspond to the corresponding environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) of the environment (U) of the trajectory (24). The driving assistance system (12) is designed to, once image data (B, B1, B2, B3, B4, B5, B6, B7, B8) involving the corresponding environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) is detected by at least one camera of the vehicle (10), display the corresponding detected image data (B, B1, B2, B3, B4, B5, B6, B7, B8). At least a portion of B5, B6, B7, B8) is stored as current image data portions (B, B1, B2, B3, B4, B5, B6, B7, B8) in association with the corresponding environmental regions (U1, U2, U3, U4, U5, U6, U7, U8). At the time point when the overview map (28) is displayed, all current image data portions (B, B1, B2, B3, B4, B5, B6, B7, B8) stored up to that time point are displayed in the display areas (A1, A2, A3, A4, A5, A6, A7, A8) of the overview map (28) corresponding to the corresponding environmental regions (U1, U2, U3, U4, U5, U6, U7, U8). Using the detection device of the motor vehicle (10), objects (30) detected in specific environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) within environmental areas (U1, U2, U3, U4, U5, U6, U7, U8) are classified as drivable or indestructible. The objects (30) and the categories (32) assigned to the objects (30) are displayed in the overview map (28) in the display areas (A1, A2, A3, A4, A5, A6, A7, A8) assigned to the specific environmental areas (U1, U2, U3, U4, U5, U6, U7, U8). The correction of the classification of object (30) is performed based on user input.

8. A motor vehicle (10) having a driving assistance system (12) according to claim 7.

Images