Method for assisting a manual parking maneuver of a motor vehicle and motor vehicle

Abstract

The invention is directed to a method for assisting a parking maneuver of a vehicle (10), which includes a driver warning system (12) with an environment sensor system (14), a memory unit (16), and a computing unit (18). First, a parking intention of the vehicle (10) in a current environment is detected, and on the basis of the detected intention, environment data describing the current environment is determined. Then, vehicle data describing a current location and / or a current orientation of the vehicle (10) in the current environment is determined, and the environment data and the vehicle data are at least temporarily stored in the memory unit (16), where they are fused to analyze a potential parking status of the vehicle (10). Furthermore, a driver warning message is generated based on the potential parking status and is provided to the driver when the vehicle (10) is actually parked in the current environment.

Description

[0001] The invention is directed to a method for assisting a manual parking maneuver of a motor vehicle, wherein the motor vehicle includes a driver warning system with an environment sensor system, a memory unit, and a computing unit. Further aspects of the invention are directed to a driver warning system, a motor vehicle with a driver warning system, and computer program products designed to execute steps of the method according to the invention.

[0002] Usually, it is quite clear to a driver of a motor vehicle where the vehicle can be parked in the vehicle's current environment. However, there are situations in which it is not obvious to the driver whether they can park the vehicle in a specific area or parking space. This can be due to various circumstances, for example, the driver may be driving the vehicle in an environment that is unfamiliar or unfamiliar to them. For example, the driver may be driving to an unfamiliar city or country where they are unfamiliar with general parking regulations. In some cities, particularly in city centers, parking is strictly prohibited or only permitted for a predetermined group of vehicles, for example, only for residents or delivery vehicles.

[0003] Such parking restrictions or regulations may be signaled by signs, which may be in a foreign language not spoken by the driver. Alternatively or additionally, the signs may be located only at the beginning of a specific street in the current vicinity and may not be visible to the driver from the actual desired parking location. The signs may also be covered, for example, by snow. The same may apply to road markings on the ground, which may indicate a parking restriction, for example, only for disabled persons, car-sharing vehicles, or electric vehicles.

[0004] For the reasons described above, a driver may inadvertently park their vehicle illegally or in violation of the regulations, for example, over a road marking that was not visible at the time of parking because it was covered with snow. This may result in unwanted action, such as the vehicle being towed or the driver being fined.

[0005] Document US 2019 / 0100139 A describes a system for assisting a driver in correctly parking their vehicle within a marked parking space. The system includes a camera that detects a road marking that marks the parking space. The system detects whether the vehicle is incorrectly parked with respect to the road marking and can indicate this to the driver via a warning signal. However, the described system relies on the visibility of the road marking and thus provides no advantage over the driver's human vision.

[0006] It is an object of the present invention to provide a reliable method for assisting a driver in correctly parking his vehicle manually in a current vehicle environment, especially when he is not familiar with the current environment.

[0007] The object is achieved by the subject matter of the independent claims. Further embodiments of the invention are covered by the dependent claims, the following description, and the figures.

[0008] The invention is directed to a method for assisting a manual parking maneuver of a motor vehicle, wherein the motor vehicle includes a driver warning system with an environment sensor system, a memory unit, and a computing unit. In other words, the invention is directed to a method for facilitating a manual parking procedure.

[0009] The method is initiated as soon as a parking intention of the motor vehicle is detected in a current environment of the motor vehicle. In other words, the method according to the invention is not active by default the entire time the vehicle is operated, but only under certain circumstances, i.e. when an intention of the motor vehicle or the driver to park the vehicle is detected. The detection can be based on various observations, for example, the current driving speed of the vehicle. In other words, a parking intention can be detected when the vehicle is traveling at a speed slower than 30 kilometers per hour or even slower. Alternatively or additionally, the parking intention can be detected when neutral is engaged and / or when a known parking space is located near the vehicle, for example within a radius of 150 meters or less.

[0010] Based on the detected parking intention—that is, when the system has detected that the vehicle or driver intends to park the vehicle in the current environment, which may be the road along which the vehicle is currently traveling—environmental data is determined that describe the current environment of the motor vehicle. In other words, as soon as the parking intention is detected, that is, before the actual parking maneuver, the system begins to collect information about the current environment that was not previously collected. Such environmental data may include information about traffic signs in the current environment and the like. Further examples of environmental data are described below.

[0011] Once the parking intention is detected, vehicle data is also determined that describes a current location and / or current orientation of the motor vehicle in the current environment. The vehicle data may include data collected by an inertial measurement unit (IMU) of the vehicle. The vehicle data may include information about a current location of the vehicle in the current environment, for example, based on GPS data that can be acquired by a GPS sensor of the vehicle's sensor system (GPS - Global Positioning System). The vehicle data may also include orientation data of the vehicle, i.e., information about the vehicle's direction of travel (including forward or backward movement).The orientation may include information about an angle between a longitudinal axis of the vehicle and a curb and / or a main orientation of a road along which the vehicle is currently moving.

[0012] In a next step, the surroundings data and the vehicle data are stored at least temporarily in the motor vehicle's memory unit, where they are merged to analyze a potential parking status of the vehicle. The potential parking status is an assessment of the parking status that would result if the vehicle were actually parked by the driver in the current surroundings. Examples of parking statuses are "illegal" or "contrary to the rules." In other words, the system acts as if the vehicle were parked in the current surroundings and provides the corresponding parking status. For example, if the vehicle is currently passing a bus stop, the system can detect a sign indicating a no-parking zone associated with the bus stop.Since the vehicle may be traveling at a slow speed, such as 20 kilometers per hour, the system may be active and calculate the potential parking status for the vehicle at the bus stop, which is "illegal." Alternatively, the vehicle may be crossing multiple road markings indicating parking spaces. The system may detect the road markings as environmental data and may infer from the vehicle's orientation and location data that the vehicle is currently crossing more than one parking space. If the vehicle were actually parked in this position, its parking status would be "illegal." Consequently, in the latter example, the vehicle's potential parking status is "illegal." The system may store this information in the vehicle's memory unit.

[0013] In a next step, the system generates or prepares a driver warning message for a driver of the motor vehicle based on the potential parking status of the vehicle. Preferably, the warning message is also stored in the vehicle's memory unit and not yet forwarded to the driver. According to the invention, the driver warning message is only provided to the driver when the vehicle is actually parked in the current environment. The system can detect the actual parking of the vehicle, for example, when the vehicle's ignition is turned off and / or when the vehicle is put into neutral and / or when the driver unfastens their seatbelt.

[0014] The method according to the invention is advantageous in that the current surroundings of the vehicle are pre-scanned before a parking maneuver is actually initiated. Consequently, a driver warning message can be prepared based on the observation of the current surroundings, which is initiated as soon as the vehicle's intention to park is detected. Even if a no-parking sign is installed at the beginning of a street, which can easily be overlooked by a driver unfamiliar with the current surroundings, the system will detect this sign and base the analysis of the potential parking status on this sign as well if the driver attempts to park the vehicle further down the street.

[0015] The invention also includes embodiments that provide additional advantages.

[0016] According to one embodiment, the parking intention is detected based on the current driving speed of the motor vehicle and / or on the currently engaged gear of the motor vehicle and / or on the presence of a known parking space in the current surroundings and / or on the basis of navigation data of the motor vehicle. In other words, if a navigation system of the vehicle knows that it is approaching a navigation destination or a planned final destination, the parking intention can be detected and the method can be initiated.

[0017] According to a further embodiment, the surroundings data include information about at least one of a traffic sign and / or a road marking and / or an orientation of at least one other motor vehicle next to the motor vehicle in the current surroundings. In other words, the orientation of at least one neighboring vehicle can be determined. If, based on the vehicle data, it is detected that the motor vehicle is oriented parallel to the neighboring vehicle along a street or road with only parallel curbside parking, the system can deduce that the motor vehicle intends to park in the second row, which can be assigned an "illegal" potential parking status.

[0018] According to a further embodiment, the environmental data are determined at least partially on the basis of camera data that are captured by camera sensors of the environmental sensor system of the motor vehicle.

[0019] A camera sensor within the meaning of the present invention or a motor vehicle camera can be considered a camera that is suitable for being mounted on a motor vehicle, in particular for capturing camera images or video data streams that represent an external environment of the motor vehicle, such as a front camera, a rear camera, a side camera, and so on. The camera sensor can be part of the vehicle's environment sensor system. An environment sensor system can, for example, be understood as a sensor system that can generate sensor data or sensor signals that depict, represent, or image an environment of the environment sensor system. In particular, the ability to capture or detect electromagnetic or other signals from the environment cannot be considered a sufficient condition for qualifying a sensor system as an environment sensor system.Cameras, lidar systems, radar systems or ultrasonic sensor systems can, for example, be considered as environmental sensor systems.

[0020] Preferably, the camera sensors capture an all-round view of the motor vehicle. Furthermore, the camera data can be analyzed using a computer vision algorithm. Computer vision algorithms, which can also be referred to as machine vision algorithms or algorithms for automatic visual perception, can be considered computer algorithms for automatically performing a visual perception task. A visual perception task, also referred to as a computer vision task, can be understood, for example, as a task for extracting visual information from image data. In particular, the visual perception task can, in principle, in several cases be performed by a human who can visually perceive an image corresponding to the image data. In the present context, however, visual perception tasks are performed automatically without requiring human assistance.

[0021] A computer vision algorithm can be understood, for example, as an image processing algorithm or algorithm for image analysis that is trained using machine learning and can be based, for example, on an artificial neural network, in particular a convolutional neural network.

[0022] The computer vision algorithm may include, for example, an object detection algorithm, an obstacle detection algorithm, an object tracking algorithm, a classification algorithm, a segmentation algorithm, and / or a depth estimation algorithm.

[0023] Corresponding algorithms can be performed analogously based on input data other than images that are visually perceptible to humans. Point clouds or images from infrared cameras, for example, can also be analyzed using appropriately adapted computer algorithms. Strictly speaking, however, the corresponding algorithms are not visual perception algorithms, since the corresponding sensors can operate in ranges that are not perceptible to the human eye, such as the infrared range. Here and in the following, such algorithms are therefore referred to as perception or automatic perception algorithms. Perception algorithms therefore include visual perception algorithms, but are not limited to these with respect to human perception.Consequently, according to this understanding, a perception algorithm can be considered a computer algorithm for automatically performing a perception task, for example, using an algorithm for sensor data analysis or processing, which can be trained, for example, using machine learning and can be based, for example, on an artificial neural network. Generalized perception algorithms can also include object detection algorithms, object tracking algorithms, classification algorithms, and / or segmentation algorithms, such as semantic segmentation algorithms.

[0024] If an artificial neural network is used to implement a visual perception algorithm, a commonly used architecture is a convolutional neural network (CNN). In particular, a 2D CNN can be applied to specific 2D camera images. CNNs can also be used for other perception algorithms. For example, 3D CNNs, 2D CNNs, or 1D CNNs can be applied to point clouds, depending on the spatial dimensions of the point cloud and the processing details.

[0025] The output of a perception algorithm depends on the specific underlying perception task. For example, an output of an object detection algorithm may include one or more bounding boxes defining a spatial location and optionally an orientation of one or more respective objects in the environment and / or corresponding object classes for the one or more objects. An output of a semantic segmentation algorithm applied to a camera image may include a pixel level class for each pixel of the camera image. Similarly, an output of a semantic segmentation algorithm applied to a point cloud may include a corresponding point level class for each of the points. The pixel level classes or point level classes may, for example, define a type of object to which the respective pixel or point belongs.

[0026] In the context of the present disclosure, an object detection algorithm can be understood as a computer algorithm that can identify and locate one or more objects within a provided input data set, for example, an input image, by defining respective bounding boxes or regions of interest (ROI), and in particular, assigning a respective object class to each of the bounding boxes, wherein the object classes can be selected from a predefined set of object classes. Assigning an object class to a bounding box can be understood as providing a corresponding confidence value or probability that the object identified within the bounding box is of the corresponding object class.For example, the algorithm may provide such a confidence value or probability for each of the object classes for a given bounding box. Assigning the object class may, for example, involve selecting or providing the object class with the highest confidence value or probability. Alternatively, the algorithm may only specify the bounding boxes without assigning a corresponding object class.

[0027] According to a further embodiment, the environmental data includes map data from a digital map of the current surroundings of the motor vehicle. A digital map can be understood as a data set or a plurality of data sets stored on a computer-readable storage medium that relate objects in a predefined spatial area to one another and / or to a predefined reference point. Optionally, the digital map can contain semantic information relating to the objects. A high-resolution digital map or HD map can be understood as a digital map that has a spatial resolution on the order of one centimeter or less, that is to say in particular a resolution of less than 10 cm. The map data can, for example, contain semantic information about a public parking space near the vehicle in which parking is legally permitted.In other words, the vehicle can be assigned a “legal” potential parking status if the map data indicates a public parking space near the vehicle, preferably within a radius of less than 100 or less than 50 meters, preferably less than 25 meters, from the vehicle.

[0028] Preferably, the map data contains information about parking regulations in the current environment of the motor vehicle. For example, the vehicle may currently be driving within a city center where only residents' vehicles can park legally. The system can retrieve this information as semantic information from the map related to the city center. Consequently, the system can assign the vehicle an "illegal" potential parking status as long as it is moving within the city center.

[0029] According to one embodiment, the storage unit includes a ring buffer, wherein the surroundings data and / or the vehicle data are stored in the ring buffer. As described, the driver warning system initiates the determination of surroundings data, vehicle data, and the potential parking status as soon as the vehicle's intention to park has been detected. To this end, the system can store the surroundings data, the vehicle data, and also the determined potential parking status in the storage unit and preferably in the ring buffer. If the vehicle is actually parked in the current surroundings, the driver warning system can deliver the warning message to the driver based on the stored data. If the vehicle is not parked, the data stored in the ring buffer is overwritten as soon as the ring buffer is full. Consequently, the stored data can be handled very efficiently.

[0030] According to one embodiment, the driver warning message includes audible and / or visual and / or haptic feedback for the driver. In other words, a beep or a series of beeps may be played to the driver to warn them of a negative, i.e., illegal or non-compliant, parking status. Preferably, the feedback varies and depends on whether the parking status is merely "non-compliant" or actually "illegal." Alternatively or additionally, the vehicle's steering wheel may vibrate and / or a visual warning sign may be displayed to the driver inside the vehicle. The warning message may follow an escalation pattern, i.e., if the driver does not respond to the warning message, a volume of the beeper(s) may be increased and / or a frequency of multiple beeps may be increased.The vibration of the steering wheel may also be increased and / or the visual warning sign may become more intense or change from green to red.

[0031] Preferably, the driver warning message includes a suggested maneuver to avoid the potential parking situation. For example, the driver warning system may suggest a nearby official parking space to the driver and may preferably suggest a route to get there. If the vehicle is parked illegally, for example, covering a road marking between two adjacent parking spaces or spaces, the driver warning system may suggest a maneuver to park the vehicle correctly.

[0032] For applications or application situations that may arise in the method according to the invention and which are not explicitly described herein, it may be provided that, according to the method, an error message and / or a request for user feedback is output and / or a default setting and / or a predetermined initial state is specified.

[0033] A further aspect of the invention is directed to a driver warning system for a motor vehicle, which includes an environment sensor system, a memory unit and a computing unit, wherein the driver warning system is designed to carry out the method according to the invention according to any of the described embodiments.

[0034] In the present disclosure, a computing unit can be understood, for example, as a data processing device with a processing circuit. A computing unit can therefore perform computing operations to process data. The computing operations can also include indexed accesses to a data structure, for example, a lookup table (LUT).

[0035] In particular, a computing unit may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems-on-chip (SoCs). The computing unit may also include one or more processors, for example, one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit may also include a physical or virtual network of computers or other of the units.

[0036] A computing unit may also include one or more hardware and / or software interfaces and / or one or more memory or storage units. A memory unit may be implemented as a volatile data memory, for example, a dynamic random access memory (DRAM), a static random access memory (SRAM), or a non-volatile data memory, for example, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or flash EEPROM, a ferroelectric random access memory (FRAM), a magnetoresistive random access memory (MRAM), or a phase-change random access memory (PCRAM).

[0037] Another aspect of the invention is directed to a motor vehicle with such a driver warning system. The motor vehicle may be an automatic or semi-automatic vehicle. The vehicle may be a passenger car, a bus, or a motorcycle.

[0038] A further aspect of the invention is directed to a computer program comprising instructions which, when executed by a computing unit, in particular by a computing unit of a driver warning system as described herein, cause the computing unit to carry out steps of a method according to any of the embodiments described above.

[0039] In other words, a computer program with instructions is provided. When executed by at least one computing unit, the instructions cause the at least one computing unit to perform a method according to the invention.

[0040] The instructions may be provided, for example, as program code. The program code may be provided, for example, as binary code or assembly code and / or as source code of a programming language, for example, C, and / or as a program script, for example, Python.

[0041] According to a further aspect of the invention, a computer-readable storage medium storing a computer program according to the invention is provided.

[0042] The computer program and the computer-readable storage medium are respective computer program products containing the instructions.

[0043] Further embodiments of the driver warning system and / or the motor vehicle according to the invention follow directly from the various embodiments of the method according to the invention and vice versa. In particular, individual features and corresponding explanations as well as advantages with regard to the various embodiments of the method according to the invention can be transferred analogously to corresponding embodiments of the further aspects according to the invention. In particular, the driver warning system according to the invention is designed or programmed to carry out the method according to the invention. In particular, the driver warning system according to the invention carries out the method according to the invention.

[0044] When it is mentioned in the present disclosure that a component of the driver warning system according to the invention, in particular the computing unit of the driver warning system, is designed, configured or configured, etc., to perform or realize a certain function, to achieve a certain effect or to serve a certain purpose, this can be understood in such a way that the component, in addition to being usable or suitable for this function, effect or purpose in principle or theoretically, is concretely and actually capable of performing or realizing the function, achieving the effect or serving the purpose through appropriate adaptation, programming, physical construction, etc.

[0045] Further features of the invention are evident from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and / or shown in the figures can be included in the invention not only in the respective specified combination, but also in other combinations. In particular, embodiments and combinations of features that do not have all the features of an originally formulated claim can also be included in the invention. Furthermore, embodiments and combinations of features that go beyond or deviate from the combinations of features set out in the backreferences of the claims can be included in the invention.

[0046] The invention will be explained in detail below with reference to specific exemplary embodiments and respective schematic drawings. In the drawings, identical or functionally equivalent elements may be designated by the same reference numerals. The description of identical or functionally equivalent elements is not necessarily repeated with reference to different figures.

[0047] In the figures: Fig. 1 shows a schematic view of a motor vehicle with a driver warning system according to an embodiment of the invention; Fig. Figure 2 shows a schematic view of a first parking scenario for such a vehicle; Fig. 3 shows a schematic view of another parking scenario for such a vehicle; Fig. 4 shows a schematic view of another parking scenario for such a vehicle; Fig. 5 shows a schematic view of another parking scenario for such a vehicle; and Fig. 6 shows a schematic view of a method for assisting a manual parking maneuver of a motor vehicle according to an embodiment of the invention.

[0048] Fig. 1 shows a schematic view of a motor vehicle 10 with a driver warning system 12. The driver warning system 12 (short: system 12) includes an environment sensor system 14, a memory unit 16 and at least one computing unit 18. The environment sensor system 14 can include a plurality of sensors 20, for example a plurality of camera sensors, which can be arranged on the vehicle 10 in order to capture an all-round view or a 360-degree view of the current vehicle surroundings.

[0049] The driver warning system 12 can be configured to detect a parking intention of the vehicle 10 or a driver of the vehicle 10. To do so, the system 12 can evaluate sensor data describing a current driving behavior of the vehicle 10 or the driver. For example, the system 12 can receive sensor data describing a current driving speed of the vehicle 10, which can be lower than a predefined threshold, for example, lower than 30 or 10 kilometers per hour (km / h). This can be evaluated as an indication of an increased probability that the vehicle 10 intends to park in the near future, for example, within the next 2 to 3 minutes. The increased probability can be assumed to be an indication of a parking intention of the motor vehicle 10.

[0050] The system 12 may also receive a signal describing the shifting into neutral of the motor vehicle 10, which may be assumed to be an indication of an intention of the vehicle 10 to park.

[0051] The driver warning system 12 can additionally or alternatively receive environmental data from the sensor system 14 that describes the current environment of the vehicle 10. The system 12 can also derive the parking intention of the vehicle 10 from the environmental data. The environmental data can, for example, include map data indicating the presence of a public parking space near the vehicle 10, for example, within a radius of approximately 100 meters around the vehicle 10. This can be assumed to be an indication of the parking intention of the vehicle 10.

[0052] Once the warning system 12 has detected the parking intention of the vehicle 10, it can start collecting (further) environmental data. This environmental data can include, for example, information about traffic signs in the current environment and / or road markings 22 in the current environment of the vehicle 10 that may indicate a parking restriction in the current environment. The system 12 can, for example, apply an object detection algorithm to the captured camera data to extract features from the detected traffic signs, for example, features that indicate a parking restriction (no parking sign, bus stop, zebra crossing, etc.).

[0053] In a parallel step, the system 12 may determine or acquire vehicle data describing a current location and / or orientation of the vehicle 10 in the current environment. To do so, the system 12 may receive orientation data from an inertial measurement unit (IMU) of the vehicle 10, which may be part of the environment sensor system 14. The system 12 may also receive location data from a location sensor of the vehicle 10, for example, from a GPS sensor of the vehicle 10. Consequently, the system 12 may localize the vehicle 10 in the current environment.

[0054] The system 12 may store the data (i.e., the surrounding data and the vehicle data) in the storage unit 16, and may merge them to analyze a potential parking status of the vehicle 10. The potential parking status describes the consequences if the vehicle 10 were to follow its detected parking intention and actually park in the current surroundings. For example, if the system 12 determines that the vehicle 10 intends to park at a bus stop, the potential parking status may be "illegal." If the system 12 determines that the vehicle 10 is about to be inaccurately parked, for example, over a road marking 22 indicating a parking space, the potential parking status may be "abnormal."

[0055] The system 12 may generate or prepare a driver warning message based on the potential parking status, wherein the message may differ between different potential parking statuses, for example, the message may be more aggressive if the potential parking status is "illegal," with respect to a more subtle message in the case of only an "irregular" potential parking status.

[0056] If the system 12 detects that the vehicle 10 is actually parked in the current environment, for example, because it detects a shift into neutral, it can provide a warning message to the driver. This can be done, for example, by displaying a visual warning sign to the driver and / or by providing haptic feedback to the driver, for example, by vibrating the steering wheel of the vehicle 10, and / or by providing an audible warning tone.

[0057] Fig. 2 to 5 show schematic views of various exemplary parking scenarios for a motor vehicle 10, wherein the potential parking status is at least “illegal” because the vehicle 10 either has road markings 22 ( Fig. 2 and Fig. 4), outside designated parking spaces ( Fig. 3) or over a zebra crossing 24 ( Fig. 5) is parked.

[0058] Fig. 6 shows a schematic view of a method for assisting a manual parking maneuver of a motor vehicle 10 according to an embodiment of the invention, wherein the motor vehicle 10 includes a driver warning system 12 with an environment sensor system 14, a memory unit 16 and a computing unit 18.

[0059] In a step S1, a parking intention of the motor vehicle 10 in a current environment of the motor vehicle 10 is detected. The following steps are carried out based on the detected parking intention. In a step S2, environmental data describing the current environment of the motor vehicle 10 is determined. In a step S3, vehicle data describing a current location and / or a current orientation of the motor vehicle 10 in the current environment are determined. In a step S4, the environmental data and the vehicle data are at least temporarily stored in the memory unit 16 of the motor vehicle 10, where they are merged to analyze a potential parking status of the motor vehicle 10. In a step S5, a driver warning message is generated for a driver of the motor vehicle 10 based on the potential parking status of the motor vehicle 10.In a step S6, the driver warning message is provided to the driver when the motor vehicle 10 is actually parked in the current environment.

[0060] As described, illegal parking or parking in an unauthorized area can cause problems for both drivers and other users of the road or parking lot. In many parking areas, it may be difficult to detect illegal parking situations, such as parking in wheelchair spaces or public service areas, or on pedestrian crossing markings. This is where the invention can help notify the driver of the situation.

[0061] For many drivers, especially novices, it can be difficult to be aware of vehicle orientation on parking space markings, so it might cause them to get out several times until they are able to park perfectly, so it is very useful that they receive some notification of such incorrect scenarios.

[0062] Aspects of the invention preferably utilize a fusion of (i) data from surround-view cameras, preferably also including front cameras capable of detecting a current lane, a curb, parking areas, and / or parking signs, e.g., no-parking signs indicating illegal parking positions, and (ii) data indicating a current vehicle position and orientation based on localization data to identify exact vehicle positions corresponding to the parking space markings or areas. Based on the fused data, an illegal parking situation can be identified. The driver can then be notified of such a situation through interfaces in the HMI and / or steering wheel (visual, acoustic, and / or haptic feedback).

[0063] One advantage of the proposed system is that the driver can be notified of an illegal parking situation, which can help reduce the number of such incidents. This can benefit society by reducing the number of cases of parking in an illegal location.

[0064] As a practical example, an inexperienced driver or a driver from another city or country might not be aware that they have parked their car in a no-parking zone. It might also happen that such drivers fail to understand a road sign and inadvertently find themselves in a situation where they park illegally. The proposed system can assess such a scenario and notify the driver through a visual interface before they exit the vehicle, which can motivate the driver to correct the situation before leaving the vehicle.

[0065] Overall, the examples demonstrate how the detection of incorrect / illegal parking can be facilitated using the fusion of camera sensors and vehicle localization sensors. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 2019 / 0100139 A

[0005]

Claims

[1] Method for supporting a manual parking maneuver of a motor vehicle (10), wherein the motor vehicle (10) includes a driver warning system (12) with an environment sensor system (14), a memory unit (16) and a computing unit (18), wherein - a parking intention of the motor vehicle (10) is detected in a current environment of the motor vehicle (10) and on the basis of the detected parking intention - environmental data are determined which describe the current environment of the motor vehicle (10), - vehicle data are determined which describe a current location and / or a current orientation of the motor vehicle (10) in the current environment, - the environmental data and the vehicle data are stored at least temporarily in the memory unit (16) of the motor vehicle (10), where they are merged to analyze a potential parking status of the motor vehicle (10), - a driver warning message is generated for a driver of the motor vehicle (10) based on the potential parking status of the motor vehicle (10), and - the driver warning message is provided to the driver when the motor vehicle (10) is actually parked in the current environment. [2] Method according to claim 1, wherein the parking intention is detected on the basis of a current driving speed of the motor vehicle (10) and / or on the basis of a currently engaged gear of the motor vehicle (10) and / or on the basis of the presence of a known parking space in the current environment and / or on the basis of navigation data of the motor vehicle (10). [3] Method according to one of the preceding claims, wherein the environmental data contain information about at least one of a traffic sign and / or a road marking (22) and / or an orientation of at least one other motor vehicle besides the motor vehicle (10) in the current environment. [4] Method according to one of the preceding claims, wherein the environmental data are determined at least partially on the basis of camera data which are detected by camera sensors (20) of the environmental sensor system (14) of the motor vehicle (10). [5] Method according to claim 4, wherein the camera sensors (20) detect an all-round view of the motor vehicle (10). [6] A method according to any one of claims 4 or 5, wherein the camera data is analyzed using a computer vision algorithm. [7] Method according to one of the preceding claims, wherein the environmental data include map data from a digital map of the current environment of the motor vehicle (10). [8] Method according to claim 7, wherein the map data contain information about parking regulations in the current environment of the motor vehicle (10). [9] Method according to one of the preceding claims, wherein the storage unit (16) includes a ring buffer, wherein the environmental data and / or the vehicle data are stored in the ring buffer. [10] Method according to one of the preceding claims, wherein the driver warning message includes an acoustic and / or a visual and / or a haptic feedback for the driver. [11] A method according to any one of the preceding claims, wherein the driver warning message includes a suggested driving maneuver to avoid the potential parking status. [12] Driver warning system (12) for a motor vehicle (10), which includes an environment sensor system (14), a memory unit (16) and a computing unit (18), wherein the driver warning system (12) is designed to carry out a method according to one of the preceding claims. [13] Motor vehicle (10) with a driver warning system (12) according to claim 12. [14] Computer program with instructions which, when executed by a computing unit (18), in particular by a computing unit (18) of a driver warning system (12) according to claim 12, cause the computing unit (18) to carry out steps of a method according to one of claims 1 to 11. [15] A computer-readable storage medium comprising a computer program according to claim 14.

Images