Method and parking assistance device for determining a parking space for a parking procedure of a vehicle

Abstract

The invention relates to a method, in particular a computer-implemented method, for determining a parking space for a parking procedure of a vehicle (1), comprising the method steps: - detecting a vehicle environment of the vehicle (1) using a sensor unit (3), - detecting a potentially usable parking space (Pgeo) in the detected vehicle environment by means of a geometric planning method, - detecting a potentially usable parking space (Pnn) in the detected vehicle environment by means of a planning method based on a trained neural network, - verifying a parking space (Pgeo) detected by the geometric planning method using a parking space (Pnn) detected by the planning method based on a trained neural network, or vice versa, - establishing the verified parking space (P) as a potential parking option.

Description

[0001] Method and parking assistance device for determining a parking space for a parking process of a vehicle

[0002] The invention relates to a method, in particular a computer-implemented method, as well as a parking assistance device for determining a parking space for a vehicle saving process.

[0003] Parking assistance systems are generally well-known. To detect a parking space with a (semi-)automatic parking assistance system, a parking space or a potential parking space is scanned as the vehicle passes by. Due to the limited range of the sensors and / or the speed of the vehicle (for example, up to 50 km / h), the exact parking situation is often not correctly detected. In particular, only the edge of the parking space adjacent to the vehicle is often detected, so that other surrounding objects, such as other parked vehicles and boundaries or lines drawn on the ground, are only detected with insufficient accuracy.

[0004] For example, the initial detection of the parking space is not exactly parallel or perpendicular to the optimal parking position of the vehicle in the space, or the vehicle does not scan the parking space at an ideal right angle. Therefore, when the parking assistance system initially determines a parking space contour, it is unclear how the parking space contour should actually be aligned to achieve optimal parking. This is particularly problematic when no imaging sensors such as cameras are used, which would allow the interpretation of information provided by other sensors (e.g., based on lines drawn on the ground).

[0005] In addition, adjacent objects, such as vehicles that were also not optimally parked, may be positioned at an angle to the actual parking space. If the parking space contour is calculated along the vehicle's own orientation, the parking assistance system will park the vehicle in such a way that it is not positioned satisfactorily between the adjacent objects, i.e., in particular, in a position that corresponds to the normal parking position assumed by the human driver.

[0006] The problem with the known parking assistance devices is that the parking space is often not used optimally and parking processes are sometimes not initiated or aborted because the parking space is considered to be impossible to park in, even though it would be possible to park by varying the parking space orientation and / or size.

[0007] From the publication DE10 2021 203 825, a generic parking assistance system is known which discloses parking space detection and optimization based on a geometric vehicle environment, which can be represented by polygons, polylines, line segments or a corresponding representation of the free space.

[0008] A disadvantage of parking space detection based on a geometric representation of the vehicle's surroundings is that not every free space between objects or line markings is actually intended for parking. This applies, for example, to driveways, sidewalks, etc. The assessment of the parking situation based on the geometric vehicle surroundings can therefore be error-prone. It is possible that an open space is detected as a parking space when in fact it is not, thus resulting in a false positive. It is also possible that an existing parking space is not detected as a parking space, thus resulting in a false negative.

[0009] Based on this, the object of the invention is to provide a method with improved parking space detection.

[0010] This object is achieved by a method having the features of independent claim 1. Preferred embodiments are the subject of the dependent claims. A computer program is the subject of independent claim 9. A machine-readable storage medium is the subject of independent claim 10. A parking assistance device is the subject of independent claim 11.

[0011] According to a first aspect, the invention relates to a method, in particular a computer-implemented method, for determining a parking space for a vehicle parking maneuver. The method comprises the following steps:

[0012] The vehicle's surroundings are recorded using a sensor unit. Any of the vehicle's sensors can be used as the sensor unit, such as ultrasonic sensors, radar sensors, cameras, LIDARs, etc.

[0013] A potentially usable parking space is then detected in the detected vehicle environment using a geometric planning method. In this context, a geometric planning method is understood to mean parking space detection using recognized geometries from the detected environment, in particular spatial geometric shapes or patterns. It is therefore, in particular, parking space detection based on a geometric representation of the vehicle's surroundings. The detected potentially usable parking space includes information such as parking space type, parking area, relative position to the vehicle, orientation, and / or parking space size. When detecting the potentially usable parking space, the length and width of the vehicle are taken into account, for example.

[0014] Preferably, the potentially usable parking space is detected step by step using the geometric planning method, in which the parking space's size is adapted to the determined geometries, i.e., to the determined boundaries. Purely as an example, an initially determined parking space orientation is changed by rotating an initial parking space contour, with a quality criterion used to check whether the usability of the parking space has changed, or in particular improved, as a result of this change. Furthermore, a potentially usable parking space in the detected vehicle environment is detected using a trained neural network-based planning method. The trained neural network-based planning method is thus particularly designed to evaluate potentially available parking spaces for a parking maneuver in the vehicle environment detected by the sensor unit.Preferably, the trained neural network for parking space determination is provided with input data containing at least one piece of information about the parking space present in the vehicle's surroundings. The trained neural network is particularly designed to output information about a potential parking space for parking space determination based on the input.

[0015] Neural networks can be used to efficiently process input data from various sensors, such as radar, lidar, or ultrasonic sensors, as well as cameras. Using the input data from the respective environmental sensors, the neural network can be trained to detect potential parking spaces in the data about the vehicle's surroundings transmitted by the environmental sensors. However, the statements made by trained neural networks are often associated with uncertainty. The input data from the respective environmental sensors can be subject to a variety of problems, such as insufficient sensor resolution, problematic weather conditions, or rarely occurring scenarios. In addition, certain objects may be obscured by other objects and thus not visible in the input data.Similar problems generally arise when making predictions using neural networks. In addition to various problems concerning the input data for the networks, errors in making the respective predictions can also result, for example, from inadequate or faulty training processes. Another issue concerns how to deal with faulty predictions from neural networks. Such misjudgments can generally have far-reaching consequences, especially if the prediction made is the starting point for further considerations. An automated parking vehicle, in particular, is particularly affected by this. It is therefore of key importance to ensure a low error rate and warning mechanisms when correct predictions cannot be reliably made.

[0016] Against this background, a parking space detected by the geometric planning method is verified using a parking space detected by the trained neural network-based planning method, or vice versa. For example, the verification of which potentially detected parking space is determined by the faster available result in order to ensure computational efficiency.

[0017] The verified parking space is defined as a potential parking option. The specified parking space can be offered to the driver, for example, via an HMI. It is also conceivable to automatically evaluate a suitable parking trajectory to the specified parking space and then automatically park the vehicle in the specified parking space.

[0018] A technical advantage of the present invention is that various types of uncertainties in the geometric and neural network-based planning process are taken into account and reliably and systematically identified through verification. This increases the robustness and, at the same time, improves the accuracy of the parking space determination for a variety of parking situations. This allows additional parking spaces to be parked that would otherwise be rejected by conventional parking assistance systems.

[0019] The neural network of the neural network-based planning method can be trained using at least one dataset containing labeled images of various parking situations. For example, the labeled images include different parking space types, such as parallel, perpendicular, and / or angled parking spaces, different ground conditions, line markings, and / or different adjacent objects. The trained neural network of the neural network-based planning method is, for example, a convolutional neural network. However, any other suitable architecture and / or training method can be used.

[0020] According to a preferred embodiment, the verification is carried out based on a comparison of the respective parking space positions, relative position of the parking space to the own vehicle, orientation and / or the parking space type.

[0021] According to a further preferred embodiment, the potentially usable parking space determined by the geometric planning method is verified, in particular optimized, based on the potentially usable parking space determined by the neural network-based planning method. A resulting advantage is the reduction of false-positive statements that frequently occur in the geometric planning method. Alternatively, the potentially usable parking space determined by the neural network-based planning method is verified, in particular optimized, based on the potentially usable parking space determined by the geometric planning method. In this way, false-negative statements can be increasingly reduced in the neural network-based planning method based on the verification.When optimizing the identified parking space, for example, a parking space contour is adjusted in size, i.e., enlarged or reduced, and / or rotated around a vertical axis, and / or adjusted in orientation, and / or shifted in the parking space contour, and / or the evaluated parking space type is changed, in particular, corrected. This achieves the most precise possible statement regarding the actually available parking space.

[0022] In an advantageous embodiment, an initial parking space contour and an initial parking space orientation are determined for the initially detected, potentially usable parking space using the geometric and trained neural network-based planning method. The initial parking space contour can, for example, be calculated from the information available from the initial detection of at least a partial area of ​​the parking space. For example, the initial parking space contour can be a rectangular area whose size is derived at least partially from the initially detected information about the parking space and preferably also partially from vehicle-specific information, for example the length and / or width of the vehicle to be parked. Initial parking space orientation is understood to mean the horizontal rotational position of this parking space contour, whereby the rotational position can refer to a reference axis.In the next step, a quality criterion for the potentially usable parking space is calculated.

[0023] To determine the verified parking space, the initial parking space orientation and / or initial parking space contour determined using the trained neural network-based planning procedure is adjusted. The adjustment is performed based on the initial parking space orientation and / or initial parking space contour determined by the geometric planning procedure. A modified quality criterion is calculated based on the adjusted parking space orientation and / or parking space contour, and it is checked whether the quality criterion has improved based on the adjusted parking space orientation and / or parking space contour.

[0024] Preferably, the above steps of changing the parking space orientation, calculating a modified quality criterion, and checking the improvement in the quality criterion are then repeated iteratively until a termination criterion is reached. The termination criterion can be, for example, that the quality criterion no longer improves significantly, i.e., a maximum or essentially a maximum of the quality criterion has been reached. As an exemplary alternative, a threshold can be specified as the termination criterion, so that the iterations are terminated when the quality criterion exceeds the threshold.

[0025] Subsequently, a parking space with a final adjusted parking space contour and / or final adjusted parking space orientation is defined as a verified parking space based on the highest quality criterion determined.

[0026] The verification steps described above with the geometric planning method can also be advantageously implemented with the neural network-based planning method for the reasons mentioned, whereby in this case the potentially determined parking space of the geometric planning method must again be used as the basis for the verification.

[0027] According to a preferred embodiment, the potentially usable parking space of the geometric or the trained neural network-based planning method is defined as a potential parking option, depending on which of the planning methods provides the potentially usable parking space more quickly. This avoids computational downtime and thus ensures that the driver can be offered a parking space as quickly as possible. Furthermore, it is preferred that the defined, potentially usable parking space be replaced by the determined, verified parking space after verification is complete, so that the originally provided parking space is output in an optimized, in particular more precise or corrected, manner.

[0028] According to another preferred embodiment, if one of the two planning methods fails, the potentially usable parking space detected by the other planning method is defined as a potential parking option. Verification is not possible in this case, but a complete failure of the parking space determination and, in particular, the parking process can be prevented.

[0029] A further subject of the invention relates to a parking assistance device which is designed to carry out the method according to the preceding description.

[0030] The parking assistance device may, in particular, comprise a microcontroller or processor, a central processing unit (CPU), a graphics processor (GPU), a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and the like, as well as software for performing the corresponding method steps. The present invention may thus be implemented in digital electronic circuits, computer hardware, firmware, or software.

[0031] Furthermore, the present invention also includes a computer program with program code for carrying out the method according to the invention when the computer program is executed in a computer or other programmable computer known from the prior art.

[0032] Furthermore, the present invention comprises a computer-readable storage medium comprising instructions which cause the computer on which they are executed to perform a method according to at least one of the preceding claims.

[0033] Further developments, advantages, and possible applications of the invention will become apparent from the following description of exemplary embodiments and from the figures. All described and / or illustrated features, individually or in any combination, are fundamentally part of the invention, regardless of their summary in the claims or their reference back to them. The content of the claims is also incorporated into the description.

[0034] The invention is explained in more detail below with reference to exemplary embodiments and the figures. They show:

[0035] Figure 1 is a schematic plan view of an ego vehicle with a

[0036] Computing unit for parking space determination;

[0037] Figure 2 is a schematic representation of an exemplary

[0038] Parking situation in which the ego vehicle from Figure 1 is parked in a parking space determined by the computing unit;

[0039] Figure 3 is a flowchart illustrating the steps of the parking space determination process. In the figures, identical elements are always provided with the same reference numerals.

[0040] Figure 1 shows a schematic plan view of an ego vehicle 1 having a parking assistance device 2. The parking assistance device 2 is configured to perform a parking process at least partially automatically. The parking assistance device 2 comprises a sensor unit 3, which is configured to detect only a partial area of ​​the surrounding area of ​​the ego vehicle 1 or the entire surrounding area of ​​the vehicle 1 (360° detection). The sensor unit can comprise any sensor type, for example, ultrasonic sensors, one or more radar sensors, one or more cameras, and / or one or more lidar sensors.

[0041] The sensor unit 3 is coupled to a computing unit 4, which has at least one processor and at least one memory unit. The computing unit 4 comprises a first parking space detection unit and a second parking space detection unit. The first parking space detection unit is designed to detect a potentially usable parking space in the detected surrounding area of ​​the ego vehicle 1 using a geometric planning method. The second parking space detection unit is designed to detect a potentially usable parking space in the detected surrounding area of ​​the ego vehicle 1 using a trained neural network-based planning method. Thus, two independent determinations of at least one potentially usable parking space for a parking maneuver of the ego vehicle 1 are carried out, based on different planning methods.The at least one parking space detected by the geometric planning method is verified using the at least one parking space detected by the trained neural network-based planning method, or vice versa. Verification in this context is understood to mean the proof provided that the potential parking space identified by one planning method has been confirmed and / or optimized based on the potential parking space identified by the other planning method. For example, verification is performed to determine whether the identified potentially usable parking space is actually a parking space or usable parking space. Alternatively or optionally, the identified parking space size, parking space orientation, and / or parking space type can be verified.

[0042] The neural network for applying the method according to the invention is thus already trained. Training can be carried out in a variety of ways known from the prior art, in particular adapted to the respective type of neural network used for a specific application.

[0043] Possible training data for training the neural network for the neural network-based planning process are manually labeled images. The labeled images comprise a vehicle environment with freely available parking spaces, taking into account different parking space types, line markings, and adjacent objects. In these images, for example, the corners of a rectangular parking space are labeled. Of course, errors cannot be ruled out during labeling. For example, if a parking space boundary is set at a wheel stop such as a curb, over which the ego vehicle 1 could protrude, the available parking space is set smaller than in reality.Combining the geometric and trained neural network-based planning methods not only achieves synchronous detection of an available parking space, but also reduces the false positives that occur in the geometric planning method and, at the same time, the false positives that occur in the neural network-based planning method. Computing unit 4 is configured to specify the verified parking space Pver as the parking option.

[0044] Figure 2 shows, by way of example, a parallel parking situation detected by the computing unit 4 of the ego vehicle 1. The parallel parking space P is bounded, purely by way of example, longitudinally on the rear side by an object O, on a first longitudinal side of the parking space P by an adjacent, additional potentially usable parking space F and an object O, and on a second longitudinal side of the parking space P by two objects O. The objects O can be any conceivable obstacles or boundaries; purely by way of example, a pillar, a curb, and another vehicle are mentioned.

[0045] To determine the parallel parking space P, the surrounding area of ​​the vehicle 1 is detected in a first step using the sensor unit 3. In this exemplary embodiment, the parallel parking space P was detected as a potentially usable parking space based on the detected surrounding area by the neural network-based planning method and the geometric planning method, respectively. When detecting a potentially usable parking space P, the parking space position, the relative position of the parking space to the ego vehicle 1, the orientation, and / or the parking space type are evaluated. The detection of the parking space thus includes an initially determined parking space contour, parking space orientation, and an initially determined parking space type, if possible.

[0046] The parking space determined by the geometric planning method is purely schematic and represented for explanatory purposes by three schematic boundary lines Pgeo. One of the three boundary lines Pgeo, which runs antiparallel to the actual long side of the parking space P, is a false statement. The orientation of this incorrectly placed boundary line would impair the orientation when determining the final parking space based on the purely geometrically based planning method and influence the evaluation of the parking space size, orientation, and type. By verifying the parking space detected using the trained neural network-based planning method, represented by the boundary lines Pnn, sources of error in the parking space determination can be excluded or at least mitigated. In this way, improved use of the available parking space is achieved.

[0047] The ego vehicle 1 is finally parked in the verified and defined parallel parking space Pver manually or at least partially automatically. Preferably, the verified parking space is continuously optimized, in particular, adapted to updated environmental information acquired by the environmental sensors. In this way, missing environmental information at the beginning of the parking process can often be compensated for during the parking process.

[0048] Preferably, trajectory planning is initiated or a parking space is offered to the driver for a parking maneuver as soon as one of the two planning methods has detected a potentially usable parking space. This ensures that the parking maneuver starts as early as possible, thus avoiding unnecessary waiting times. As soon as the second planning method has completed the detection of a potentially usable parking space, the offered parking space or the parking space determined for trajectory planning is verified against the parking space detected by the other planning method. This confirms and / or optimizes the parking area.

[0049] Figure 3 shows a schematic flow chart illustrating the processes of the parking space determination method for a parking process of the ego vehicle 1.

[0050] In the method, the vehicle 1 comprises the sensor unit 3, by means of which the vehicle surroundings of the vehicle 1 are detected S100. The sensor unit 3 is coupled to the computing unit 4. The computing unit 4 detects a potentially usable parking space Pgeo in the vehicle surroundings using the geometric planning method S200 on the basis of the data transmitted by the sensor unit 3. Furthermore, the computing unit 4 detects a potentially usable parking space Pnn in the detected vehicle surroundings using a trained neural network-based planning method S300 on the basis of the data transmitted by the sensor unit 3. The computing unit verifies a parking space detected by the geometric planning method using a parking space detected by the trained neural network-based planning method or vice versa S400.Verification can be performed based on a value comparison of the determined parking space positions, the relative position of the parking space to the ego vehicle, the orientation, and / or the determined parking space type. If the comparison of one, several, or all determined values ​​of the geometric and trained neural network-based planning process results in a deviating actual result outside a tolerance range, the potentially usable parking space is discarded and recalculated. In the next step, the verified parking space is defined as a potential parking option (S500).

[0051] The invention has been described above using exemplary embodiments. It is understood that numerous changes and modifications are possible without departing from the scope of protection defined by the patent claims.

[0052] List of reference symbols

[0053] 1 Vehicle 2 Parking assistance device

[0054] 3 Sensor unit

[0055] 4 Computing unit

[0056] P parking space

[0057] Pnn Potentially usable parking space determined using a trained neural network-based planning method

[0058] Pgeo Potentially usable parking space determined by means of a geometric

[0059] Planning process

[0060] Pver Verified Parking

[0061] O Object F Additional potentially usable parking space

Claims

Patent claims 1. Method, in particular a computer-implemented method, for determining a parking space for a parking process of a vehicle (1), comprising the method steps: - detecting a vehicle environment of the vehicle (1) using a sensor unit (3) (S100), - Detection of a potentially usable parking space (Pgeo) in the detected vehicle environment using a geometric planning method (S200), - Detection of a potentially usable parking space (Pnn) in the detected vehicle environment using a trained neural network-based planning method (S300), - Verification of a parking space detected by the geometric planning method (Pgeo) using a parking space detected by the trained neural network-based planning method (Pnn) or vice versa (S400), - Specifying the verified parking space (P) as a potential parking option (S500).

2. Method according to claim 1, characterized in that the verification is carried out based on a comparison of the respective parking space positions, relative position to the own vehicle (1), orientation and / or the parking space type.

3. Method according to claim 1 or 2, characterized in that the potentially usable parking space (Pgeo) determined by the geometric planning method is optimized on the basis of the potentially usable parking space (Pnn) determined by the neural network-based planning method.

4. Method according to claim 1 or 2, characterized in that the potentially usable parking space (Pnn) determined by the neural network-based planning method is calculated on the basis of the geometric The potentially usable parking space (Pgeo) determined in the planning process is optimized.

5. Method according to claim 3 or 4, characterized in that in order to optimize the determined parking space (Pnn / Pgeo), a parking area contour is adjusted in size and / or rotated about a vertical axis and / or the parking area contour is shifted and / or the parking space type is changed.

6. Method according to claim 4 or 5, characterized in that by means of the geometric and neural network-based planning method, an initial parking area contour and an initial parking area orientation are determined for the initially detected, potentially usable parking space (Pnn / Pgeo), wherein a quality criterion for the potentially usable parking space (Pnn / Pgeo) is calculated, wherein to determine the verified parking space, the initial parking area orientation and / or initial parking area contour determined by means of the neural network-based planning method is adapted, wherein the adaptation is carried out on the basis of the initial parking area orientation and / or initial parking area contour determined by the geometric planning method, wherein a modified quality criterion is calculated based on the adapted parking area orientation and / or parking area contour, wherein it is checked,whether the quality criterion has improved based on the adjusted parking area orientation and / or parking area contour, whereby a parking space with a finally adjusted parking area contour and / or finally adjusted parking area orientation is defined as a verified parking space (Pver) based on the highest quality criterion determined.

7. Method according to one of the preceding claims, characterized in that the potentially usable parking space (Pgeo, Pnn) of the geometric or neural network-based planning method is defined as a potential parking option, depending on which of the Planning procedure provides the initially determined parking space more quickly, whereby the potentially usable parking space (Pgeo, Pnn) is replaced by the verified parking space (Pver) after verification has been completed.

8. Method according to one of the preceding claims, characterized in that if one of the two planning methods fails, the detected potentially usable parking space of the other planning method is defined as a potential parking option.

9. A computer program with program code for carrying out a method according to one of the preceding claims, when the computer program is executed by a computer.

10. A machine-readable storage medium on which the computer program according to claim 9 is stored.

11. Parking assistance device (2) which is configured to carry out the method according to one of claims 1 to 8.

Images