Parking system for one vehicle and vehicle

Abstract

The present invention relates to a parking system (5) for a vehicle (10) and a vehicle (10), wherein the parking system (5) has an evaluation unit (20) which is configured to receive image data captured by at least one camera (12) of the vehicle (10) and representing the vehicle's (10) surroundings, to determine a length (32) and a midpoint (34) of an entry line (30) of a parking space (40) within the image data, to determine an angle (50) between the entry line (30) and at least one side line (60) of the parking space (40) adjacent to the entry line (30), and, based on the length (32) and midpoint (34) of the entry line (30) and the angle (50) between the entry line (30) and the side line (60), to determine a lateral extent and a position of the parking space (40), wherein the position of the parking space (40) relates to a predefined reference coordinate system refers to,to compare information about the size of the vehicle (10) with the lateral dimensions of the parking space (40) in order to determine the suitability of the parking space (40) for a parking maneuver by the vehicle (10) and, based on information about the position of the vehicle (10) with respect to the reference coordinate system, to determine a trajectory (70) for parking the vehicle (10) into the parking space if the parking space (40) is suitable for the parking maneuver by the vehicle (10).

Description

State of the art

[0001] The present invention relates to a parking system for a vehicle and a vehicle with such a parking system.

[0002] In the prior art, parking systems for vehicles are known that are able to identify parking spaces in the vicinity of the vehicles based on environmental sensors of the vehicles and to carry out a parking process for the vehicles semi- or fully automatically.

[0003] Furthermore, it is known from the prior art to use neural networks trained using a supervised learning method to implement various vehicle functions. Disclosure of the invention

[0004] According to a first aspect of the present invention, a parking system for a vehicle is proposed, which includes an evaluation unit. The evaluation unit is designed, for example, as an ASIC, microcontroller, CPU, or as a different computing unit.

[0005] The vehicle in which the parking system according to the invention can be used is, for example, a car, a truck, a bus, a van, a motorcycle, a rail vehicle or a vehicle other than the above.

[0006] The evaluation unit is configured to receive image data captured by at least one camera of the vehicle, representing the vehicle's surroundings. Preferably, the image data is received via the vehicle's on-board network, which connects the evaluation unit directly or indirectly (e.g., via a vehicle control unit) to the at least one camera of the vehicle.

[0007] The evaluation unit is further configured to determine the length and center point of a parking space entry line within the image data. This is preferably done based on the recognition of the parking space's marking lines. The entry line is understood to be a marking of the parking space over which a vehicle typically enters the space.

[0008] The evaluation unit is further equipped to determine an angle between the entrance line and at least one side line of the parking lot adjacent to the entrance line, and, based on the length and center point of the entrance line and the angle between the entrance line and the side line, to determine a lateral extent and a position of the parking lot, wherein the position of the parking lot refers to a predefined reference coordinate system.

[0009] It should be noted that the determination of the aforementioned parameters (length and center of the entry line, angle between entry line and side line) can be carried out on the basis of image analysis methods known from the prior art and / or in a manner deviating from them.

[0010] The predefined reference system can, in principle, be any reference system, such as one related to the vehicle, its surroundings, or the parking space itself, without thereby restricting the reference system to the aforementioned variants. The lateral extent of the parking space can, for example, be determined under the assumption that the parking space is laterally bounded by two essentially parallel sidelines, so that, based on the position of the sideline determined by the evaluation unit, a second sideline opposite the first can be determined.

[0011] Furthermore, it is possible to determine the lateral extent of the parking lot based on additional information about the parking lot and / or its surroundings, even if the side lines do not run parallel to each other.

[0012] The evaluation unit is also designed to compare information about the vehicle's size with the lateral dimensions of the parking space to determine the suitability of the parking space for the vehicle to park. The vehicle's size information can be stored, for example, in a storage unit connected to the evaluation unit, either as model data representing the vehicle's geometry or in a different format. For instance, the vehicle's size could be represented by its maximum length, width, and / or height. Furthermore, it is conceivable that the evaluation unit could also receive information about the vehicle's maneuverability, which could be determined based on factors such as the vehicle's wheelbase and turning radius.

[0013] Finally, the evaluation unit is set up to determine a trajectory (i.e., a path of movement) for parking the vehicle in the parking space, based on information about the vehicle's position in relation to the reference coordinate system, if the parking space is suitable for the parking operation by the vehicle.

[0014] It should be noted that the evaluation unit may also be configured to perform one or more preprocessing steps on the received image data before determining the length and / or midpoint of the entry line and / or the angle to the at least one side line. Such preprocessing steps could include, for example, denoising and / or rectification and / or merging of image data and / or removing irrelevant areas from the image data, etc. Furthermore, it is possible to determine the aforementioned parameters (length and midpoint of the entry line and the angle between the entry line and the at least one side line) based on features that were automatically and / or manually extracted from the image data (e.g., the respective corner points of the parking lot, from which the aforementioned parameters can be derived).

[0015] The parking system according to the invention offers the particular advantage that, by using the length of the entry line, the midpoint of the entry line, and the angle between the entry line and at least one side line, it enables a particularly reliable determination of the suitability of a given parking space for the vehicle. This is due, among other things, to the fact that even parking spaces whose markings, which are usually circumferential, are partially obscured (e.g., due to being covered by other vehicles parked in adjacent spaces) can be reliably assessed, since the method does not require information about the length of the side line and / or about the side line opposite the entry line.

[0016] By assessing the suitability of a parking space as soon as the driver approaches a potentially usable parking space, comfort for the driver can be increased and / or time saved, as the driver can decide on a suitable parking space early on based on the result determined by the evaluation unit.

[0017] The dependent claims describe preferred embodiments of the invention.

[0018] Preferably, the angle between the entrance line and the side line is a first angle, and the side line is a first side line. The evaluation unit is also configured to determine a second angle, which is the angle between the entrance line and a second side line of the parking lot adjacent to the entrance line. Furthermore, the evaluation unit is configured to determine the lateral extent and / or the location of the parking lot based on this second angle. In this way, it is possible to determine the lateral extent and the location of the parking lot with greater reliability, since the second side line does not need to be estimated in this case.

[0019] In a preferred embodiment of the present invention, the evaluation unit is configured to determine the lateral extent and / or the location of the parking space and / or the suitability of the parking space for the vehicle based on a neural network, wherein the neural network is a neural network trained on the basis of a large number of image data, and wherein the image data in particular represent a large number of different parking spaces. By using the parameters described above to evaluate the suitability of the parking space, the requirements placed on the neural network with regard to its complexity and / or its memory requirements, etc., are relatively low, so that the neural network can be implemented based on a cost-effective evaluation unit and / or a cost-effective storage unit.

[0020] In the case where, as described above, a neural network is used to determine the suitability of a parking space for a vehicle, the neural network advantageously has a common input part (also called a "backbone") which, in a deeper layer of the neural network, branches into a first head and a second head. The first head is trained on the basis of information about the length and midpoint of the entry line and at least one angle to at least one side line (preferably, however, both angles to the respective side lines), while the second head, in addition to the information from the first head, is trained on the length of at least one side line adjacent to the entry line and is configured to determine the longitudinal extent of the parking space.In this way, the suitability of the parking space for the vehicle can be determined with particularly high reliability.

[0021] Furthermore, the second head of the neural network is preferably trained, at least partially, on image data whose sidelines are only partially visible due to occlusion. Based on this, the second head is configured to estimate the longitudinal extent of the parking lot in the case of partial occlusion of at least one sideline.

[0022] It should be noted that the heads of the neural network can be trained separately. Alternatively, both heads of the neural network can be trained using the same image data by implementing an optimizer that weights the outputs of the first and second heads differently, according to their respective functions.

[0023] In a further preferred embodiment of the present invention, the evaluation unit is configured, based on information from at least one additional sensor of the vehicle, to determine and / or validate the longitudinal extent of the parking space and / or to monitor the remaining distance to a longitudinal boundary of the parking space during a parking maneuver. The at least one sensor corresponds, for example, to an ultrasonic sensor or a plurality of ultrasonic sensors designed to determine the distance to obstacles in the vicinity of the vehicle. By using the at least one additional sensor, the suitability of a parking space for the vehicle can be assessed more reliably and / or the parking maneuver can be carried out more quickly.

[0024] The evaluation unit is preferably configured to receive the information described above from the at least one additional sensor, namely the ultrasonic sensor and / or radar sensor and / or lidar sensor and / or another camera of the vehicle described above by way of example. For this purpose, the evaluation unit is preferably connected to the one or more additional sensors directly and / or indirectly (e.g., via a sensor control unit) via the vehicle's electrical system.

[0025] Furthermore, the evaluation unit is advantageously designed to identify a large number of suitable parking spaces in the vicinity of the vehicle based on the image data and to output information about these spaces to a user of the parking system. This output can be displayed, for example, on a screen (e.g., a touchscreen) of the vehicle's on-board computer system, instrument cluster, and / or head-up display.

[0026] According to a second aspect of the present invention, a vehicle is proposed which has a parking system according to the first aspect of the invention. As already described above, the vehicle can be, for example, a passenger car, a truck, a bus, a van, a motorcycle, a rail vehicle, or a vehicle other than these. The vehicle has at least one camera (and preferably a plurality of cameras) and a system for at least semi-automatic or fully automatic driving operation. The parking system is configured to receive the image data from the at least one camera and to transmit the trajectory determined according to the invention for a parking maneuver to the system for semi-automatic or fully automatic driving operation.The features, combinations of features and the advantages arising from them correspond so clearly to those described in connection with the first-mentioned aspect of the invention that reference is made to the above statements to avoid repetition.

[0027] In an advantageous embodiment, the at least one camera of the vehicle according to the invention is a component of a surround-view system of the vehicle, which consists of a plurality of cameras distributed around the vehicle, while the parking system is configured to receive the image data from the surround-view system and process it according to the invention. Brief description of the drawings

[0028] Exemplary embodiments of the invention are described in detail below with reference to the accompanying drawings. The drawing shows: Fig. 1 a schematic view of an exemplary embodiment of a vehicle according to the invention in conjunction with a parking system according to the invention; and Fig. 2 a block diagram of a further exemplary embodiment of a parking system according to the invention. Embodiments of the invention

[0029] Fig. Figure 1 shows a schematic view of an exemplary embodiment of a vehicle 10 according to the invention, which is designed here as a passenger car, in conjunction with a parking system 5 according to the invention.

[0030] In the vicinity of vehicle 10 there are a number of parking spaces 40, 40', 40".

[0031] The parking system 5 has an evaluation unit 20 designed as an ASIC, which is configured to receive image data captured by a multitude of cameras 12, 14, 16, 18 of the vehicle 10 and representing the vehicle 10's surroundings. For this purpose, the evaluation unit 20 is connected via the vehicle 10's onboard network (not shown) to a surround-view system 100 (not shown) (see Fig. 2) connected, which in turn is connected to the individual cameras 12, 14, 16, 18 and is set up to combine the image data of the individual cameras 12, 14, 16, 18 into a 360° image.

[0032] The evaluation unit 20 is set up to determine a length 32 and a midpoint 34 of an entry line 30 of the parking lot 40 within the image data.

[0033] The evaluation unit 20 is further set up to determine a first angle 50 between the entry line 30 and a first side line 60 of the parking lot 40 adjacent to the entry line 30 and a second angle 55 between the entry line 30 and a second side line 65 of the parking lot 40 adjacent to the entry line 30.

[0034] The evaluation unit 20 is further equipped to determine a lateral extent and a position of the parking space 40 on the basis of the length 32 and the midpoint 34 of the entry line 30 and on the basis of the angles 50, 55 between the entry line 30 and the respective side lines 60, 65, wherein the position of the parking space 40 refers to a predefined reference coordinate system, which here corresponds to a vehicle coordinate system of the vehicle 20.

[0035] Furthermore, the evaluation unit 20 is equipped to compare information about the size of the vehicle 10 with the lateral extent of the parking space 40 in order to determine the suitability of the parking space 40 for a parking maneuver by the vehicle 10.

[0036] For this purpose, a neural network 80 is implemented using the evaluation unit 20. The evaluation unit 20 is configured to determine the lateral extent and location of parking space 40 and its suitability for vehicle 10. The neural network 80 is trained on a large number of image data, with the image data used for training representing a large number of different parking spaces 40. Furthermore, the neural network 80 is partially trained on image data whose side lines 60, 65 are only partially visible due to occlusion.

[0037] Since, due to the current position of the vehicle 10 in relation to the potentially suitable parking space 40, there is a hidden area 110 which is not currently detectable by any of the cameras 12, 14, 16, 18, the suitability of the parking space 40 for the vehicle 10 can nevertheless be determined with high reliability on the basis of the parking system 5 according to the invention, since according to the invention not all parking space markings and at least not in their entirety need to be detected.

[0038] This offers the advantage that the parking system 5 according to the invention can determine, even when approaching a potentially usable parking space (here parking space 40), whether it is suitable for the vehicle 10 or not.

[0039] Since parking space 40 is deemed suitable for vehicle 10 in this case, the evaluation unit 20 is configured to determine a trajectory 70 for parking vehicle 10 in parking space 40. Preferably, vehicle 10 is configured to transmit the determined trajectory 70 to a system 90 (see Fig. 2) to transmit for an automatic driving operation which is set up on the basis of trajectory 70, to carry out the parking process automatically.

[0040] The evaluation unit 20 is configured, based on information from an ultrasonic sensor 140 of the vehicle 10, to determine the longitudinal extent 42 of the parking space 40 during the parking maneuver and to assess the suitability of the parking space 40 for the vehicle 10 based on this determined longitudinal extent 42. If it turns out that the parking space 40 is not suitable for the vehicle 10 due to its length, the evaluation unit 20 is also configured to abort any parking maneuver that has already begun and to issue a corresponding notification to the driver of the vehicle 10.

[0041] Fig. Figure 2 shows a block diagram of a further exemplary embodiment of a parking system 5 according to the invention for a vehicle 10 (see Figure 2). Fig. 1), which is equipped to receive image data from a surround view system 100 of the vehicle 10, wherein the surround view system 100 is connected to a multitude of cameras 12, 14, 16, 18 via information technology.

[0042] An evaluation unit 20 of the parking system 5 comprises a preprocessing unit 120, a neural network 80, and a postprocessing unit 150. The preprocessing unit 120 is configured here to preprocess image data received from the surround view system 100 for further processing by the neural network 80 (e.g., by removing irrelevant image areas, etc.).

[0043] The neural network 80 has an input part 82, which in a deeper layer of the neural network 80 branches into a first head 84 and a second head 86, wherein the first head 84 is trained on the basis of information about a length 32 and a midpoint 34 of an entry line 30 of a parking lot 40 and about respective angles 50, 55 to respective side lines 60, 65 of the parking lot (see each Fig. 1) and wherein the second head 86, in addition to the information of the first head 84, is trained and equipped on the basis of length information of at least one side line 60, 65 adjacent to the entry line 30, to determine a longitudinal extent 42 of the parking lot 40.

[0044] The evaluation unit 20 is finally set up, a trajectory 70 (see Fig.1) to determine the parking maneuver of the vehicle in parking space 40 by means of the post-processing unit 150 on the basis of the respective outputs of the first head 84 and the second head 86 and to transmit these to a system 90 for automatic driving operation.

Claims

[1] Parking system (5) for a vehicle (10) comprising an evaluation unit (20), wherein the evaluation unit (20) is configured, - To receive image data which were captured by at least one camera (12) of the vehicle (10) and represent an environment of the vehicle (10), - to determine a length (32) and a midpoint (34) of an entry line (30) of a parking lot (40) within the image data, - to determine an angle (50) between the entrance line (30) and at least one side line (60) of the parking lot (40) adjacent to the entrance line (30), - to determine a lateral extent and a position of the parking space (40) on the basis of the length (32) and the midpoint (34) of the entrance line (30) and on the basis of the angle (50) between the entrance line (30) and the side line (60), wherein the position of the parking space (40) refers to a predefined reference coordinate system, - to compare information about the size of the vehicle (10) with the lateral dimensions of the parking space (40) in order to determine the suitability of the parking space (40) for a parking maneuver by the vehicle (10), and - based on information about the position of the vehicle (10) in relation to the reference coordinate system, to determine a trajectory (70) for parking the vehicle (10) in the parking space, if the parking space (40) is suitable for the parking operation by the vehicle (10). [2] Parking system (5) according to claim 1, wherein - the angle between the entry line (30) and the side line (60) is a first angle (50) and the side line is a first side line (60), and - the evaluation unit (20) is set up, - to determine a second angle (55) which is an angle between the entry line (30) and a second side line (65) of the parking lot (40) adjacent to the entry line (30), and - to determine the lateral extent and / or the location of the parking space (40) additionally on the basis of the second angle (55). [3] Parking system (5) according to one of the preceding claims, wherein - the evaluation unit (20) is set up to determine the lateral extent and / or the location of the parking space (40) and / or the suitability of the parking space (40) for the vehicle (10) on the basis of a neural network (80), and - the neural network (80) is a neural network (80) trained on the basis of a large number of image data, where the image data in particular represent a large number of different parking spaces (40). [4] Parking system (5) according to claim 3, wherein - the neural network (80) has an input part (82) which branches into a first head (84) and a second head (86) in a deeper layer of the neural network (80), - the first head (84) is trained on the basis of information about the length (32) and the midpoint (34) of the entry line (30) and about at least one angle (50, 55) to at least one sideline (60, 65), and - the second head (86) is trained and equipped to determine a longitudinal extent (42) of the parking lot (40) in addition to the information of the first head (84) based on length information of at least one sideline (60, 65) adjacent to the entry line. [5] Parking system (5) according to claim 4, wherein the second head (86) of the neural network (80) - is trained at least partially on image data whose sidelines (60, 65) are only partially visible due to occlusion, and - is set up to estimate the longitudinal extent (42) of the parking space (40) in the case of partial obstruction (110) of at least one side line (60, 65). [6] Parking system (5) according to one of the preceding claims, wherein the evaluation unit (20) is set up based on information from at least one additional sensor of the vehicle (10), - to determine and / or to validate the longitudinal extent (42) of the parking lot (40), and / or - to monitor the remaining distance to a longitudinal boundary of the parking space (40) during a parking maneuver of the vehicle (10). [7] Parking system (5) according to claim 6, wherein the evaluation unit (20) is configured to receive information from at least one additional sensor of - an ultrasonic sensor (140), and / or - a radar sensor, and / or - a lidar sensor, and / or - to be received by another camera of the vehicle (10). [8] Parking system (5) according to one of the preceding claims, wherein the evaluation unit (20) is set up, - to identify a large number of suitable parking spaces (40, 40', 40") in the vicinity of the vehicle (10) based on the image data, and - To provide information about the multitude of suitable parking spaces (40, 40', 40") to a user of the parking system (5). [9] Vehicle (10) comprising: - a parking system (5) according to any one of the preceding claims, - at least one camera (12), and - a system (90) for at least semi-automatic or fully automatic driving operation, wherein the parking system is set up, - to receive the image data from at least one camera (12, 14, 16, 18), and - to transmit the determined trajectory (70) for a parking operation to the system (90) for semi-automatic or fully automatic driving operation. [10] Vehicle (10) according to claim 9, wherein - the camera (12) is a component of a surround view system (100) of the vehicle (10), which consists of a large number of cameras (12, 14, 16, 18) distributed around the vehicle (10), and - the parking system (5) is set up to receive image data from the surround view system (100).

Images