Method and device for operating a parking assistance system, parking lot, and vehicle

Abstract

A method is proposed for operating a parking assistance system (105) of a vehicle (100), comprising the steps of a) projecting (S1) a predetermined pattern (PAT1-PAT6) onto a predetermined area (205), in particular an area (205) beside the vehicle (100), b) capturing (S2) an image such that at least a part of the predetermined area (205) with the projection (220) is visible in the captured image, c) determining (S3) an object (210) located in the predetermined area (205) based on the captured image, and d) updating (S4) a digital map (MAP0, MAP1) of the surroundings using the captured object (210).

Description

[Technical field]

[0001] The present invention relates to a method and a device for operating a parking assistance system of a vehicle, a parking lot having such a device, and a vehicle. [Background technology]

[0002] To make the parking process more efficient for the vehicle user, it is desirable to automate parking. This can be called automated valet parking. In this case, the user transfers the vehicle at a waypoint to the automated valet parking system, which takes over control of the vehicle and autonomously controls the vehicle to an available parking space and parks it there. Thus, the user can also take over the vehicle again at the waypoint. Such automated valet parking systems use, for example, sensors located on the exterior of the vehicle, in particular cameras, radar devices, and / or lidar devices, to capture the vehicle and its surroundings. In this way, control signals are output to the vehicle based on the captured data, which controls the vehicle. These systems are advantageous not only for the user, but also for the operator of the parking lot or parking area, since they can optimize space utilization. Furthermore, any remotely controllable vehicle can be used in such systems, the vehicle itself does not require complex technology for capturing and controlling the surroundings.

[0003] One known problem in such systems is that smaller and / or moving obstacles, especially living beings such as children or animals, are poorly or inaccurately captured by the sensors. Nevertheless, to ensure a sufficient level of safety, a very high technical price must be paid, for example a very large number of cameras are used, which makes the system very complex and expensive.

[0004] If the vehicle itself has sensors and an autonomous control unit for capturing the surroundings, the vehicle can drive autonomously. In the case of automated valet parking, the control unit does not have an orientation determination, so knowledge of a map of the parking area or parking lot is additionally required. Even if such a map is provided, localization of the vehicle is only achieved by the use of complex means, for example when the vehicle enters the parking area or parking lot. In particular, notices or signs that can be captured by sensors and that identify a unique position in the parking area or parking lot must be distributed and placed. Localization by means such as GPS is not possible or not possible with sufficient accuracy inside the building, especially if the parking lot has several floors and they are arranged one above the other.

[0005] U.S. Patent Application Publication No. 2020 / 0209886 discloses a system and method in which laser scanners located on the ceiling of a parking lot project a path onto a road that is used to guide an autonomous vehicle. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US Patent Application Publication No. 2020 / 0209886 Summary of the Invention

[0007] Against this background, one of the objects of the present invention is to improve the operation of vehicle parking assistance systems.

[0008] According to a first aspect, a method for operating a parking assistance system of a vehicle is proposed, the method comprising: a) projecting a predetermined pattern onto a predetermined area, in particular an area beside a vehicle; b) capturing an image, where at least a portion of the predetermined area having the projection is visible in the captured image; c) identifying an object located in the predetermined area according to the captured image; and d) updating the digital surroundings map using the captured objects.

[0009] This method has the advantage that objects located within the lane of the vehicle can be captured with greater reliability and accuracy, thus increasing the safety level during the operation of the parking assistance system, in particular during the autonomous driving of the vehicle, such as during the automatic parking process.

[0010] The term "parking assistance system" is in this case understood to mean any system which assists and / or controls a vehicle during a parking process, in particular during an autonomously performed parking process. The parking assistance system may comprise a unit integrated in the vehicle, may comprise a unit located in an infrastructure, for example in a parking garage, and / or may comprise several decentrally located units having a functional and / or communication connection to each other.

[0011] The parking assistance system is particularly configured to autonomously control and / or drive a vehicle. If the parking assistance system is located outside the vehicle, it can also be referred to as remote control. The parking assistance system preferably has automation level 4 or 5 according to the SAE classification scheme. The SAE classification scheme was published in 2014 by the automotive standardization organization SAE International as J3016, "Classification and definition of terms related to road vehicle automated driving systems". It is based on six different degrees of automation, taking into account the level of required intervention of the system and the required attention of the driver. The SAE degrees of automation range from level 0, which corresponds to a fully manual system, to levels 1-2 driver assistance systems, partially autonomous systems (levels 3 and 4), and fully autonomous systems where a driver is no longer required (level 5). An autonomous vehicle (also known as a driverless car, self-driving car, or robotic car) is a vehicle that can sense its surroundings and navigate its surroundings without human input, conforming to SAE automation level 5.

[0012] A first step a) of the method comprises projecting a predetermined pattern onto a predetermined area, for example an area in a car park, in particular an area next to a vehicle.

[0013] The predetermined pattern includes optical features arranged in a predetermined manner, for example lines arranged according to geometric rules, which may be straight or curved, open or form a closed shape. Predetermined patterns include, for example, checkerboard patterns, diamond patterns, circles, triangles, wavy lines, etc. Different patterns of these patterns can be combined to form new patterns. The predetermined pattern does not necessarily include lines as optical features, but may be a dot pattern, etc.

[0014] The predetermined pattern is preferably projected such that the distance between two adjacently located optical features of the pattern, e.g. two lines, is between 5 and 30 cm, preferably between 5 and 20 cm, preferably between 5 and 15 cm, more preferably less than 13 cm, even more preferably less than 11 cm. The closer the optical features are to each other, the smaller the object can be captured. However, the number of optical features required to fully illuminate an area increases, which increases the resolution required when capturing the projection and the computational power required to identify the object.

[0015] In a preferred embodiment, the predetermined pattern is designed such that an object having a minimum size of 11 cm is captured by the pattern.

[0016] The predetermined pattern may be generated and projected by a projection unit, in particular a laser projector, located on the vehicle or on the exterior of the vehicle in an infrastructure. The projection unit may comprise an LCD unit, a microlens array, and / or a micromirror array. The projection unit may be configured to scan a laser beam to project the predetermined pattern.

[0017] The predefined area onto which the pattern is projected includes in particular the future lane or trajectory of the vehicle. The projection may be independent of the presence of the vehicle. The predefined area is preferably located by the vehicle, but is for example located in front of the vehicle or behind the vehicle. The area may also extend laterally around the vehicle. For example, the area extends over several meters, for example 5 meters, in front of the vehicle. The area may in particular extend up to the vehicle and include the vehicle (or more precisely, the projection of the vehicle on the ground).

[0018] By projecting this predetermined pattern, the area is said to be scanned.

[0019] The predefined pattern is in particular projected at wavelengths in the spectral range of 250 nm to 2500 nm. Depending on the embodiment of the projection unit, the pattern can be projected with a broadband spectrum, a narrowband spectrum and / or a spectrum comprising multiple narrowband lines.

[0020] A second step b) of the method includes capturing an image, where at least a portion of the predetermined area having the projection is visible in the captured image.

[0021] The images may be captured by a capture unit, in particular a camera, located on the vehicle or outside the vehicle within the infrastructure.

[0022] The image is preferably captured using a certain minimum parallax relative to the light beam generating the projection, thus ensuring that changes in a given pattern due to objects located within a given area can be seen with high reliability and precision.

[0023] A third step c) of the method includes identifying an object located in the predetermined area according to the captured images.

[0024] Thus, when an object is located within an area having a projection, the projection of a given pattern will be altered or affected by the object, for example, shadowing will occur (i.e., individual optical features of the pattern will be absent in some parts of the image of the projection), some parts of one or more of the optical features of the pattern will be distorted (i.e., the affected optical features will extend to points other than those expected in the image of the projection), and / or local variations in the intensity of the optical features will occur due to changes in the angle of reflection.

[0025] Based on these changes in a predefined pattern that can be captured in the image of the projection, the presence of an object can be confirmed with little computational effort.

[0026] In an embodiment, a captured image of the projection is compared to a predetermined pattern, and a change in the predetermined pattern is indicative of an object within the area of ​​the projection.

[0027] A fourth step d) involves updating the digital surroundings map using the captured objects.

[0028] The digital surroundings map includes, inter alia, a digital representation of the vehicle's actual surroundings. The digital surroundings map is preferably based on a map reflecting construction conditions for the location, such as a site plan, a building plan, etc. The digital surroundings map may further include other road users, in particular other vehicles and moving objects such as pedestrians, captured by the sensors. Furthermore, the digital surroundings map may include road markings and / or other traffic management instructions, captured by the sensors. Furthermore, the digital surroundings map may include items of information of the underlying surface, such as configurations.

[0029] The digital surroundings map contains, inter alia, a coordinate system, the origin of which is for example permanently specified (world coordinate system) or the origin of which is fixed to a point on the vehicle.

[0030] The parking assistance system is in particular configured to perform path planning for the vehicle on the basis of the digital surroundings map, i.e. the parking assistance system plans the future trajectory of the vehicle on the basis of the digital surroundings map.

[0031] According to one embodiment of the method, step c) comprises: This includes capturing the distortion of the projected pattern in the captured image.

[0032] According to a further embodiment of the method, step a) comprises It involves projecting a predetermined pattern using a laser projector.

[0033] According to a further embodiment of the method, step a) comprises projecting a predetermined pattern using a predetermined color; Step b) is This involves capturing an image using a filter that is transparent to a given color.

[0034] A given color may, for example, include one or more specific wavelength ranges. Each wavelength range preferably includes a narrow range having a half-width of at most 20 nm, preferably at most 15 nm, more preferably at most 10 nm. Thus, a "specific color" may include multiple narrow wavelength ranges, for example corresponding to emission lines of a laser or the like.

[0035] This embodiment has the advantage that the signal-to-noise ratio with which the projection of the pattern can be captured by the capture unit can be increased, in particular if the filters used are narrow-band filters that are transparent only to one or more narrow wavelength ranges.

[0036] The term "transparent" is understood in this case to mean that the filter has a transmittance for the corresponding wavelength of more than 10%, preferably more than 50%, preferably more than 70%, more preferably more than 90%. The filter is preferably not transparent for colors other than the given color.

[0037] According to a further embodiment of the method, step a) comprises Sequential projection of a predetermined pattern in chronologically successive projections, where the patterns are projected displaced relative to each other in the different projections, is included.

[0038] It can also be said that the pattern is "scanned" over the area. This has the advantage that the area between two optical features of the pattern of the projection, where an object not captured by the projection may be located, can be captured by one of the subsequent projections, since the optical feature of the subsequent projection extends through the area. Thus, the scanning of the area with the pattern can be increased in succession. This is advantageous when a given pattern has, for example, a rather large spacing between the optical features, for example more than 11 cm.

[0039] Step b) in this case in particular involves capturing an image of each projection of the pattern, and step c) is performed for each captured image.

[0040] According to a further embodiment of the method, step a) comprises It involves projecting a plurality of different predetermined patterns in a time sequence according to a predetermined order.

[0041] For example, the sequences may include a checkerboard pattern, a diamond pattern, a triangle pattern, and a wave pattern, which are projected in succession.

[0042] Step b) in this case in particular involves capturing an image of each projection of the pattern, and step c) is performed for each captured image.

[0043] According to a further embodiment of the method, the method comprises: This involves determining the vehicle's trajectory based on a digital map of the surrounding area.

[0044] The trajectory is checked, taking into account, among other things, objects in the digital surrounding map to avoid collisions.

[0045] According to a further embodiment of the method, the method comprises: determining a location of the vehicle based on the projected pattern;

[0046] In this embodiment, the projection unit is arranged in particular outside the vehicle and fixed in a fixed position. The vehicle can therefore move relative to the projection. Furthermore, the projection of the pattern can capture the vehicle itself. The vehicle can then be identified as an object. Due to the fixed arrangement of the projection unit, the pattern can be projected at a defined and specified relative position relative to the infrastructure. It is thus possible, for example, to project a particular optical feature that appears at a defined, fixed position. Fixed coordinates in the digital surroundings map correspond to the fixed position. The respective position of the further optical feature can be concluded from the relative position of the further optical feature relative to the defined optical feature. The position of the vehicle can therefore be concluded from the relative position of the vehicle relative to the defined optical feature or further optical features of the projection whose position is defined.

[0047] Visually, a fixed projection can be viewed as a coordinate system within which the vehicle moves, with each position in the coordinate system being uniquely assigned to a position in the digital surroundings map.

[0048] According to a further embodiment of the method, the method comprises: Projecting a light notification signal.

[0049] The optical notification signal may be useful to other road users, for example if it contains a notification that an autonomously controlled vehicle is driving, and may also be used to control the vehicle itself. The notification signal may be used here in terms of a "follow me" function. The vehicle preferably includes for this purpose a sensor configured to capture the notification signal and has a control unit configured to drive the vehicle autonomously according to the captured notification signal.

[0050] According to a further embodiment of the method, the method comprises: capturing the projection of the notification signal by a camera of the vehicle; Check the information contained in the notification signal, Operating the vehicle in response to the ascertained information.

[0051] The information may include, among other things, directional information. Additionally, the information may include a stop signal.

[0052] The method of the first aspect can be carried out, for example, in the scenario described below. In this scenario, the devices are distributed, the projection unit and the capture unit are arranged outside the vehicle in an infrastructure designed as a parking lot, and the verification unit and the update unit are arranged in the vehicle, for example as part of a parking assistance system for the vehicle configured to drive the vehicle autonomously. Both the vehicle and the parking lot each include a communication unit and are therefore able to communicate with each other. The user of the vehicle drives with the vehicle to the entrance of the parking lot. A communication connection is established and the vehicle registers at the parking lot. In this case, for example, a digital surroundings map including the contours of the parking lot is transmitted to the vehicle, as well as free parking spaces and a path that leads the vehicle to the free parking space. The user leaves the vehicle and the autonomous driving mode is initiated. The parking assistance system takes over control of the vehicle and verifies the trajectory that extends along the transmitted path. The movable object is not included in the digital surroundings map. In order to avoid collisions with the object, a predetermined pattern is projected in each case in a defined area in front of and / or around the autonomous vehicle and the projection is captured. The captured image is transmitted to a verification unit in the vehicle, which verifies whether the object is located within the area of ​​the projection. Thus, the digital surroundings map based on which the parking assistance system plans the trajectory is updated. Thus, in particular, the movable object is respectively captured at the current time and can be taken into account in the planning of the trajectory. Thus, the vehicle can reach a free parking space autonomously and safely. When the free parking space is reached, the vehicle can park, for example using ultrasonic sensors for this purpose.

[0053] According to a second aspect, a device for operating a parking assistance system of a vehicle is proposed. The parking assistance system is arranged to automatically drive the vehicle. The device comprises: a projection unit for projecting a predetermined pattern onto a predetermined area; a capture unit for capturing an image in which at least a portion of the predetermined area having the projection is visible in the captured image; a confirmation unit for confirming an object located in a predetermined area according to the captured image; and an update unit for updating the digital surroundings map using the captured objects.

[0054] This device has the same advantages as described for the method according to the first aspect, therefore the embodiments and features described for the proposed method apply to the proposed device.

[0055] Each unit, in particular the verification unit and the update unit, can be implemented in hardware and / or software. In case of implementation in hardware, each unit may be in the form of, for example, a computer or a microprocessor. In case of implementation in software, each unit may be in the form of a computer program product, a function, a routine, an algorithm, a piece of program code, or an executable object. Furthermore, each of the units mentioned herein may also be in the form of a part of a higher-level control system of a vehicle and / or a building, such as a parking lot. The higher-level control system may be in the form of, for example, a central electronic control unit, such as a server and / or a domain computer, and / or an engine control unit (ECU).

[0056] The various units of the device may in particular be distributed and have functional and / or communication connections with one another: the device may comprise a unit integrated in the vehicle, may comprise a unit located in an infrastructure such as a car park, and / or may comprise several units in a distributed arrangement.

[0057] The vehicle includes a parking assistance system operable by the device. In this case, the parking assistance system may integrate some or all of the units of the device. The parking assistance system includes at least one control device, which is configured to receive control signals from the device and operate the vehicle according to the control signals (remote control of the vehicle).

[0058] According to one embodiment of the device, the projection unit is located outside the vehicle, and the capture unit, the verification unit and the update unit are located in or on the vehicle.

[0059] According to a further embodiment of the device, the projection unit and the capture unit are arranged outside the vehicle and the verification unit and the update unit are arranged inside the vehicle.

[0060] In a further embodiment, the validation unit is additionally located outside the vehicle, so that only the update unit is located within the vehicle.

[0061] According to a third aspect, a parking lot is proposed, comprising a device according to the second aspect and having a communication unit for establishing a communication connection with a parking assistance system of the vehicle for transmitting an updated digital surroundings map and / or control signals to the parking assistance system.

[0062] The parking lot is configured to perform an automatic parking process by the vehicle if the vehicle includes at least one control device, also called a parking assistance system, configured to receive control signals from at least one device and to operate the vehicle according to the control signals (remote control of the vehicle).

[0063] Optionally, the vehicle's parking assistance system may be configured to ascertain a suitable trajectory to an available parking spot itself based on the received digital surrounding map, and to autonomously drive the vehicle along the trajectory.

[0064] According to a fourth aspect, a vehicle is proposed having a parking assistance system for autonomously driving the vehicle and having a device according to the second aspect.

[0065] This is possible, in particular, by the device and parking assistance system that performs an automatic parking process. The parking process includes driving to a free parking space, which can include parking and starting, and the user of the vehicle leaves the vehicle, for example, in a transfer area, and activates the autonomous parking function. The vehicle then drives autonomously to a free parking space and parks. For example, the vehicle can be activated via a call signal received via a mobile radio network or another wireless data network, and the vehicle then drives autonomously from the parking space to the transfer area, where the user takes over the vehicle again. This can also be called an automatic valet parking system.

[0066] The vehicle is, for example, a car or even a truck. Preferably, the vehicle comprises several sensor units configured to capture the driving state of the vehicle and capture the surroundings of the vehicle. In particular, the vehicle comprises a projection unit and a capture unit that are part of the device. Further examples of the sensor units of the vehicle are image capture devices such as cameras, radar (radio detection and ranging) or lidar (light detection and ranging), ultrasonic sensors, position sensors, wheel angle sensors, and / or wheel speed sensors. Each of the sensor units is configured to output a sensor signal, for example, to a parking assistance system or a driving assistance system that performs partially autonomous or fully autonomous driving based on the captured sensor signal.

[0067] Further possible implementations of the invention also include combinations not expressly mentioned of the features or embodiments described above or below with respect to the exemplary embodiments, the skilled person will also in this case add individual aspects as improvements or additions to the respective basic forms of the invention.

[0068] Further advantageous configurations and aspects of the invention are the subject of the dependent claims and exemplary embodiments of the invention described below. The invention is explained in more detail below on the basis of preferred embodiments with reference to the attached drawings. [Brief description of the drawings]

[0069] [Figure 1] A schematic diagram showing an overhead view of the vehicle is shown. [Diagram 2] 3 shows an example of a projection of a predetermined pattern. [Diagram 3] 1 illustrates an example embodiment of updating a digital surroundings map. [Figure 4A] 1 shows an exemplary embodiment of a device for operating a parking assistance system. [Figure 4B] 2 shows another exemplary embodiment of a device for operating a parking assistance system. [Figure 4C] 4 shows yet another exemplary embodiment of a device for operating a parking assistance system. [Figure 4D] 4 shows yet another exemplary embodiment of a device for operating a parking assistance system. [Figure 5A] An example of a predetermined pattern is shown below. [Figure 5B] 13 shows another example of a predetermined pattern. [Figure 5C] 13 shows yet another example of a predetermined pattern. [Figure 5D] 13 shows yet another example of a predetermined pattern. [Figure 5E] 13 shows yet another example of a predetermined pattern. [Figure 5F] 13 shows yet another example of a predetermined pattern. [Figure 6] 4 shows further examples of projection of a predetermined pattern. [Figure 7] 1 shows a schematic diagram of a projection with an obstacle. [Figure 8A] 1 shows an exemplary embodiment of the projection of a light notification signal. [Figure 8B] 4 shows another exemplary embodiment of the projection of a light notification signal. [Figure 9A] 4 shows a schematic diagram of a further exemplary embodiment of a device for operating a parking assistance system; [Figure 9B] 4 shows another schematic diagram of a further exemplary embodiment of a device for operating a parking assistance system. [Figure 10] 1 shows a schematic block diagram of an exemplary embodiment of a method for operating a parking assistance system. [Figure 11] 1 shows a schematic block diagram of an exemplary embodiment of a device for operating a parking assistance system; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0070] Unless otherwise noted, identical or functionally identical elements are designated by the same reference numbers in the drawings.

[0071] FIG. 1 shows a schematic bird's-eye view of a vehicle 100. The vehicle 100 is, for example, an automobile arranged in a surrounding 200. The automobile 100 has a parking assistance system 105, for example in the form of a control unit. Furthermore, the vehicle 100 includes a device 110 configured to operate the parking assistance system 105. The device 110 comprises, in this example, two projection units 112, namely one projection unit 112 directed forward and one projection unit 112 directed backward, a number of acquisition units 114, a checking unit 116 and an updating unit 118. The projection units 112 are in particular in the form of laser projectors and are configured to project predetermined patterns PAT1 to PAT6 (see FIGS. 5A to 5F) by the vehicle 100 in a predetermined area 205 (see FIG. 2). The acquisition unit 114 comprises, for example, a visual camera, a radar and / or a lidar. Each of the capture units 114 can capture an image of a respective area from the surroundings 200 of the automobile 100 and output it as an optical sensor signal. Furthermore, a number of surrounding sensor devices 130 are arranged on the automobile 100, which may be, for example, ultrasonic sensors. The ultrasonic sensors 130 are configured to detect a distance from an object arranged in the environment 200 and output a corresponding sensor signal. The sensor signals captured by the capture units 114 and / or the ultrasonic sensors 130 enable the parking assistance system 105 and / or the device 110 to drive the automobile 100 partially autonomously or even fully autonomously. In addition to the capture units 114 and the ultrasonic sensors 130 shown in FIG. 1, the vehicle 100 can have various other sensor devices. Examples of these are microphones, acceleration sensors, wheel speed sensors, steering angle sensors, antennas with coupled receivers for receiving electromagnetically transmissible data signals, etc.

[0072] The device 110 is, for example, designed as described in more detail with reference to FIG. 11 and configured to carry out the method described with reference to FIG.

[0073] FIG. 2 shows an example of a projection 220 of predefined patterns PAT1 to PAT6 (see FIGS. 5A to 5F) on a predefined area 205 by a vehicle 100. For example, this is the vehicle 100 described on the basis of FIG. 1. The projection 220 can be generated by a projection unit 112 arranged on the vehicle 100 (see FIGS. 1, 4, 7, 9 or 11) and / or by a projection unit 112 arranged outside the vehicle 100. It should be noted that the projection can be generated on another predefined area independently of the vehicle (not shown), in which case the projection unit is arranged outside the vehicle. This example relates, for example, to the predefined pattern PAT1 shown in FIG. 5A. This pattern PAT1 comprises two line families with parallel running lines arranged perpendicular to each other. This pattern can also be called a chessboard pattern. In a planar area, the projection 220 of the predefined pattern PAT1 corresponds to the predefined pattern PAT1, i.e. the lines run parallel and perpendicular to each other. However, the projection 220 does not necessarily correspond to the predetermined pattern PAT1 in that the surface onto which the pattern is projected extends in a curved line, for example because of the position of the object 210. This is shown by way of example in Figure 2, where the projection 220 appears distorted in the area of ​​the object 210, as indicated by the line 225 which extends in a curved line.

[0074] The projection 220 of the pattern is captured as an image, for example, by a capture unit 112 (see Fig. 1, 4, 7, 9 or 11) arranged on the vehicle 100. Based on the distortion of the pattern 225, it can be determined that the object 210 causing this distortion must be located in the corresponding region. Thus, the verification unit 116 (see Fig. 1, 4, 7, 9 or 11) is configured to verify the object 210 based on the captured image.

[0075] In an embodiment (not shown), the shape of the object 210 can be concluded based on the distortion of the pattern 225. Alternatively or additionally, an object classification can be performed based on the distortion 225 (not shown), which is preferably performed by a neural network, in particular a GAN (generative adversarial network) and / or a CNN (convolutional neural network).

[0076] 3 shows an exemplary embodiment of updating the digital surroundings maps MAP0, MAP1. The digital surroundings maps MAP0, MAP1 are a representation of the actual surroundings 200 (see FIG. 1) of the vehicle 100 at a defined time, and optionally include some or all captured features of the surroundings 200. In the example of FIG. 3, the digital surroundings maps MAP0, MAP1 show an overhead view of the surroundings 200.

[0077] In this example, the vehicle 100 is located, for example, in a parking lot, and the digital surroundings map MAP0 includes a parked vehicle 310 and a pillar 304. In an embodiment, the digital surroundings maps MAP0, MAP1 may be specified at least in part by a system located outside the vehicle 100, such as a parking guidance system. The specified digital surroundings map MAP0 includes, for example, the outline of the parking lot, and already includes therein the lanes and building structures such as the pillar 304.

[0078] The vehicle 100 is autonomously controlled, for example, by the parking assistance system 105 (see FIG. 1) of the vehicle 100, which can be operated by the device 110 (see FIG. 1 or FIG. 11). For example, as described based on FIG. 2, the object 210 is located in front of the vehicle 100 by the confirmation unit 116 (see FIG. 1, FIG. 4, FIG. 7, FIG. 9 or FIG. 11) based on the captured image of the projection 220 (see FIG. 2). At that time, the update unit 118 (see FIG. 1, FIG. 4, FIG. 7, FIG. 9 or FIG. 11) updates the digital surroundings map MAP0, and the updated surroundings map MAP1 includes the confirmed object 210. Thus, a collision between the autonomous vehicle 100 and the object 210 can be avoided.

[0079] 4A-4D show four different exemplary embodiments of a device 110 for operating a parking assistance system 105 (see FIG. 1) of a vehicle 100. In all four examples, the device 100 comprises a projection unit 112, a capture unit 114, a confirmation unit 116 and an update unit 118. Each projection unit 112 is configured to project 220 a predefined pattern PAT1-PAT6 (see FIG. 5A-5F) in a predefined area 205 (see FIG. 2), in particular in an area beside the vehicle 100, and the capture unit 114 is configured to capture an image of the projection 220. The confirmation unit 116 is configured to confirm an object 210 (see FIG. 2 or FIG. 3) according to the captured image, and the update unit 118 is configured to update a digital surrounding map MAP0, MAP1 (see FIG. 3).

[0080] 4A shows a first embodiment in which the device 110 with all units is placed in the vehicle 100. This can also be called a "standalone" solution.

[0081] 4B shows a second embodiment in which the projection unit 112 is located outside the vehicle 100 in an infrastructure, which in this example is a car park 300, and the projection unit 112 is specifically located on the ceiling of the car park 300.

[0082] FIG. 4C shows a third embodiment in which the projection unit 112 and the capture unit 114 are arranged outside the vehicle 100 in an infrastructure. The infrastructure is a parking lot 300 in this example, and the projection unit 112 and the capture unit 114 are arranged offset from each other on the ceiling of the parking lot 300. The offset arrangement creates a parallax between the projection unit 112 and the capture unit 114, which improves the capture of the object. Furthermore, in this example, the vehicle 100 and the parking lot 300 have respective communication units 102, 302 configured to establish a wireless communication connection COM with each other. In particular, in this example, the captured image by the capture unit 114 is transmitted via the communication connection COM, so that the confirmation unit 116 arranged in the vehicle can confirm the object 210 (see FIG. 2 or FIG. 3) according to the captured image.

[0083] Fig. 4D shows a fourth embodiment in which the entire device 100 is located in the infrastructure. As in Fig. 4C, in this example the vehicle 100 and the parking lot 300 have respective communication units 102, 302 configured to establish a wireless communication connection COM with each other. However, in this case, instead of a captured image, for example, an updated surroundings map MAP0, MAP1 (see Fig. 3) is transmitted to the vehicle 100. The vehicle 100 or the parking assistance system 105 (see Fig. 1) of the vehicle 100 is configured to check the trajectory based on the digital surroundings map MAP0, MAP1, for example, to reach an available parking space. Alternatively, the trajectory of the vehicle 100 can also be checked by a corresponding unit in the infrastructure and only control signals are transmitted to the vehicle 100 (remote control of the vehicle 100).

[0084] 5A to 5F show different examples of the predetermined patterns PAT1 to PAT6.

[0085] The predetermined pattern PAT1 in Fig. 5A is, for example, a chessboard pattern. The predetermined pattern PAT2 in Fig. 5B is, for example, a diamond pattern. The predetermined pattern PAT3 in Fig. 5C is, for example, a triangular pattern. The predetermined pattern PAT4 in Fig. 5D is, for example, a wave pattern. The predetermined pattern PAT5 in Fig. 5E is, for example, a further triangular pattern. The predetermined pattern PAT6 in Fig. 5F is, for example, a circular pattern.

[0086] It should be noted that the predefined patterns shown on the basis of Figs. 5A-5F are merely examples. Any other predefined patterns are conceivable, for example combinations of the predefined patterns PAT1-PAT6. The predefined patterns PAT1-PAT6 are projected in particular such that the spacing between adjacent optical features, for example the line spacing between two adjacent lines, is at most 11 cm in a static projection. In case of dynamic projection, i.e. the projection is displaced at a defined time interval and / or different patterns are projected at different time intervals, the line spacing of a single pattern can also be greater than 11 cm. The line spacing of two successive patterns is preferably at most 11 cm, i.e. the maximum line spacing is 11 cm when chronologically successive projected patterns are superimposed. It is thus ensured that an object of at least 11 cm in size is captured by the projection 220 and thus can be seen by the device 110.

[0087] FIG. 6 shows a further example of the projection 220 of predefined patterns PAT1-PAT6 (see FIGS. 5A-5F), for example the chessboard pattern PAT1 of FIG. 5A. In this example, it is explained how the projection 220 of the pattern can be used to assist in locating the vehicle 100. Locating the vehicle 100 in a closed space such as a parking lot 300 (see FIGS. 4B-4D or 9) is difficult due to the lack of a location signal such as GPS, so it is particularly advantageous to ascertain the location of the vehicle 100 in the parking lot 300 as described below. In this example, the projection unit 112 (see FIGS. 1, 4, 7, 9 or 11) is fixedly arranged in a fixed position in the infrastructure, for example as shown based on FIGS. 4B-4D.

[0088] Due to the fixed arrangement of the projection unit 112, the pattern can be projected at a defined and specified relative position to the infrastructure. It is thus possible to project lines with a precisely predetermined distance, for example 2 meters from the side wall. In FIG. 6, the lines of the projection 220 are numbered H1-H10 and V1-V15. For example, the position of each line is specified in the digital surrounding maps MAP0, MAP1 (see FIG. 3). Based on the horizontal lines H1-H10, the transverse position of the vehicle 100 can be ascertained, and based on the vertical lines V1-V15, the longitudinal position of the vehicle 100 can be ascertained.

[0089] Here, as the vehicle 100 moves along the projection 220, the vehicle 100 passes through fixed lines of the pattern, i.e., parts of the projection 220 are not generated on the ground but rather reside on the vehicle 100 (not shown for clarity). Furthermore, Fig. 6 shows that the projection 220 protrudes at several points below the body of the vehicle 100, depending on the height of the body from the ground and the projection angle of the projection 220 relative to the vehicle 100. In particular, with the wheels HR, VR of the vehicle 100 in contact with the ground, precise localization is possible, since the transition points where the projection 220 changes from the ground to the respective wheels HR, VR mark the current position of the respective wheels HR, VR of the vehicle 100 precisely.

[0090] In this example, the position of the front wheel VR in the longitudinal direction is ascertained based on lines V10, V11, the ascertained position corresponds to the value between the positions of lines V10, V11, the position of the rear wheel HR in the longitudinal direction is ascertained based on lines V2, V3, the ascertained position corresponds to the value between the positions of lines V2, V3. The transverse position of the vehicle 100 is ascertained based on lines H3, H4 on the right side of the vehicle, the ascertained position corresponds to the value between the positions of lines H3, H4.

[0091] The localization can be performed using a capture unit 114 placed on the vehicle 100 (see FIG. 1, FIG. 4, FIG. 7, FIG. 9 or FIG. 11) or using a capture unit 114 placed at a fixed location in the infrastructure. If the capture unit 114 is placed on the vehicle 100, it is necessary that at least one of the projected optical features is identified so that it can be distinguished from the other features. This optical feature can then be seen in the captured image of the projection 220, and a position corresponding to the position of the optical feature is defined and specified in the digital surroundings map MAP0, MAP1. The relative position of the vehicle 100 with respect to the optical feature, and therefore also the absolute position of the vehicle 100 in the digital surroundings map MAP0, MAP1, can then be seen.

[0092] Figure 7 shows a schematic diagram of a projection 220 with two obstacles 210. The projection unit 112 is located in this example on the ceiling of the car park 300. The capture unit 114 is located at another position on the ceiling of the car park 300. Figure 7 is used to explain how each position 210 can change the projection 220 of the predefined patterns PAT1-PAT6 (see Figures 5A-5F).

[0093] The first beam R1 of the projection 220 is incident on a side of the object 210. The side of the object 210 is not visible from the perspective of the capture unit 114. Therefore, optical features that should be generated by the first beam R1 on the floor of the parking lot 300 are not included in the image of the projection 220, from which the presence of the object 210 can be concluded.

[0094] The second beam R2 of the projection 220 is incident on the upper side of the object 210, which is visible to the capture unit. The optical features to be generated by the second beam R2 on the floor of the car park 300 therefore appear displaced with respect to their expected positions. It can also be said that in this area the projection 220 appears distorted with respect to the predefined patterns PAT1-PAT6. From there the presence of the object 210 can be concluded.

[0095] The third beam R3 of the projection 220 is incident on an inclined surface of the object 210. The reflection angle of the beam R3 is thus affected, which is perceptible, for example, due to a change in the brightness of the optical features to be generated by the third beam R3, from which the presence of the object 210 can be concluded.

[0096] The three examples mentioned above do not form an exhaustive list of optical effects based on which the object 210 is visible in the image of the projection 220 of the predetermined patterns PAT1 to PAT6, but are used merely for illustration.

[0097] 8A-8B show an exemplary embodiment of the projection of a light notification signal POS containing defined information. In this example, the projection unit 112 is arranged outside the vehicle 100, which includes a capture unit 114, using which to capture the light notification signal POS. A parking assistance system 105 (see FIG. 1 ) of the vehicle 100 is configured to ascertain the information contained in the light notification signal POS based on the captured image and to autonomously control the vehicle 100 according to this information.

[0098] 8A, the light notification signal POS is used in particular to indicate to the vehicle 100 a confirmed trajectory along which the vehicle 100 should be driven. The parking assistance system 105 is configured to autonomously drive the vehicle 100 along the indicated trajectory.

[0099] The light notification signal POS may also develop a signal effect for other road users, among other things. For example, the dashed line of the light notification signal POS indicates the lane of the autonomous vehicle 100. Thus, other road users are informed that it is the autonomous vehicle 100 and can keep the lane of the vehicle 100 clear. In an embodiment (not shown), a visually clearly perceptible light notification signal POS may be projected in a predefined area around the vehicle 100, which clearly indicates the autonomous vehicle 100.

[0100] 8B, the light notification signal POS is used to stop the vehicle specifically in front of the identified object 210 to avoid a collision with the object 210. The parking assistance system 105 is configured to stop the vehicle 100 according to the light notification signal POS.

[0101] 9A-9B respectively show schematic diagrams of a further exemplary embodiment of a device 110 for operating a parking assistance system 105 (see FIG. 1) of a vehicle 100. FIG. 9A shows a side view and FIG. 9B shows a view from above, the vehicle 100 is located in a parking lot 300. In this example, the projection unit 112 is arranged outside the vehicle 100, just above the roadway, on a pillar 304, for example at a height of 1-5 cm. The capture unit 114 is arranged on the ceiling of the parking lot 300.

[0102] The projection unit 112 in this example projects only one line as the predetermined pattern PAT1-PAT6 (see FIG. 5A-FIG. 5F), and the light propagates just above the ground. A reflector 306 is placed at a predetermined position on the ground, and the light enters the reflector 306 and is reflected from the reflector 306. The reflector 306 has a height corresponding to the height of the projection unit 112, for example. If the area between the projection unit 112 and the reflector does not include an object 210, the projection 220 is a line extending along the path of the reflector 306.

[0103] However, if the object 210 is located in the area, light will enter the object and be reflected from the object. This can be seen as a distorted pattern 225 in the image of the projection 220. Furthermore, a shadow 222 will appear in the area of ​​the reflector 306 that corresponds to the alignment position from the projection unit 112 through the object 210. Thus, the presence of the object 210 can be concluded.

[0104] FIG. 10 shows a schematic block diagram of an exemplary embodiment of a method for operating a parking assistance system 105, for example the parking assistance system 105 of the vehicle 100 of FIG. 1. In a first step S1, a predefined pattern PAT1-PAT6 (see FIG. 5A-5F) is projected onto a predefined area 205 (see FIG. 2), in particular onto an area beside the vehicle 100 (see FIG. 1-4, FIG. 6, FIG. 8 or FIG. 9). In a second step S2, an image of the surroundings 200 (see FIG. 1) of the vehicle 100 is captured, and at least a part of the predefined area 205 with the projection 220 (see FIG. 2, FIG. 4 or FIG. 6-9) is visible in the captured image. In a third step S3, an object 210 (see FIG. 2, FIG. 3, FIG. 7 or FIG. 9) located in the predefined area (205) is identified according to the captured image. In a fourth step S4, the digital surroundings map MAP0, MAP1 (see FIG. 3) is updated using the captured object 210.

[0105] Fig. 11 shows a schematic block diagram of an exemplary embodiment of a device 110 for operating a parking assistance system 105, for example the parking assistance system 105 of the vehicle 100 of Fig. 1. The device 110 comprises a projection unit 112 for projecting predetermined patterns PAT1 to PAT6 (see Fig. 5A to 5F) in a predetermined area 205 (see Fig. 2), in particular in an area beside the vehicle 100, a capture unit 114 for capturing an image of the surroundings 200 (see Fig. 1) of the vehicle 100, where at least a part of the predetermined area 205 with a projection 220 (see Fig. 2, Fig. 4 or Fig. 6 to 9) is visible in the captured image, a confirmation unit 116 for confirming an object 210 located in the predetermined area 205 (see Fig. 2, Fig. 3, Fig. 7 or Fig. 9) according to the captured image, and an update unit 118 for updating a digital surroundings map MAP0, MAP1 using the captured object 210.

[0106] The device 110 is particularly configured to carry out the method described with reference to FIG.

[0107] While the present invention has been described with reference to exemplary embodiments, the present invention is susceptible to many variations. [Explanation of symbols]

[0108] 100 vehicles 102 Communication unit 105 Parking Assistance System 110 Devices 112 Projection Unit 114 Capture Unit 116 Confirmation Unit 118 Update Unit 130 Sensors 200 perimeter 205 Area 210 Object 220 Projection 222 Shade 225 Distorted Pattern 300 Parking 302 Communication Unit 304 Obstacles 306 Reflector 310 Parked Vehicles COM communication connection H1~H10 line HR rear wheel MAP0 Digital surrounding map MAP1 Digital surrounding map PAT1 Predefined pattern PAT2 Predefined pattern PAT3 Predefined pattern PAT4 Predefined pattern PAT5 Predefined pattern PAT6 Predefined pattern POS optical notification signal R1 Light Beam R2 Light Beam R3 Light Beam S1 Method Steps S2 Method Steps S3 Method Steps S4 Method Steps V1-V15 line VR front wheel

Claims

1. A method for operating a parking assistance system (105) of a vehicle (100), comprising: a) projecting (S1) a predetermined pattern (PAT1 to PAT6) onto a predetermined area (205), in particular an area (205) beside said vehicle (100); b) capturing an image (S2), where at least a portion of the predetermined area (205) having the projection (220) is visible in the image; c) identifying (S3) an object (210) located in said predetermined area (205) according to said image; d) updating (S4) the digital surrounding maps (MAP0, MAP1) using said object (210); Including, step a) comprises sequentially projecting said predetermined patterns (PAT1 to PAT6) in chronologically successive projections (220), said patterns being projected displaced with respect to each other in the different projections; method.

2. 2. The method of claim 1, wherein step c) comprises capturing the distortion (225) of the pattern in the image.

3. A method according to claim 1 or 2, characterized in that step a) comprises projecting said predetermined patterns (PAT1 to PAT6) using a laser projector.

4. Step a) comprises projecting said predetermined patterns (PAT1-PAT6) using predetermined colors; Step b) includes capturing the image using a filter transparent to the predetermined color. The method according to any one of claims 1 to 3, characterized in that

5. Step a) includes sequentially projecting a plurality of different predetermined patterns (PAT1 to PAT6) in a time series according to a predetermined order. The method according to any one of claims 1 to 4, characterized in that

6. Check the trajectory of the vehicle (100) based on the digital surrounding maps (MAP0, MAP1). The method according to any one of claims 1 to 5, characterized in that

7. Identifying the location of the vehicle (100) based on the pattern. The method according to any one of claims 1 to 6, characterized in that

8. Projecting a light notification signal (POS) The method according to any one of claims 1 to 7, characterized in that

9. capturing the projection of the light notification signal (POS) by a camera of the vehicle; Check the information contained in the light notification signal (POS), and operating the vehicle (100) in response to the ascertained information.

9. The method according to claim 8.

10. A device (110) for operating a parking assistance system (105) of a vehicle (100), said parking assistance system (105) being configured to automatically drive said vehicle (100), said device (110) comprising: - a projection unit (112) for sequentially projecting predetermined patterns (PAT1 to PAT6) in chronologically successive projections (220) onto a predetermined area (205), in particular an area (205) beside said vehicle (100), said patterns being projected displaced with respect to each other in the different projections; a capture unit (114) for capturing an image, wherein at least a portion of the predetermined area (205) having the projection (220) is visible in the image; a recognition unit (116) for recognizing an object (210) located in the predetermined area (205) according to the image; an updating unit (118) for updating a digital surrounding map (MAP0, MAP1) using said object (210); The device (110).

11. The device of claim 10, wherein the projection unit (112) is located outside the vehicle (100), and the capture unit (114), the verification unit (116), and the update unit (118) are located within the vehicle (100).

12. The device of claim 10, wherein the projection unit (112) and the capture unit (114) are located outside the vehicle (100), and the verification unit (116) and the update unit (118) are located within the vehicle (100).

13. A parking lot (300) comprising a device (110) according to claim 10 and a communication unit (302) for establishing a communication connection (COM) with the parking assistance system (105) of the vehicle (100) for transmitting the updated digital surrounding maps (MAP0, MAP1) and / or control signals to the parking assistance system (105).

14. A vehicle (100) having a parking assistance system (105) for automatically driving said vehicle (100), said vehicle having a device (110) according to claim 10.

Images