Method for providing assistance for parking process of motor vehicle

Abstract

The invention relates to a method for assisting a parking process of a motor vehicle (1), which projects an illumination marking (14) by means of an illumination device (2, 3) onto a wall surface (10) that can be seen by a parking person, comprising the following steps: recording at least one image (10) of the wall surface; -analysing the image in order to determine at least one three-dimensional structure (12) in or on the wall surface (10) shown in the image; -determining a structure boundary (13) extending at least partially around the structure (12),-determining the illumination marking (14) in such a way that a marking boundary (15) of the illumination marking is a negative of the structure boundary (13); projecting an illumination marking (14) onto the wall surface (10), the position of the marking boundary (15) relative to the structural boundary (13) varying as a function of the distance and / or position of the vehicle (1).

Description

Technical Field

[0001] The invention relates to a method for assisting a parking process of a motor vehicle by projecting an illuminated marking onto a wall surface visible to a parking person using an illumination device. Background Art

[0002] Today's motor vehicles have assistance systems that assist or facilitate parking. Known sensors are installed in the front and rear aprons and can be used to detect the distance to obstacles. During the parking process, the vehicle approaches an obstacle and, starting from below a distance threshold, a warning, usually a buzzer, is issued after the obstacle is detected, and sometimes a graphic display is also output, whereby the person driving the vehicle can position himself. As the approach continues, the frequency of the buzzer increases to a continuous sound when a further distance threshold (e.g. 40 cm) is reached. Further approach to the obstacle is then almost uncontrolled. A camera mounted on the vehicle, which captures the surroundings and provides a real-time image of the surroundings, can also be used in such an assistance system. The image is displayed on a display. This can improve the visual perception of the situation, but in this illustration, due to the installation position of the camera, it is not possible to see, for example, the edge area of ​​the bumper, so that an accurate positioning relative to the obstacle cannot be achieved. In addition, these assistance systems cannot check the extent to which the motor vehicle is currently parked in the parking space, that is, whether the motor vehicle is almost perpendicular to the wall that delimits the parking space with its longitudinal axis, or is inclined relative to its wall.

[0003] DE 10 2018 220 855 B3 discloses a motor vehicle in which, in order to assist the parking process, two illuminated markings are projected onto a wall delimiting a parking area by means of a lighting device mounted at the front of the vehicle. The two illuminated markings are initially vertically spaced apart from one another. As the vehicle approaches the wall, the illuminated markings come closer and closer and touch one another when a certain, defined distance from the wall is reached.

[0004] A similar system is known from US 2005 / 0099821 A1. Here, two illuminated markings are projected onto a wall by means of an illumination device, the two illuminated markings being offset from each other in the horizontal direction and approaching each other as the vehicle approaches the wall. When a defined distance is reached, the two illuminated markings merge into a common illuminated marking. Summary of the invention

[0005] The problem underlying the present invention is to specify an improved method for assisting a person when parking a vehicle.

[0006] In order to solve the above problems, the present invention provides a method for assisting the parking process of a motor vehicle by using a lighting device to project a lighting mark onto a wall surface visible to a parking person, the method comprising the following steps:

[0007] - take at least one image of the wall,

[0008] - analyzing the image to determine at least one three-dimensional structure in or on the wall surface shown in the image,

[0009] - determining a structure boundary extending at least partially around the structure,

[0010] - the illuminated marking is determined in such a way that the marking boundary of the illuminated marking is the negative of the structure boundary,

[0011] - Projecting an illuminated marking onto the wall surface, wherein the position of the marking boundary relative to the structure boundary varies as a function of the distance and / or the position of the vehicle.

[0012] The method according to the invention provides for projecting an illuminated marking onto a wall surface to assist the parking process, the illuminated marking having a specific marking boundary, which is determined based on a three-dimensional structure in or on the wall surface. To this end, in a first step, an image of the wall surface is captured by means of a suitable camera device mounted on the vehicle. The captured image is processed and analyzed by means of a suitable image analysis device or control device in order to determine whether a three-dimensional structure on or in the wall surface is contained in the image. If such a structure is detected, a structure boundary is determined by means of the image analysis device or control device, which structure boundary at least partially borders the edge of the structure, i.e. partially extends around the structure. The image analysis device has a suitable contour or boundary detection algorithm that allows such a boundary to be detected.

[0013] In the next step, the specific geometry of the illuminated marking to be projected onto the wall is then determined. That is, the illuminated marking is determined in such a way that the marking boundary of the illuminated marking is the negative of the structure boundary. The determination of the illuminated marking can be carried out, for example, in a control device which also performs the image analysis and is preferably also used to control the lighting device, but it can also be carried out in a separate control device which only controls the lighting device.

[0014] If the corresponding boundaries of the illuminated marking have now been determined, i.e. its specific geometry has been determined, the illuminated marking is projected onto the wall with the aid of a lighting device. The projection is carried out at the start of the assistance, i.e. immediately after the illuminated marking has been determined, so that the marking boundary is spaced from the structure boundary. As the vehicle approaches the wall, the relative position of the illuminated marking and the structure, i.e. the relative position of the marking boundary and the structure boundary, changes continuously. The marking boundary approaches the fixed structure boundary more and more. Once the ideal distance is reached (which can also be defined by the user as appropriate), the marking boundary will be exactly adapted to the structure boundary, i.e. the two are parallel or directly adjacent to each other. Since the person driving the vehicle looks forward through the windshield, he can see the illuminated marking and its approach to the three-dimensional structure and can understand the moment when the best match between the marking boundary and the structure boundary is achieved.

[0015] The assistance system can be activated either automatically or manually. For example, it can be activated automatically based on the determination of the vehicle's surroundings and the detection of the basic parking situation, combined with the detection of potential wall distances. When the corresponding minimum distance is reached, the image acquisition is then started, followed by the determination of the structure and the determination and projection of the illuminated marking, wherein the distance measurement is continuously continued, so that as the distance decreases, the illuminated marking can get closer and closer to the three-dimensional structure until the boundaries interlock when a defined, ideal parking distance is reached.

[0016] The system not only helps to achieve the best possible parking distance, but also helps to orient the vehicle as accurately as possible relative to the wall, that is, preferably to orient the longitudinal axis of the vehicle perpendicular to the plane of the wall. Because by projecting the illuminated marking, of course, the corresponding angular offset between the longitudinal axis of the vehicle and the plane of the wall can also be optically indicated. By a suitable sensor system of the vehicle, the spatial direction in which the longitudinal axis of the vehicle extends can be detected on the one hand, and the position and direction of the plane of the wall can be detected on the other hand, so that the relative position of the longitudinal axis of the vehicle and the plane of the wall can also be detected. When the person driving the vehicle sees the illuminated marking as described above, he can not only recognize the vertical offset between the marking boundary and the structure boundary (which is an indication of the distance between the vehicle and the wall), but also the horizontal offset between the marking boundary and the structure boundary. Since the illuminated marking is projected several meters before reaching the ideal distance position, the person can change the longitudinal axis of the vehicle by steering movement during the slow approach to the wall, so that the tilting state can be compensated as much as possible. That is, the horizontal position of the illuminated marking moves with the change of the relative position of the longitudinal axis of the vehicle and the wall. Ideally, when the longitudinal axis of the vehicle is ideally oriented, the geometric shape of the marking boundary is positioned to match the structure boundary in the horizontal direction, but is still spaced from the structure boundary in the vertical direction. However, horizontal displacement of the illuminated marking and thus providing assistance for assuming a precise position of the vehicle longitudinal axis relative to the plane of the wall is optional and does not necessarily have to be performed.

[0017] Thus, the person driving the vehicle is shown an illuminated marking whose geometry or boundaries are determined relative to the actual geometry of the structures provided on the wall. This allows illuminated markings to be displayed on walls that are uneven in certain areas due to three-dimensional structures, wherein these structures in particular are used in the assistance according to the method.

[0018] The lighting device can be controlled accordingly via the control device. The lighting device is, for example, a dot matrix / matrix LED headlight, which has an array composed of a plurality of individually controllable individual LEDs and equipped with individually controllable micromirrors, which allow corresponding projection of illuminated markings and corresponding generation of determined marking boundaries, as well as corresponding tracking based on continuously changing distances and, if appropriate, a continuously determined offset in the longitudinal direction of the vehicle relative to the wall plane.

[0019] According to a first alternative of the invention, one or more objects arranged in front of a wall can be determined as a three-dimensional structure. These objects can be, for example, garden equipment or tools, i.e., objects that are usually arranged on the garage wall forming the wall surface in a garage (whether a single garage, a double garage or a multi-garage). These objects can stand at the wall surface or in front of the wall surface, or lean against the wall surface, or hang on the wall surface, etc. Therefore, if a person drives a vehicle into the garage, an image of the wall surface and one or more objects is taken, the image analysis device or a control device having such an image analysis device in the form of a program device is set up to determine the structural boundary of the object, i.e. the object boundary, as soon as the object is detected as a three-dimensional structure. Then the lighting mark is determined in such a way that the lighting mark has a negative of the determined object boundary. The person parking the car then sees the wall surface with one or more objects on the one hand and the lighting mark projected onto the wall surface on the other hand, wherein as the person approaches the wall surface, the mark boundary gets closer and closer to the structure or object boundary, and when the ideal distance position is reached, the mark boundary and the structure or object boundary perfectly coincide.

[0020] As an alternative to detecting objects located on the wall surface, it is also conceivable to determine one or more elevations and / or depressions in the wall surface as three-dimensional structures. When parking, for example in a parking garage or the like, corresponding elevations / depressions, such as wall niches, doors or door frames, windows or window frames, projections, etc., can be determined as such structures. These structures are also bordered accordingly, so that the boundaries of these structures can also be determined in a suitable manner and used as a basis for determining the illuminated markings.

[0021] As already described, it is preferred to use one or two headlights as the lighting device, since headlights are usually correspondingly controllable lighting devices. Today's headlights usually have correspondingly large LED arrays with a plurality of individually controllable individual LEDs and associated, likewise individually controllable micromirrors, which allow corresponding generation of bordered illuminated symbols and variable projection of illuminated symbols, wherein such lighting devices can also be used to project illuminated symbols at a sufficient height on the wall so that people who are actually sitting high up can also see the illuminated symbols as well as possible.

[0022] An advantageous development of the invention provides that even after the start of the projection of the illuminated marking, the wall surface image is continuously captured and the matching accuracy of the marking boundary with the structure boundary is determined, wherein the driving operation is automatically controlled at least in part according to the result of this determination. The wall surface is continuously captured by means of the camera device during the entire assistance process, and the image then shows the three-dimensional structure and the illuminated marking. By means of the image analysis device or the control device, the image analysis is continuously performed to determine the matching accuracy of the illuminated marking with the structure, or the matching accuracy of the marking boundary with the structure marking. Thus, it is possible to continuously monitor how the marking boundary approaches the structure boundary. As it continues to approach, the driving operation can then be automatically intervened, for example, by automatically slowing down the vehicle until the vehicle is automatically stopped when the marking boundary and the structure boundary are perfectly aligned. Thus, the vehicle can at least automatically stop when the ideal distance position is reached, and if appropriate, automatic vehicle intervention can also be performed before this by the assistance system. In this automatic vehicle guidance, whether only automatic parking or additional speed adjustment, the distance measurement by sensors installed on the vehicle, that is, usually distance sensors installed in the front apron, can also be used, that is, the continuous acquisition of distance measurement values ​​can also be combined in the automatic vehicle guidance. If the system not only assists in achieving the ideal distance, but also guides the vehicle in a lateral direction so that the vehicle is optimally oriented relative to the wall, it is also conceivable that the assistance system automatically intervenes in the steering to automatically achieve the best matching accuracy of the marking boundary and the structure boundary, that is, the system can also automatically guide the vehicle in a lateral direction so that the vehicle is automatically and optimally oriented. That is, the person driving the vehicle ultimately does not need to actively participate during the parking process, so that neither longitudinal guidance and deceleration nor lateral guidance and steering movements need to be performed. Instead, everything is automatically performed by the assistance system, that is, it is controlled by one or more control devices integrated in the assistance system, so that at least from the projection of the illuminated marking, the driving operation is automatically controlled until the final position is reached, that is, the vehicle parks itself. It is conceivable here that the assistance system is also a self-learning system, that is, for example, it can learn the desired and ideal parking distance of the person through one or more parking processes and also through the lateral parking position, especially when parking the vehicle in a larger private garage.

[0023] In addition to the method itself, the invention also relates to a motor vehicle comprising at least one control device, at least one lighting device and at least one camera device, which are configured to carry out a method of the type described above. The lighting device is preferably a headlight, but two headlights that are usually present in motor vehicles can also be used. The camera device is preferably a camera device installed in the area of ​​the front skirt or in the area of ​​the windshield, which can record the area in front of the vehicle. The control device can be a central control device that controls all relevant system elements or communicates with all relevant system elements. However, the control device can also be a control device with separate, decentralized controllers. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Further advantages and details of the invention are apparent from the following description of the exemplary embodiments and from the accompanying drawings. The drawings show:

[0025] Figure 1 A schematic diagram showing the principle of a motor vehicle according to the present invention in a parking situation,

[0026] Figure 2 A schematic diagram showing a wall and a partial three-dimensional structure arranged thereon, wherein the three-dimensional structure is in the form of an object located on the wall.

[0027] Figure 3 A schematic diagram showing a principle of lighting marking determined for a structure perimeter is shown, and

[0028] Figure 4 An illuminated marker is shown projected onto a wall, wherein the marker boundaries match the structure boundaries in the parked position. DETAILED DESCRIPTION

[0029] Figure 1A motor vehicle 1 according to the invention is shown, which in the example shown comprises two lighting devices 2, 3 in the form of two headlights 4, 5, which are designed as dot matrix LED headlights, i.e., have an array formed by a plurality of individually controllable individual LEDs and a mirror array formed by a plurality of individually controllable micro mirrors. The lighting devices 3, 4 can be individually controlled by a common control device 6. In addition, a camera device 7 in the form of a front camera is provided, which can be mounted on the front apron of the vehicle or on the windshield of the vehicle. In this example, the camera device 7 communicates with the control device 6, but separate control devices can also be provided, which in turn communicate with each other. In addition, distance sensors 8, 9 are provided on the front side, which also communicate with the control device 6, and through which the distance between the motor vehicle 1 and the wall 10 also shown can be continuously detected. These distance sensors 8, 9 can be, for example, ultrasonic sensors. In the example shown, the distance sensors also communicate with the control device 6, but alternatively, a separate control device can also be provided for this purpose, which communicates with other control devices.

[0030] Also shown is a wall 10, in front of which the motor vehicle 1 is located and which it wants to approach for parking (see arrow P). In the example shown, it is assumed that the motor vehicle 1 is already aligned with its longitudinal axis L exactly relative to the plane of the wall 8, i.e. the longitudinal axis L of the vehicle is perpendicular to the plane of the wall.

[0031] To assist the parking process, the person driving the vehicle can, for example, activate an assistance system which assists the parking process by projecting illuminated markings onto the wall 10. Alternatively, the assistance system can also be activated automatically at a correspondingly low speed, for example ≤10 km / h. With the activation of the assistance system, the distance sensors 8, 9 are activated, i.e. the distance sensors measure the distance between the motor vehicle 1 and the wall 10 and send corresponding measurement signals to the control device 6, which processes and evaluates these signals. If, for example, a threshold value related to the distance of the motor vehicle 1 from the wall 10 is reached as the vehicle approaches, the actual guidance function of the assistance system is activated, unless this has already been done by activating the assistance system. In any case, the camera device 7 then continuously captures images of the area in front of the vehicle, i.e. the wall 10. With the receipt of the first camera image, the control device 6, which has a suitable image analysis device in the form of suitable program software, immediately evaluates the first camera image in order to check whether one or more three-dimensional structures are present in or on the wall 10 in the camera image. If such a structure is present, the control device 6 detects a structure boundary of the determined three-dimensional structure, which extends at least partially along the structure. The control device 6, which, as described, also controls the two lighting devices 2, 3, then determines an illuminated mark to be projected onto the wall surface 10 via at least one of the lighting devices 2, 3, so that the illuminated mark has a mark boundary, which at least partially borders the illuminated mark and is the negative of the detected structure boundary. If the illuminated mark is determined in this way, the illuminated mark is projected onto the wall surface 10 by one lighting device, optionally by both lighting devices 2, 3. Since the motor vehicle 1 continuously moves in the direction of the wall surface 10, the relative position of the illuminated mark and the structure also continuously changes, that is, the position of the mark boundary relative to the structure boundary changes with increasing approach. Here, depending on the detected structure, the lighting mark can be projected at a relatively long distance so that the mark boundary is obviously far away from the structure boundary, whether viewed in the vertical direction, the horizontal direction, or the complete surrounding of the structure. As it continues to approach, the mark boundary gradually approaches the fixed structure boundary until the lighting mark and the structure mark perfectly match when the defined, ideal parking distance is reached, because the two have a positive and negative film relationship.

[0032] A person driving the vehicle looking through the windshield at the wall can see the structure and the projected illuminated marking as well as the change in the relative position of the marking boundary to the structure boundary. Thus, the person can also recognize the moment when the marking boundary and the structure boundary meet ideally and can then end the longitudinal control of the motor vehicle, i.e. stop the motor vehicle, because the ideal parking position with the ideal parking distance has been reached.

[0033] By means of a horizontal (i.e. lateral) offset of the marking boundary from the structure boundary, information can optionally also be provided to the person driving the vehicle about the course of the vehicle longitudinal axis L relative to the plane of the wall 10. If the vehicle longitudinal axis and the wall plane are orthogonal to one another, there will not be any lateral offset of the marking boundary from the structure boundary, except that they are only spaced apart from one another. However, if the vehicle longitudinal axis L is not orthogonal to the plane of the wall 10, a corresponding lateral offset of the marking boundary relative to the structure boundary can occur, which can of course be compensated by a steering maneuver that results in a change in the position of the vehicle longitudinal axis L relative to the plane of the wall 10. This means that assistance to lateral control can also be achieved by means of the illuminated marking.

[0034] Figure 2-Figure 4 Showing an example for a given wall situation ( Figure 2 ), examples of the lighting signatures determined ( Figure 3 ) and an illustration of the illuminated mark when the ideal distance position is reached ( Figure 4 ) example.

[0035] Figure 2 The schematic diagram shows a wall 10 on which, in the example shown, a plurality of individual objects 11 in the form of garden equipment and tools are arranged, which are suspended from fixtures formed on the wall. The objects 11 form a three-dimensional structure 12, since they protrude from the wall 10 in three dimensions and allow a structure boundary 13, i.e. an object edge, to be determined, which partially borders the structure 12. Figure 2 The structural boundary 13 is shown by way of example in FIG. Figure 2 The image shown in is a camera image which is evaluated in the control device 6 so that the control device 6 can determine the structure boundary 13 .

[0036] The control device 6 then determines an illuminated marking 14 having a marking boundary 15 which is the negative of the structure boundary 13 of the structure 12 forming the positive. Figure 3 A schematic diagram of the principle of an illuminated marking 14 with a marking boundary 15 is shown, which corresponds exactly as a negative to a structure boundary 13 forming a positive, such as Figure 2 As shown. Obviously, the structure boundary 13 does not have to enclose the entire three-dimensional structure, but only a part of it can be considered. The lighting mark 14 does not have to illuminate the entire structure 12 with the corresponding marking boundary 15, but it is sufficient to illuminate the part, especially when there is such a representative boundary shape or edge as in the example.

[0037] If the geometry of the illuminated marking 14 has been determined, such as Figure 3As shown, an illuminated marker 14 is then projected onto the wall surface 10 by one or both lighting devices 2, 3. In the case where the distance between the motor vehicle 1 and the wall surface 10 is large, the illuminated marker 14 is correspondingly large, that is, the marker boundary 15 is farther away from the structure boundary 13. The illuminated marker 14 is determined at a specific time when there is a certain distance between the motor vehicle 1 and the wall surface 10. The assistance system also knows the ideal distance to be occupied, so that the assistance system can determine how large the illuminated marker 14 should be projected onto the wall surface 10 by the lighting devices 2, 3 based on the actual distance, so that the illuminated marker becomes smaller as it approaches continuously, and when the ideal parking distance is reached, the illuminated marker has a size such that the marker boundary 15 is equivalent to the size of the structure boundary 13. At the beginning of the projection, a correspondingly larger illuminated mark will be displayed, and the mark boundary of the larger illuminated mark is far away from the structure boundary. As it gets closer, the mark boundary gets closer and closer to the structure boundary. This is because the size of the projected illuminated mark becomes smaller and smaller due to the distance relationship, until the illuminated mark has the following size when the ideal distance position is reached, that is, at this size, the mark boundary 15 and the structure boundary 13 perfectly match.

[0038] exist Figure 4 This situation is shown in FIG. 1 , which also shows a wall 10 and a structure 12 provided thereon, i.e., a plurality of individual objects 11, as well as an illuminated marking 14 and its marking boundary 15, which clearly coincides exactly with the structure boundary 13. The person driving the vehicle thus sees that the boundary or the two edges coincide exactly at this point and knows that the ideal distance position has been reached so that he can park the vehicle.

[0039] However, it is also conceivable to provide a fully automatic longitudinal control operation of the motor vehicle 1. Since the image of the wall surface 10 shows the structure 12 and the projected illuminated marking 14 when the camera device 7 continuously records the wall surface 10, the control device 6 can continuously determine the relative position of the marking boundary 15 and the structure boundary 13, thereby detecting the continuous approach of the marking boundary to the structure boundary. With increasing matching accuracy, an automatic longitudinal intervention can be carried out in that the assistance system automatically slows down the motor vehicle further until the motor vehicle reaches the ideal distance position more and more slowly, at which the system automatically stops the motor vehicle 1.

[0040] It is also conceivable to provide lateral control information by means of the illustrated illuminated markings 14. The illuminated markings are projected onto the wall 10, for example, along the exact extension of the longitudinal axis L of the vehicle. Of course, if the longitudinal axis L of the vehicle is not perpendicular to the plane of the wall, a lateral offset will occur, i.e. the marking boundary 15 is significantly offset to one side of the structure boundary 13. The person can achieve the orientation by personally performing the steering operation, wherein the distance of the marking boundary 15 relative to the marking boundary 13 becomes increasingly symmetrical as the longitudinal axis L approaches a position perpendicular to the plane of the wall. This lateral control can also be performed automatically by an assistance system. When the optimal lateral orientation is achieved, the lateral control can be stopped until the desired distance position is reached.

Claims

1. A method for assisting a parking process of a motor vehicle (1), the method comprising projecting an illuminated marking (14) onto a wall (10) visible to a parking person by means of an illumination device (2, 3), the method comprising the following steps: - taking at least one image of the wall (10), - analyzing the image to determine at least one three-dimensional structure (12) in or on the wall surface (10) shown in the image, - determining a structure boundary (13) extending at least partially around the structure (12), - determining the illuminated mark (14) in such a way that the mark boundary (15) of the illuminated mark is the negative of the structure boundary (13), - projecting an illuminated marking (14) onto the wall surface (10), wherein: The position of the marking boundary (15) relative to the structure boundary (13) varies depending on the distance and / or position of the vehicle (1).

2. The method according to claim 1, characterized in that One or more objects (11) located in front of the wall (10) are determined as the three-dimensional structure (12).

3. The method according to claim 1, characterized in that One or more protrusions and / or depressions in the wall surface are defined as the three-dimensional structure (12).

4. The method according to any one of the preceding claims, characterized in that One headlamp (4, 5) or two headlamps (4, 5) are used as the lighting device (2, 3).

5. The method according to any one of the preceding claims, characterized in that Even after the start of projecting the illuminated marking (14), an image of the wall surface (10) is continuously recorded and the matching accuracy of the marking boundary (15) and the structure boundary (13) is determined, whereby the driving operation is automatically controlled at least partially based on the determination result.

6. The method according to claim 5, characterized in that In case a sufficient matching accuracy is determined, the motor vehicle (1) is automatically stopped.

7. The method according to claim 5 or 6, characterized in that: The travel operation is automatically controlled at least from the start of the projection until the end position is reached.

8. A motor vehicle comprising at least one control device (6), at least one lighting device (2, 3), and at least one camera device (7) which are arranged to carry out the method according to any of the preceding claims.

Images