Colour-augmented lane marking for obstacle warning

Abstract

The invention relates to a system for a vehicle (1) for visually augmenting a vehicle environment, wherein a control unit (7) detects another road user (9) in data from a sensor unit (3) and continuously updates a distance range to the other road user (9) which indicates that the lane change of the vehicle (1) is not to be carried out if the distance lies within the distance range, and displays, on a display unit (5), a marking (11) which visualises the distance range and is moved, together with a movement of the other road user (9), along a corridor which lies contact-analogously on an actual or virtually determined lane marking between the lane of the vehicle (1) and the adjacent lane.

Description

[0001] Color-augmented lane marking for obstacle warning

[0002] The invention relates to a system for a vehicle for visually augmenting a vehicle environment and a method for visually augmenting a vehicle environment.

[0003] When a vehicle changes lanes, the driver is typically responsible for not obstructing other road users in adjacent lanes, especially the lane being changed. To do this, the driver typically has to estimate the approach speed of another road user in the intended lane and the distance between that other road user and their own vehicle. While the driver is thus reliant on an imprecise estimate, it can also be difficult for them to accurately judge the required distances to other road users in the intended lane.

[0004] Modern vehicles increasingly feature augmentation systems that can overlay additional information onto the vehicle's screen or project augmented reality onto mirrors or glass surfaces. It is known in the prior art that displaying a lane change recommendation can depend on a check to determine whether the change is possible without a collision. Specifically, it is known that when changing lanes and no hazard is detected in the adjacent lane, a simple arrow is discreetly displayed on the head-up display. However, if a vehicle is present in the adjacent lane, the graphic can be modified to display a message to stop the lane change, for example, by flashing the indicator or changing it to red.

[0005] DE 10 2010 013 398 A1 relates to a method for displaying graphic images on a transparent windshield head-up display of a vehicle, describing the vehicle's journey in relation to a lane, wherein the method comprises: monitoring information inputs describing an operating environment of the vehicle, including monitoring a desired lane and the vehicle's current position; determining a relationship between the desired lane and the vehicle's current position; determining a graphic describing the relationship between the desired lane and the vehicle's current position; and displaying the graphic describing the relationship between the desired lane and the vehicle's current position on the essentially transparent windshield head-up display.and wherein the substantially transparent windscreen head-up display comprises light-emitting particles or microstructures over a predefined area of ​​the windscreen which permit a luminescent display while allowing a view through the windscreen.

[0006] WO 2017 / 162278 A1 also relates to a method for supporting a complete lane-changing process using an augmented reality device, wherein the method uses at least five cameras provided on a vehicle, a computing device and the augmented reality device, and wherein the method comprises the steps: a) capturing images with the at least five cameras; b) generating lane-changing information from the captured images; c) displaying the lane-changing information with the augmented reality device; wherein the step of generating lane-changing information comprises the further steps: b1) detecting lanes, b2) detecting the position and speeds of other vehicles surrounding the vehicle, and b3) generating at least one lane-changing recommendation in a lane-changing recommendation unit.

[0007] The object of the invention is to provide a visual augmentation of the environment in the vehicle, which allows the driver of the vehicle to better assess the safety of an intended lane change.

[0008] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0009] A first aspect of the invention relates to a system for a vehicle for visually augmenting a vehicle's surroundings, comprising a sensor unit for detecting the vehicle's surroundings, a display unit, and a control unit configured to recognize another road user in the sensor unit's data and to determine whether the other road user must be taken into account when the vehicle changes lanes to an adjacent lane with regard to a distance between the road user and the vehicle, and if so, to continuously calculate a distance range to the other road user, indicating that the vehicle should refrain from changing lanes if the distance is within the distance range, and to display a marker on the display unit visualizing the distance range, which extends along a corridor.which lies on an actual or virtually determined lane marking between the vehicle's lane and the adjacent lane, and is moved along with the movement of another road user.

[0010] The system is particularly helpful for a driver considering changing lanes from their current lane to an adjacent lane if there is a possibility that another vehicle is approaching from behind in the adjacent lane. However, the system can also be used to warn of oncoming traffic.

[0011] In particular, when a vehicle changes lanes to an adjacent lane, the distance between the other road user and the vehicle must be taken into account if the other road user is already in the adjacent lane or is entering it.

[0012] The driver of their own vehicle must maintain a certain distance from other road users who are already in or entering the intended lane. This distance serves to ensure that the other road user is not unnecessarily forced to brake, or has the opportunity to reduce their speed in good time. In any case, the distance must ensure that no collision occurs between the other road user and the driver's own vehicle when the driver's vehicle changes from its current lane into an adjacent lane in which the other road user is already located or entering.

[0013] To assist the driver in assessing whether a lane change from the current lane to the adjacent lane is safe, ensuring a necessary distance to other road users, the following procedure is followed, whereby the control unit can perform this procedure simultaneously for a large number of other road users:

[0014] The control unit uses data from a vehicle sensor unit, which provides information about the vehicle's surroundings, to detect the presence of other road users. Each detected road user is then checked to determine if they are relevant to a lane change potentially intended by the driver. For example, other road users who are stopped or moving slowly on the hard shoulder, or who are on an acceleration lane, may be disregarded. The control unit's primary function is to take into account other road users approaching from behind who are about to overtake the vehicle; however, it can also alternatively or additionally consider oncoming traffic in an adjacent lane.

[0015] The control unit continuously updates and determines a specific distance zone assigned to each other road user. This distance zone generally moves with the other road user, but its extent can be adjusted during updates, for example, if the relative speed between the other road user and the vehicle increases or decreases. Minimum limits can be specified for the distance zone, such as 5 meters in front of and behind another road user.

[0016] In the area between another road user and the driver's own vehicle, when the other road user is moving towards the driver's vehicle, the required distance is correspondingly greater than in the area where the other road user is moving away from this intermediate area. For example, if the vehicle is about to overtake a truck by changing lanes, while another road user is approaching from behind at a higher speed, the driver must take into account the speed and distance of the other road user before initiating the overtaking maneuver.However, once another road user has passed the vehicle, the vehicle can merge into the adjacent lane with the other road user already there, maintaining a smaller distance compared to the approaching vehicle, as the other road user is moving away from the vehicle and the distance tends to increase. However, minimum distances must still be observed, which are primarily determined by the vehicle's speed.

[0017] The distance zone defines a zone relative to other road users into which the vehicle should not, or may not, enter, as otherwise a required minimum distance to the other road user would be violated. The control unit visualizes this distance zone for the driver by means of a marker superimposed on an image of the vehicle's real-world surroundings. This marker moves along a corridor, i.e., on or within a corridor, which is analogous to an actual or virtual lane marking between the vehicle's current lane and an adjacent lane. This corridor extends along the lane marking and is preferably not displayed itself unless the entire corridor is color-coded, using a first color for the marking and a second color outside the marking.The corridor provides a spatial reference for augmenting the driver's view of their surroundings in a contact-analogous manner. The marking itself is generally not contact-analogous to the surroundings, as the marking indicates the distance range, and this distance range is relative to the position of a moving road user. Therefore, the marking moves relative to the road and relative to the lane markings when the other road user to which the marking refers moves.

[0018] The lane marking can be a real, actual lane marking, for example, characterized by a series of longitudinal, bar-shaped white or orange stripes. The corridor can then be defined, for example, by image recognition of this lane marking, so that it always follows the bar-shaped stripes; it can also be smoothed to a certain extent.

[0019] The virtually determined lane marking can, for example, compensate for gaps in an actual lane marking and continue it virtually. This makes it advantageous to display the marking to indicate the distance between two lanes, even without a continuous and complete physical lane marking.

[0020] The marking itself is preferably colored, for example red. The marking extends lengthwise along the virtual or actual lane marking and has a width that essentially corresponds to the width of typical or currently existing bar-shaped marking stripes. Preferably, the marking does not protrude significantly into its own lane or the adjacent lane. The marking is also preferably represented as a continuous, uninterrupted line.

[0021] At a front and a rear end of the marking, the marking may fade or change to a different color, for example to a green colored area of ​​the lane marking, to indicate to the driver of the vehicle the possibility of a safe lane change if he finds himself outside the marking at the level of the green area or at the level of the faded area.

[0022] If several other road users are traveling in or entering an adjacent lane, the control unit generates multiple markings, which can also overlap. Gaps are visible between the markings for the driver, which they can use to change lanes from their current lane to the adjacent lane.

[0023] In a preferred embodiment, a lane marking is continuously colored between the vehicle's own lane and an adjacent lane, the choice of color depending on whether and where a marking for a distance area is displayed.

[0024] The system is advantageously able to accurately indicate a gap in traffic approaching from the front and / or from behind with other road users, thus significantly simplifying the driving task for the driver of their own vehicle in order to anticipate the dangers of a lane change.

[0025] By having the control unit take into account, in particular, a distance and a relative speed between the respective other road user and the driver's own vehicle when determining the length, i.e., the extent, the distance range and thus the marking, it is no longer necessary for the driver of the own vehicle to estimate the safety of a lane change from an estimated relative speed of the other road user to the driver's own vehicle and its distance to it; rather, comparing the driver's own position with a respective displayed marking is sufficient.If, as in the example above, the lane marking is indicated by a red area and areas outside the marking by a green area, then for the driver of the vehicle, when deciding on the safety of an intended lane change to an adjacent lane, it is sufficient to determine whether the position of his own vehicle is at the level of the green area and preferably to estimate whether he can execute and complete the lane change in the green area.

[0026] According to an advantageous embodiment, the marking extends longitudinally along the corridor over a length of the distance range and is colored over this length.

[0027] According to another advantageous embodiment, the marking transitions smoothly into a different color in the corridor at its front and rear ends.

[0028] According to a further advantageous embodiment, the display unit comprises a screen, the control unit being configured to display an image of the vehicle's surroundings, derived from the data of the sensor unit, on the screen, the image including at least part of the adjacent lane. Alternatively, the marking can be projected onto a mirror, for example, a rearview or side mirror of the vehicle. The marking can also be projected onto a head-up display of the vehicle. Further configurations of the display unit are possible, for example, AR glasses or VR glasses. Accordingly, an augmented mirror image can be viewed on the display unit, or a synthetic image, such as a camera image, can be displayed on a screen.The latter can be used, for example, when a camera-monitor system is used instead of physical exterior mirrors with mirror glass, in which exterior cameras are directed backwards against the direction of travel towards the vehicle and transmit a video data stream to a respective exterior image monitor.

[0029] According to another advantageous embodiment, the screen is part of a camera-monitor system that replaces a side mirror.

[0030] According to a further advantageous embodiment, the control unit is designed to display the marking only when an adjacent lane and / or an actual lane marking is detected in the data of the sensor unit by the control unit, which meets a predetermined condition.

[0031] One such condition might be that a regular lane marking exists between the driver's own lane and the adjacent lane, and that the lane marking does not delineate a hard shoulder or acceleration lane. The marking might also only be displayed if changing lanes is permitted. For example, solid lines are often temporarily installed between lanes in construction zones to indicate that changing lanes is prohibited.

[0032] According to a further advantageous embodiment, the control unit is designed to display a respective crossbar directed towards the actual or virtually determined lane marking, analogous to contact with a front end of the road user and analogous to contact with a rear end of the vehicle.

[0033] The crossbar helps the driver to assess the position of their vehicle relative to the lane markings, as well as the actual distance between their vehicle and other road users, especially when they are close together. This is particularly relevant in scenarios where the markings are used to visualize the distance of another road user approaching from behind, against the vehicle's direction of travel. According to a further advantageous embodiment, the crossbar ends at a distance before the actual or virtually determined lane marking.

[0034] This provides a better overview of the displayed graphic elements (the marking along the lane marking and the respective crossbar), as these graphic elements do not overlap each other.

[0035] According to a further advantageous embodiment, the control unit is designed to display a single marking for each road user superimposed on a single actual or virtually determined lane marking on each side of the vehicle. The vehicle sides are a left side and a right side.

[0036] Another aspect of the invention relates to a vehicle with a system as described above and below.

[0037] Another aspect of the invention relates to a method for visually augmenting a vehicle's surroundings, wherein a sensor unit detects the vehicle's surroundings, at least part of the vehicle's surroundings is visible to the driver of the vehicle on a display unit, and a control unit detects another road user in the sensor unit's data and determines whether the other road user must be taken into account when the vehicle changes lanes to an adjacent lane with regard to a distance between the road user and the vehicle, and if so, continuously calculates a distance range to the other road user, indicating that the vehicle should refrain from changing lanes if the distance is within the distance range, and wherein a marker visualizing the distance range is displayed on the display unit along a corridor,which lies on an actual or virtually determined lane marking between the vehicle's lane and the adjacent lane, and moves along with the movement of the road user.

[0038] Advantages and preferred further developments of the proposed procedure result from an analogous and substantive transfer of the above statements made in connection with the proposed system.

[0039] Further advantages, features, and details will become apparent from the following description, in which—possibly with reference to the drawing—at least one embodiment is described in detail. The drawing shows:

[0040] Fig. 1: An interior of a vehicle with a system for rear-view mirrors according to an embodiment of the invention.

[0041] Fig. 2: An interior of a vehicle with a system for a digital rearview mirror system according to an embodiment of the invention.

[0042] Fig. 3: The vehicle of Fig. 1 or Fig. 2 in a bird's-eye view.

[0043] Fig. 4: A scenario and a display of the system according to Fig. 1 or Fig. 2.

[0044] Fig. 5: A marked representation according to an embodiment of the invention.

[0045] Fig. 6: A method for visually augmenting a vehicle environment according to an embodiment of the invention.

[0046] The representations in the figures are schematic and not to scale.

[0047] Fig. 1 shows the interior of a vehicle 1 with a system for visually augmenting the vehicle's surroundings as viewed by the driver of the vehicle 1. Not shown in Fig. 1, but illustrated by way of example in Fig. 3, the vehicle 1 has a sensor unit 3 for detecting the vehicle's surroundings. Data about the vehicle's surroundings are continuously transmitted from the sensor unit 3 to a control unit 7, which analyzes this data. In this data analysis, other road users 9 are continuously detected and recognized as such. The control unit 9 also performs an examination to determine whether, in the event of a lane change by the vehicle 1 to an adjacent lane, these road users 9 must be taken into account in such a way that a predetermined distance to the other road user 9 must be maintained. This check can be performed conservatively, i.e.,This check also takes place for other road users 9 who are very far away, but not for other road users 9 who are, for example, traveling on other roads. In particular, this check is always and only triggered if the other road user 9 is in an adjacent lane into which the vehicle 1 could potentially change lanes. The control unit 7 then calculates a distance range to the other road user 9, which indicates that the lane change by the vehicle 1 should be avoided if the distance between the vehicle 1 and the other road user 9 is within this range. In the example shown in Fig. 1, the driver of the vehicle 1 views the area behind and partially to the side of the vehicle 1 using the side mirrors.These form the display unit 5, on which the marker 11, which visualizes the distance range, is superimposed on the actual mirror image. The display of the marker 11 is described in more detail with reference to Fig. 4.

[0048] Fig. 2 shows, analogous to Fig. 1, the interior of a vehicle 1 from the perspective of the driver of the vehicle 1, but with a design variation: Instead of side mirrors, the display unit 5 is formed by a camera-monitor system. Rear-view cameras are used instead of the side mirrors, and their images are displayed on monitors of the display unit 5 in the interior of the vehicle 1. These monitors are located on the left and right sides of the interior of the vehicle 1. The marking 11 (see Fig. 4 and Fig. 5 for exemplary markings 11) is therefore superimposed on the monitors of the display unit 5 in the image of the vehicle 1's surroundings.

[0049] Fig. 3 shows a vehicle 1, either that of Fig. 1 or Fig. 2. Fig. 3 depicts an exemplary sensor unit 3 for supplying data about the vehicle 1's surroundings to the vehicle 1's control unit 7. Since this is a modern vehicle with a range of driver assistance and automation systems, the vehicle 1 is equipped with highly accurate environmental sensors. The sensor unit 3 comprises a multitude of individual camera systems, radar systems, and / or lidar systems distributed along the circumference of the vehicle 1's exterior. These allow the control unit 7 to continuously determine the distance and speed of each other road user 9 relative to the vehicle 1.Based on the respective distance and relative speed of another road user 9, the control unit 7 can continuously update and determine a distance range assigned to each other road user 9 and accordingly display the marking 11 on a display unit 5, however it may be designed.

[0050] Fig. 4 shows an exemplary view of a driver in a real rearview mirror (variant of the display unit 5 according to Fig. 1) or optionally in a digital side mirror (variant of the display unit 5 according to Fig. 2). When the driver of vehicle 1 looks in the side mirror or in a monitor of a camera-monitor system equivalent to a side mirror, he sees, on the one hand, the other road user 9, on the other hand, areas of his own vehicle 1, and, on the other hand, a lane marking between his own lane and the adjacent lane in which the other road user 9 is located. While this lane marking is generally displayed in green on the display unit 5, the green color is replaced by a red color in the distance area; this is marking 11.

[0051] Fig. 5 schematically shows, from a bird's-eye view, the area of ​​marking 11 relative to an exemplary position of vehicle 1 and an exemplary position of the other road user 9. The distance range, visualized by marking 11, is calculated such that a lane change can only be carried out safely if vehicle 1 is not at the level of marking 11. In the example of Fig. 5, this is not the case; the driver of vehicle 1 can still perform a lane change as long as road user 9 does not approach vehicle 11 any further.

[0052] Fig. 6 shows a method for visually augmenting a vehicle's surroundings, wherein a sensor unit 3 detects a vehicle's surroundings S1, at least part of the vehicle's surroundings is visible to a driver of the vehicle 1 on a display unit 5, and a control unit 7 detects another road user 9 in data from the sensor unit 3 S2 and determines S3 whether the other road user 9 must be taken into account when the vehicle 1 changes lanes to an adjacent lane with regard to a distance between the road user 9 and the vehicle 1, and if so, a distance range to the other road user 9 is continuously updated and calculated S4, which indicates that the lane change by the vehicle 1 should be avoided if the distance is within the distance range, and wherein a marker 11 visualizing the distance range is displayed on the display unit 5 S5, which runs along a corridor,which lies on an actual or virtually determined lane marking between the lane of vehicle 1 and the adjacent lane, and is moved along with a movement of road user 9.

Claims

Patent claims 1. System for a vehicle (1) for visually augmenting a vehicle's surroundings, comprising a sensor unit (3) for detecting the vehicle's surroundings, a display unit (5), and a control unit (7) configured to detect another road user (9) in the data from the sensor unit (3) and to determine whether the other road user (9) must be taken into account when the vehicle (1) changes lanes to an adjacent lane with respect to a distance between the road user (9) and the vehicle (1), and if so, to continuously calculate a distance range to the other road user (9) indicating that the lane change by the vehicle (1) should be avoided if the distance is within the distance range, and to display a marker (11) visualizing the distance range on the display unit (5), which extends along a corridor,which lies on an actual or virtually determined lane marking between the lane of the vehicle (1) and the adjacent lane, and is moved along with a movement of the other road user (9).

2. System according to claim 1, wherein the marking (11) extends elongatedly along the corridor over a length of the distance range and is colored over this length.

3. System according to claim 2, wherein the marking (11) transitions to a different color in the corridor at its front and rear ends with a smooth transition.

4. System according to one of the preceding claims, wherein the display unit (5) comprises a screen, wherein the control unit (7) is configured to display an image of the vehicle environment obtained from the data of the sensor unit (3) on the screen, wherein the image includes at least part of the adjacent lane.

5. System according to claim 4, wherein the screen is part of a camera-monitor system that replaces a side mirror.

6. System according to one of the preceding claims, wherein the control unit (7) is configured to display the marking (11) only when an adjacent lane and / or an actual lane marking is detected in the data of the sensor unit (3) by the control unit (7) that meets a predetermined condition.

7. System according to one of the preceding claims, wherein the control unit (7) is configured to display a respective transverse bar directed towards the actual or virtually determined lane marking, analogous to contact with a front end of the road user (9) and analogous to contact with a rear end of the vehicle (1).

8. System according to claim 7, wherein the crossbar ends at a distance in front of the actual or virtually determined lane marking.

9. System according to one of the preceding claims, wherein the control unit (7) is configured to display a respective marking (11) for a respective road user (9) superimposed on only one actual or virtually determined lane marking per side of the vehicle.

10. Method for visually augmenting a vehicle's surroundings, wherein a sensor unit (3) detects a vehicle's surroundings (S1), at least part of the vehicle's surroundings is visible to a driver of the vehicle (1) on a display unit (5), and a control unit (7) detects another road user (9) in data from the sensor unit (3) (S2) and determines (S3) whether the other road user (9) must be taken into account when the vehicle (1) changes lanes to an adjacent lane with regard to a distance between the road user (9) and the vehicle (1), and if so, continuously calculates a distance range to the other road user (9) (S4) which indicates that the lane change of the vehicle (1) should be avoided if the distance is within the distance range, and wherein a marker (11) visualizing the distance range is displayed on the display unit (5) (S5).which lies along a corridor that is contact-analogous to an actual or virtually determined lane marking between the lane of the vehicle (1) and the adjacent lane, with a movement of the road user (9), is moved along with it.

Images