Lane-deviation warning method for vehicle and device

Abstract

This lane deviation warning device detects a roadway center line (11) from a camera image, and displays a virtual deviation warning line (15) on the roadway center line (11) via an AR-HUD (2) of a vehicle when the state has been determined to be a state in which the lane departure warning should be issued. The display position of the deviation warning line (15) with respect to the roadway center line (11) is corrected by adding an offset amount (D) corresponding to a display distance (X) in the vehicle front-rear direction, such that the display position is superimposed on the roadway center line (11) in the lane width direction at a distal end (15b) far from the vehicle, and offset from the roadway center line (11) in the lane width direction at a proximal end (15a) close to the vehicle.

Description

Lane departure warning method and device for vehicle

[0001] The present invention relates to a lane departure warning method and device for a vehicle, which warns of lane departure by drawing virtual departure warning lines superimposed on actual lane markings via an AR-HUD (Augmented Reality Head-Up Display).

[0002] One known driving assistance technology for automobiles is a lane departure warning (LDW) system that issues a warning to the driver when the vehicle deviates or is about to deviate from the lane it is traveling in. Patent Document 1 discloses a technology that detects lane markings of the lane in which the vehicle is traveling from a camera image, and when it is determined that a lane departure warning should be issued, issues a warning by displaying a white line icon that resembles the lane markings on a display in the driver's seat and by flashing the icon.

[0003] Furthermore, Patent Document 2 discloses an AR-HUD that displays virtual information consisting of virtual images superimposed on real information visible through the windshield, and describes that one piece of virtual information can be a line image that resembles lane markings on the left and right.

[0004] When virtual deviation warning lines are displayed via an AR-HUD superimposed on actual lane markings for lane departure warning, the lane markings are generally bright white lines, which provide low contrast with the deviation warning lines displayed in red, for example, making it difficult for the driver to notice the deviation warning lines. Furthermore, the superimposed deviation warning lines may obscure the lane markings, making it difficult to recognize the direction in which the lane markings extend.

[0005] International Publication WO2012 / 169029A1 JP 2021-104794 A

[0006] The lane departure warning method for a vehicle according to the present invention detects lane markings of the lane in which the vehicle is traveling from a camera image, and when it is determined that a lane departure warning should be issued, displays, as at least part of the lane departure warning, a virtual departure warning line on the road surface ahead, as seen through the windshield, via the vehicle's AR-HUD, so as to be superimposed on the lane marking on the side to which the vehicle is approaching; the deviation warning line is displayed with an offset amount added according to the display distance in the fore-and-aft direction of the vehicle, so that the deviation warning line is superimposed on the lane marking in the lane width direction at the distal end far from the vehicle, and is offset from the lane marking in the lane width direction at the proximal end close to the vehicle.

[0007] In other words, the departure warning line, which is a virtual display by the AR-HUD, is basically displayed superimposed on the lane markings, but the proximal end closer to the vehicle is displayed offset (i.e., shifted to the left or right) from the lane markings in the lane width direction (i.e., left-right direction).

[0008] The entire length of the deviation warning line is not superimposed on the lane markings, but is displayed offset from the lane markings at the proximal end, making it easier for the driver to see both the lane markings and the deviation warning line.

[0009] 1 is an explanatory diagram of the configuration of a lane departure warning device according to one embodiment; a flowchart showing the processing flow of one embodiment; an explanatory diagram of lane departure determination criteria; an explanatory diagram of a departure warning line seen by a driver through the windshield; an explanatory diagram showing a plan view of the positional relationship between real lane markings and virtual departure warning lines; an explanatory diagram of a departure warning line displayed as virtual information as seen from the side of the vehicle; a characteristics diagram showing the relationship between the display distance from the vehicle and the offset amount; an explanatory diagram of a reference example in which the entire departure warning line overlaps with the lane markings; an explanatory diagram of a second reference example in which the entire departure warning line is offset from the lane markings in the lane width direction.

[0010] An embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0011] 1 is an explanatory diagram showing the configuration of a lane departure warning device for a vehicle according to one embodiment. The lane departure warning device according to one embodiment is configured to include a forward camera 1 that captures an image of the area in front of the vehicle, an AR-HUD (Augmented Reality Head-Up Display) 2 that serves as one of the vehicle's information display units, and a controller 3 that controls the display on the AR-HUD 2. The device may also include a GPS (Global Positioning System) 4 that includes detailed map data for obtaining road radius information.

[0012] The front camera 1 may be a camera dedicated to a lane departure warning system, but it is also possible to use a forward-facing camera included in a so-called ADAS (Advanced Driver Assistance System). The front camera 1 may be configured to include multiple cameras. Image data of the area in front of the vehicle acquired by the front camera 1 is input to the controller 3, which then analyzes the image to detect lane markings (such as the center line and outer perimeter lines) consisting of white lines on both the left and right sides of the lane in which the vehicle is traveling.

[0013] The AR-HUD 2 is a type of head-up display that displays virtual information superimposed on real information visible through the windshield. In the present invention, the AR-HUD 2 may take any form, but may, for example, be configured with a projector mounted below the windshield. Light reflected from the surface of the windshield reaches the driver's eyes, generating a virtual image of the virtual information on a virtual focal plane located an appropriate distance (e.g., several meters) in front of the driver. The virtual information may include a departure warning line, as described below, in accordance with the present invention, as well as other appropriate information such as vehicle speed information and route guidance information from a car navigation system. The AR-HUD 2 also includes a viewpoint detection camera (not shown) that detects the driver's viewpoint in order to optimize the position of the virtual image generated by the projector.

[0014] The controller 3 determines whether a lane departure warning should be issued based on the distance between the lane markings detected via the forward camera 1 and the host vehicle in the lane width direction. If a lane departure warning is required, the controller 3 issues the lane departure warning to the driver. In this embodiment, as at least part of the lane departure warning, a virtual departure warning line is displayed via the AR-HUD 2 so as to be superimposed on the actual lane markings on the side the host vehicle is approaching, so as to draw the driver's attention to the lane markings. The lane departure warning may also be issued in the form of an audible warning. Furthermore, the controller 3 may be configured to provide a function to assist steering in a direction to eliminate lane departure simultaneously with the warning. The condition in which a lane departure warning should be issued includes both a case in which the vehicle has already deviated from the lane and a case in which the vehicle is about to deviate from the lane. The stage at which the vehicle is deemed to have deviated from the lane when it is drifting to one side is determined to be "lane departure."

[0015] Next, the deviation warning lines superimposed on the actual road surface via the AR-HUD 2 will be described with reference to Figures 4 and 5. Figure 4 is an explanatory diagram illustrating a case where the host vehicle drifts to the right while traveling forward on a curved road that curves gently to the left, and a lane deviation warning is issued for the right lane marking (the center line 11 in the illustrated example), and shows a simplified view of what the driver sees through the windshield. The example in Figure 4 is a two-lane road with one lane in each direction, having two outside lane markings 12, 13 and a center line 11, with the center line 11 on the right side of the host vehicle and the outside lane marking 12 on the left side.

[0016] In the illustrated example, the controller 3 determines that the host vehicle has deviated from the roadway centerline 11 on the right side of the host vehicle, and a departure warning line 15 is displayed via the AR-HUD 2 as part of the lane departure warning. As shown in FIG. 4 , a rectangular display area 16 of the AR-HUD 2 is present in a portion of the area visible through the windshield, and the departure warning line 15, which is virtual information, is displayed within this display area 16. Note that, for the sake of explanation, the display area 16 is illustrated surrounded by a frame in FIG. 4 , but in reality, such a frame is not necessary. The display area 16 is basically positioned directly in front of the driver. Various virtual information required during driving is displayed in this display area 16, and in addition to the above-mentioned departure warning line 15, information such as the speed limit and actual speed of the road being traveled may also be displayed.

[0017] The departure warning line 15 is displayed so as to be superimposed on the roadway centerline 11, which is a white line that is real information. In one example, it is displayed as a single red line having a width approximately equal to the painted width of the roadway centerline 11. As will be described later, the AR-HUD 2 generates a virtual image so that the departure warning line 15 appears at a predetermined display distance ahead of the vehicle (for example, within a range of 40 to 70 m ahead of the vehicle) when viewed by the driver.

[0018] For the purpose of explanation, FIG. 5 is an explanatory diagram showing the positional relationship between actual lane markings (the roadway centerline 11 and the roadway outer lane lines 12 and 13) and the virtual departure warning lines 15 in a plan view. As described above, the host vehicle is traveling on a curved road that curves gently to the left, and therefore the roadway centerline 11 is also gently curved. Correspondingly, the departure warning lines 15 are also generated as gently curved lines. More specifically, the radius of curvature R of the roadway centerline 11 is obtained based on the camera image captured by the forward camera 1 or map information from the GPS 4. Based on this radius of curvature R, basic display position data (global coordinates X, Y) of the departure warning lines 15 in a planar view as shown in FIG. 5 is calculated. Then, by converting these global coordinates into the coordinate system of the AR-HUD 2 so as to be consistent with the road surface as seen diagonally from above by the driver, the departure warning lines 15 appear to the driver on the roadway centerline 11. The X direction of the global coordinate system corresponds to the longitudinal direction of the vehicle, and the Y direction corresponds to the transverse direction of the vehicle.

[0019] Here, the departure warning line 15 is displayed within a range from a virtual distance X1 (e.g., 40 m) to a virtual distance X2 (e.g., 70 m) ahead of the vehicle. At the distal end 15b far from the vehicle, the departure warning line 15 is correctly superimposed on the roadway centerline 11 in the lane width direction (in other words, the left-right direction), whereas at the proximal end 15a close to the vehicle, the departure warning line 15 is offset in the lane width direction from the roadway centerline 11. More specifically, if the distance between the center of the width direction of the departure warning line 15 and the center of the width direction of the roadway centerline 11 (the distance in the Y direction) in a plan view as shown in FIG. 5 is defined as the offset amount D, then the offset amount D is 0 at the distal end 15b and a maximum offset amount Dmax at the proximal end 15a. The offset amount D varies continuously between the display distance, i.e., the virtual distance X between X1 and X2.

[0020] In a preferred embodiment, the maximum offset amount Dmax at the proximal end 15a is set so that the departure warning lines 15 are adjacent to each other at the proximal end 15a without being completely separated from the roadway centerline 11. However, the present invention is not limited to this, and the maximum offset amount Dmax may be set so that the departure warning lines 15 and the roadway centerline 11 are appropriately separated from each other at the proximal end 15a.

[0021] In a preferred embodiment, the offset amount D is applied to the outer periphery of the radius of curvature R. In other words, as shown in Fig. 5, the proximal end 15a of the departure warning line 15 is located on the outer periphery of the radius of curvature R with respect to the roadway centerline 11. However, the present invention is not limited to this, and the departure warning line 15 may be offset to the inner periphery of the radius of curvature R at the proximal end 15a.

[0022] In a preferred embodiment, the radius of curvature R of the roadway centerline 11 is calculated based on the camera image or map information as described above, and the basic display position (X, Y coordinates) of the departure warning line 15 superimposed on the roadway centerline 11 along this radius of curvature R is calculated, and the display position is corrected by adding the above-mentioned offset amount D corresponding to the display distance X.

[0023] 7 is a characteristic diagram showing several examples of the characteristic of the offset amount D with respect to the display distance X. The characteristic line L1 is an example in which the offset amount D changes proportionally with the display distance X. In other words, the offset amount D at each position is determined by the following formula:

[0024] D=(X2-X) / (X2-X1)*Dmax Furthermore, characteristic line L2 is an example where "a=2" in the following formula, and characteristic line L3 is an example where "a=1 / 2".

[0025] D=((X2-X) / (X2-X1)) ａ×Dmax Figure 6 is an explanatory diagram of a deviation warning line 15, provided as virtual information, virtually displayed on a road 21 (which is real information) on which the vehicle is traveling, as viewed from the side of the vehicle. As shown in Figure 6, a virtual focal plane 23 of the AR-HUD 2 exists in front of the driver 22 (outside the windshield), and a virtual image is generated on this focal plane 23 by the AR-HUD 2. The triangle labeled 24 indicates the viewing angle from which the driver 22 views the display area of ​​the AR-HUD 1. The driver 22 generally looks down at the road surface 21, and the driver 22 recognizes the distance to an object based on the look-down angle at this time. Therefore, by generating virtual information, i.e., a virtual image of the deviation warning line 15, on the focal plane 23 so as to obtain a look-down angle corresponding to the virtual distance, the driver 22 recognizes that the deviation warning line 15 exists within the range of virtual distances X1 to X2. The same is true for the angle in the left-right direction; the closer an object is to the eye, i.e., the viewpoint, the more it appears to extend to the left and right, so a departure warning line 15 having a shape corresponding to the virtual distance is generated on the focal plane 23 so as to basically overlap with the roadway centerline 11, which is real information. Note that the AR-HUD 1 preferably has a viewpoint detection camera that detects the eye position of the driver 22 as described above, and a virtual image is generated at a position corresponding to the individual eye position.

[0026] By displaying the departure warning line 15 as described above as one of the lane departure warnings when the vehicle deviates from a lane, the driver can easily recognize that the vehicle is deviating from a lane and which lane the vehicle is deviating from. In particular, the distal end 15b is aligned with the target lane marking (e.g., the roadway centerline 11) while the proximal end 15a is displayed offset from the lane marking without completely overlapping with the lane marking, making it easier for the driver to notice the display of the departure warning line 15. In other words, the entire length of the departure warning line 15 is not superimposed on the lane marking, and the proximal end 15a is displayed offset from the lane marking, making it easier for the driver to see both the lane marking and the departure warning line 15.

[0027] 8 shows a reference example in which the entire length of the departure warning line 15 is superimposed on the roadway centerline 11 without applying an offset as in the above-described embodiment. In this case, the roadway centerline 11 is generally painted with highly luminous white paint, and the departure warning line 15 is displayed on top of it as a red virtual image, so the contrast between the two is low and it is difficult for the driver to notice the departure warning line. Furthermore, the superimposed departure warning line 15 may hide the roadway centerline 11, which may make it difficult to recognize the direction in which the roadway centerline 11 extends.

[0028] 9 shows a second reference example in which the entire length of the departure warning line 15 is offset by a fixed amount in the lane width direction from the roadway centerline 11. In such a case, for example, if the driver's viewpoint shifts to the left or right, the entire length of the departure warning line 15 will appear to overlap with the roadway centerline 11, causing the same problem as in the reference example of FIG. 8. There is also a risk that the driver will mistakenly believe that the direction in which they should steer to trace is the direction of the departure warning line 15, which is away from the roadway centerline 11.

[0029] In the above embodiment, in which the deviation warning line 15 is offset at the proximal end 15a but coincides with the roadway centerline 11 at the distal end 15b, the direction in which the deviation warning line 15 and the roadway centerline 11 extend can be correctly recognized while making the deviation warning line 15 easier to see.

[0030] Next, Fig. 2 is a flowchart showing the flow of processing for lane departure warning control executed by the controller 3. First, in step 1, the position Y Ｌ Specifically, the distance Y from the center of the vehicle body to the lane marking in the vehicle width direction Y is detected based on the camera image. Ｌ In step 2, the reference distance Y Ｔ The distance to the lane marking Y Ｌ For example, "Y Ｌ -Y Ｔ If " is negative, it is determined that a lane departure warning should be issued (see FIG. 3), and the process proceeds from step 2 to step 3. Note that in the present invention, the lane departure determination may be based on other criteria or methods.

[0031] In step 3, the road radius, more specifically the radius of curvature R of the lane marking on the deviation side (e.g., roadway centerline 11) is calculated from the camera image. Note that the radius of curvature R may also be obtained from map information from the GPS 4. Then, in step 4, based on this radius of curvature R, the coordinates in the global coordinate system of the deviation warning line 15 (referred to as the LDW line in the figure) that can be superimposed on the target lane marking are calculated. In other words, the basic display position is determined.

[0032] Next, in step 5, an offset amount D is added to the coordinates of the basic deviation warning line 15. In other words, the basic display position is corrected using the offset amount D according to the display distance X in the forward / backward direction.

[0033] Next, the process proceeds to step 6, where the global coordinates of the deviation warning line 15 after correction, which reflects the offset amount D, are converted into the coordinate system of the AR-HUD 2. Then, in step 7, the deviation warning line 15 is displayed via the AR-HUD 2.

[0034] In the next steps 8 and 9, similarly to steps 1 and 2, the position Y of the lane marking is calculated. Ｌ That is, the distance Y in the vehicle width direction Y from the center of the vehicle body to the lane marking Ｌ is calculated and used as the reference distance Y Ｔ If the state in which a lane departure warning should be issued continues, the process returns to step 3, and the display of the departure warning line 15 continues.

[0035] If it is determined in step 9 that the vehicle has left the lane departure state, the process proceeds from step 9 to step 10, where the display of the departure warning line 15 is terminated.

[0036] The process of FIG. 2 is repeatedly executed in an appropriate calculation cycle.

[0037] While one embodiment of the present invention has been described above in detail, the present invention is not limited to the above embodiment and various modifications are possible. For example, in the above embodiment, the departure warning line 15 is displayed so as to be superimposed on the roadway center line 11 because the vehicle is approaching the side of the roadway center line 11. However, if the vehicle approaches on the left side in Figure 4, the departure warning line 15 is displayed so as to be superimposed on the roadway outer edge line 12 on the left. Furthermore, while the above embodiment has been described using a curved road as an example, the same applies to a straight road. The departure warning line 15 is drawn so as to be superimposed on a straight lane marking, and an offset amount D in the lane width direction corresponding to the display distance X is given so as to have a maximum offset amount Dmax at the proximal end 15a.

Claims

A lane departure warning method for a vehicle, which, when detecting lane markings of a lane in which the vehicle is traveling from a camera image and determining that a lane departure warning should be issued, displays, as at least a part of the lane departure warning, a virtual departure warning line on a road surface ahead that is visible through a windshield of the vehicle via an AR-HUD of the vehicle so as to be superimposed on the lane marking on the side that the vehicle is approaching, As a display position of the deviation warning line relative to the lane markings, the deviation warning line is displayed by adding an offset amount according to the display distance in the front-rear direction of the vehicle so that the deviation warning line is superimposed on the lane markings in the lane width direction at a distal end far from the vehicle and is offset from the lane markings in the lane width direction at a proximal end close to the vehicle. A method for warning a vehicle from lane departure. 　 an offset amount at the proximal end is set so that the departure warning lines are adjacent to the lane markings at the proximal end without being separated from the lane markings; The lane departure warning method for a vehicle according to claim 1. 　 A radius of curvature of a lane marking is calculated based on a camera image or map information, and a basic display position of a departure warning line superimposed on the lane marking along the radius of curvature is calculated, and the display position is corrected by adding the offset amount. The lane departure warning method for a vehicle according to claim 1. 　 At the proximal end, the lane marking is offset toward an outer periphery of the radius of curvature of the lane marking. The lane departure warning method for a vehicle according to claim 1. 　 At the proximal end, the lane marking is offset toward an inner periphery of the radius of curvature of the lane marking. The lane departure warning method for a vehicle according to claim 1. 　 the offset amount is maximum at the proximal end and gradually decreases from the proximal end to the distal end such that the offset amount is zero at the distal end; The lane departure warning method for a vehicle according to claim 1. 　 A camera that detects lane markings of the lane in which the vehicle is traveling; An AR-HUD that displays virtual information overlaid on the real information seen through the windshield; A controller for controlling the AR-HUD; Equipped with The above controller is When it is determined that a lane departure warning should be issued, as at least a part of the lane departure warning, a virtual departure warning line is displayed on the road surface ahead, which can be seen through the windshield, via the AR-HUD so as to be superimposed on the lane marking on the side to which the vehicle is approaching; As a display position of the deviation warning line relative to the lane markings, the deviation warning line is displayed by adding an offset amount according to the display distance in the front-rear direction of the vehicle so that the deviation warning line is superimposed on the lane markings in the lane width direction at a distal end far from the vehicle and is offset from the lane markings in the lane width direction at a proximal end close to the vehicle. Vehicle lane departure warning system.

Images