Vehicle driving assistance control device

Abstract

Vehicle driving assistance control device, comprising: a first environment detection device for detecting lane line information on a preceding lane based on image information, a second environmental sensing device for detecting information regarding a three-dimensional object next to a road in the preceding lane based on transmitted and received radio wave information, a vision range estimation device for estimating the vision range (Lv) of a driver based on lane line information, a lane estimation device for estimating the lane based on at least one of the lane line information and the three-dimensional object information, a lane estimation device for estimating the lane of travel of the vehicle (1), a deviation position estimation device for estimating a deviation position (Pt) at which the vehicle (1) will deviate from the lane, based on the lane estimated by the lane estimation device and the lane estimated by the lane estimation device, and a lane departure prevention control device for performing lane departure prevention control of at least one of a notification to the driver and automatic braking according to a possibility of departure from the lane, wherein the lane departure prevention control device determines that a departure point is detected by the driver and the lane departure prevention control is not executed if the departure point is equal to or smaller than the field of vision.

Description

[0001] The present invention relates to a vehicle travel support control device for executing warning and automatic braking to prevent a vehicle from deviating from a lane ahead.

[0002] In recent years, various types of driving assistance control devices for vehicles have been developed and put into practice. These devices detect a lane in front of the vehicle, a three-dimensional object such as a vehicle, and an obstacle in front of the vehicle using a camera, a radar, and the like, and prevent the vehicle from deviating from the lane and from colliding with the three-dimensional object. In such a driving assistance control device, highly accurate detection of the driving environment in front of the vehicle directly results in accurate execution of the driving assistance control. Therefore, for example, a device disclosed in Japanese Unexamined Patent Application Publication (JP-A) No.A vehicle surroundings monitoring device disclosed in JP 2003-217099 A determines the position of an object based on information in a captured image and calculates the position of the object based on a reflected wave from an emitted radio wave. The device then calculates an observation position obtained as a result of observation based on both thus calculated positions and determines a possibility that the object in front of the subject vehicle is approaching the vehicle at a predetermined distance or less. If the possibility is determined to be high, this possibility is notified to the driver.

[0003] However, this vehicle driving assistance control device for preventing collisions has a major problem in driving assistance technology, in which it is difficult to determine how to extract, present, and control the necessary information when required by the driver. Specifically, even if a three-dimensional object such as an obstacle in front of the vehicle is detected with a high degree of accuracy, as in the device disclosed in JP-A No.The vehicle environment monitoring device disclosed in JP 2003-217099 A and described above has the following disadvantage: if a hazard of which the driver is already aware is subject to a warning or control to a greater extent than necessary, the driver will become accustomed to this warning or control, and this may lead to a delayed reaction when the hazard actually occurs, or the driver may be disturbed or annoyed by the warning or control and disable the driving assistance function. This may result in the driving assistance control device being unable to perform its function.

[0004] The present invention has been made in view of the above-mentioned circumstances, and has an object to provide a vehicle travel support control device capable of appropriately preventing a vehicle from colliding with an obstacle on the side of the road and from deviating from a travel lane by issuing a notification to the driver, activating the brakes, and the like, with reference to a travel lane ahead of the vehicle which is assumed not to be recognized by the driver.

[0005] According to one aspect of the present invention, there is provided a vehicle travel support control device comprising: first environment detection means for detecting lane line information on a preceding lane based on image information; second environment detection means for detecting information regarding a three-dimensional object adjacent to a road of the preceding lane based on radio wave information that is transmitted and received; visibility range estimation means for estimating the driver's visibility range based on the lane line information; lane estimation means for estimating the driving lane based on at least one of the lane line information and the three-dimensional object information; travel lane estimation means for estimating the travel lane of the vehicle; deviation position estimation means for estimating a deviation position;at which the vehicle will deviate from the lane, based on the lane estimated by the lane estimation means and the lane estimated by the lane estimation means, and lane departure prevention control means for executing lane departure prevention control of at least one of a driver notification and automatic braking according to a possibility of deviating from the lane, wherein the lane departure prevention control means determines that a departure point is detected by the driver and the lane departure prevention control is not executed when the departure point is equal to or smaller than the visibility range.

[0006] The invention will be further explained below using an embodiment with reference to the drawings in which Fig. 1 is a schematic configuration diagram showing a vehicle travel support control device according to an embodiment of the present invention mounted on a vehicle, Fig. 2 is a functional block diagram showing a control unit according to an embodiment of the present invention, Fig. 3 is a flowchart showing a vehicle travel support control program according to the embodiment of the present invention, and Fig. 4 is a diagram showing a lane departure prevention control according to the embodiment of the present invention.

[0007] In the following, an embodiment of the present invention will be described with reference to the drawings.

[0008] In Fig. 1, a vehicle (subject vehicle) 1 such as an automobile includes a driving assistance control device 2. The driving assistance control device 2 mainly includes a stereo camera unit 3 serving as the first environment recognition device according to the present invention, a millimeter-wave radar unit 4 serving as the second environment recognition device according to the present invention, and a control unit 5. The driving assistance control device 2 is connected to various types of sensors such as a vehicle speed sensor 6 for detecting a vehicle speed V0 of the vehicle 1 and a steering angle sensor 7 for detecting a steering angle θH. The control unit 5 outputs a control signal to a warning lamp 9 and a brake drive unit 10 when necessary.

[0009] The stereo camera unit 3 includes a stereo camera 31 and a first environment detection unit 32 for processing signals from the stereo camera 31.

[0010] The stereo camera 31, as a stereo optical system, includes a pair (left and right) of CCD cameras with semiconductor image sensors such as charge-coupled devices (CCDs). The CCD cameras constituting the stereo camera 31 are mounted at a predetermined distance from each other on the front portions of a roof lining of a passenger compartment of the vehicle 1, capture stereo images of external objects from different viewing angles, and output the captured image information to the first environment detection unit 32.

[0011] The first environment recognition unit 32 receives the image information from the stereo camera 31 and estimates the driving lane by recognizing lane line data in front of the vehicle, etc. The first environment recognition unit 32 processes the image information from the stereo camera 31, for example, as follows: The first environment recognition unit 32 generates distance information for a pair of (right and left) images (stereo images) obtained by capturing the environment in front of the vehicle 1 in the traveling direction with the stereo camera 31, using differences between corresponding positions according to a principle of triangulation.The first environment detection unit 32 first divides the reference image (for example, the right image) into small regions, compares the luminance or color pattern of each small region with that of the comparison image to find a region in the comparison image that corresponds to it, and obtains a distance distribution over the entire reference image. The first environment detection unit 32 also checks the luminance difference between neighboring pixels for each pixel in the reference image, extracts pixels with luminance differences greater than a threshold value (contrast threshold) as edges, and adds distance information to the extracted pixels (edges) to generate a distribution image (distance image) of the edges with the distance information.The first environment recognition unit 32 then performs, for example, a well-known grouping process on the distance image and performs pattern matching with various types of predetermined templates to thereby recognize the lane line in front of the vehicle. This lane line recognition is continuously monitored through corresponding image frames. When the first environment recognition unit 32 recognizes the lane line, it also stores, as lane line data, the width W of the lane line from the position coordinates of the right and left lane lines and the widthwise position of the vehicle 1 within the lane (the distance of the vehicle 1 from the left lane line and the distance of the vehicle 1 from the right lane line).For a lane line different from the above lane line obtained based on the luminance and contrast of the image information given by the stereo camera 31, for example, a lane line section that is interrupted for some reason and a lane line of a self-lane that is assumed to continue to exist, the first environment recognition unit 32 estimates by further extending the lane line obtained from the image information from the stereo camera 31, thus estimating it as the coordinate data. The estimated coordinate data of the lane line can be obtained using data of a map and the like of a navigation system, not shown. Then, the lane line information on the lane thus obtained from the stereo camera unit 3 is output to the control unit 5.

[0012] The millimeter wave radar unit 4 includes a millimeter wave transmission-reception unit 41 and a second environment detection unit 42 for processing a signal output from the millimeter wave transmission-reception unit 41.

[0013] The millimeter-wave transmission-reception unit 41 is provided at the front end of the vehicle 1 and configured to transmit a predetermined millimeter wave (for example, a radio wave of 30 GHz to 100 GHz) toward the front of the vehicle and receives the reflected millimeter wave, thus outputting the transmitted and received data to the second environment detection unit 42.

[0014] The second environment detection unit 42 detects a three-dimensional object by processing the transmitted and received data from the millimeter-wave transmission-reception unit 41 as follows: The second environment detection unit 42 measures a relative distance from the vehicle 1 to a target based on a time period between the time when the transmission wave is reflected from the target and the time when the transmission wave returns. Then, a portion where the same distance value continuously appears, as well as the size (width), and coordinate data as a part of a three-dimensional object are extracted from the distribution state of the distance values.For each three-dimensional object thus extracted, the distance from the vehicle 1 is derived, and the three-dimensional object located on the roadside of the lane in front of the vehicle is detected and extracted by performing a well-known grouping process on the distance information and performing pattern matching with various types of predetermined templates. The three-dimensional object information regarding the three-dimensional object is output to the control unit 5.

[0015] The control unit 5 receives the lane line information regarding the lane from the first environment recognition unit 32 of the stereo camera unit 3, receives the three-dimensional object information regarding the three-dimensional object from the second environment recognition unit 42 of the millimeter-wave radar unit 4, receives the vehicle speed V0 from the vehicle speed sensor 6, and receives the steering angle θH from the steering angle sensor 7.

[0016] Then, the control unit 5 estimates the driver's field of vision based on the lane line information based on the input information as explained above, estimates the lane based on at least one of the lane line information and the three-dimensional object information, estimates the traveling lane of the vehicle 1, and estimates the departure position at which the vehicle 1 will deviate from the lane based on the estimated traveling lane and the estimated traveling lane. When the departure position is beyond the driver's field of vision, the control unit 5 executes at least one of notifying the driver or automatically braking according to the possibility of lane departure.

[0017] As in Fig. 2, the control unit 5 mainly includes a visibility range estimation unit 51, a travel lane estimation unit 52, a first deviation point distance calculation unit 53, a second deviation point distance calculation unit 54, a control determination unit 55, a warning control unit 56, and a brake control unit 57.

[0018] The visibility range estimation unit 51 receives the lane line information regarding the driving lane from the first environment detection unit 32, then estimates the driver's visibility range Lv, and outputs the driver's visibility range Lv to the control determination unit 55. The driver's visibility range Lv is estimated, for example, as follows: a human-vision average contrast value for detecting the lane line on the road surface is set as a threshold value in advance through experiments. The visibility range estimation unit 51 compares the threshold value and the lane line information on the driving lane received from the first environment detection unit 32 (distance information ahead of the lane line), and the furthest distance of the lane line detected with a contrast value greater than the threshold value is estimated as the driver's visibility range Lv.The visual range estimation unit 51 thus serves as the visual range estimation means according to the present invention.

[0019] The lane estimation unit 52 receives the vehicle speed V0 from the vehicle speed sensor 6 and receives the steering angle θH from the steering angle sensor 7. Then, as exemplified in Fig. 4, the travel path of the vehicle 1 is estimated based on the vehicle speed V0 and the steering angle θH, for example, by approximation using the travel path of a curve radius ρa according to the following equation (1): ρa=(1+A⋅V02)⋅|⋅n / θHe where A denotes steering characteristics, I denotes the wheelbase, n denotes a steering gear ratio, and θHe denotes a predicted steering angle value. The predicted steering angle value θHe is calculated, for example, by the following equation (2): θHe=θH+(dθH / dt)⋅Δt where (dθH / dt) denotes a steering angular velocity and Δt denotes a predicted time.

[0020] The travel lane of the vehicle 1 estimated by the travel lane estimation unit 52, as well as the curve radius ρa, are output to the first deviation point distance calculation unit 53. The travel lane estimation unit 52 is thus provided as the travel lane estimation device according to the present invention.

[0021] The first deviation point distance calculation unit 53 receives the lane line information regarding the travel lane from the first environment recognition unit 32 and receives the curve radius ρa of the travel lane of the vehicle 1 from the travel lane estimation unit 52. Then, based on the width direction distance Wl to the approach direction travel lane of the vehicle 1 and the curve radius ρa of the travel lane, the distance from the travel lane of the vehicle 1 to the deviation position (indicated by point Pd in Fig. 4) as a distance Ld to the first deviation point calculated according to, for example, the following equation (3): Ld=ρa⋅cos(sin−1((ρa−Wl) / ρa))

[0022] The first deviation point distance calculation unit 53 is thus provided as the lane estimation device and the deviation position estimation device according to the present invention. The thus calculated distance Ld to the first deviation point is output to the control determination unit 55, the warning control unit 56, and the braking control unit 57.

[0023] The second deviation point distance calculation unit 54 receives the lane line information regarding the lane from the first environment recognition unit 32 and receives the three-dimensional object information regarding the three-dimensional object from the second environment recognition unit 42. Then, the distance to the deviation position (indicated by a point Pt in Fig. 4) at which the vehicle 1 travels straight ahead without change and then deviates from the lane after passing a three-dimensional object (for example, a guardrail, a curb, or the like) on one side of the road, is calculated as a distance Lt to the second deviation point. In this case, assuming that, as in Fig. 4, the deviation position Pt is located between an obstacle P01 (x1, y1) on one side of the road at the top of the figure and an obstacle P02 (x2, y2) on one side of the road at the bottom of the figure, the distance Lt to the second deviation point, which is a straight line passing therebetween, is calculated, for example, using the following equation (4): Lt=x1+(y1 / (y2−y1))⋅(x2−x1)

[0024] The second deviation point distance calculation unit 54 is thus provided as the lane estimation device and the deviation position estimation device according to the present invention. The thus calculated distance Lt to the second deviation point is output to the brake control unit 57.

[0025] The control determination unit 55 receives the driver's visual range Lv from the visual range estimation unit 51 and receives the distance Ld to the first deviation point from the first deviation point distance calculation unit 53. Then, the driver's visual range Lv and the distance Ld to the first deviation point are compared. If the distance Ld to the first deviation point is equal to or smaller than the driver's visual range Lv (Ld ≤ Lv), it is determined that the deviation position at which the vehicle 1 will deviate from the lane (the point Pd in Fig. 4) is recognized by the driver. Then, the determination result indicating that the lane departure prevention control is not executed is output to the warning control unit 56 and the braking control unit 57. If notification and automatic braking are also performed for a lane departure recognized by the driver, the driver becomes accustomed to such notification and automatic braking, and this may lead to a delay in response when the danger actually arises, or the driver may be disturbed and annoyed by the notification and automatic braking and turn off the driving support function, whereby the driving support control device 2 cannot achieve its function. Therefore, in such a case, the activation of the warning and automatic braking is suppressed.

[0026] The warning control unit 56 receives the vehicle speed V0 from the vehicle speed sensor 6, receives the distance Ld of the first departure point from the first departure point distance calculation unit 53, and receives the control execution determination result from the control determination unit 55. When the control execution is permitted, a notification is output, for example, as follows. When the following equation (5) is satisfied, a warning lamp 9 is turned ON, thereby notifying the driver of the possibility of lane departure. Ld <Lal where Lal = V02 / (2 · a1) and a1 is a delay set in advance by experiments, calculation and the like.

[0027] The brake control unit 57 receives the vehicle speed V0 from the vehicle speed sensor 6, receives the distance Ld of the first deviation point from the first deviation point distance calculation unit 53, receives the distance Lt of the second deviation point from the second deviation point distance calculation unit 54, and receives the execution determination result of the control from the control determination unit 55. When the execution of the control is permitted and, for example, at least one of the following equations (6) and (7) is satisfied, the brake drive unit 10 is activated to execute automatic braking. Ld <Lb1 Lt <Lb2 where Lb1 = V02 / (2 · a2) and a2 is a delay set in advance by experiments, calculation, and the like, while Lal > Lb1. Lb2 = V02 / (2 · a3) and a3 is a delay set in advance by experiments, calculation, and the like, where Lb1 < Lb2.

[0028] The warning control unit 56 and the brake control unit 57 are thus provided as the lane departure prevention control device according to the present invention.

[0029] The vehicle travel support control executed by the control unit 5 configured as described above will be described with reference to the flowchart shown in Fig. 3 described.

[0030] In step 101 (hereinafter abbreviated as "S"), required information is read. The required information includes the lane line information of the lane, the three-dimensional object information of the three-dimensional object, the vehicle speed V0, and the steering angle θH.

[0031] The program then proceeds to S102, where the driver's visual range Lv is estimated, for example, as follows: a human-vision average contrast value for detecting the lane line on the road surface is set in advance as a threshold value through experiments. The visual range estimation unit 51 compares the threshold value and the lane line information of the lane received from the first environment detection unit 32 (distance information in front of the lane line), and the furthest distance of the lane line detected with a contrast value higher than this threshold value is estimated as the driver's visual range Lv.

[0032] The program then proceeds to S103, where the travel lane estimation unit 52 estimates the travel lane of the vehicle 1, for example, by approximation using the travel path of a curve radius ρa according to the equation (1) described above.

[0033] Then, the program proceeds to S104, where the first deviation point distance calculation unit 53 calculates, as the first deviation point distance Ld, the distance to the deviation point at which the vehicle 1 will deviate from the lane (indicated by the point Pd in Fig. 4) according to, for example, equation (3) above.

[0034] The program then proceeds to S105, where the control determination unit 55 compares the driver's visual range Lv and the distance Ld of the first departure point to determine whether the distance Ld of the first departure point is beyond the driver's visual range Lv (Ld > Lv). If the distance Ld of the first departure point is farther than the driver's visual range Lv (Ld > Lv), the program proceeds to S106. However, if the distance Ld of the first departure point is equal to or less than the driver's visual range Lv (Ld ≤ Lv), it is determined that the departure position at which the subject vehicle 1 will leave the lane (the point Pd in Fig. 4) is recognized by the driver and the program is exited.

[0035] If, as a result of the determination in S105, Ld > Lv and the program proceeds to S106, the warning control unit 56 determines whether the above equation (5) is satisfied or not. If the above equation (5) is satisfied, the program proceeds to S107 to execute the notification control, and the warning lamp 9 is turned ON to inform the driver that the vehicle is about to deviate from the lane. If equation (5) is not satisfied, it is not necessary to inform the driver, and the program is therefore exited without further processing.

[0036] When the notification control is executed in step S107, the program proceeds to S108, where the second deviation point distance calculation unit 54 calculates, as the distance Lt to the second deviation point, the distance to the deviation position (indicated by the point Pt in Fig.4) is calculated at which, when the subject vehicle 1 is traveling straight, it will deviate from the lane after passing a three-dimensional object (for example, a guardrail or a curb) on one side of the road, for example, according to equation (4) explained above.

[0037] Then, the program proceeds to S109, where, for example, the brake control unit 57 determines whether or not at least one of the above equations (6) and (7) is satisfied. If this determination is satisfied, the program proceeds to S110, where the brake control is executed and the brake drive unit 10 is activated to perform automatic braking. Then, the program exits. If this determination is not satisfied, that is, Ld ≥ Lb1 and Lt ≥ Lb2, the program exits without further processing.

[0038] As described above, according to the driving assistance device of the present embodiment, the driver's visual range is estimated based on the lane line information, the driving lane is estimated based on at least one of the lane line information and the three-dimensional object information, the traveling lane of the vehicle 1 is estimated, and the deviation position at which the vehicle 1 will deviate from the lane is estimated based on the estimated traveling lane and the estimated traveling lane. When the traveling deviation is beyond the visual range, at least one of notifying the driver and executing automatic braking is performed according to the possibility of the lane deviation.If notification and automatic braking are also performed for lane deviations detected by the driver, the driver will become accustomed to such notifications and automatic braking, and this may lead to a delay in response when the danger actually arises. Or the driver may be disturbed and annoyed by the notification and automatic braking, and turn off the driving support function, thereby failing to fulfill the original function of the driving support control device 2. Therefore, in such a case, the activation of the warning and automatic braking is suppressed.Such control prevents the vehicle from colliding with an obstacle on one side of the road and from deviating from a lane by issuing a notification to the driver, activating brakes, and the like, with respect to a lane in front of the vehicle when it is assumed that this will not be recognized by the driver.

[0039] In the above-described embodiment of the present invention, the lane line detection and the view range setting are performed based on image information provided by the stereo camera unit 3. Alternatively, the lane line detection and the view range setting may be performed based on image information provided by a monocular camera unit or a color camera unit. In the present embodiment, the second environment recognition unit is configured to detect the three-dimensional object information using radio wave information provided by the millimeter-wave radar. Alternatively, the three-dimensional object information may be detected using optical wave information provided by a laser radar. SUMMARY OF REVELATION

[0040] A vehicle travel support control device is provided. A control unit 5 receives lane line information provided by a stereo camera unit 3 and information about a target three-dimensional object provided by a millimeter-wave radar unit 4. The control unit 5 estimates the driver's field of vision based on the lane line information, estimates the driving lane based on at least one of the lane line information and the target three-dimensional object information, and estimates the traveling lane of the subject vehicle 1. Based on the estimated traveling lane and traveling lane, the control unit 5 estimates a departure position at which the subject vehicle 1 will deviate from the traveling lane.When the deviation position is beyond the visual range, the control unit 5 performs at least either a driver notification or automatic braking in accordance with the possibility of lane deviation.

Claims

[1] A vehicle travel support control device comprising: a first environment detection device for detecting lane line information on a lane ahead based on image information, a second environment detection device for detecting information regarding a three-dimensional object adjacent to a road of the lane ahead on the basis of transmitted and received radio wave information, a visibility range estimator for estimating the visibility range (Lv) of a driver on the basis of the lane line information, a lane estimating device for estimating the lane based on at least one of the lane line information and the three-dimensional object information, a lane estimation device for estimating the lane of the vehicle (1), a deviation position estimating means for estimating a deviation position (Pt) at which the vehicle (1) will deviate from the lane, on the basis of the lane estimated by the lane estimating means and the lane estimated by the lane estimating means, and a lane departure prevention control device for executing a lane departure prevention control of at least one of a driver notification and automatic braking according to a possibility of departure from the lane, wherein the lane departure prevention control means determines that a departure point is detected by the driver and the lane departure prevention control is not executed when the departure point is equal to or smaller than the visibility range. [2] The vehicle travel support control device according to claim 1, wherein the visibility range estimation means previously sets, as a threshold value, a human vision average contrast value for detecting the lane line and estimates, as the driver's visibility range (Lv), a distance to a lane line detected with a contrast value higher than the threshold value. [3] The vehicle travel support control device according to claim 1 or 2, wherein the lane departure prevention control means notifies the possibility of lane departure to the driver when the departure position is beyond the visual range and the departure position is closer than a predetermined threshold value. [4] A vehicle travel support control device according to any one of claims 1 to 3, wherein a lane departure prevention control unit executes the automatic braking when the departure position is beyond the visual range and a distance to a position where the vehicle (1) collides with the three-dimensional object on the side of the road is closer than a predetermined threshold value.

Images