Driving environment detection device

Abstract

Driving environment detection device comprising: - an information acquisition section (310) mounted in a vehicle and configured to acquire information about at least one front area of ​​the vehicle with respect to one direction of travel of the vehicle; and - a road surface calculation section (1, 100) configured to calculate a road surface of a roadway and an area lower than the road surface in the front area of ​​the vehicle from the information acquired by the information acquisition section (310), wherein: - the information includes a plurality of first visually characteristic points present on the road surface and a plurality of second visually characteristic points present in the area below the road surface, - the road surface calculation section (1, 100) is designed to recognize a boundary area between the majority of first visually characteristic points and the majority of second visually characteristic points as a boundary area between the road surface and the area lying lower than the road surface, - the identified boundary area is the end of the road, - the road surface calculation section (1, 100) includes a distance data collection section (340) designed to store information about distance at predetermined intervals, - the road surface calculation section (1, 100) is designed to capture the first and second regions by mutual superposition of a plurality of stored distance data and - the road surface calculation section (1, 100) captures the first and second regions based on an average of a portion of current environment data and a portion of previous environment data, if the portion of current environment data differs from that of previous environment data while the majority of stored distance data are superpositioned.

Description

BACKGROUND OF THE INVENTION

[0001] The present invention relates to a driving environment detection device configured to detect a driving environment in which a vehicle is moving.

[0002] Japanese patent application JP 2009-286222 A discloses a prior art device. In this technique, the height of a road surface is detected by sensing specific positions on the lateral side of a vehicle using a plurality of ultrasonic sensors.

[0003] WO 2004 / 021 546 A2 describes a means of transport with a 3D distance image camera and a method for constructing and operating the 3D distance image camera in means of transport, in particular motor vehicles. In particular, the method and device are suitable for use in motor vehicles as a brake assistant, damage reduction system, closing velocity sensor system, preferably for impact detection in the front, side, and / or rear, accident avoidance, pedestrian protection, stop-and-go or traffic jam assistant, parking aid, for example, parking space measurement, parking assistant, and / or autonomous parking. Furthermore, the proposed objects can advantageously be used for blind spot monitoring, as a lane departure warning system, lane change assistant, in particular with blind spot monitoring and / or lane position detection, lane keeping assistant, rollover and / or pitch angle detection.Furthermore, the proposed items can be advantageously used for night vision, road condition detection, as a doorstop, for traffic sign detection, as a light sensor, weather sensor, trafficability sensor and / or as a docking aid or reversing aid.

[0004] DE 10 2004 057 296 A1 discloses a driver assistance device for warning a driver of a motor vehicle of an impending departure from the roadway or lane, comprising at least one imaging sensor, an evaluation device connected to the imaging sensor for detecting road edge and / or lane markings and / or road edges in the area detected by the imaging sensor, and a warning device connected to the evaluation device, in which at least one distance-providing sensor is additionally connected to the evaluation device, with which the distance to objects raised above the road surface in the area of ​​the road edge, in particular a structural boundary of the road edge, can be determined.

[0005] DE 10 2005 039 167 A1 relates to a driver assistance device for warning a driver of a vehicle of an impending departure from a roadway and / or a lane departure, which comprises means for detecting a road edge and / or lane marking and / or a road edge, means for detecting objects in the vicinity of the vehicle that are raised above the roadway, means for identifying structural boundaries of the roadway and / or the lane from the detected raised objects, and means for issuing a warning in the event of an impending departure from the roadway and / or a lane departure, and in which means for assessing a current environmental situation at least by comparing the spatial position of the identified structural boundary,the roadway and / or lane with the spatial position of the detected road edge marking and / or lane marking or the detected road edge as well as means for adapting the output of a warning to the current environmental situation.

[0006] DE 102 18 924 A1 defines a method for course prediction in motor vehicles which have a positioning system for objects located in front of the vehicle, in which method a function which describes the course of the road edge is calculated on the basis of measured distance and angle data for stationary targets at the edge of the road, characterized in that a plurality of stationary targets are identified and tracked, that for various subsets of the set of tracked stationary targets, under the assumption that these stationary targets are located at the edge of the road, the road edge course is estimated and that on the basis of the plausibility of the possible road edge courses obtained, a distinction is made between road edge targets and interfering objects and the most probable road edge course is determined on the basis of the road edge targets.

[0007] JP 2009-286222 A discloses an accident prediction device that can correctly predict the possibility of accidents such as a vehicle drifting into a ditch, a vehicle overturning caused by the ditch, or a vehicle falling into the bottom of a cliff. The accident prediction device is provided with a first sensor for irradiating the device with a supersonic wave so that it forms an oblique first angle from a side of a passenger car with respect to a roadway and for measuring a first required time required to detect a reflection wave; a second sensor for irradiating the device with the supersonic wave at a second angle smaller than the first angle and for measuring a second required time required to detect a reflection wave;a third sensor for irradiating the device with the supersonic wave at a third angle smaller than the second angle and for measuring a third required time required to detect a reflected wave; and a prediction part for predicting the possibility of the occurrence of the accident based on whether the respective required times at which the reflected wave is detected are longer than respective predetermined reference times; and an information part for informing the possibility of the occurrence of the accident predicted by the prediction part. SUMMARY OF THE INVENTION

[0008] However, since the specific positions on the lateral side are monitored using the above-mentioned prior art technology, the frontal area and the distant area in front of the vehicle cannot be detected. The above-mentioned prior art technology does not adequately improve safety through vehicle control.

[0009] It is an object of the present invention to provide a driving environment detection device which is designed to detect a driving environment more accurately.

[0010] The object is alternatively achieved by features of independent claims 1, 9 and 14. The subclaims each relate to advantageous developments of the invention.

[0011] According to one aspect of the present invention, there is provided a driving environment detection device comprising, among other things: an information detection section mounted in a vehicle and configured to detect information about at least a front area of ​​the vehicle with respect to a traveling direction of the vehicle; and a road surface calculation section configured to calculate a road surface of a traveling road and an area lower than the road surface in the front area of ​​the vehicle from the information detected by the information detection section.

[0012] According to another aspect of the present invention, there is provided a driving environment detection device comprising, among other things: a camera mounted in a vehicle and configured to detect an environment of a front area of ​​the vehicle with respect to a traveling direction of the vehicle; and a road surface calculation section configured to calculate a boundary area between a road surface of a driving road and an area lower than the road surface beside the driving road, using distance data based on a distance to each object located in the front area of ​​the vehicle in the environment detected by the camera.

[0013] According to a further aspect of the present invention, there is provided a driving environment detection device, comprising, inter alia: a stereo camera configured to detect image information comprising a road surface of a driving road located in a front area of ​​a vehicle with respect to a traveling direction of the vehicle, with a plurality of cameras mounted in the vehicle, wherein the stereo camera is configured to measure a distance in the image information by means of a parallax caused during respective recording of images of the same object with the plurality of cameras;and a road surface calculation section configured to detect a first region corresponding to the road surface of the travel road and a second region corresponding to an area lower than the road surface adjacent to the travel road using the distance information measured by the stereo camera, wherein the road surface calculation section is configured to calculate a boundary area between the first and second regions as the end of the travel road.;

[0014] The further objects and features of this invention will become apparent from the following description with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a structural view of a vehicle to which a driving environment detection device according to a first embodiment of the present invention has been applied. Fig. 2 is a control block diagram of an electronic control unit according to the first embodiment. Fig. 3 is a flowchart showing the operation of a lane departure prevention control unit according to the first embodiment. Fig. 4 is a view showing a relationship between fixed coordinates of a vehicle and left and right lane lines. Fig. 5A to 5C are views each showing an imaginary lane boundary line calculated from the lane boundary line. Fig. 6A to 6C are views each showing an imaginary lane boundary line calculated from representative points. Fig. 7A to 7C are views each showing an imaginary lane boundary line calculated from the lane boundary line in an alternative manner. Fig. 8A to 8C are views each showing an imaginary lane boundary line calculated from representative points in an alternative manner. Fig. 9 is a block diagram showing a configuration of a driving environment detection device according to the first embodiment. Fig. 10 is a flowchart showing image processing executed by the driving environment detection apparatus according to the first embodiment. Fig. Figure 11 is a schematic view of a road shoulder constructed by steeply sloping areas. Fig. Figure 12 is a picture showing a schematic diagram obtained when the road shoulder constructed by steeply sloping areas was photographed from the vehicle. Fig. Figure 13 is a schematic view showing characteristic points which are simultaneously recorded when a real road is photographed. Fig. 14 is a schematic view showing superposition processing of image data according to the first embodiment. Fig. 15 is a schematic view showing a result acquired by imaging the road shoulder with respect to a cross-sectional direction of the road according to the first embodiment. Fig. Figure 16 is a schematic view showing a road shoulder constructed by moderately sloping areas. Fig. Figure 17 is a picture showing a schematic diagram obtained when the road shoulder constructed by the moderately sloping areas was photographed from the vehicle. Fig. Fig. 18 is a schematic view showing a result obtained by imaging the road shoulder with respect to a cross-sectional direction of the road. Fig. 19 is a systematic structural view of a vehicle equipped with a driving environment detection device according to a second embodiment of the present invention. Fig. 20 is a schematic view showing superposition processing of image data according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] In the following, embodiments according to the present invention will be explained with reference to the drawings. [First embodiment]

[0016] Fig. 1 is a structural view of a vehicle to which a driving environment detection device according to a first embodiment is applied. The vehicle according to the first embodiment includes a driving environment detection device 1, an electric steering unit 2, an electro-hydraulic brake unit 3, a brake booster 4, a steering wheel 5, a left front road wheel 6, a right front road wheel 7, a left rear road wheel 8, a right rear road wheel 9, an electronic control unit 10, and a vehicle motion detection sensor 11. The driving environment detection device 1 photographs (captures an image of) an area in front of the vehicle (vehicle with driving environment detection device) and a front area of ​​the vehicle using cameras or the like to generate data about the driving environment. These cameras are mounted at a central position mentioned below at the front end of the vehicle.The electronic steering unit 2 turns (steers) the left and right front wheels 6 and 7 based on a command corresponding to a steering angle of the steering wheel 5, but can also turn (steer) the left and right front wheels 6 and 7 independently of the steering angle of the steering wheel 5. The electro-hydraulic brake unit 3 controls a braking force of each of the four wheels 4-7 independently, in accordance with a driver's braking input or in accordance with a vehicle condition. The brake booster 4 amplifies (increases) the driver's braking input and transmits the amplified braking input to the electro-hydraulic brake unit 3. The vehicle motion detection sensor 11 detects a vehicle speed (vehicle velocity), longitudinal acceleration (front-rear directional acceleration), lateral acceleration (left-right directional acceleration), yaw rate, steering angle, steering torque, and the like.The electronic control unit 10 controls the driving environment detection device 1, the electric steering unit 2, and the electro-hydraulic braking unit 3 based on respective detection values ​​of the vehicle motion detection sensor 11. In a case where the vehicle tends to deviate from a driving lane (a lane boundary line) as detected from the image captured by the driving environment detection device 1, the electronic control unit 10 controls the electric steering unit 2 and / or the electro-hydraulic braking unit 3 as lane departure prevention control (or lane departure suppression control). Thereby, a yaw moment and / or deceleration is applied to the vehicle, so that the vehicle is prevented from leaving the lane.The electro-hydraulic brake unit 3 applies an equal amount of braking force to the left front wheel 6 and the right front wheel 7, and also applies an equal amount of braking force to the right rear wheel 8 and the left rear wheel 9 when the electro-hydraulic brake unit 3 is operated by the driver's braking input. On the other hand, at the time of lane departure prevention control, the electro-hydraulic brake unit 3 can apply different amounts of braking force to the respective left and right front wheels 6 and 7, and can also apply different amounts of braking force to the respective right and left rear wheels 8 and 9 when the electro-hydraulic brake unit 3 is operated by the lane departure prevention control unit.That is, the electro-hydraulic brake unit 3 can cause the braking forces of the front wheels 6 and 7 to become different from each other, and can also cause the braking forces of the rear wheels 8 and 9 to become different from each other. Thus, under lane departure prevention control, a yaw moment can be generated on the vehicle. Furthermore, if the electric control unit 2 receives both a driver-input command and a lane departure prevention control command, the electric control unit 2 executes control of the left and right front wheels 6 and 7 in accordance with a sum of the driver-input command and the lane departure prevention control command. (Lane Departure Prevention Control / Regulation System)

[0017] Fig. 2 is a control block diagram of the electronic control unit 10 according to the first embodiment. The electronic control unit 10 includes a deviation tendency calculation section 20 and a lane departure prevention control unit (lane departure suppression control unit) 21. The deviation tendency calculation section 20 calculates a tendency of deviation from a traveling lane of the vehicle. When the deviation tendency calculation section 20 detects a tendency of deviation from the traveling lane of the vehicle, the lane departure prevention control unit 21 controls the electric steering unit 2 and / or the electro-hydraulic braking unit 3 to apply a yaw moment and / or deceleration to the vehicle, thereby suppressing the tendency of the vehicle to deviate from the traveling lane.Note that the lane departure prevention control unit 21 corresponds to a vehicle state suppression control unit according to the present invention, which executes vehicle state suppression control to suppress a driving state of the vehicle based on a relationship between the vehicle and the lane boundary line. The departure tendency calculation section 20 includes a lane boundary detection section 22, a current vehicle position detection section (vehicle end region detection section) 23, an expected vehicle position detection section 24, an imaginary lane boundary calculation section 25, and an action-need decision section 26.The lane boundary detection section 22 detects lane boundary lines on the left and right sides of the road on which the vehicle is traveling from the image of the front of the vehicle captured by the driving environment detection device 1. The lane boundary lines are, for example, white lines, guardrails, or curbs. The current vehicle position detection section 23 detects the current positions of two predetermined points on the vehicle. These two predetermined points on the vehicle are predetermined points located inside or near the vehicle. The two predetermined points on the vehicle are designated as the left and right lane boundary lines, respectively. (The two predetermined points on the vehicle are designated as the left and right lane boundary lines for comparison, respectively.)For example, these predetermined points are right and left end portions of the front of the vehicle with respect to a traveling direction of the vehicle. The expected vehicle position detecting section 24 detects two predicted positions of the two predetermined points of the vehicle. These predicted positions are positions advanced (shifted forward) from the current position of the two predetermined points by a predetermined amount. Specifically, each of these predicted positions is a position at which the predetermined point arrives from the current position when a predetermined time interval has just elapsed. That is, the predetermined point of the vehicle is predicted to reach this predicted position when, from a current time point, the predetermined time interval has just elapsed.This predetermined time interval can be set shorter as the vehicle speed increases. The predicted positions of the two predetermined points of the vehicle are also provided for the left and right lane boundary lines (or for comparison with the respective left and right lane boundary lines), i.e., they are set on the left and right sides according to the current left and right positions of the two predetermined points.

[0018] The imaginary lane boundary calculation section 25 calculates an imaginary lane boundary line. The imaginary lane boundary line is essentially a tangent to an arbitrary point located on the lane boundary line near the predicted position (predetermined point) of the vehicle. The imaginary lane boundary line is calculated for each of the left and right current positions (the two predetermined points) of the vehicle. That is, the imaginary lane boundary line set to the left lane boundary line is located near the left predicted position (the left predetermined point) of the vehicle; on the other hand, the imaginary lane boundary line set to the right lane boundary line is located near the right predicted position (the right predetermined point) of the vehicle.

[0019] The action-need decision section 26 decides whether or not to activate the lane departure prevention control, that is, whether or not to execute intervention of the lane departure prevention control, based on a relationship between the current position of the vehicle, the predicted position of the vehicle, and the imaginary lane boundary line. The action-need decision section 26 includes a sign decision section 26a. The sign decision section 26a calculates a first line and a second line, and decides signs detected from the directional characteristics of these first and second lines with respect to the imaginary lane boundary line. More specifically, the first line is a straight line connecting the imaginary lane boundary line and the current position (the predetermined point) of the vehicle.The first line is perpendicular to the imaginary lane boundary line and passes through the current position (predetermined point) of the vehicle. The second line is a straight line connecting the imaginary lane boundary line and the current position (predetermined point) of the vehicle. The second line is perpendicular to the imaginary lane boundary line and passes through the predicted position (predetermined point) of the vehicle. The front area image captured by the driving environment detection device 1 is divided (spatially differentiated) into an inner lane area and an outer lane area, between which the imaginary lane boundary line is located. If the first and second lines are located in the same inner lane area or the same outer lane area, the aforementioned sign of the first line is identical to that of the second line.On the other hand, if the first line is located in the inner area of ​​the travel lane or the outer area of ​​the travel lane, and the second line is located in the other of the inner area of ​​the travel lane or the outer area of ​​the travel lane, the sign of the first line is different from that of the second line. The action-need decision section 26 determines that the lane departure prevention control is not necessary if the sign decision section 26a determines that the signs of the first and second lines are identical to each other when the vehicle is traveling in an area between the left and right imaginary lane boundary lines (i.e., in an imaginary roadway defined by the left and right imaginary lane boundary lines).On the other hand, the action-need decision section 26 determines that the lane departure prevention control is necessary if the sign decision section 26a determines that the signs of the first and second lines are different from each other when the vehicle is traveling within a defined range between the left and right imaginary lane boundary lines. That is, it is determined that the lane departure prevention control is necessary when the predicted position (predetermined point) of the vehicle is outside the imaginary lane boundary line (i.e., outside the imaginary roadway), even though the current position (predetermined point) of the vehicle is inside the imaginary lane boundary line (i.e., inside the imaginary roadway). At the times of the other cases, it is determined that the lane departure prevention control is not necessary.

[0020] The lane departure prevention control unit 21 executes the lane departure prevention control when the action-need decision section 26 has determined that the lane departure prevention control is necessary. On the other hand, the lane departure prevention control unit 21 does not execute the lane departure prevention control when the action-need decision section 26 has determined that the lane departure prevention control is not necessary. [Lane departure prevention control / regulation]

[0021] Fig. 3 is a flowchart showing the flow of lane departure prevention control executed in the electronic control unit 10 according to the first embodiment. Relevant steps of the flowchart will now be explained. This processing is repeated, for example, with a calculation period of 10 ms (milliseconds) during vehicle travel.

[0022] At step S1, the lane departure prevention control unit 21 reads the detection values ​​received from the vehicle motion detection sensor 11, such as the vehicle speed, the longitudinal acceleration, the lateral acceleration, the yaw rate, the steering angle, and the steering torque.

[0023] At step S2, the lane boundary detecting section 22 detects the position of the (real) lane boundary line from the front area image of the vehicle obtained from the driving environment detecting device 1. The position of the (real) lane boundary line is calculated and displayed in fixed coordinates of the vehicle. In addition, various positions mentioned below are basically calculated and displayed in the fixed coordinates of the vehicle. For example, as shown in Fig. 4, the fixed coordinates have an origin O, an X-axis, and a Y-axis. The origin O is a mounting position of the driving environment detection device 1 in the vehicle 31. A traveling direction (moving direction) of the vehicle is defined as the X-axis. A direction perpendicular to the X-axis, namely, a vehicle width direction, is defined as the Y-axis. The left and right lane boundary lines 32 and 33 are located on the left and right sides of the road, respectively. In this embodiment, the lane boundary line can be detected by some representative points 34, 35, 36, 37, 38, and 39, taking into account the calculation load, as shown in Fig. 4 shown.

[0024] At step S3, the current vehicle position detecting section 23 considers two arbitrary points inside or near the vehicle as the position of the vehicle. The two points considered as the position of the vehicle are set corresponding to the left and right lane boundary lines because these two points are compared with the positions of the left and right lane boundary lines as shown below. For example, the two points may be a left front end and a right front end of the vehicle. At this time, the current vehicle position detecting section 23 calculates current positions of the two points (current position of the vehicle).Then, the expected vehicle position detecting section 24 calculates a position (from each of the two points) of the vehicle as a predicted position of the vehicle, which is obtained when a predetermined time interval has just elapsed from the current position of the vehicle. At this time, the expected vehicle position detecting section 24 calculates the predicted position (from each of the two points) of the vehicle from the values ​​detected at step S1 and previously stored vehicle specifications. That is, the predicted position of the vehicle is calculated based on the vehicle speed, the steering angle, the yaw rate, the vehicle width, the vehicle length, the mounting position of the cameras, and the like.Therefore, the two current positions and the two predicted positions of the two points of the vehicle are set corresponding to the left and right lane boundary lines in order to compare these two current positions and these two predicted positions with the positions of the left and right lane boundary lines.

[0025] At step S4, the imaginary lane boundary calculation section 25 calculates an imaginary lane boundary line. The imaginary lane boundary line is a tangent to the lane boundary line drawn near the predicted position (from each of the two points) of the vehicle. For example, as shown in Fig. 5A-5C, each of the imaginary lane boundary lines 42 may be tangent to a point on the lane boundary line that is closest to the predicted position 41 of the vehicle. This predicted position 41 is a position that the vehicle will reach when a predetermined time interval has just elapsed from the current position 40 of the vehicle. In the case of Fig. 5A, the vehicle 31 has a tendency to deviate (leave) from the left lane boundary line 32 of a straight path. In the case of Fig. 5B, the vehicle 31 has a tendency to deviate (leave) from the left lane boundary line 32 positioned on a radially outer side of a curved path (with respect to the center of curvature of the curved path). In the case of Fig. 5C, the vehicle 31 has a tendency to deviate from the right lane boundary line 33 positioned on a radially inner side of a curved path. Alternatively, in the case where the lane boundary line is represented by multiple representative points, in order to reduce the computation load, the two representative points closest to the predicted position 41 of the vehicle are selected. Then, a straight line passing through these two selected points is regarded as the imaginary lane boundary line, as shown in Fig. 6A-6C. In the case of Fig. 6A, the imaginary carriageway boundary line is a straight line that includes (passes through) two representative points 44 and 45. In the case of Fig. 6B, the imaginary lane boundary line is a straight line containing two representative points 47 and 48. In the case of Fig. 6C, the imaginary lane boundary line is a straight line containing two representative points 50 and 51. A distance between the predicted position 41 of the vehicle and the imaginary lane boundary line 42 is defined as a deviation height (departure height) d.

[0026] Furthermore, as in Fig. 7A-7C, the imaginary lane boundary line 42 can be set as a tangent at an intersection point 53 positioned on the lane boundary line 32 (or 33). This intersection point 53 is the intersection point of the lane boundary line 32 (or 33) and a straight line 52 passing through the predicted position 41 of the vehicle and at right angles to the vehicle's direction of travel. Alternatively, in the case where the lane boundary line is represented by multiple representative points, in order to reduce the computation load as mentioned above, the two representative points closest in distance to the straight line 52 are selected from the plurality of representative points. The straight line 52 passes through the predicted position 41 of the vehicle and is at right angles to the vehicle's direction of travel in the same way as in the case of Fig. 7A-7C. Then, a straight line passing through the two selected representative points is regarded as the imaginary lane boundary line 42, as shown in Fig. 8A-8C. In the case of Fig. 8A, the imaginary carriageway boundary line is a straight line that includes (passes through) two representative points 44 and 45. In the case of Fig. 8B, the imaginary carriageway boundary line 42 is a straight line that includes two representative points 47 and 48. In the case of Fig. 8C, the imaginary carriageway boundary line 42 is a straight line containing two representative points 50 and 51. In such cases, as shown in Fig. 8A-8C, a distance between the predicted position 41 of the vehicle and the intersection point 53 formed by the imaginary lane boundary line 42 and the straight line 52 is defined as the deviation height d.

[0027] Since the imaginary lane boundary line 42 is set as mentioned above, the lane boundary line is identical to (equal to) the imaginary lane boundary line in the case of a straight path. On the other hand, in the case of a radially outer side of the curved path with respect to the center of curvature of the curved path, the imaginary lane boundary line is calculated to be located in a radially outer region beyond the lane boundary line. That is, in this case, the imaginary lane boundary line is located outside the lane boundary line (i.e., as if it were not located in the roadway defined by the lane boundary line).However, in the case of a radially inner side of the curved path relative to the center of curvature of the curved path, the imaginary lane boundary line is calculated to be located in a radially outer region beyond the lane boundary line. That is, in this case, the imaginary lane boundary line is located within the roadway defined by the lane boundary line. In this embodiment, the calculated imaginary lane boundary line can be rotated, slightly spaced, or shifted—that is, it can be adjusted.

[0028] At step S5, the action-need decision section 26 decides whether or not the predicted position (each of the two points) is outside the imaginary lane boundary line calculated in step S4 (outside the imaginary roadway), that is, whether or not the predicted position of the vehicle is in the lane outer region formed by the imaginary lane boundary line. If the predicted position of the vehicle is outside the imaginary lane boundary line, that is, in the lane outer region defined by the imaginary lane boundary line, the program proceeds to step S6. If the predicted position of the vehicle is inside the imaginary lane boundary line (inside the imaginary roadway), that is, in the lane inner region defined by the imaginary lane boundary line, the program is reset.

[0029] At step S6, the action decision section 26 decides whether the current position (each of the two points) of the vehicle is within the imaginary lane boundary line (inside the imaginary roadway) or not, that is, whether the current position of the vehicle is within the lane inner area defined by the imaginary lane boundary line. If the current position of the vehicle is within the imaginary lane boundary line (inside the imaginary roadway), the program proceeds to step S7. If the current position of the vehicle is outside the imaginary lane boundary line (outside the imaginary roadway), the program is reset.

[0030] At step S7, the lane departure prevention control unit 21 calculates an operation amount (influenced variable) M by multiplying a gain factor K by the deviation amount d calculated at step S4, as shown in the following formula (1). M=K×d

[0031] The lane departure prevention control unit 21 operates the electric steering unit 2 and / or the electro-hydraulic brake unit 3 based on the operation height M. Thereby, the yaw moment and / or the deceleration is applied to the vehicle to execute the lane departure prevention control. (Driving environment detection device)

[0032] Next, the driving environment detection device is explained below. Fig. 9 is a block diagram showing a configuration of a driving environment detection device 1 according to the first embodiment. The driving environment detection device 1 includes a stereo camera formed of a pair of cameras (a pair of camera lenses) 310a and 310b (hereinafter also referred to as camera 310) as an image forming means. The stereo camera may be formed of three or more cameras. By using these cameras 310a and 310b, the driving environment detection device 1 detects an environment around the vehicle with the driving environment detection device 1. In the case of the first embodiment, the pair of cameras 310a and 310b are mounted at positions that are the same distance away from the center of the vehicle in the left and right vehicle width directions. That is, the camera 310a has a distance from the center of the vehicle equal to that of the camera 310b in the vehicle width direction.In this example, the driving environment detection device 1 performs processing for the images captured by these cameras. However, according to the first embodiment, the image processing and the like may be performed by the other control units. Note that the pair of cameras 310a and 310b corresponds to an information acquisition section according to the present invention.

[0033] The driving environment detection device 1 calculates a distance to each object of the image using the principle of triangulation, that is, by utilizing a disparity (difference) between the viewing angles that arises when the images are captured by the plurality of cameras 310a and 310b (hereinafter referred to as parallax). For example, the following relationship is satisfied. Z=(B×f) / δ

[0034] Where Z denotes a distance to the object, B denotes a distance between the camera pair 310a and 310b, f denotes a focal length of each of the cameras 310a, 310b and δ denotes the parallax.

[0035] The driving environment detection device 1 includes a RAM 320, a CPU 330, a data ROM 340, and a program ROM 350. The RAM 320 stores the images captured by the cameras 310. The CPU 330 performs arithmetic processing. The data ROM 340 stores various data. The program ROM 350 stores detection processing programs. The cameras 310 are attached to a rearview mirror mounted inside the vehicle. Each of these cameras 310 captures an image of the environment ahead of the vehicle from a predetermined inclination angle from the predetermined mounting location of this camera 310. The vehicle front area image captured by the camera 310 is read into the RAM 320. Then, a road surface(s) and a three-dimensional object(s) located in front of the vehicle are detected from the vehicle front area images, and a road shape is estimated from the vehicle front area images.It is noted that the driving environment detecting device 1 corresponds to the road surface calculating section according to the present invention.

[0036] Fig. 10 is a flowchart showing image processing performed by the driving environment detection device according to the first embodiment. In step S201, the image from the left-side camera 310a is read (into the RAM 320). In step S202, the image from the right-side camera 310b is read. In step S203, a corresponding point(s) of the image is calculated. In step S204, the distance to the corresponding point is calculated. In step S205, information about the distance is output. In step S206, it is decided whether or not an image read signal is present. If the image read signal is present, the program returns to step S201, so that the flow from Fig. 10 is repeated. If the image input signal is not present, the arithmetic processing is terminated, so that the program enters the idle state and remains there until an image input signal appears. (Detection processing on roads with steep slopes)

[0037] Image processing is explained below for a case where an off-road area (e.g., both sides outside the road on which the vehicle is traveling) is lower than the road surface. Fig. Figure 11 is a schematic view of an elevated road (embanked road) constructed with steep embankments. This elevated road has a substantially trapezoidal shape in cross-section as viewed on a plane perpendicular to a propagation direction of the elevated road. The road surface is formed at an upper base portion (upper edge portion) of this trapezoidal shape. The embankment portion is formed at a region between the road surface (i.e., the road interior portion) and the road exterior portion. Both exterior portions of the road (both sides laterally outside the road) are lower than the road interior portion and the embankment portion. Hereinafter, the road is also referred to as the road surface. Fig. 12 is an obtained image showing a schematic diagram obtained when the elevated road constructed by steep embankment portions is photographed from the vehicle. In this image, both ends of the road abut (are adjacent to) the road outer portions (i.e., the low portions lower than the road surface). In the case of this road, an angle of the embankment is larger than the inclination angle of the camera 310 because the embankment has a steep incline. Therefore, blind spots (i.e., spots not photographed due to a blind spot) are generated in the embankment portion. Thus, the embankment portions are not present in the captured images. Therefore, in the captured image, each road end is in contact with the road outer portion (the lower portion). Accordingly, in this case, the road inner portion and the area representing the other lower portions are captured in the image.Therefore, a road side point of a boundary area between the road interior area and the area representing the other lower portions in the image is extracted as a true road end (one of the two true ends of the road). This allows detection processing to be achieved in accordance with the real road environment. (Improving the accuracy of image processing)

[0038] In a case where the street interior or the street exterior are visually completely homogeneous, it is difficult to extract image locations belonging to an identical area of ​​the respective images captured by the two cameras. Fig. Figure 13 is a schematic view showing characteristic points that are simultaneously recorded when a real road is photographed. As in Fig. As shown in Fig. 13, the actual road includes a grain of asphalt concrete used for the road surface, a road marking, a joint in the road surface, cracks in the road surface, tire marks caused by vehicles, and the like as visually characteristic portions throughout the road in the case of the asphalt road. Even in the case of unpaved roads, the actual road includes visually characteristic portions such as ruts throughout the actual road. Note that the road interior corresponds to a first region having a first visually characteristic point according to the present invention. Furthermore, in the area lower than the road, there are visually characteristic portions such as grass throughout the lower area. Note that the area lower than the road corresponds to a second region having a second visually characteristic point according to the present invention.That is, while the road surface has been paved or flattened for vehicle travel, the area lower than the road surface has not been paved or flattened. Therefore, there is a visual difference between the road surface and the area lower than the road surface, so there is a high probability that a boundary area between the road surface and the area lower than the road surface will become visually characteristic. Thus, there are a plurality of visually characteristic points in the road, the road exterior, and the boundary area between the road and the road exterior. Therefore, the images captured by the two cameras 310 are compared with each other for each characteristic point, and thereby a direction and a distance to each characteristic point from the camera 310 are calculated.Therefore, a position of each characteristic point can be obtained. Accordingly, a collection of characteristic points actually located on the road can be considered to be essentially on the same plane, and those actually located in the area lower than the road can be considered to be located outside the road. (Superposition processing)

[0039] To obtain the shape of the road surface, the characteristic components such as road marking paint, small cracks in the asphalt, and tire marks present on the road surface are extracted from the vehicle front area images captured by the cameras 310. Then, a distance of each of the extracted characteristic components from the camera 310 is measured by a positional deviation (deviation of each characteristic component) between the images captured by the two cameras 310. However, such characteristic components do not necessarily exist uniformly on the entire road surface. Furthermore, even if such characteristic components exist on the entire road surface, it is not certain that such characteristic components are always detectable. Similarly, it is not certain that the characteristic components are always detectable in the entire area lower than the road surface.Therefore, it is necessary to further improve the accuracy. Therefore, in this embodiment, the acquired distance data is accumulated in the data ROM 340. This accumulated distance data is superimposed with distance data obtainable from images taken at a subsequent position (at the next time period) or later. Note that the data ROM 340 corresponds to a distance data accumulation section according to the present invention. Fig. 14 is a schematic view showing superposition processing of the image data (distance information) in the first embodiment. For example, portions that can be detected on images taken at the previous position (at the previous time) are superposed on portions that can be detected on images taken at the current position. That is, some portions whose distance information could not be obtained from the images taken at the previous position can have this distance information because this distance information was newly obtained from the images taken at the current position. This can improve detection accuracy of the road and the surrounding environment. As shown in Fig. As shown in Fig. 14, even if the vehicle is traveling with the traveling environment detection device 1, a plurality of images comprising the same area (i.e., overlapping areas) are obtained over time in a case where the vehicle speed is relatively low, i.e., in a case where vehicle movement between two photographing times is relatively short. Therefore, information about such an overlapping area is superposed among the plurality of images acquired over time. In this embodiment, the number of superpositions is not limited to two times. As many superpositions as possible is effective.

[0040] If the distance data of a specific location included in the overlapping area obtained from the images acquired at the first time point is different from the distance data of that specific location obtained from the images acquired at the second time point, priority can be given to the more recent distance data from the first and second time points. This can improve detection accuracy by using new data. Alternatively, an average of the data acquired at the first time point and the data acquired at the second time point (or average data from multiple time points) can be applied. This can eliminate the influence of noise and the like contained in the data, thus achieving stable detection.Furthermore, the data collected at the first time point and the data collected at the second time point (or multiple times) can be used alternatively, each with a smaller disparity compared to the environmental data. This allows the arithmetic calculation to be performed based on stable data, thus improving detection accuracy. Although numerous processing methods can be mentioned above, any one or a combination of such processing methods can be used. (Detection processing for the end of the road)

[0041] Fig. 15 is a schematic view showing a result obtained by capturing images of the elevated road with respect to a cross-sectional direction of the road. In this case, the slope is steep, so there is a blind spot (causing blind spots) of the camera. Therefore, the slope is not visible in the captured image. In these images, the road area is in direct contact with the area lower than the road. More specifically, a point (location) 601 positioned at an end part of the road and a point (location) 602 positioned outside the road are adjacent (approximately abut) each other in the image. However, in reality, the point 601, as shown in Fig. 15, is not adjacent (does not abut) to point 602, but is positioned somewhat away from point 602. Therefore, because it is not precise to output point 602 as an end point of the road, point 601 is output as the end point of the road.

[0042] In Fig. 15, the following process is performed when, for example, it is assumed that the position data corresponding to point 601 has not yet been acquired, and that a point (location) 603 located further inside the road than point 601 has been acquired as an outermost point located on the road surface. In this case, the acquired image includes an undetected area between (a point corresponding to) point 602 and (a point corresponding to) point 603. Therefore, a location where the road end is positioned between point 602 and point 603 is not clear. However, because point 602 located in the area lower than the road surface has been acquired, it can be estimated that the road is not located on a line of sight along which the camera 31 sees point 602.Therefore, it can be estimated that a road end is located at least between point 603 and point 601 (which, in this case, has not yet been discovered but is obtainable from the above-mentioned line of sight). Accordingly, in this case, a point located between point 602 and point 603 and located at a position further inward of the road is output as a position (near point 601) corresponding to the boundary area, and is output as the road end. (Detection processing for the end of the road in the case of a road constructed with a moderately sloping embankment)

[0043] Fig. 16 is a schematic view showing an elevated road (embanked road) constructed with a moderately sloping embankment. This elevated road has a substantially trapezoidal shape in cross section as viewed on a plane perpendicular to a propagation direction of the elevated road. The road surface is formed at an upper base portion (upper edge portion) of this trapezoidal shape. The embankment portion is formed at a region between the road surface (i.e., the road interior portion) and the road exterior portion. Both exterior portions of the road (both sides laterally outside the road) are lower than the road interior portion and the embankment portions. Fig. 17 is a captured image showing a schematic diagram obtained when the elevated road constructed by moderately sloping embankment portions is photographed from the vehicle. In this image, both ends of the road abut (are adjacent) the embankment portions, and these embankment portions abut (are adjacent) the road outer portions (i.e., the low portions lower than the road surface). In the case of this road, an angle of the embankment is smaller than the inclination angle of the camera 310 because the embankment has a moderate inclination. Therefore, no blind portions (i.e., portions not photographed due to a blind spot) are generated.

[0044] Fig. 18 is a schematic view showing a result obtained by image capturing the elevated road with respect to a cross-sectional direction of the road. In this case, the embankment portion slopes gently and is therefore photographed by the camera. Therefore, the road portion is in contact with (adjacent to) the embankment portion, and the embankment portion is in contact with (adjacent to) the road outer portion (the lower portion) in the captured image. Since it is important to detect the road end, it is unnecessary to distinguish the embankment portion from the road outer portion (the lower portion) in this embodiment. Therefore, all portions whose heights (elevation levels) are different from a height (level) of the road surface are treated as road outer portions. Therefore, hereinafter, the embankment portion is also referred to as the road outer portion.Therefore, a point (location) 901 is detected as being located in an end part of the road area (road interior), and a point (location) 902 of the road exterior is detected as a closest point to the road. Therefore, it can be estimated that a true road end is located between point 901 and point 902. (Improved accuracy of road end detection)

[0045] In the case where the road is connected (linked) to the area lower than the road surface by a moderately steep embankment, an image of this embankment can be captured by the camera, so that distance information of the embankment can be obtained. This can detect that the embankment is unsuitable for vehicle travel. Furthermore, a boundary between the embankment and the road area can be regarded as a road boundary (i.e., road end).

[0046] Even if the lower area cannot be detected, for example in the case where the lower area is significantly lower than the road (e.g., road is built along a cliff) or in the case where the lower area does not contrast clearly; it can be determined that the lower area is not the road (i.e., is not the road interior).

[0047] There is a possibility that the detected road end has a detection error compared with the actual road end. Furthermore, there is a possibility that the actual road end has a fragile substructure, and therefore, it is inappropriate for the vehicle to travel on the road end. As a countermeasure, for example, a position of the road that is further inward by an appropriate amount compared to the detected road end can be output as a modified road end. Note that the further inward position of the road compared to the detected road end corresponds to a second predetermined position according to the present invention.On the other hand, in the case where this driving environment detection device is combined with the lane departure prevention control of the first embodiment, a position of the road further outward by an appropriate amount compared to the detected road end can be output as a modified road end to avoid excessive control or excessive warnings. (Countermeasure at the time when false images are generated)

[0048] When attempting to extract and determine the area lower than the road as the road exterior area, there is a possibility that a false image will be captured due to reflection from a puddle. In such a case, there is a risk that the puddle will be mistakenly detected as being below the road surface because the erroneous image is below the road surface in a false view. Therefore, in this embodiment, the puddle is distinguished from the area truly lower than the road surface and is not determined as the area truly lower than the road surface using the following characteristics of the erroneous image reflected on the puddle, which are different from characteristics of real images. (a) Because the defective image consists of a far-away object(s), a road surface portion of the captured image, which is detected to have a shorter distance than an apparent distance of the defective image, is regarded in a front area of ​​a defective image area of ​​the captured image. (b) Because a water surface of a puddle is not completely flat, there is a case where the faulty image is highly distorted, so that distance information is scattered in the puddle area. (c) If the water surface of the puddle is not stable, an apparent area of ​​the faulty image varies over time. (d) Object(s) that appear to be on the road are also located at a location that is symmetrical to that object(s) with respect to the road surface (water surface). (e) In the case of a faulty image of a moving vehicle, the moving vehicle is still moving despite being considered to be below the road surface. There is an extremely low probability of this phenomenon occurring in the case of a genuine image.

[0049] By detecting such characteristics of defective images, an area of ​​a defective image can be determined, ie, it can be distinguished from an area of ​​a true image.

[0050] Some technical structures graspable from the above first embodiment according to the present invention will now be listed with their advantageous effects. [a] Driving environment sensing device comprising: an information acquisition section (e.g., the camera 310) mounted in a vehicle and configured to acquire information about at least a front portion of the vehicle with respect to a traveling direction of the vehicle; and a road surface calculation section (e.g., the driving environment detection device 1, 100) configured to calculate a road surface of a driving road and an area lower than the road surface in the front area of ​​the vehicle from the information acquired by the information acquisition section (e.g., 310).

[0051] According to such a structure, a position lower than the road surface can be detected, so that the information of the front area in the traveling direction of the vehicle can be accurately detected.

[0052] [b] The driving environment detection device as described in item [a], wherein the information detection section (e.g., 310) is a camera configured to capture an image of the front portion of the vehicle with respect to a traveling direction of the vehicle.

[0053] According to such a structure, a driving environment in front of the vehicle can be captured as an image.

[0054] [c] The driving environment sensing device as described in item [b], wherein the information acquiring section (e.g., 310) is a stereo camera including a plurality of cameras (e.g., 310a, 310b), the stereo camera being configured to measure distance using parallax caused during respective image capture of the same object with the plurality of cameras (e.g., 310a, 310b), and the information relates to distances. According to such a structure, a distance to an obstacle ahead of the vehicle and a road surface condition can be obtained based on the distance information.

[0055] [d] The driving environment detection device as described in item [c], wherein the road surface calculation section (e.g. 1, 100) is formed for detecting a first region (street interior) with a first visually characteristic point and a second region (street exterior) with a second visually characteristic point from the distance-related information, and for calculating the road surface and the area lower than the road surface, relative to the respective first and second regions detected.

[0056] According to such a structure, the road surface and the area below the road surface are detected based on both the visual characteristic point and the distance information. This can improve the detection accuracy.

[0057] [e] Driving environment detection device as described in item [d], wherein the first and second regions are directly adjacent to each other, wherein the road surface calculation section (e.g. 1, 100) is designed to calculate a position (e.g. 601, 901) which corresponds to a boundary area between the first and second regions as the end of the road surface.

[0058] According to such a structure, the road end can be detected with high accuracy.

[0059] [f] The driving environment detection device as described in item [e], wherein the road surface calculation section (e.g., 1, 100) is configured to calculate a second predetermined position located between a first predetermined position of the first region and the position corresponding to the boundary area as the end of the road surface. According to such a structure, a danger resulting from weak underpinning of the road end can be avoided.

[0060] [g] The driving environment detection device as described in item [d], wherein the road surface calculation section (e.g. 1, 100) comprises a distance data accumulation section (e.g. data ROM 340) configured to store the information about distance at predetermined intervals, wherein the road surface calculation section (e.g. 1, 100) is configured to detect the first and second regions by mutual superposition of a plurality of stored distance data.

[0061] According to such a structure, the detection accuracy can be improved.

[0062] [h] The driving environment detection device as described in item [g], wherein the road surface calculation section (e.g., 1, 100) detects the first and second regions based on the part of the current surrounding data if a part of the current surrounding data differs from that of previous surrounding data when the plurality of stored distance data are superimposed against each other. According to such a structure, the detection accuracy can be improved because more recent data is used.

[0063] [i] The driving environment detection device as described in item [g], wherein the road surface calculation section (e.g. 1, 100) detects the first and second regions based on an average of the part of the current environment data and the part of the previous environment data if a part of the current environment data is different from that of previous environment data when the plurality of stored distance data are superposed against each other.

[0064] According to such a structure, since noise influence and the like included in the data are removed, stable detection can be achieved. [Second embodiment]

[0065] Next, a second embodiment according to the present invention will be explained. Since a basic structure of the second embodiment is the same as the first embodiment, only the areas different from the first embodiment will be explained. Although the driving environment detection device is combined with the lane departure prevention control in the first embodiment, a driving environment detection device in the second embodiment is simply combined with a warning system for giving warnings. Furthermore, although a stereo camera 310 is used in the first embodiment, a single camera is used in the second embodiment.

[0066] Fig. 19 is a structural view of a vehicle equipped with a driving environment detection device according to a second embodiment. A driving environment detection device 100 includes the single camera 310 as image generation means, and thereby detects an environment around the vehicle equipped with the driving environment detection device 100. Note that the single camera 310 corresponds to the information acquisition section according to the present invention. The driving environment detection device 100 detects an image area difference of a measured identical location between a plurality of images captured by the single camera 310 at different times from each other, and thereby detects or determines a true position (distance) of the location.That is, when the plurality of images captured by the camera 310 are compared with each other, a point that is actually positioned near the vehicle moves at a high speed between the images (i.e., it has a large positional difference between one of the plurality of images and another of the plurality of images), and a point that is actually far from the vehicle moves at a low speed between the images, along with the actual movement of the vehicle. From such a fact, the actual distance and direction to the measured point can be inferred. This makes it possible to capture a three-dimensional shape of the environment captured by the camera 310.

[0067] The driving environment detection device 100 includes a RAM 320, a CPU 330, a data ROM 340, and a program ROM 350. The RAM 320 temporarily stores the images captured by the camera 310. The CPU 330 performs arithmetic processing. The data ROM 340 stores various data. The program ROM 350 stores detection processing programs. The camera 310 is attached to a rearview mirror mounted inside the vehicle. The camera 310 captures an image of the environment in the front of the vehicle at a predetermined inclination angle from the predetermined mounting position of this camera 310. The vehicle front image captured by the camera 310 is read into the RAM 320. Then, a road surface(s) in front of the vehicle and a three-dimensional object(s) are detected from the vehicle front area images, and a road shape is estimated from the vehicle front area images.It is noted that the driving environment detection device 100 corresponds to the road surface calculation section according to the present invention.

[0068] The driving environment sensing device 100 is connected (communicates) with a control unit 200 for controlling a warning system, and is also connected to a display device 115 for displaying various images and information. The display device 115 is mounted inside the vehicle. Furthermore, the control unit 200 is connected to a speaker 119, an ignition switch 120, and a start switch 121. The speaker 119 generates a warning sound when the vehicle equipped with the driving environment sensing device 100 is at risk of colliding with an obstacle. The ignition switch 120 is turned on when an engine is started. The start switch 121 starts or activates the warning system.

[0069] Although the detection processing in the first embodiment is executed based on the images captured by the left and right cameras, the detection processing in the second embodiment is executed based on the images captured by a camera at different times. Since the other basic detection logics are the same as those of the first embodiment, the corresponding explanations will be omitted for the sake of simplicity of this publication. Therefore, only parts different from the first embodiment will be explained below. (Improving the accuracy of image processing)

[0070] In a case where the street interior or the street exterior are visually completely homogeneous, it is difficult to extract the locations belonging to the same area (for the purpose of area differentiation) from the respective time-differentiated images taken by the individual camera 310. As above with regard to Fig. As explained in Figure 13, in reality, there are several visually characteristic points located on the road, the road exterior, and the boundary area formed between the road and the road exterior. Therefore, the images captured at different times by the individual camera 310 are compared with each other for characteristic points, and a direction and distance to each characteristic point from the camera 310 are calculated. This allows a position of each characteristic point to be obtained. Accordingly, a collection of characteristic points that are actually located on the road can be detected as being substantially on the same plane, and characteristic points that are actually located in the area lower than the road can be detected as being located on the road exterior. (Superposition processing)

[0071] To obtain the shape of the road surface, the characteristic components such as road marking paint, small cracks in the asphalt, and tire marks present on the road surface are extracted from the vehicle front area images captured by the camera 310. Then, a distance to each of the extracted characteristic components is measured from the camera 310 by a positional deviation of each of the characteristic components between the plurality of images captured by the camera 310 at different times. However, such characteristic components do not necessarily exist uniformly throughout the road surface. Furthermore, even if such characteristic components exist uniformly throughout the road surface, it is not certain that these characteristic components will always be detectable.Likewise, it is not certain that the characteristic portions in the entire area lower than the road surface are always detectable. Therefore, it is necessary to further improve the accuracy. Therefore, in this embodiment, the acquired distance data is accumulated in the data ROM 340. This accumulated distance data is superimposed with distance data obtainable from an image taken at a subsequent position (the next time point) or later. Note that the data ROM 340 corresponds to a distance data accumulation section according to the present invention. Fig. 20 is a schematic view showing superposition processing of image data (distance information) in the second embodiment. Since images captured at mutually different timings are needed to detect a distance from each location, in the case of the single camera 310, the following procedure is performed in this second embodiment. That is, in a previous control period, distance data was calculated using a difference between corresponding locations detected on an image captured in the previous control period (i.e., the consecutive number of the previous control period minus one) and the corresponding locations detected on an image captured in the previous control period. Then, the calculated distance data of the previous control period is stored.Next, in a now control period, distance data is calculated using a difference between corresponding locations detected in an image taken in the previous control period and the corresponding locations detected in an image taken in the now control period. Then, the calculated distance data of the now control period is stored. Then, the stored distance data of the now control period is superposed on the stored distance data of the previous control period. This makes it possible to obtain distance information for some parts for which distance information could not be obtained in the previous control period because this distance information is newly acquired in the now control period. This can improve detection accuracy of the road and the surrounding environment. As shown in . Fig.As shown in FIG. 20, even if the vehicle equipped with the driving environment detection device 100 is traveling, a plurality of images acquired over time include the same area (i.e., overlapping areas) in a case where the vehicle speed is relatively low, that is, in a case where vehicle movement between two photographing times is relatively short. Therefore, information about such an overlapping area is superposed among the plurality of images acquired over time. In this embodiment, the number of superpositions is not limited to two. As many superpositions as possible are effective.

[0072] Some technical structures encompassable by the above-described second embodiment according to the present invention will now be listed with their advantageous effects.

[0073] [j] The driving environment detection device as described in item [b], wherein the information detection section (e.g., 310) is a single camera, and the road surface calculation section (e.g., 1, 100) is configured to calculate a height difference in the captured image in accordance with a difference in an interframe moving speed of a photographed object caused between a region near the single camera and a region far from the single camera. According to such a structure, the driving environment in front of and near a vehicle can be efficiently detected by a small number of cameras. (Further examples)

[0074] Although the invention has been described above with reference to the first and second embodiments of the invention, the invention is not limited to the above-described embodiments. Modifications and variations of the above-described embodiments will become apparent to those skilled in the art in light of the above teachings.

[0075] For example, although the camera(s) is used as the information acquiring section in the first and second embodiments, a laser radar (lidar) or a millimeter-wave radar may also be used as the information acquiring section. In such a case, distance information acquired by waves reflected from a road surface and a region lower than the road surface in front of the vehicle equipped with the driving environment sensing device may be used. Furthermore, the above-mentioned various systems may be appropriately combined according to the present invention.

[0076] Some technical structures encompassable from the above embodiments according to the present invention will now be listed with their advantageous effects.

[0077] [k] The driving environment detection device as described in item [a], wherein the information acquisition section (e.g. 310) is a laser radar or a millimeter wave radar, and the information relates to a distance and is obtained by reflected waves from the road surface and the area lower than the road surface in the front area of ​​the vehicle.

[0078] According to such a structure, the distance information can be detected without being affected by sunlight or the like.

[0079] [l] A driving environment detection device comprising: a camera (e.g., 310) mounted in a vehicle and configured to detect an environment of a front portion of the vehicle with respect to a traveling direction of the vehicle; and a road surface calculation section (e.g., 1, 100) configured to calculate a boundary area between a road surface of a travel road and an area lower than the road surface adjacent to the travel road, using distance data based on a distance to each object located in the front area of ​​the vehicle in the environment captured by the camera (e.g., 310).

[0080] According to such a structure, a position lower than the road surface can be detected, so that the information of the front area in the traveling direction of the vehicle can be accurately detected.

[0081] [m] The driving environment sensing device as described in item [I], wherein the camera (e.g., 310) is a stereo camera comprising a plurality of cameras (e.g., 310a, 310b), the stereo camera being configured to measure distance using parallax caused during respective image capture of the same object with the plurality of cameras (e.g., 310a, 310b), and the environment is distance-related information. According to such a structure, a distance to an obstacle ahead of the vehicle and a road surface condition can be obtained based on the distance information.

[0082] [n] The driving environment detection device as described in item [m], wherein the road surface calculation section (e.g., 1, 100) is configured to detect a first region having a first visual characteristic point and a second region having a second visual characteristic point from the distance-related information, and calculate the road surface and the area lower than the road surface with respect to the detected respective first and second regions. According to such a structure, the road surface and the area lower than the road surface are detected based on both the visual characteristic point and the distance information. This can improve detection accuracy.

[0083] [o] The driving environment detection device as described in item [n], wherein the first and second regions are directly adjacent to each other, wherein the road surface calculation section (e.g. 1, 100) is configured to calculate a position (e.g. 601, 901) corresponding to a boundary area between the first and second regions as the end of the road surface.

[0084] According to such a structure, the road end can be detected with high accuracy.

[0085] [p] The driving environment detection device as described in item [n], wherein the road surface calculation section (e.g., 1, 100) is configured to estimate a second predetermined position located between a first predetermined position in the first region and the position corresponding to the boundary between the first and second regions as the end of the road surface. According to such a structure, a risk arising from weak underpinning of the road end can be avoided.

[0086] [q] The driving environment detection device as described in item [p], wherein the road surface calculation section (e.g., 1, 100) includes a distance data accumulation section (e.g., data ROM 340) configured to store the information about distance at predetermined intervals, wherein the road surface calculation section (e.g., 1, 100) is configured to detect the first and second regions by mutually superimposing a plurality of stored distance data. According to such a structure, the detection accuracy can be improved.

[0087] [r] The driving environment detection device as described in item [q], wherein the road surface calculation section (e.g., 1, 100) detects the first and second regions based on the part of the current environment data if a part of the current environment data differs from that of previous environment data when the plurality of stored distance data are superimposed on each other. According to such a structure, the detection accuracy can be improved because more recent data is used.

[0088] [s] The driving environment detection device as described in item [r], wherein the road surface calculation section (e.g., 1, 100) detects the first and second regions based on an average of a part of the current environment data and a part of the previous environment data, if a part of the current environment data differs from that of the previous environment data when the plurality of stored distance data are superimposed on each other. According to such a structure, since an influence of noise and the like included in the data is removed, stable detection can be achieved.

[0089] [t] A driving environment detection device comprising: a stereo camera (e.g. 310) configured to capture image information comprising a road surface of a driving road located in a front area of ​​a vehicle with respect to a traveling direction of the vehicle, with a plurality of cameras (e.g. 310a, 310b) mounted in the vehicle, wherein the stereo camera (e.g. 310) is configured to measure a distance in the image information by means of a parallax caused during the respective recording of images of the same object with the plurality of cameras (e.g. 310a, 310b); and a road surface calculation section (e.g. 1, 100) configured to capture a first region corresponding to the road surface of the driving road and a second region corresponding to an area lower than the road surface beside the driving road, using the image information acquired by the stereo camera (e.g.310) measured distance information, wherein the road surface calculation section (e.g., 1, 100) is configured to calculate a boundary area between the first and second regions as the end of the road surface. According to such a structure, a position lower than the road surface can be detected, so that information about the front area in the traveling direction of the vehicle can be accurately detected. Furthermore, a distance to an obstacle in front of the vehicle and a condition of the road surface can be obtained based on the distance information. Furthermore, since the road surface and the area lower than the road surface can be detected using both the visual characteristic point and the distance information, the detection accuracy can be improved. Furthermore, the road end can be accurately detected.

Claims

[1] Driving environment detection device, comprising: - an information acquisition section (310) mounted in a vehicle and configured to acquire information about at least a front portion of the vehicle with respect to a traveling direction of the vehicle; and - a road surface calculation section (1, 100) which is designed to calculate a road surface of a driving road and a region lower than the road surface in the front area of ​​the vehicle from the information acquired by the information acquisition section (310), wherein: - the information comprises a plurality of first visually characteristic points present on the road surface and a plurality of second visually characteristic points present in the area lower than the road surface, - the road surface calculation section (1, 100) is designed to recognize a boundary area between the plurality of first visually characteristic points and the plurality of second visually characteristic points as a boundary area between the road surface and the area lower than the road surface, - the detected boundary area is an end of the route, - the road surface calculation section (1, 100) comprises a distance data accumulation section (340) configured to store the information on distance at predetermined intervals, - the road surface calculation section (1, 100) is designed to detect the first and second regions by mutual superposition of a plurality of stored distance data and - the road surface calculation section (1, 100) detects the first and second regions based on an average of a piece of current surrounding data and a piece of previous surrounding data when the piece of current surrounding data is different from that of previous surrounding data while the plurality of stored distance data are superposed on each other. [2] The driving environment detection device according to claim 1, wherein the information detection section (310) is a camera configured to capture an image of the front portion of the vehicle with respect to a traveling direction of the vehicle. [3] Driving environment detection device according to claim 1 or 2, wherein - the information acquisition section (310) is a stereo camera comprising a plurality of cameras (310a, 310b), wherein the stereo camera is designed to measure a distance by means of a parallax caused during the respective recording of images of the same object with the plurality of cameras (310a, 310b), and - the information relates to a distance. [4] The driving environment detection device according to any one of claims 1 to 3, wherein the road surface calculation section (1, 100) is formed - for detecting a first region with a first visually characteristic point and a second region with a second visually characteristic point from the distance-related information, and - to calculate the road surface and the area lower than the road surface in relation to the respective first and second regions detected. [5] Driving environment detection device according to claim 4, - where the first and second regions are directly adjacent to each other, - wherein the road surface calculation section (1, 100) is designed to calculate a position (601, 901) corresponding to a boundary area between the first and second regions as the end of the road surface. [6] The driving environment detection device according to claim 5, wherein the road surface calculation section (1, 100) is configured to calculate a second predetermined position located between a first predetermined position of the first region and the position corresponding to the boundary area as the end of the road surface. [7] Driving environment detection device according to claim 2, - wherein the information acquisition section (310) is a single camera, - wherein the road surface calculation section (1, 100) is configured to calculate a height difference in the captured image in accordance with a difference in an image-to-image moving speed of a photographed object caused between a region near the single camera and a region far from the single camera. [8] Driving environment detection device according to one of the preceding claims, wherein - the information acquisition section (310) is a laser radar or a millimeter wave radar, and - the information relates to a distance and is obtained by reflected waves from the road surface and the area in front of the vehicle which is lower than the road surface. [9] A driving environment detection device comprising: - a camera (310) mounted in a vehicle and configured to capture an environment of a front area of ​​the vehicle with respect to a direction of travel of the vehicle; and - a road surface calculation section (1, 100) which is designed to calculate a boundary area between a road surface of a travel road and an area lower than the road surface beside the travel road using distance data based on a distance to each object located in the front area of ​​the vehicle in the environment captured by the camera (310), wherein: - the distance data comprise a plurality of first visually characteristic points present on the road surface and a plurality of second visually characteristic points present in the area lower than the road surface, - the road surface calculation section (1, 100) is designed to recognize a boundary area between the plurality of first visually characteristic points and the plurality of second visually characteristic points as a boundary area between the road surface and the area lower than the road surface, - the detected boundary area is an end of the route, - the road surface calculation section (1, 100) comprises a distance data accumulation section (340) configured to store the information on distance at predetermined intervals, - the road surface calculation section (1, 100) is designed to detect the first and second regions by mutual superposition of a plurality of stored distance data and - the road surface calculation section (1, 100) detects the first and second regions based on an average of a piece of current surrounding data and a piece of previous surrounding data when the piece of current surrounding data is different from that of previous surrounding data while the plurality of stored distance data are superposed on each other. [10] Driving environment detection device according to claim 9, wherein - the camera (310) is a stereo camera comprising a plurality of cameras (310a, 310b), wherein the stereo camera is designed to measure the distance by means of a parallax caused during the respective recording of images of the same object with the plurality of cameras (310a, 310b), and - the environment is information related to a distance. [11] A driving environment detection device according to claim 9 or 10, wherein the road surface calculation section (1, 100) is formed - for detecting a first region with a first visually characteristic point and a second region with a second visually characteristic point from the distance-related information, and - to calculate the road surface and the area lower than the road surface, related to the respective first and second regions detected. [12] Driving environment detection device according to claim 11, - where the first and second regions are directly adjacent to each other, - wherein the road surface calculation section (1, 100) is designed to calculate a position (601, 901) corresponding to a boundary area between the first and second regions as the end of the road surface. [13] The driving environment detection device according to claim 11, wherein the road surface calculation section (1, 100) is configured to estimate a second predetermined position located between a first predetermined position in the first region and the position corresponding to the boundary area between the first and second regions as the end of the road surface. [14] Driving environment detection device comprising: - a stereo camera (310) configured to capture image information comprising a road surface of a road located in a front area of ​​a vehicle with respect to a direction of travel of the vehicle, having a plurality of cameras (310a, 310b) mounted in the vehicle, wherein the stereo camera (310) is configured to measure a distance in the image information by means of a parallax caused during the respective recording of images of the same object with the plurality of cameras (310a, 310b); and - a road surface calculation section (1, 100) configured to detect a first region corresponding to the road surface of the travel road and a second region corresponding to an area lower than the road surface adjacent to the travel road, using the distance information measured by means of the stereo camera (310), wherein: - the distance information comprises a plurality of first visually characteristic points present on the road surface and a plurality of second visually characteristic points present in the area lower than the road surface, - the road surface calculation section (1, 100) is designed to calculate a boundary area between the plurality of first visually characteristic points and the plurality of second visually characteristic points as a boundary area between the first and second regions, - the detected boundary area is an end of the road surface, - the road surface calculation section (1, 100) comprises a distance data accumulation section (340) configured to store the information on distance at predetermined intervals, - the road surface calculation section (1, 100) is designed to detect the first and second regions by mutual superposition of a plurality of stored distance data and - wherein the road surface calculation section (1, 100) detects the first and second regions based on an average of a piece of current surrounding data and a piece of previous surrounding data when the piece of current surrounding data is different from that of previous surrounding data while the plurality of stored distance data are superposed on each other.

Images