Vehicle driving assistance device

Abstract

To provide a drive support apparatus which can maintain an automatic drive support function being executed when image recognition accuracy of an on-vehicle camera is reduced.SOLUTION: A drive support apparatus of a vehicle comprises: a camera device 10 which acquires a circumference image of an own vehicle M as first travel environment information; a radar device 37 which acquires three-dimensional object data of the circumference of the own vehicle as second travel environment information; an image recognition unit 13 which recognizes a lane division line 102 on the basis of the first travel environment information, recognizes a road end 104 on the basis of the second travel environment information and calculates a distance C between the lane division line and the road end from respective relative distances A, B of the own vehicle, the lane division line and the road end; and a travel control unit 14 which performs vehicle travel control on the basis of the lane division line and road end information. The image recognition unit sets an estimated road section line on the basis of the distance between the lane division line and the road end immediately before recognition becomes impossible and the relative distance between the road end and the own vehicle while the radar device continues recognition in such a situation that recognition of the lane division line becomes impossible on the basis of the first travel environment information. The travel control unit controls travel of the own vehicle on the basis of the estimated road section line.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device for a vehicle that provides driving assistance based on surrounding environment information acquired by an on-board camera device and an on-board radar device. [Background technology]

[0002] In recent years, development of automatic driving control technology for vehicles such as automobiles that allows the vehicle to travel automatically without the need for driver operation has been progressing. In addition, various driving assistance devices that can perform various driving controls to assist the driver in driving operations using this type of automatic driving control technology have been proposed and are becoming generally put into practical use.

[0003] In this type of conventional driving assistance device, a sensing device such as an in-vehicle camera device or an in-vehicle radar device is used as a surrounding environment recognition device for recognizing the surrounding environment of the vehicle and acquiring the surrounding information.

[0004] Of these, the in-vehicle camera device acquires electronic images and, based on the acquired electronic images, recognizes the vehicle's surrounding environment, such as road markings (hereinafter simply referred to as road markings, etc.), three-dimensional objects on the road such as curbs, as well as pedestrians, other vehicles, etc.

[0005] In addition, the on-board radar device outputs radio waves toward the area around the vehicle, receives reflected waves from objects, and analyzes the received waves to recognize three-dimensional objects around the vehicle, such as curbs and guardrails on the side of the road, as well as pedestrians and other vehicles.

[0006] Various technologies for installing this type of surrounding environment recognition device, recognizing road shapes, dividing lines, etc., and assisting the vehicle in autonomous driving based on the acquired information have been disclosed, for example, in JP 2015-45622 A, JP 2004-139338 A, and the like.

[0007] The driving assistance device disclosed in the above-mentioned JP 2015-45622 A and the like includes an on-board radar device and an on-board camera device. The device recognizes the road edge shape based on detection information obtained using the on-board radar device. The device recognizes the lane shape, which is the lane boundary line on the road, based on image information obtained using the on-board camera device. The device then compares the recognized road edge shape with the recognized lane shape, and identifies the road shape on which the vehicle is traveling based on the degree of match between the two.

[0008] The driving assistance device disclosed in the above-mentioned Japanese Patent Laid-Open No. 2004-139338 and the like includes an on-board camera device and an on-board radar device, and when selecting a lane recognition method between a method of recognizing lanes based on luminance changes in image information acquired by the on-board camera device and a lane recognition method using pattern matching based on images acquired by the on-board camera device, the appropriate selection is made based on light reception amount information from the on-board radar device. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-45622 [Patent Document 2] Japanese Patent Application Laid-Open No. 2004-139338 Summary of the Invention [Problem to be solved by the invention]

[0010] However, an on-board camera device as a surrounding environment recognition device has a problem in that the image recognition accuracy of lane markings and surrounding three-dimensional objects (road curbs, etc.) decreases depending on the weather conditions and lighting conditions of the surrounding environment. Specifically, it is known that the image recognition accuracy of on-board camera devices tends to decrease in bad weather such as rain, snow, and fog, and in low light or low brightness conditions such as backlight and twilight. As such, if the surrounding environment cannot be recognized sufficiently, it becomes difficult to continue the assistance function of the driving assistance device.

[0011] However, conventional driving assistance devices disclosed in the above-mentioned Patent Publication No. 2015-45622, Patent Publication No. 2004-139338, etc., lack consideration for continuing the driving assistance function in the event of a decrease in the accuracy of image recognition by the vehicle-mounted camera device or a temporary failure of image recognition for some reason.

[0012] The present invention aims to provide a vehicle driving assistance device that can maintain and continue the automatic driving assistance function that is currently being executed, even if the accuracy of image recognition by an on-board camera device temporarily decreases or image recognition temporarily fails for some reason while the vehicle's automatic driving assistance function is being executed. [Means for solving the problem]

[0013] In order to achieve the above object, a driving assistance device for a vehicle according to one aspect of the present invention comprises: The surrounding environment recognition device A camera device that acquires image data of the surroundings of the vehicle as first driving environment information, a radar device that senses reflected waves from objects of radio waves output around the vehicle and acquires recognized three-dimensional object data as second driving environment information, and the The aforementioned First, recognize road markings based on driving environment information. death , acquired by the radar device The aforementioned Recognizing road edges based on second driving environment information ,before The relative distance between the vehicle and the road dividing line Get The relative distance between the vehicle and the road edge and obtain the information on the relative distances. an image recognition unit that calculates the distance between the road dividing line and the road edge from the and Recognized by the image recognition unit the host vehicle and the road dividing line and the road edge The relative distance between Informed The above Vehicle Autonomous driving The driving control unit that controls driving FurthermoreWhen the road dividing line cannot be recognized based on the first driving environment information, the image recognition unit calculates the distance between the road dividing line and the road edge immediately before the road dividing line cannot be recognized. of information, and a distance between the road edge and the vehicle that is continuously recognized by the radar device. The above Relative Distance of and the driving control unit sets an estimated road dividing line corresponding to an extension position of the road dividing line based on the estimated road dividing line. The automatic driving Control the ride. [Effects of the Invention]

[0014] According to the present invention, it is possible to provide a vehicle driving assistance device that can maintain and continue the automatic driving assistance function that is currently being executed, even if the accuracy of image recognition by the on-board camera device decreases or image recognition temporarily fails for some reason while the vehicle's automatic driving assistance function is being executed. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a driving assistance device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram conceptually illustrating a monitoring area of ​​a sensor device (camera, radar, sonar) provided in a driving assistance device according to an embodiment of the present invention. [Figure 3] FIG. 1 is an explanatory diagram conceptually illustrating a state in which a vehicle equipped with a driving assistance device according to an embodiment of the present invention is traveling on a road. [Figure 4] 3 is a flowchart showing the operation of the driving assistance device 1 according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present invention will be described below with reference to the illustrated embodiments. The drawings used in the following description are schematic, and the dimensional relationships and scales of the components may be different for each component in order to show each component at a size that allows it to be recognized on the drawing. Therefore, the present invention is not limited to the illustrated embodiments in terms of the number of components shown in the drawings, the shapes of the components, the size ratios of the components, the relative positional relationships of the components, and so on.

[0017] In the following description of this embodiment, a road system is exemplified in which vehicles basically keep to the left, with the lane being on the left side of the road as viewed in the direction of travel. Therefore, to apply the configuration of the present invention to a road system in which vehicles basically keep to the right, it can be easily applied by simply switching the left and right directions.

[0018] First, the schematic configuration of a driving assistance device according to one embodiment of the present invention will be described below with reference to Figures 1 and 2. Figure 1 is a block diagram showing the schematic configuration of a driving assistance device according to one embodiment of the present invention. Figure 2 is a diagram conceptually showing the monitoring area of ​​sensor devices (camera, radar, sonar) provided in the driving assistance device according to one embodiment of the present invention.

[0019] As shown in FIG. 1, the driving assistance device 1 is configured to include, for example, a camera unit 10, which is an in-vehicle camera device fixed to the upper center of the front part of the passenger compartment of a vehicle (referring to a vehicle equipped with the driving assistance device 1; hereinafter referred to as the subject vehicle M; see FIGS. 2 and 3).

[0020] The camera unit 10 includes a stereo camera 11, an image processing unit (IPU) 12, an image recognition unit (image recognition_ECU) 13, and a driving control unit (driving_ECU) 14.

[0021] The stereo camera 11 has a main camera 11a and a sub-camera 11b. The main camera 11a and the sub-camera 11b are arranged, for example, at symmetrical positions across the center of the vehicle width. The main camera 11a and the sub-camera 11b are configured, for example, with CMOS image sensors, and capture stereo images of the driving environment in an area Af (see FIG. 2) outside the vehicle from different viewpoints at a predetermined imaging period that is synchronized with each other.

[0022] IPU 12 performs predetermined image processing on driving environment image data (image data representing the surrounding environment while the vehicle is traveling) captured by stereo camera 11, and detects edges of various objects, such as three-dimensional objects depicted in the image and lane markings (hereinafter simply referred to as lane markings, etc.) marked on the road surface. In this way, IPU 12 recognizes three-dimensional objects and lane markings around the vehicle. IPU 12 then calculates distance information from the amount of positional deviation of corresponding edges on the left and right images, and generates image information including the distance information (distance image information).

[0023] Based on distance image information received from the IPU 12, the image recognition_ECU 13 calculates the road curvature [1 / m] of the marking lines dividing the left and right sides of the roadway on which the vehicle M is traveling (the host vehicle roadway) and the width between the left and right marking lines (lane width). Various methods are known for calculating the road curvature and lane width. For example, the image recognition_ECU 13 recognizes the left and right marking lines by binarizing the road curvature based on the driving environment information using brightness differences, and calculates the curvatures of the left and right marking lines for each predetermined section using a curve approximation formula based on the least squares method. Furthermore, the image recognition_ECU 13 calculates the lane width from the difference in curvature between the left and right marking lines.

[0024] Then, the image recognition_ECU 13 calculates the lane center, the lateral position deviation of the vehicle M, which is the distance from the lane center to the center of the vehicle M in the vehicle width direction, etc. based on the curvature of the left and right lane markings and the lane width.

[0025] Furthermore, the image recognition_ECU 13 performs predetermined pattern matching on the distance image information to recognize three-dimensional objects such as guardrails extending along the road, curbs, and surrounding vehicles. Here, the recognition of three-dimensional objects by the image recognition_ECU 13 recognizes, for example, the type of the three-dimensional object, the height of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, the relative speed between the three-dimensional object and the vehicle M, and the relative distance between three-dimensional objects (for example, the lateral distance between a curb at the edge of the road and a marking line nearby).

[0026] The various pieces of information recognized by the image recognition_ECU 13 are output to the traveling_ECU 14 as first traveling environment information.

[0027] In this manner, in this embodiment, the image recognition_ECU 13, together with the stereo camera 11 and the IPU 12, serves as a driving environment recognition means for recognizing the first driving environment information around the vehicle, and realizes the function of a surrounding environment recognition device.

[0028] The travel_ECU 14 is a control unit for overall control of the driving assistance device 1. Various control units, such as a cockpit control unit (CP_ECU) 21, an engine control unit (E / G_ECU) 22, a transmission control unit (T / M_ECU) 23, a brake control unit (BK_ECU) 24, and a power steering control unit (PS_ECU) 25, are connected to the travel_ECU 14 via an in-vehicle communication line such as a CAN (Controller Area Network).

[0029] Furthermore, various sensors, such as a locator unit 36, an on-board radar device 37 (left front side sensor 37lf, right front side sensor 37rf, left rear side sensor 37lr, and right rear side sensor 37rr), and a rear sensor 38, are connected to the travel_ECU 14.

[0030] A human-machine interface (HMI) 31 arranged near the driver's seat is connected to the CP_ECU 21. The HMI 31 includes, for example, a switch for issuing an instruction to execute various driving assistance controls, a mode selector switch for switching driving modes, a steering touch sensor for detecting the driver's steering state, a driver monitoring system (DMS) for detecting the driver's facial recognition and line of sight, a touch panel display, a combination meter, a speaker, and the like.

[0031] When the CP_ECU 21 receives a control signal from the travel_ECU 14, it notifies the driver appropriately by displaying, sounding, or the like via the HMI 31 various types of information such as various warnings for preceding vehicles, the implementation status of driving assistance controls, and the driving environment of the host vehicle M. In addition, the CP_ECU 25 outputs various types of input information, such as the on / off operation status of various driving assistance controls input by the driver via the HMI 31, to the travel_ECU 14.

[0032] The output side of the E / G_ECU 22 is connected to a throttle actuator 32 of an electronically controlled throttle, etc. The input side of the E / G_ECU 22 is connected to various sensors such as an accelerator sensor (not shown).

[0033] The E / G_ECU 22 controls the operation of the throttle actuator 32 based on a control signal from the travel_ECU 14 or detection signals from various sensors. In this way, the E / G_ECU 22 adjusts the amount of intake air into the engine to generate a desired engine output. The E / G_ECU 22 also outputs signals such as the accelerator opening detected by the various sensors to the travel_ECU 14.

[0034] An output side of the T / M_ECU 23 is connected to a hydraulic control circuit 33. Furthermore, various sensors such as a shift position sensor (not shown) are connected to an input side of the T / M_ECU 23. The T / M_ECU 23 performs hydraulic control for the hydraulic control circuit 33 based on an engine torque signal estimated by the E / G_ECU 22 and detection signals from various sensors. As a result, the T / M_ECU 23 operates friction engagement elements, pulleys, and the like provided in the automatic transmission, and shifts the engine output at a desired gear ratio. Furthermore, the T / M_ECU 23 outputs signals such as the shift position detected by the various sensors to the travel_ECU 14.

[0035] A brake actuator 34 for adjusting the brake fluid pressure output to the brake wheel cylinders provided on the respective wheels is connected to the output side of the BK_ECU 24. In addition, various sensors such as a brake pedal sensor, a yaw rate sensor, a longitudinal acceleration sensor, and a vehicle speed sensor (not shown) are connected to the input side of the BK_ECU 24.

[0036] The BK_ECU 24 performs drive control on the brake actuator 34 based on control signals from the travel_ECU 14 or detection signals from various sensors. As a result, the BK_ECU 24 appropriately generates braking force on each wheel to perform forced braking control, yaw rate control, etc. on the host vehicle M. In addition, the BK_ECU 24 outputs signals of the brake operation state, yaw rate, longitudinal acceleration, vehicle speed (host vehicle speed), etc. detected by the various sensors to the travel_ECU 14.

[0037] An electric power steering motor 35, which applies steering torque to the steering mechanism by the rotational force of the motor, is connected to the output side of the PS_ECU 25. In addition, various sensors such as a steering torque sensor and a steering angle sensor are connected to the input side of the PS_ECU 25.

[0038] The PS_ECU 25 controls the drive of the electric power steering motor 35 based on control signals from the travel_ECU 14 or detection signals from various sensors. As a result, the PS_ECU 25 generates a steering torque for the steering mechanism. The PS_ECU 25 also outputs signals of the steering torque, steering angle, etc. detected by the various sensors to the travel_ECU 14.

[0039] The locator unit 36 ​​includes a GNSS sensor 36a and a high-precision road map database (road map DB) 36b.

[0040] The GNSS sensor 36a receives positioning signals transmitted from a plurality of positioning satellites to determine the position (latitude, longitude, altitude, etc.) of the vehicle M.

[0041] The road map DB 36b is a large-capacity storage medium such as an HDD or SSD, and stores high-precision road map information (dynamic map). The road map DB 36b stores lane data required for autonomous driving, such as lane width data, lane center position coordinate data, lane travel azimuth data, and speed limits. This lane data is stored at intervals of several meters for each lane on the road map. The road map DB also stores information on various facilities, parking lots, and the like. For example, based on a request signal from the traveling_ECU 14, the road map DB 36b outputs road map information for a set range based on the vehicle position measured by the GNSS sensor 36a to the traveling_ECU 14 as third traveling environment information.

[0042] In this manner, in this embodiment, the road map DB 36b, together with the GNSS sensor 36a, serves as a driving environment recognition means that recognizes third driving environment information around the vehicle, and realizes the function of a surrounding environment recognition device.

[0043] The left front side sensor 37lf, the right front side sensor 37rf, the left rear side sensor 37lr, and the right rear side sensor 37rr are a plurality of sensors that make up the on-vehicle radar device 37, and are configured by, for example, millimeter wave radar.

[0044] Here, each millimeter wave radar analyzes the reflected waves from objects in response to the output radio waves to detect mainly three-dimensional objects such as pedestrians and vehicles traveling alongside, as well as structures and the like (for example, curbs, guardrails, walls of buildings, plants, and other three-dimensional objects) provided on the edge of the road (for example, the edge on the shoulder side). Specifically, each radar detects information about the three-dimensional object, such as the width of the three-dimensional object, the position of a representative point of the three-dimensional object (relative position and relative distance to the vehicle M), and the relative speed.

[0045] The left front side sensor 37lf and the right front side sensor 37rf are disposed, for example, on the left and right sides of a front bumper, respectively. The left front side sensor 37lf and the right front side sensor 37rf detect, as second driving environment information, three-dimensional objects present in areas Alf, Arf (see FIG. 2) diagonally forward and to the left and right of the vehicle M, which are difficult to recognize in the image from the stereo camera 11.

[0046] The left rear side sensor 37lr and the right rear side sensor 37rr are disposed, for example, on the left and right sides of the rear bumper, respectively. The left rear side sensor 37lr and the right rear side sensor 37rr detect, as second driving environment information, three-dimensional objects present in areas Alr, Arr (see FIG. 2) diagonally to the left and right and behind the vehicle M, which are difficult to recognize with the left front side sensor 37lf and the right front side sensor 37rf.

[0047] In this manner, in this embodiment, the on-board radar device 37 (the front side sensor 37lf, the right front side sensor 37rf, the left rear side sensor 37lr, and the right rear side sensor 37rr) functions as a driving environment recognition means that recognizes second driving environment information around the vehicle, and realizes the function of a surrounding environment recognition device. The information acquired by these sensors 37lf, 37rf, 37lr, and 37rr is sent to the image recognition_ECU 13.

[0048] The rear sensor 38 is configured by, for example, a sonar device. The rear sensor 38 is disposed, for example, on the rear bumper. The rear sensor 38 detects, as the fourth driving environment information, three-dimensional objects present in an area Ar (see FIG. 2) behind the host vehicle M that are difficult to recognize with the left rear side sensor 37lr and the right rear side sensor 37rr.

[0049] In this manner, in this embodiment, the rear sensor 38 is a driving environment recognition means that recognizes the fourth driving environment information around the vehicle, and realizes the function of a surrounding environment recognition device.

[0050] In addition, the coordinates of each object outside the vehicle included in the first driving environment information recognized by the image recognition_ECU 13, the third driving environment information recognized by the locator unit 36, the second driving environment information recognized by the left front side sensor 37lf, the right front side sensor 37rf, the left rear side sensor 37lr, and the right rear side sensor 37rr, and the fourth driving environment information recognized by the rear sensor 38 are all converted by the driving_ECU 14 into coordinates of a three-dimensional coordinate system (see Figure 2) with the center of the vehicle M as the origin.

[0051] The driving modes set in the travel_ECU 14 include a manual driving mode, a first driving control mode and a second driving control mode for driving control, and an evacuation mode. These driving modes can be selectively switched in the travel_ECU 14 based on, for example, the operation status of a mode selector switch provided in the HMI 31.

[0052] Here, the manual driving mode is a driving mode that requires the driver to maintain steering, and is a driving mode in which the vehicle M is driven according to driving operations such as steering, accelerator, and brake operations by the driver.

[0053] Similarly, the first driving control mode is a driving mode that requires the driver to maintain steering. That is, the first driving control mode is a so-called semi-automatic driving mode in which the host vehicle M travels along a target driving route by appropriately combining mainly adaptive cruise control (ACC), active lane keep centering (ALKC), and active lane keep bouncing (ALKC) controls through control of the E / G_ECU 22, BK_ECU 24, PS_ECU 25, etc., while reflecting the driving operation by the driver.

[0054] Here, the control for following the preceding vehicle is basically performed based on the first traveling environment information input from the image recognition_ECU 13. That is, the control for following the preceding vehicle is performed based on, for example, the preceding vehicle information included in the first traveling environment information from the image recognition_ECU 13.

[0055] Furthermore, the lane centering control and lane departure prevention control are basically performed based on the first and third driving environment information input from at least one of the image recognition_ECU 13 and the locator unit 36. That is, the lane centering control and lane departure prevention control are performed based on, for example, lane marking information included in the third driving environment information from the image recognition_ECU 13 or the locator unit 36.

[0056] The second driving control mode is an autonomous driving mode that realizes a so-called hands-off function in which the vehicle M is driven along a target route (route map information) without requiring the driver to maintain steering, operate the accelerator, or operate the brakes, mainly by appropriately combining preceding vehicle following control, lane centering control, and lane departure prevention control through control of, for example, the E / G_ECU22, BK_ECU24, PS_ECU25, etc.

[0057] The evacuation mode is a mode for automatically stopping the vehicle M on a roadside or the like, for example, when, while driving in the second driving control mode, driving in that mode cannot be continued and the driver is unable to take over driving operations (i.e., when it is not possible to transition to manual driving mode or the first driving control mode).

[0058] In addition, in each of the above-mentioned driving modes, the driving_ECU 14 appropriately performs emergency braking (AEB (Autonomous Emergency Braking): collision damage mitigation braking) control against obstacles such as preceding vehicles on the vehicle's driving path that are highly likely to collide with the vehicle M.

[0059] All or part of the locator unit 36, image recognition_ECU 13, driving_ECU 14, CP_ECU 21, E / G_ECU 22, T / M_ECU 23, BK_ECU 24, PS_ECU 25, etc. are configured by a processor including hardware.

[0060] Here, the processor is configured by a well-known configuration including, for example, a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM), a non-volatile memory, a non-volatile storage, a non-transitory computer readable medium, and peripheral devices thereof.

[0061] Software programs to be executed by the CPU, fixed data such as data tables, etc. are stored in advance in ROM, nonvolatile memory, nonvolatile storage devices, etc. The CPU reads out the software programs stored in ROM, etc., expands them into RAM, and executes them, and the software programs refer to various data, etc. as appropriate, thereby realizing the functions of the above-mentioned components and components units (36, 13, 14, 21, 22, 23, 24, 25), etc.

[0062] The processor may be configured with a semiconductor chip such as an FPGA (Field Programmable Gate Array), etc. The above components and units (36, 13, 14, 21, 22, 23, 24, 25) may be configured with electronic circuits.

[0063] Furthermore, the software program may be in a form in which it is recorded in whole or in part as a computer program product on a portable disk medium such as a flexible disk, CD-ROM, or DVD-ROM, or on a non-transitory computer readable medium such as a card-type memory, HDD (Hard Disk Drive) device, or SSD (Solid State Drive) device.

[0064] The operation of the driving assistance device 1 of this embodiment configured as described above will be described below with reference to Figures 3 and 4. Figure 3 is an explanatory diagram conceptually showing a state in which a host vehicle M equipped with the driving assistance device 1 of this embodiment is traveling on a road. Figure 4 is a flowchart showing the operation of the driving assistance device 1 of this embodiment.

[0065] In FIG. 3, the host vehicle equipped with the driving assistance device 1 of this embodiment is indicated by the symbol M. Furthermore, symbol 101 in FIG. 3 indicates the host lane in which the host vehicle M is traveling. Symbol 102 in FIG. 3 indicates a dividing line on the left side of the host lane 101 (hereinafter referred to as the left dividing line 102). Symbol 103 in FIG. 3 indicates a dividing line on the right side (the center of the road) of the host lane 101. Note that FIG. 3 illustrates a road with one lane in each direction. Therefore, the right dividing line 103 of the host lane 101 indicates the center line of the road. Symbol 104 in FIG. 3 indicates the road edge on the left side of the host lane 101 (hereinafter referred to as the left road edge 104). Here, specifically, the left road edge 104 corresponds to a three-dimensional object such as a curb, guardrail, or wall provided at the boundary between a road and a sidewalk, etc. Reference numeral 105 in FIG. 3 indicates an area between the left lane marking 102 and the left road edge 104 (the shoulder or side strip).

[0066] In FIG. 3, symbol A indicates the relative distance (lateral distance) between the host vehicle M and the left lane marking 102. In FIG. 3, symbol B indicates the relative distance (lateral distance) between the host vehicle M and the left road edge 104. In FIG. 3, symbol C indicates the lateral distance between the left lane marking 102 and the left road edge 104. In FIG. 3, symbol D indicates the lateral distance between the host vehicle M and the host lane 101, i.e., the lane width of the host lane 101.

[0067] First, it is assumed that the host vehicle M equipped with the driving assistance device 1 of this embodiment is traveling on the host vehicle lane 101 as shown in FIG. 3. At this time, it is assumed that the driving assistance device 1 of the host vehicle M is set to the first driving control mode or the second driving control mode. In other words, it is assumed that the host vehicle M is executing a predetermined automatic driving assistance function, such as adaptive cruise control (ACC), lane centering control (ALKC), or lane departure prevention control (ALKB). Furthermore, the host vehicle M may be executing a hands-off function (second driving control mode).

[0068] In this state, the driving assistance device 1 operates the camera unit 10 to acquire lane marking information (mainly information related to the left lane marking 102) in step S11 of Fig. 4. At the same time, the driving assistance device 1 operates the in-vehicle radar device to acquire road edge information (mainly information related to the left road edge 104). Furthermore, the driving assistance device 1 acquires information related to the road the vehicle is currently traveling on (road information; various information including lane width D, etc.) using the locator unit 36.

[0069] Here, for example, the camera unit 10 calculates and acquires information regarding the relative distance (lateral distance A) between the vehicle M and the left-side lane marking 102 by performing a predetermined calculation process in the IPU 12 based on the pair of image data acquired.

[0070] The relative distance (lateral distance B) between the vehicle M and the left road edge 104 is calculated and acquired by a predetermined calculation process performed in the image recognition_ECU 13 based on information acquired by the left front lateral sensor 37lf or the left rear lateral sensor 37lr of the on-board radar device.

[0071] The lateral distance (lane width D) between the vehicle M and the lane 101 is calculated and acquired by a predetermined calculation process performed in the image recognition_ECU 13 based on the distance information between the left lane marking 102 and the right lane marking 103 acquired by the camera unit 10. The information on the lane width D may be acquired from the road map information DB 36b of the locator unit 36.

[0072] Next, in step S12, the image recognition_ECU 13 calculates the relative relationships between the objects (for example, distance information between the objects) based on the information acquired in the processing of step S11 described above. Specifically, the lateral distance C between the left lane marking 102 and the left road edge 104 is calculated and acquired by performing a predetermined arithmetic process in the image recognition_ECU 13 based on the information acquired by the camera unit 10 (distance A) and the information acquired by the on-board radar device (distance B).

[0073] Next, in step S13, the image recognition_ECU 13 checks whether the image recognition accuracy of the camera unit 10 has decreased. In this case, whether the image recognition accuracy of the camera unit 10 has decreased is determined, for example, by determining whether edge detection of the left-side lane marking 102 has become unstable or whether detection has become impossible. If it is determined that the image recognition accuracy has decreased, the process proceeds to the next step S14. If it is determined that the image recognition accuracy has not decreased (is normal), the process proceeds to step S19.

[0074] In step S14, the image recognition_ECU 13 stops the image recognition process by the camera unit 10 or does not refer to the process result, and continues the process of acquiring information about the left road edge 104 by the in-vehicle radar device.

[0075] Next, in step S15, the image recognition_ECU 13 calculates an estimated position of the left lane marking 102 based on the relative distance information calculated in the process of step S12 described above.

[0076] In step S16, the image recognition_ECU 13 transfers the estimated position information of the left lane marking 102 calculated in the process of step S15 to the camera unit 10. Running _ECU 14 This allows the position of the left-side lane marking 102 from the point where it becomes impossible to recognize by the camera unit 10 to be estimated. road This is called a dividing line.

[0077] In step S17, the image recognition_ECU 13 determines the stability of the position of the left road edge 104 in the lateral direction relative to the traveling direction of the host vehicle. As described above, the left road edge 104 may be, for example, a curb, a guardrail, or a wall of a building. Specifically, on a high-standard trunk road such as an expressway or a motorway, the curb or guardrail of the left road edge 104 extends continuously and has a relatively stable shape.

[0078] Even on high-standard trunk roads, in areas near entrances / exits, junctions, service areas, parking areas, and toll booths (hereinafter referred to as road facilities, etc.), there are places where the left-hand lane markings of the vehicle's travel lane are temporarily interrupted due to the existence of branching or merging roads from the main road and the existence of a decrease or increase in the number of lanes. Furthermore, on high-standard trunk roads, there are places where emergency parking bays, route bus stops, etc. (hereinafter, these will also be included in road facilities, etc.) are installed at predetermined intervals. In places where such road facilities, etc. exist, the position of the left road edge 104 is considered to be temporarily unstable. However, because information on these road facilities, etc. is included in the road map information, the driving assistance device 1 can recognize them in advance.

[0079] On the other hand, in the case of general roads, the curbs and guardrails at the roadside 104 are often formed in an intermittent manner to allow access to commercial facilities and residential areas facing the road. Also, in the case of general roads, buildings and walls may be constructed at the roadside 104. In these cases, too, the walls and the like are often formed in an intermittent manner along the road.

[0080] Furthermore, for example, in the case of a general road, there may be a situation where there are no structures around the road, making it impossible to recognize the road edge 104. Specifically, this may be a road on a river bank or a wide-area agricultural road. In such a case, the road edge 104 cannot be recognized by either the camera unit 10 or the on-board radar device.

[0081] Taking these factors into consideration, the stability of the position of the left road edge 104 is defined in stages, and the corresponding stability level is determined based on the detection results of the on-board radar device and locator information.

[0082] Specifically, for example, when the vehicle is traveling on a high-standard trunk road, the detection results of the onboard radar device are stable, and the locator information indicates that there are no road facilities or the like in the vicinity of the vehicle M, the stability level is defined as 0.

[0083] Also, for example, when the vehicle is traveling on a high-standard trunk road, the detection results of the onboard radar device are stable, and the locator information indicates that road facilities, etc. are present near the vehicle M, the stability level is set to 1.

[0084] Furthermore, for example, when traveling on an ordinary road, if the detection results of the on-board radar device are intermittently unstable but the road edge 104 can be continuously recognized, the stability level is defined as 2.

[0085] For example, when the vehicle is traveling on an ordinary road and the road edge 104 cannot be recognized by the on-board radar device, the stability level is set to 3. Note that the stability level setting shown here is merely an example and is not limited to the above example.

[0086] In step S18, the image recognition_ECU 13 checks whether the above-mentioned determination result is level = 0. If level = 0, the shape of the left road edge 104 is stable and the estimated position of the left lane marking 102 is determined to be reliable, so the process proceeds to step S19. If level = other than 0, the process proceeds to step S20.

[0087] In step S19, the image recognition_ECU 13 maintains the execution of the currently running automatic driving assistance functions, such as lane departure prevention (ALKB) control and adaptive cruise control (ACC) control. Also, if the second driving control mode is set, the execution of the hands-off function is maintained. In other words, when the road edge 104 is in a state where the stability level = 0, these automatic driving assistance functions are permissible. Note that these automatic driving assistance functions are merely examples. After that, the process returns to the original processing (return).

[0088] In step S20, the image recognition_ECU 13 checks whether the above-mentioned determination result is level = 1. If level = 1, the process proceeds to step S21. If level = 0 or other than level = 1, the process proceeds to step S22.

[0089] In step S21, the image recognition_ECU 13 maintains the execution of, for example, lane departure prevention (ALKB) control and adaptive cruise control (ACC) control among the currently executing automatic driving assistance functions. Also, if the second driving control mode is set, the execution of the hands-off function is turned off. In other words, when the road edge 104 is in a situation where the stability level = 1, some of these automatic driving assistance functions may be allowed (kept on), while some functions may not be allowed (turned off). Note that these automatic driving assistance functions are merely examples. Then, the process returns to the original processing (return).

[0090] When the automatic driving assistance function is to be turned off, it is desirable to display a warning to the driver in advance. The same applies when the function is turned off in the following processing steps.

[0091] In step S22, the image recognition_ECU 13 checks whether the above-mentioned determination result is level = 2. If level = 2, the process proceeds to step S23. If level = 0, level = 1, or level = 2, the process proceeds to step S24.

[0092] In step S23, the image recognition_ECU 13 maintains the execution of, for example, adaptive cruise control (ACC) among the automatic driving assistance functions currently being executed, and turns off the execution of lane departure prevention (ALKB) control. Also, if the second driving control mode is set, the execution of the hands-off function is turned off. In other words, when the road edge 104 is in a situation where the stability level is 2, some of these automatic driving assistance functions may be allowed (kept on), while some functions may not be allowed (turned off). Note that these automatic driving assistance functions are merely examples. Then, the process returns to the original processing (return).

[0093] In step S24, the image recognition_ECU 13 confirms that the above-mentioned determination result is level = 3, and turns off the execution of all automatic driving assistance functions. In other words, when the road edge 104 is in a situation where the stability level is 3, none of these automatic driving assistance functions can be permitted. Note that these automatic driving assistance functions are merely examples. Then, the process returns to the original process (return).

[0094] As described above, according to the embodiment, the image recognition_ECU 13 recognizes the left lane marking 102 based on the first driving environment information acquired by the camera unit 10, and recognizes the left road edge 104 based on the second driving environment information acquired by the on-board radar device. At the same time, the image recognition_ECU 13 acquires relative distance information A between the host vehicle M and the left lane marking 102 and relative distance information B between the host vehicle M and the left road edge 104. do. Then, the image recognition_ECU 13 calculates the distance C between the left lane marking 102 and the left road edge 104 .

[0095] When the camera unit 10 is unable to recognize the left-side lane marking 102 based on the first driving environment information, the image recognition_ECU 13 sets an estimated road-delimiting line corresponding to the extended position of the left-side lane marking 102 based on the distance C between the left-side lane marking 102 and the road edge 104 immediately before the left-side lane marking 102 became unrecognizable, and the relative distance B between the road edge 104, which is continuously recognized by the on-board radar device, and the vehicle M. At this time, the setting of the estimated road-delimiting line may also take into account the third driving environment information acquired by the locator unit 36.

[0096] The cruise control unit (travel_ECU) 14 controls the traveling of the host vehicle M based on the estimated road dividing lines thus set.

[0097] In this case, the driving control unit (driving_ECU) 14 determines the stability of the road edge 104 recognized based on the second driving environment information acquired by the on-board radar device, and sets the continuation or cancellation of the automatic driving assistance function according to the stability level.

[0098] With this configuration, the driving assistance device 1 of this embodiment can maintain and continue the automatic driving assistance function that is currently being executed even if the image recognition accuracy of the camera unit 10 temporarily decreases or image recognition temporarily fails for some reason while the automatic driving assistance function of the vehicle M is being executed.

[0099] In addition, depending on the surrounding environment of the vehicle M, safer driving can be continued by continuing the automatic driving function that is currently running, or by stopping and canceling the automatic driving function that is currently running.

[0100] In the above-described embodiment, the host vehicle M is traveling on a road with one lane in each direction, as shown in FIG. 3 , but the present invention is not limited to this. For example, the present invention can be similarly applied to a case where the host vehicle M is traveling in a passing lane near the center of a two-lane road. That is, in this case, the left-side lane marking of the host vehicle M is not the lane marking closest to the road edge, but the right-side lane marking in the lane (driving lane) adjacent to the left of the lane (passing lane) in which the host vehicle M is traveling. In this case, the left-side lane marking of the host vehicle M can be estimated based on distance information between the left-side lane marking recognized by the host vehicle M and the road edge (curb, etc.), just like in the above-described embodiment.

[0101] For this reason, the present invention can be applied in the same way even when the vehicle M is traveling in a lane from the center of a road with three lanes in each direction. In this case, if the control is performed to recognize the right-side median strip as the road edge, the present invention can be applied in the same way by switching the left and right sides of the above-described embodiment.

[0102] The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit and scope of the invention. Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple constituent elements. For example, if the problem to be solved by the invention can be solved and the effects of the invention can be obtained even if some constituent elements are deleted from all the constituent elements shown in one embodiment, the configuration from which these constituent elements are deleted can be extracted as the invention. Furthermore, constituent elements from different embodiments may be appropriately combined. The present invention is not limited by specific embodiments other than as limited by the appended claims. [Explanation of symbols]

[0103] 1...Driving assistance device 10...Camera unit 11...Stereo camera 11a...Main camera 11b...Sub camera 13...Image recognition unit (Image recognition ECU) 14...Travel control unit (Travel_ECU) 21...Cockpit control unit (CP_ECU) 22...Engine control unit (E / G_ECU) 23...Transmission control unit (T / M_ECU) 24...Brake control unit (BK_ECU) 25...Power steering control unit (PS_ECU) 31...Human Machine Interface (HMI) 32...Throttle actuator 33...Hydraulic control circuit 34...Brake actuator 35...Electric power steering motor 36...Locator unit 36a...GNSS sensor 36b...Road map information DB 37...Automotive radar device 37lf...Front left side sensor 37lr...Left rear side sensor 37rf...Right front side sensor 37rr...Right rear side sensor 38...Rear sensor 101…own lane 102...Left lane marking 103...Right lane marking 104…Left side of the road M...own vehicle

Claims

1. A surrounding environment recognition device, a camera device that captures image data of the surroundings of the vehicle and acquires the image data as first driving environment information; a radar device that senses reflected waves from objects around the vehicle using radio waves and acquires recognized three-dimensional object data as second driving environment information; an image recognition unit that recognizes a road dividing line based on the first driving environment information acquired by the camera device, recognizes a road edge based on the second driving environment information acquired by the radar device, acquires a relative distance between the vehicle and the road dividing line, acquires a relative distance between the vehicle and the road edge, and calculates the distance between the road dividing line and the road edge from information on both relative distances; and a driving control unit that controls autonomous driving of the host vehicle based on information on the relative distance between the host vehicle and the road dividing line and the road edge recognized by the image recognition unit; Further comprising: when the image recognition unit is unable to recognize the road-dividing line based on the first driving environment information, it sets an estimated road-dividing line corresponding to an extension position of the road-dividing line based on information on the distance between the road-dividing line and the road edge immediately before the road-dividing line becomes unrecognizable and information on the relative distance between the road edge and the vehicle, which is continuously recognized by the radar device; The driving control unit controls the autonomous driving of the vehicle based on the estimated road dividing line.

2. The surrounding environment recognition device further includes: a locator unit having road map information and locating a vehicle position based on a positioning signal to acquire the vehicle position as third driving environment information; 2. The vehicle driving assistance device according to claim 1, wherein the image recognition unit sets the estimated road dividing line taking into account the third driving environment information.

3. The image recognition unit determines a stability level according to the degree to which the position of the road edge is recognized based on the second driving environment information acquired by the radar device; The vehicle driving assistance device described in claim 1, characterized in that the driving control unit sets the continuation of the function controlling the autonomous driving of the vehicle when the stability level indicates that the road edge is recognized, and when the stability level indicates that the road edge cannot be recognized, gradually sets the cancellation of the function controlling the autonomous driving of the vehicle according to the stability level at which the road edge cannot be recognized.

Images