Vehicle driving assistance device

Abstract

To provide a driving support device for a vehicle capable of executing smooth traveling control all the time without making a driver feel uncomfortable when aerial floating objects, etc. are recognized.SOLUTION: A driving support device 1 for a vehicle, which can execute at least emergency brake control for avoiding collision with a recognized object includes: an ambient environment recognition device 10 having a recognition section for recognizing an ambient environment of the vehicle and a feature information acquisition section for acquiring feature information on a target object in the recognized ambient environment; and a traveling control unit 14 performing overall control of the vehicle in an integrated manner, having a determination section determining whether or not there is a possibility that the target object might inhibit traveling of the vehicle on the basis of the acquired feature information and continuing normal traveling control of the vehicle when a determination that there is no possibility that the target object might inhibit traveling of the vehicle is made.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device for a vehicle that performs driving assistance control to avoid obstacles and the like based on surrounding environment information acquired using an in-vehicle camera device or the like. [Background technology]

[0002] In recent years, development of automatic driving control technology for vehicles such as automobiles that allows the vehicle to travel automatically without the need for driver operation has been progressing. In addition, various driving assistance devices that can perform various driving controls to assist the driver in driving operations using this type of automatic driving control technology have been proposed and are becoming generally put into practical use.

[0003] In conventional driving assistance systems, sensing devices such as an on-board camera device or an on-board radar device are used as a surrounding environment recognition device to recognize the surrounding environment of the vehicle and acquire the surrounding information. Among these, an on-board camera device recognizes the surrounding environment of the vehicle based on acquired image data. Meanwhile, an on-board radar device recognizes the surrounding environment of the vehicle by emitting radio waves around the vehicle, receiving reflected waves from objects, and analyzing the received waves.

[0004] Conventional driving assistance devices use these sensing devices to recognize the vehicle's surrounding environment while driving the vehicle. If an obstacle that may impede the vehicle's travel is recognized on the vehicle's path, emergency braking control or emergency steering control is performed to prevent the vehicle from colliding with the obstacle. This allows the vehicle to continue traveling safely. Various proposals have been made in the past regarding driving assistance devices equipped with this type of control technology, such as obstacle avoidance control for avoiding obstacles, as disclosed in, for example, JP 2019-18733 A, and these have been put into practical use. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-18733 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, objects that can be recognized by conventional driving assistance devices include, for example, various road markings such as lane markings, and planar structures such as manholes (hereinafter simply referred to as lane markings, etc.). Furthermore, on roads, there are various types of stationary objects, such as road curbs, guardrails, road signs, utility poles, and commercial signs, as well as moving objects such as pedestrians, bicycles, other vehicles, and animals. In addition to these, there are various types of three-dimensional objects on roads, including temporarily installed stationary objects such as road cones and standing signs installed near construction sites, and various objects that have fallen from the loading platforms of other vehicles (hereinafter, these will be collectively referred to as obstacles, etc.).

[0007] Furthermore, for example, so-called plastic bags or vinyl bags, when they are a single bag with no contents inside, have an extremely light mass and may exist as an object floating in the air due to the influence of wind, etc. (Hereinafter, this type of object will be referred to as an airborne object, etc.) The surrounding environment recognition device of the driving assistance device can recognize these various objects.

[0008] When a three-dimensional object that satisfies a predetermined condition is detected on the path of the vehicle, the conventional driving assistance device recognizes the object as an object that may obstruct the vehicle's travel. In this case, obstacle avoidance control including emergency braking control and emergency steering control is executed to avoid a collision with the recognized obstacle.

[0009] In such cases, conventional driving assistance devices may execute predetermined obstacle avoidance control even if the recognized object is, for example, an object floating in the air.

[0010] However, it is considered extremely unlikely that this type of airborne object will impede vehicle travel. Therefore, if this type of airborne object is recognized and obstacle avoidance action is taken, the driver of the vehicle may feel uncomfortable.

[0011] The present invention aims to provide a vehicle driving assistance device that performs driving assistance control to avoid obstacles, etc. based on surrounding environment information acquired using an on-board camera device, etc., and that, when a so-called airborne object, etc. is recognized while the vehicle is traveling, does not immediately perform obstacle avoidance control including emergency braking control, etc., and can always perform smooth driving control without causing any discomfort to the driver. [Means for solving the problem]

[0012] In order to achieve the above object, a driving assistance device for a vehicle according to one aspect of the present invention is provided. Obstacle Avoidance a surrounding environment recognition device that recognizes a surrounding environment of the vehicle and acquires feature information of a target object in the recognized surrounding environment; Tori a determination unit that determines whether the target object has a possibility of obstructing the travel of the vehicle based on the acquired feature information, and when it is determined that the target object has no possibility of obstructing the travel of the vehicle, No obstacle avoidance control A driving control unit and The feature information of the target object acquired by the feature information acquisition unit includes information on the size of the target object, information on the falling speed of the target object, and information on the amount of change in size of the target object. . [Effects of the Invention]

[0013] According to the present invention, in a vehicle driving assistance device that performs driving assistance control to avoid obstacles, etc. based on surrounding environment information acquired using an on-board camera device, etc., when a so-called airborne object, etc. is recognized while the vehicle is traveling, it is possible to provide a vehicle driving assistance device that can always perform smooth driving control without immediately performing obstacle avoidance control including emergency braking control, etc., without causing any discomfort to the driver. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a driving assistance device according to a first embodiment of the present invention; [Figure 2] FIG. 1 is a conceptual diagram illustrating a state in which a vehicle equipped with a driving assistance device according to a first embodiment of the present invention recognizes an obstacle or the like ahead; [Figure 3] FIG. 1 is a diagram showing an example of a display of a forward image captured by an in-vehicle camera device of a vehicle equipped with the driving assistance device of the first embodiment of the present invention; [Figure 4] 1 is a flowchart showing the operation flow of the driving assistance device according to the first embodiment of the present invention; [Figure 5] 6 is a flowchart showing the flow of operation of a driving assistance device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] The present invention will be described below with reference to the illustrated embodiments. The drawings used in the following description are schematic, and the dimensional relationships and scales of the components may be different for each component in order to show each component at a size that allows it to be recognized on the drawing. Therefore, the present invention is not limited to the illustrated embodiments in terms of the number of components shown in the drawings, the shapes of the components, the size ratios of the components, the relative positional relationships of the components, and so on.

[0016] First, a schematic configuration of a driving assistance device according to a first embodiment of the present invention will be described below with reference to FIG. 1. FIG. 1 is a block diagram showing a schematic configuration of the driving assistance device according to the first embodiment of the present invention. As shown in FIG. 1, the basic configuration of the driving assistance device 1 according to this embodiment is substantially the same as that of a conventional driving assistance device of this type. Therefore, the following description will be limited to a schematic description of the driving assistance device 1 according to this embodiment.

[0017] The driving assistance device 1 of this embodiment has a camera unit 10, which is an in-vehicle camera device fixed to the front upper center part of the interior of the vehicle (hereinafter referred to as the host vehicle) in which the driving assistance device 1 is mounted.

[0018] The camera unit 10 includes a stereo camera 11, an image processing unit (IPU) 12, an image recognition unit (image recognition_ECU) 13, and a driving control unit (driving_ECU) 14.

[0019] The stereo camera 11 is a device that functions as a recognition unit that recognizes the surrounding environment of the vehicle. The stereo camera 11 has a main camera 11a and a sub-camera 11b. The main camera 11a and the sub-camera 11b are arranged, for example, in symmetrical positions on either side of the center in the vehicle width direction in the cabin of the vehicle, facing forward (in the direction of travel). The main camera 11a and the sub-camera 11b are configured, for example, with a CMOS image sensor or the like, and generate a stereo image by capturing two images of the surrounding environment of a predetermined range of an area in front of the vehicle from different viewpoints at a predetermined imaging period that is synchronized with each other.

[0020] IPU 12 performs predetermined image processing on the surrounding environment image data (image data representing the surrounding environment while the vehicle is traveling) captured by stereo camera 11, and detects the edges of various objects such as objects shown in the image and lane markings (hereinafter simply referred to as lane markings, etc.) marked on the road surface. In this way, IPU 12 recognizes three-dimensional objects, lane markings, etc. around the vehicle. IPU 12 then obtains distance information from the amount of positional deviation of corresponding edges on the left and right images, and generates image information including the distance information (distance image information).

[0021] Based on distance image information received from the IPU 12, the image recognition ECU 13 calculates the road curvature [1 / m] of the marking lines dividing the left and right sides of the roadway on which the vehicle is traveling (the host vehicle roadway) and the width between the left and right marking lines (lane width). Various methods are known for calculating the road curvature and lane width. For example, the image recognition ECU 13 recognizes the left and right marking lines by binarizing the road curvature based on the surrounding environment information using brightness differences, and calculates the curvatures of the left and right marking lines for each predetermined section using a curve approximation formula based on the least squares method. Furthermore, the image recognition ECU 13 calculates the lane width from the difference in curvature between the left and right marking lines.

[0022] Then, the image recognition_ECU 13 calculates the lateral position deviation of the host vehicle, which is the distance from the center of the lane to the center of the host vehicle in the vehicle width direction, based on the curvature of the left and right lane markings and the lane width.

[0023] Furthermore, the image recognition_ECU 13 performs predetermined pattern matching on the distance image information to recognize three-dimensional objects such as guardrails extending along the road, curbs, and surrounding vehicles. Here, the recognition of three-dimensional objects by the image recognition_ECU 13 recognizes, for example, the type of the three-dimensional object, the height of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, the relative speed between the three-dimensional object and the vehicle, and the relative distance between three-dimensional objects (for example, the lateral distance between a curb at the edge of the road and a dividing line nearby).

[0024] The various pieces of information recognized by the image recognition_ECU 13 are output to the traveling_ECU 14 as first surrounding environment information.

[0025] In this way, in the driving assistance device 1 of this embodiment, the image recognition_ECU 13, together with the stereo camera 11 and the IPU 12, realizes the function of a surrounding environment recognition device that recognizes the first surrounding environment around the vehicle.

[0026] The travel_ECU 14 is a control unit for overall control of the driving assistance device 1. Various control units, such as a cockpit control unit (CP_ECU) 21, an engine control unit (E / G_ECU) 22, a transmission control unit (T / M_ECU) 23, a brake control unit (BK_ECU) 24, and a power steering control unit (PS_ECU) 25, are connected to the travel_ECU 14 via an in-vehicle communication line such as a CAN (Controller Area Network).

[0027] Furthermore, various sensors, such as a locator unit 36, an on-board radar device 37 (left front side sensor 37lf, right front side sensor 37rf, left rear side sensor 37lr, right rear side sensor 37rr), and a rear sensor 38, are connected to the travel_ECU 14.

[0028] A human-machine interface (HMI) 31 arranged near the driver's seat is connected to the CP_ECU 21. The HMI 31 is configured to include, for example, a switch for issuing an instruction to execute various driving assistance controls, a mode selector switch for switching driving modes, a steering touch sensor for detecting the driver's steering state, a driver monitoring system (DMS) for detecting the driver's facial recognition and line of sight, a touch panel display (display panel), a combination meter, a speaker, and the like.

[0029] When the CP_ECU 21 receives a control signal from the travel_ECU 14, it notifies the driver of various warnings for preceding vehicles, the implementation status of driving assistance controls, and various information related to the surrounding environment of the vehicle, etc., as appropriate, by display, audio, etc. via the HMI 31. In addition, the CP_ECU 21 outputs various input information, such as the on / off operation status of various driving assistance controls input by the driver via the HMI 31, to the travel_ECU 14.

[0030] The output side of the E / G_ECU 22 is connected to a throttle actuator 32 of an electronically controlled throttle, etc. The input side of the E / G_ECU 22 is connected to various sensors such as an accelerator sensor (not shown).

[0031] The E / G_ECU 22 controls the operation of the throttle actuator 32 based on a control signal from the travel_ECU 14 or detection signals from various sensors. In this way, the E / G_ECU 22 adjusts the amount of intake air into the engine to generate a desired engine output. The E / G_ECU 22 also outputs signals such as the accelerator opening detected by the various sensors to the travel_ECU 14.

[0032] An output side of the T / M_ECU 23 is connected to a hydraulic control circuit 33. Furthermore, various sensors such as a shift position sensor (not shown) are connected to an input side of the T / M_ECU 23. The T / M_ECU 23 performs hydraulic control for the hydraulic control circuit 33 based on an engine torque signal estimated by the E / G_ECU 22 and detection signals from various sensors. As a result, the T / M_ECU 23 operates friction engagement elements, pulleys, and the like provided in the automatic transmission, and shifts the engine output at a desired gear ratio. Furthermore, the T / M_ECU 23 outputs signals such as the shift position detected by the various sensors to the travel_ECU 14.

[0033] A brake actuator 34 for adjusting the brake fluid pressure output to the brake wheel cylinders provided on the respective wheels is connected to the output side of the BK_ECU 24. In addition, various sensors such as a brake pedal sensor, a yaw rate sensor, a longitudinal acceleration sensor, and a vehicle speed sensor (not shown) are connected to the input side of the BK_ECU 24.

[0034] The BK_ECU 24 controls the driving of the brake actuator 34 based on control signals from the travel_ECU 14 or detection signals from various sensors. As a result, the BK_ECU 24 appropriately generates braking force on each wheel to perform forced braking control on the host vehicle, yaw rate control, etc. The BK_ECU 24 also outputs signals to the travel_ECU 14 indicating the brake operation state, yaw rate, longitudinal acceleration, vehicle speed (host vehicle speed), etc., detected by various sensors.

[0035] An electric power steering motor 35, which applies steering torque to the steering mechanism by the rotational force of the motor, is connected to the output side of the PS_ECU 25. In addition, various sensors such as a steering torque sensor and a steering angle sensor are connected to the input side of the PS_ECU 25.

[0036] The PS_ECU 25 controls the drive of the electric power steering motor 35 based on control signals from the travel_ECU 14 or detection signals from various sensors. As a result, the PS_ECU 25 generates a steering torque for the steering mechanism. The PS_ECU 25 also outputs signals of the steering torque, steering angle, etc. detected by the various sensors to the travel_ECU 14.

[0037] The locator unit 36 ​​includes a GNSS sensor 36a and a high-precision road map database (road map DB) 36b.

[0038] The GNSS sensor 36a receives positioning signals transmitted from a plurality of positioning satellites to determine the position (latitude, longitude, altitude, etc.) of the vehicle.

[0039] The road map DB 36b is a large-capacity storage medium such as an HDD or SSD, and stores high-precision road map information (dynamic map). The road map DB 36b stores lane data required for autonomous driving, such as lane width data, lane center position coordinate data, lane travel azimuth data, and speed limits. This lane data is stored at intervals of several meters for each lane on the road map. The road map DB also stores information on various facilities, parking lots, and the like. For example, based on a request signal from the traveling_ECU 14, the road map DB 36b outputs road map information of a set range based on the vehicle position measured by the GNSS sensor 36a to the traveling_ECU 14 as third surrounding environment information.

[0040] In this way, in the driving assistance device 1 of this embodiment, the road map DB 36b, together with the GNSS sensor 36a, realizes the function of a surrounding environment recognition device that recognizes the third surrounding environment around the vehicle.

[0041] The left front side sensor 37lf, the right front side sensor 37rf, the left rear side sensor 37lr, and the right rear side sensor 37rr are a plurality of sensors that constitute the on-vehicle radar device 37, and are configured by, for example, millimeter wave radars.

[0042] Here, each millimeter-wave radar receives and analyzes the reflected waves from objects in response to the emitted radio waves, thereby detecting mainly three-dimensional objects such as pedestrians and vehicles traveling alongside, as well as structures (e.g., curbs, guardrails, walls of buildings, plants, and other three-dimensional objects) installed on the edge of the road (e.g., the edge of the shoulder). Furthermore, each millimeter-wave radar also detects three-dimensional obstacles present on the road. In this case, each radar detects specific information about the three-dimensional object, such as the width of the three-dimensional object, the position of its representative point (relative position and distance to the vehicle), and the relative speed.

[0043] The left front side sensor 37lf and the right front side sensor 37rf are disposed, for example, on the left and right sides of the front bumper, respectively. The left front side sensor 37lf and the right front side sensor 37rf detect, as second surrounding environment information, three-dimensional objects present in areas diagonally forward and to the left and right and to the sides of the vehicle, which are difficult to recognize in the image from the stereo camera 11.

[0044] The left rear side sensor 37lr and the right rear side sensor 37rr are disposed, for example, on the left and right sides of the rear bumper, respectively. The left rear side sensor 37lr and the right rear side sensor 37rr detect, as second surrounding environment information, three-dimensional objects present in areas diagonally to the left and right sides and rear of the vehicle that are difficult to recognize with the left front side sensor 37lf and the right front side sensor 37rf.

[0045] In this manner, in the driving assistance device 1 of this embodiment, the on-board radar device 37 (the front side sensor 37lf, the right front side sensor 37rf, the left rear side sensor 37lr, and the right rear side sensor 37rr) realizes the function of a surrounding environment recognition device that recognizes the second surrounding environment around the vehicle. The information acquired by these sensors 37lf, 37rf, 37lr, and 37rr is sent to the image recognition_ECU 13.

[0046] The rear sensor 38 is configured by, for example, a sonar device, etc. The rear sensor 38 is disposed, for example, on the rear bumper. The rear sensor 38 detects, as the fourth surrounding environment information, three-dimensional objects present in the area behind the vehicle that are difficult to recognize with the left rear side sensor 37lr and the right rear side sensor 37rr.

[0047] In this way, in the driving assistance device 1 of this embodiment, the rear sensor 38 realizes the function of a surrounding environment recognition device that recognizes the fourth surrounding environment around the vehicle.

[0048] In addition, the coordinates of each object outside the vehicle included in the first surrounding environment information recognized by the camera unit 10 including the image recognition_ECU 13, the third surrounding environment information recognized by the locator unit 36, the second surrounding environment information recognized by the on-board radar device 37 (left front side sensor 37lf, right front side sensor 37rf, left rear side sensor 37lr, right rear side sensor 37rr), and the fourth surrounding environment information recognized by the rear sensor 38 are all converted by the driving_ECU 14 into coordinates of a three-dimensional coordinate system with the center of the vehicle as the origin.

[0049] The driving modes set in the travel_ECU 14 include a manual driving mode, a first driving control mode and a second driving control mode for driving control, and an evacuation mode. These driving modes can be selectively switched in the travel_ECU 14 based on, for example, the operation status of a mode selector switch provided in the HMI 31.

[0050] Here, manual driving mode is a driving mode that requires the driver to maintain steering, and is a driving mode in which the vehicle is driven according to driving operations such as steering, accelerator, and brake operations by the driver.

[0051] Similarly, the first driving control mode is a driving mode that requires the driver to maintain steering. That is, the first driving control mode is a so-called semi-automated driving mode or a driving assistance mode in which the vehicle is driven along a target driving route by appropriately combining mainly adaptive cruise control (ACC), active lane keep centering (ALKC), and active lane keep bouncing (ALKC) controls through control of the E / G_ECU 22, BK_ECU 24, PS_ECU 25, etc., while reflecting the driving operation by the driver.

[0052] Here, the adaptive cruise control (ACC) is basically performed based on the first ambient environment information input from the image recognition ECU 13. That is, the adaptive cruise control (ACC) is performed based on, for example, the preceding vehicle information included in the first ambient environment information from the image recognition ECU 13.

[0053] Furthermore, the lane centering control and lane departure prevention control are basically performed based on the first and third surrounding environment information input from at least one of the image recognition_ECU 13 and the locator unit 36. That is, the lane centering control and lane departure prevention control are performed based on, for example, lane marking information included in the third surrounding environment information from the image recognition_ECU 13 or the locator unit 36.

[0054] The second driving control mode is an autonomous driving mode that realizes a so-called hands-off function in which the vehicle travels along a target route (route map information) without the driver needing to maintain steering, operate the accelerator, or operate the brakes, by appropriately combining mainly preceding vehicle following control, lane centering control, and lane departure prevention control through control of, for example, the E / G_ECU22, BK_ECU24, PS_ECU25, etc.

[0055] The evacuation mode is a mode for automatically stopping the vehicle on a roadside or the like when, for example, while driving in the second driving control mode, driving in that mode cannot be continued and the driver is unable to take over driving operations (i.e., when the vehicle cannot transition to manual driving mode or the first driving control mode).

[0056] In addition, in each of the above-mentioned driving modes, when an obstacle such as a preceding vehicle on the vehicle's driving path that is likely to collide with the vehicle or a three-dimensional object such as a fallen object is recognized, the traveling_ECU 14 determines whether to execute emergency braking (AEB (Autonomous Emergency Braking): collision damage mitigation brake) control or obstacle avoidance control involving emergency steering control, and executes predetermined control as appropriate and necessary.

[0057] All or part of the locator unit 36, image recognition_ECU 13, driving_ECU 14, CP_ECU 21, E / G_ECU 22, T / M_ECU 23, BK_ECU 24, PS_ECU 25, etc. are configured by a processor including hardware.

[0058] Here, the processor is configured by a well-known configuration including, for example, a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM), a non-volatile memory, a non-volatile storage, a non-transitory computer readable medium, and peripheral devices thereof.

[0059] Software programs to be executed by the CPU, fixed data such as data tables, etc. are stored in advance in ROM, nonvolatile memory, nonvolatile storage devices, etc. Then, the CPU reads out the software programs stored in ROM, etc., expands them into RAM, and executes them, and the software programs appropriately refer to various data, etc., thereby realizing the functions of the above-mentioned components and components units (13, 14, 21 to 25, 36), etc.

[0060] The processor may be configured with a semiconductor chip such as a Field Programmable Gate Array (FPGA), etc. The above components and component units (13, 14, 21 to 25, 36) may be configured with electronic circuits.

[0061] Furthermore, the software program may be in a form in which it is recorded in whole or in part as a computer program product on a portable disk medium such as a flexible disk, CD-ROM, or DVD-ROM, or on a non-transitory computer readable medium such as a card-type memory, HDD (Hard Disk Drive) device, or SSD (Solid State Drive) device.

[0062] As the surrounding environment recognition device, for example, a monocular camera may be applied instead of (or in addition to) the stereo camera 11 included in the camera unit 10. Also, instead of (or in addition to) the in-vehicle radar device 37, for example, a LiDAR (Light Detection and Ranging) device or the like may be applied.

[0063] The operation of the driving support device 1 of this embodiment configured as above will be described below with reference to FIGS.

[0064] As described above, the driving assistance device 1 of this embodiment has the function of assisting the driver in driving operations by performing so-called adaptive cruise control (ACC), lane centering control (ALKC), lane departure prevention control (ALKB), etc.

[0065] Furthermore, the driving assistance device 1 of this embodiment has a function of performing obstacle avoidance control to assist driving when an obstacle or the like is recognized on the road while the vehicle is traveling. In this case, the driving assistance device 1 of this embodiment determines whether the recognized obstacle or the like is an obstacle or the like that is unlikely to impede the traveling of the host vehicle. If it is determined that the recognized obstacle or the like is an obstacle or the like that is unlikely to impede the traveling of the host vehicle, the driving assistance device 1 suppresses the execution of obstacle avoidance control such as emergency braking.

[0066] Note that objects that can be recognized by the driving assistance device 1 include, for example, various road markings such as lane markings, flat structures such as manholes (hereinafter simply referred to as lane markings, etc.), etc. These flat structures, i.e., lane markings, etc., are also treated in conventional driving assistance devices as having no possibility of obstructing the travel of the vehicle.

[0067] Furthermore, roads include three-dimensional objects such as permanently installed stationary objects, such as road curbs, guardrails, road signs, utility poles, and commercial signs, as well as moving objects, such as pedestrians, bicycles, other vehicles, and animals. Furthermore, various other three-dimensional objects exist on roads, including temporarily installed stationary objects, such as road cones and standing signs near construction sites, and objects that have fallen from the loading platforms of other vehicles. These three-dimensional objects are recognized as objects that may impede the travel of the vehicle. Therefore, hereinafter, these three-dimensional objects will be collectively referred to as obstacles, etc.

[0068] Furthermore, for example, so-called plastic bags or vinyl bags, when they are a single bag with no contents inside, have an extremely light mass and may exist as an object floating in the air due to the influence of wind, etc. (Hereinafter, this type of object will be referred to as an airborne object, etc.) And, even though this type of airborne object, etc. is a three-dimensional object, it is recognized based on its characteristic information as something that is not likely to obstruct the travel of the vehicle.

[0069] Here, the feature information of the airborne object etc. may be, for example, It exists in the air, that is, away from the road surface. The apparent size (area occupied in the recognized image) is extremely small (minuscule), Because it is lightweight, the falling speed is extremely low (it falls at a slow speed), Also, the behavior of the object when it falls is not limited to the direction of gravity (it can fluctuate up and down and left and right due to factors such as wind, and therefore the falling speed may also change). The apparent size and shape change as it falls (its shape changes depending on the wind, etc.). Some bags are highly transparent (some plastic bags are colorless and transparent), The following characteristics can be mentioned:

[0070] This feature information of the airborne object, etc., can be acquired, for example, from the results of predetermined image processing performed on images acquired by the camera unit 10. In this case, the camera unit 10, which functions as a surrounding environment recognition device, also functions as a feature information acquisition unit that acquires feature information of the target object in the recognized surrounding environment. In this case, the feature information acquired by the feature information acquisition unit is sent to the driving_ECU 14. In response to this, the driving_ECU 14 functions as a determination unit that determines whether the target object has the potential to obstruct the vehicle's driving based on the received feature information.

[0071] For example, FIGS. 2 and 3 are diagrams conceptually showing a situation in which an object present ahead in the traveling direction is recognized while a vehicle equipped with the driving assistance device of this embodiment is traveling on a road.

[0072] In FIG. 2, reference numeral 100 denotes a road on which a vehicle is traveling. Reference numeral M in FIG. 2 denotes the host vehicle. The host vehicle M is equipped with a camera unit 10, which constitutes part of the driving assistance device 1 and is an on-board camera device that recognizes the surrounding environment. The camera unit 10 is fixed at a predetermined position inside the cabin of the host vehicle M. In FIG. 2, reference numeral V conceptually denotes the range of the field of view (imaging range) of the stereo camera 11 of the camera unit 10. This field of view V corresponds to the angle of view of the image acquired by the camera unit 10. Note that in FIG. 2, the range of the field of view indicated by reference numeral V only shows the range in the vertical direction.

[0073] Here, it is assumed that the vehicle M equipped with the driving assistance device 1 is traveling on a road 100. At this time, the driving assistance device 1 continuously acquires images of the surrounding environment (mainly ahead in the traveling direction) at predetermined time intervals using the camera unit 10, and recognizes the surrounding environment by performing predetermined image processing based on the acquired image data.

[0074] In this case, it is assumed that an airborne object 101 exists at a position away from the surface of the road 100 (at a height H from the road surface) within the range of the field of view V ahead in the traveling direction of the host vehicle M. At this time, the driving assistance device 1 of the host vehicle M recognizes this airborne object 101. Figure 2 conceptually illustrates such a situation.

[0075] FIG. 3 shows an example of a display of an image acquired by the camera unit 10 of the driving assistance device 1 of the host vehicle M under the circumstances shown in FIG.

[0076] In FIG. 3, reference numeral 10A denotes an image captured by the camera unit 10. Within the range of this image 10A, an image of the surroundings of the host vehicle M (mainly ahead in the direction of travel) is captured. In FIG. 3, reference numeral 100 denotes the road on which the host vehicle M is traveling, as in FIG. 2. Reference numeral 101 denotes an airborne object, etc., as in FIG. 2. Reference numeral 101a denotes the shadow of the airborne object, etc. 101. Reference numeral 102 denotes a manhole, etc., as an installation on the road 100. Reference numeral 103 denotes a marking (white line, etc.) on the road 100. Reference numeral 104 denotes a curb, etc., installed along the left edge of the road 100. Reference numeral 105 denotes plants, etc., or a sidewalk area, etc., installed along the left edge of the road 100. Reference numeral 106 denotes a boundary wall, etc., installed along the right edge of the road 100. Reference numeral 107 denotes a utility pole, etc., installed on the right edge of the road 100. Reference numeral 108 denotes the shadow of a utility pole, electric wire, etc. (not shown). Reference numeral 109 denotes a bicycle or the like traveling on the road 100 toward the host vehicle M. Reference numeral 110 denotes a plurality of road cones or the like installed along both side edges of the road 100.

[0077] 2 and 3 indicate coordinate axes representing the space in which the host vehicle M exists. In this case, the arrow X is a coordinate axis along the width direction (lateral direction) of the host vehicle M. The direction indicated by the arrow X is the positive (+) direction toward the left when viewed from the host vehicle M looking forward. The arrow Y is a coordinate axis along the height direction (vertical direction) of the host vehicle M. The direction indicated by the arrow Y is the positive (+) direction toward the upward direction when viewed from the host vehicle M looking forward. The arrow Z is a coordinate axis along the traveling direction of the host vehicle M. The direction indicated by the arrow Z is the positive (+) direction toward the host vehicle M from the front of the host vehicle M.

[0078] The driving assistance device 1 recognizes various objects present in the surrounding environment of the vehicle M based on image data representing such an image 10A.

[0079] Next, the operation of the driving assistance device 1 of the vehicle M in such a situation will be described below with reference to the flowchart in Fig. 4. Note that the processing flow described below shows only characteristic processing specific to the present invention among the various driving assistance controls executed by the driving assistance device 1 of this embodiment.

[0080] First, it is assumed that the vehicle M is traveling on a road 100 with the driving assistance control of the driving assistance device 1 in an on state. At this time, it is assumed that the driving assistance device 1 is continuously recognizing the surrounding environment using a surrounding environment recognition device (mainly the camera unit 10, etc.).

[0081] 4, the driving assistance device 1 first checks whether or not an object floating in the air (airborne object 101) has been recognized among the various recognized objects. Here, the determination of whether or not the object is an airborne object 101 is performed as follows.

[0082] For example, as shown in Fig. 2, the relative positional relationship between the height position of a sensor (camera unit 10) installed on the vehicle M and the height position of a target object (airborne object, etc. 101) is estimated based on image data. This makes it possible to calculate the height H of the target object (airborne object, etc. 101) from the road surface. If the height H is equal to or greater than a predetermined threshold, it can be estimated that the target object is an airborne object, etc. 101.

[0083] Furthermore, as shown in FIG. 3, for example, it is checked whether a shadow corresponding to the target object (airborne object, etc. 101) is projected below the target object in image 10A (on the surface of road 100), and if a shadow is confirmed, it can be inferred that the target object is an airborne object, etc. 101.

[0084] Next, in step S12, the driving assistance device 1 acquires characteristic information of the recognized airborne object 101. Here, as described above, the characteristic information of the airborne object 101 is, for example, information such as the size, falling speed, amount of change in size, transparency, etc. of the airborne object 101. The characteristic information of the airborne object 101 can be acquired, for example, from the result of predetermined image processing performed based on image data acquired by the camera unit 10.

[0085] The relationship between the mass of an object and its falling speed is Falling speed of an object = (object mass x gravitational acceleration) / air resistivity According to this, the lighter the mass of an object, the slower the object's falling speed. In other words, the slower the falling speed, the smaller the object's mass.

[0086] Next, in step S13, the driving assistance device 1 checks whether the size of the recognized airborne object 101 (the area of ​​the object image in the image 10A) is less than a predetermined threshold. If the size of the airborne object 101 (the area occupied in the image) is less than the predetermined threshold, the process proceeds to step S14. If not, the process proceeds to step S18.

[0087] In step S14, the driving assistance device 1 checks whether the falling speed of the recognized airborne object 101 is less than a predetermined threshold. If the falling speed of the airborne object 101 is less than the predetermined threshold, the process proceeds to step S15. If not, the process proceeds to step S18.

[0088] In step S15, the driving assistance device 1 checks whether the size of the recognized airborne object 101 has changed. If the size has changed, the driving assistance device 1 checks whether the amount of change in size is equal to or greater than a predetermined threshold. If the size of the airborne object 101 has changed and the amount of change in size is equal to or greater than the threshold, the driving assistance device 1 proceeds to the processing of step S16.

[0089] If the size of the airborne object 101 has not changed, or if a change in size is confirmed and the amount of change in size is less than the threshold, the process proceeds to step S18.

[0090] In step S16, the driving assistance device 1 determines that the recognized airborne object 101 is an object that the host vehicle M can pass through even if it is not an object to be avoided (hereinafter referred to as a passable object).

[0091] Subsequently, in step S17, the driving assistance device 1 checks whether the transparency of the recognized airborne object 101 is equal to or greater than a predetermined threshold (high transparency = close to colorless and transparent).

[0092] The reason for providing this confirmation step is as follows. That is, generally, so-called plastic bags, vinyl bags, etc. have a wide range of transparency, such as colorless and transparent ones as well as translucent or colored ones. Focusing on this point, in the driving assistance device 1 of this embodiment, the transparency of an object determined to be a passable object after checking the above-mentioned conditions (size, falling speed, size change) is further confirmed in the processing of steps S13 to S15. Then, if the result shows that the object has low transparency (colored), it is used as a reference condition for displaying a warning to alert the driver in the processing of the subsequent step S20. That is, in the processing of step S17, the low transparency of the target object is confirmed and added as a reference condition for determining whether to display a warning.

[0093] The transparency of the target object can be confirmed by the following means. For example, a possible means is to compare the pixel values ​​of the target object (airborne object, etc. 101) with the pixel values ​​of its surroundings based on image data acquired by the camera unit 10. Another possible means is to detect the reflection of light that has passed through the target object (airborne object, etc. 101) based on surrounding environment data acquired using LiDAR or the like, and confirm the level of reflectance.

[0094] In the process of step S17, if the transparency of the airborne object 101 is equal to or greater than a predetermined threshold, the reliability of the determination that it is a passable object increases. Therefore, in this case, no control action for avoidance is taken, and normal driving control continues as is. Then, the series of processes ends, and the process returns to the original processing step (return).

[0095] On the other hand, if the transparency of the airborne object 101 is less than the predetermined threshold in the process of step S17, the process proceeds to step S20.

[0096] In step S20, the driving assistance device 1 displays a warning to the driver that an airborne object or the like 101 is present ahead. This warning display may be a visual warning such as an image display on the HMI 31 (display panel or the like) or an auditory warning such as a sound using the HMI 31 (speaker or the like). In this case, the display panel, speaker or the like included in the HMI 31 functions as a notification device that notifies the driver of predetermined information. Thereafter, the series of processes is terminated, and the process returns to the original processing step (return).

[0097] The reason why the processing of step S20 (warning display processing) is performed in this manner is as follows: That is, the object of the warning display in the processing of step S20 is an airborne object 101 that has been identified as an airborne object, etc. 101, and a passable object in the processing of steps S13 to S16 described above, but that has been determined to have low transparency (below a predetermined threshold) in the processing of step S17 described above. Although such an airborne object 101 is a passable object that is unlikely to impede the traveling of the host vehicle M, due to its low transparency, there is a possibility that it may, for example, obstruct the forward visibility of the host vehicle M. Therefore, when it is determined to be such an airborne object 101, no control action for avoidance is taken, but the driver is warned just in case.

[0098] On the other hand, if the processing of step S18 is proceeded to in each of the above-mentioned steps S13, S14, and S15, in this step S18, the driving assistance device 1 determines that the recognized airborne object 101 is an object that may obstruct the travel of the vehicle M (hereinafter referred to as an impassable object).

[0099] Then, in the next step S19, the driving assistance device 1 executes a predetermined emergency braking control. After that, the series of processes ends and the process returns to the original processing step (RETURN). The emergency braking control in this case is substantially the same as the control executed when a general obstacle is recognized.

[0100] As described above, according to the first embodiment, when the host vehicle M equipped with the driving assistance device 1 travels on a road while recognizing the surrounding environment ahead in the traveling direction using the surrounding environment recognition device (10, 37), it acquires feature information of a target object from the recognized surrounding environment. Then, the cruise control unit 14 determines whether the target object has the potential to obstruct the traveling of the host vehicle M based on the acquired feature information. At this time, if it is determined that the target object has no potential to obstruct the traveling of the host vehicle M, normal cruise control of the host vehicle M is continued.

[0101] With this configuration, unnecessary obstacle avoidance control can be suppressed when the target object among the recognized three-dimensional objects is an airborne object 101 that is unlikely to obstruct the travel of the vehicle M. Therefore, smooth driving assistance control can be always executed without giving the driver any discomfort.

[0102] Furthermore, in this embodiment, the transparency of an object determined to be an airborne object or the like 101 is confirmed. If the transparency of the airborne object or the like 101 is less than a predetermined threshold, the recognized target object is an airborne object or the like 101 but is not transparent (e.g., a colored object), and there is a possibility that it may obstruct the forward visibility of the vehicle M. Therefore, in this case, unnecessary obstacle avoidance control is suppressed and the driver is alerted. As a result, the driving assistance device 1 of this embodiment can achieve more reliable and smooth driving control without giving the driver a sense of discomfort.

[0103] Next, a driving assistance device 1 according to a second embodiment of the present invention will be described. The basic configuration of the driving assistance device 1 itself according to this embodiment is the same as that of the first embodiment described above, with only a slight difference in the processing flow. Therefore, in the following description, the configuration of the driving assistance device itself and the situation in which it is operated will be explained using FIGS. 1 to 3 used in the first embodiment described above, and illustrations and explanations will be omitted. The operation of the driving assistance device according to this embodiment will be explained below using FIG. 5.

[0104] The situation in which the processing flow in Fig. 5 is executed is the same as that in the first embodiment described above. Furthermore, the processing in steps S11 and S12 in the processing flow in this embodiment (see Fig. 5) is exactly the same as that in the processing flow in the first embodiment described above (see Fig. 4). That is, in step S11 in Fig. 5, the driving assistance device 1 recognizes the airborne object 101. Subsequently, in step S12 in Fig. 5, the driving assistance device 1 acquires feature information of the recognized airborne object 101.

[0105] Next, in step S21 of FIG. 5, the driving assistance device 1 calculates a predetermined evaluation value for each condition (size, fall speed, size change, transparency of the airborne object 101) based on the characteristic information of the airborne object 101 acquired in the processing of step S12 described above.

[0106] Here, the evaluation values ​​for each condition are, for example, as follows: For example, the evaluation value for the size of the target object is small = 0, medium = 1, large = 2; the evaluation value for the falling speed of the target object is slow = 0, medium = 1, high = 2; and the evaluation value for the amount of change in size of the target object is small = 0, medium = 1, large = 2. Note that the evaluation value for each condition shown here is a three-level evaluation value, but this is merely an example and the evaluation values ​​are not limited to this example.

[0107] Next, in step S22, the driving assistance device 1 checks whether the total evaluation value calculated in the processing of step S21 described above is less than a first threshold value. In this case, the first threshold value is set to, for example, "2." This example of the first threshold value = "2" is merely an example, and is not limited to this.

[0108] If the total evaluation value is less than the first threshold value "2", the process proceeds to step S23. If the total evaluation value is equal to or greater than the first threshold value "2", the process proceeds to step S24.

[0109] In step S23, the driving assistance device 1 determines that the recognized airborne object 101 is a passable object. This processing step is exactly the same as step S16 in FIG.

[0110] In step S24, the driving assistance device 1 checks whether the total evaluation value calculated in the processing of step S21 described above is equal to or greater than a second threshold value. In this case, the second threshold value is set to, for example, "5." This example of setting the second threshold value to "5" is merely an example, and the present invention is not limited to this.

[0111] If the total evaluation value is equal to or greater than the second threshold value "5", the process proceeds to step S25. If the total evaluation value is less than the second threshold value "5", the process proceeds to step S27.

[0112] That is, the process proceeds from step S24 to step S27 when the total evaluation value is equal to or greater than the first threshold value "2" and less than the second threshold value "5" (the total evaluation value is 2 to 4).

[0113] In this case, in step S27, the driving assistance device 1 checks whether the transparency of the recognized airborne object 101 is equal to or greater than a predetermined threshold. If the transparency of the airborne object 101 is equal to or greater than the predetermined threshold, normal driving control is continued without taking any control action for avoidance. Then, the series of processes is ended, and the process returns to the original processing step (RETURN). If the transparency of the airborne object 101 is less than the predetermined threshold, the process proceeds to step S28. The process of step S27 is exactly the same as step S17 in FIG. 4.

[0114] In step S28, the driving assistance device 1 displays a warning to the driver that an airborne object 101 is present ahead. After that, the series of processes ends, and the process returns to the original process step (RETURN). The process of step S28 is exactly the same as step S20 in FIG. 4.

[0115] On the other hand, if the total evaluation value is equal to or greater than the second threshold value in the process of step S24, the process proceeds to step S25, in which the driving assistance device 1 determines that the recognized airborne object 101 is an impassable object. The process of step S25 is exactly the same as step S18 in FIG. 4.

[0116] Subsequently, in step S26, the driving assistance device 1 executes a predetermined emergency braking control. After that, the series of processes ends, and the process returns to the original process step (RETURN). The process of this step S26 is exactly the same as step S19 in FIG. 4.

[0117] As explained above, according to the second embodiment, it is possible to obtain the same effects as the first embodiment. Furthermore, according to this embodiment, a predetermined evaluation value is calculated for each condition (size, fall speed, size change, transparency of the airborne object, etc. 101) based on the characteristic information of the airborne object, etc. 101, and it is determined whether or not the object is a passable object based on the total evaluation value. This allows for a more reliable determination.

[0118] The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit and scope of the invention. Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple constituent elements. For example, if the problem to be solved by the invention can be solved and the effects of the invention can be obtained even if some constituent elements are deleted from all the constituent elements shown in the above-described embodiments, the configuration from which these constituent elements are deleted can be extracted as the invention. Furthermore, constituent elements from different embodiments may be appropriately combined. The present invention is not limited by specific embodiments other than as limited by the appended claims. [Explanation of symbols]

[0119] 1...Driving assistance device 10...Camera unit 11...Stereo camera (surrounding environment recognition device) 12...Image Processing Unit (IPU) 13...Image recognition unit (Image recognition ECU) 14...Travel control unit (Travel_ECU) 21...Cockpit control unit (CP_ECU) 22...Engine control unit (E / G_ECU) 23...Transmission control unit (T / M_ECU) 24...Brake control unit (BK_ECU) 25...Power steering control unit (PS_ECU) 31...Human Machine Interface (HMI) 32...Throttle actuator 33...Hydraulic control circuit 34...Brake actuator 35...Electric power steering motor 36...Locator unit 36a...GNSS sensor 36b…High-precision road map database (road map DB) 37...In-vehicle radar device (surrounding environment recognition device) 37lf...Front left side sensor 37rf...Right front side sensor 37lr...Left rear side sensor 37rr...Right rear side sensor 38...Rear sensor 100...Travel path 101...Floating objects, etc. M...own vehicle

Claims

1. A driving assistance device for a vehicle that can at least perform obstacle avoidance control to avoid a collision with a recognized object, a surrounding environment recognition device including a recognition unit that recognizes the surrounding environment of the vehicle, and a feature information acquisition unit that acquires feature information of a target object in the recognized surrounding environment; a travel control unit having a determination unit that determines whether or not the target object has a possibility of obstructing travel of the vehicle based on the acquired feature information, and that does not execute the obstacle avoidance control when it is determined that the target object has no possibility of obstructing travel of the vehicle; Equipped with The feature information of the target object acquired by the feature information acquisition unit is information relating to the size of the target object, information relating to the falling speed of the target object, and information relating to the amount of change in size of the target object; A vehicle driving assistance device characterized by:

2. 2. The vehicle driving assistance device according to claim 1, wherein the surrounding environment recognition device is a stereo camera.

3. Further provided is a notification device that notifies the driver of predetermined information, the feature information acquisition unit further acquires information regarding the transparency of the target object as feature information of the target object; A vehicle driving assistance device as described in any one of claims 1 to 2, characterized in that when it is determined that the transparency of the target object is less than a predetermined threshold based on the acquired information about the transparency, the notification device notifies the driver of predetermined warning information.

4. 4. The vehicle driving assistance device according to claim 3, wherein the notification device includes at least one of a display panel and a speaker.

5. The determination unit of the driving control unit If the information on the size of the target object indicates that the size of the target object is less than a predetermined threshold, and the information on the falling speed of the target object indicates that the falling speed of the target object is less than a predetermined threshold, and the information on the amount of change in the size of the target object indicates that the change in the size of the target object is equal to or greater than a predetermined threshold, it is determined that the target object is not likely to obstruct the travel of the vehicle.

2. The vehicle driving assistance device according to claim 1.

Images