Vehicle driving assistance systems

Abstract

To provide a driving support device for a vehicle that executes smooth driving control without giving a driver a sense of discomfort.SOLUTION: A driving support device for a vehicle includes: an ambient environment recognition device having a recognition section for recognizing an ambient environment and a feature information acquisition section for acquiring feature information of a three-dimensional object; a driving control unit having a driving route setting section, an identification section that recognizes aerial objects based on the feature information and identifies an object type based on the feature information of the aerial object, and a determination section that determines whether or not the aerial object may interfere with driving a vehicle; and a steering control section. The driving control unit continues normal driving control when there is no possibility of the aerial object interfering with driving the vehicle, estimates the falling position of the aerial object when the aerial object may impede driving of the vehicle, and sets a novel driving route to avoid the estimated falling position and performs driving control along the novel driving route when the estimated falling position is on the driving route.SELECTED DRAWING: Figure 1

Description

Technical Field

[0005] ,

[0004] , , , ,

[0003]

[0001] The present invention relates to a driving support device for a vehicle that performs driving support control to avoid a collision with an object recognized based on surrounding environment information acquired using an in-vehicle camera device or the like.

Background Art

[0002] In recent years, in vehicles such as automobiles, the development of automatic driving control technology for automatically driving a vehicle without requiring a driver's driving operation has been underway. In addition, various driving support devices capable of executing various driving controls for assisting a driver's driving operation using this type of automatic driving control technology have been proposed and are generally being put into practical use.

[0003] In conventional driving support devices, as a surrounding environment recognition device for recognizing the surrounding environment of a vehicle and acquiring it as surrounding information, sensing devices such as an in-vehicle camera device or an in-vehicle radar device are used. Among these, the in-vehicle camera device recognizes the surrounding environment of the vehicle based on the acquired image data. In addition, the in-vehicle radar device outputs radio waves toward the surroundings of the vehicle, receives reflected waves from an object, and analyzes the received waves to recognize the surrounding environment of the vehicle.

[0004] In conventional driving support devices, the vehicle is driven while recognizing the surrounding environment of the vehicle using these sensing devices. At this time, when an object that may obstruct the vehicle's travel, for example, is recognized on the travel route on which the vehicle is traveling, emergency braking control, emergency steering control, etc. are performed to prevent the vehicle from colliding with the object. Thereby, the vehicle's travel can be safely continued. Regarding this type of control technology, for example, a driving support device having a function such as an obstacle avoidance control for avoiding obstacles such as three-dimensional objects, various proposals have been made in the past, for example, by Japanese Patent Laid-Open No. 2019-18733, and practical use is generally progressing.

[0005] Incidentally, objects that can be recognized by conventional driver assistance systems include, for example, various road markings such as lane markings and planar structures such as manholes (hereinafter simply referred to as lane markings, etc.). In addition, there are stationary objects that are permanently installed on the road, such as road edge curbs, guardrails, road signs, utility poles, and commercial billboards, as well as three-dimensional objects that move, such as pedestrians, bicycles, other vehicles, and animals. Furthermore, there are various other three-dimensional objects of various forms on the road, such as road cones and billboards that are temporarily installed near construction sites, as well as objects that have fallen from the cargo beds of other vehicles, etc.

[0006] Furthermore, for example, so-called plastic bags and vinyl bags, when empty, are extremely light in mass and may float in the air (away from the road surface) due to the influence of wind, etc. (hereinafter, these types of objects are referred to as "floating objects, etc."). Conventional driver assistance systems' ambient environment recognition devices can recognize these various types of objects.

[0007] Furthermore, even before objects that have fallen from the cargo beds of other vehicles (relatively heavy objects) or objects that are being blown towards the road by strong winds (relatively lightweight objects, such as empty cardboard boxes) fall onto the road, that is, when they are still in the air (away from the road surface) (hereinafter referred to as "falling objects"), the surrounding environment recognition devices of conventional driver assistance systems can recognize these objects.

[0008] Furthermore, conventional driver assistance systems recognize objects that meet certain conditions among the various objects mentioned above as potentially obstructing the vehicle's movement if they are detected on the vehicle's travel path. In this case, the driver assistance system executes obstacle avoidance control, including emergency braking control and emergency steering control, to avoid a collision with the recognized object.

[0009] However, the possibility of the aforementioned airborne objects obstructing vehicle movement is considered extremely low. Furthermore, falling objects are unlikely to obstruct vehicle movement unless they fall directly onto the vehicle's path. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Publication No. 2019-18733 [Overview of the project] [Problems that the invention aims to solve]

[0011] However, with conventional driver assistance systems, it has been difficult to determine whether or not the vehicle should avoid an object that is in the air (away from the road surface). As a result, conventional driver assistance systems sometimes performed predetermined obstacle avoidance control even when they detected objects that were unlikely to obstruct the vehicle's movement, such as floating objects or objects that are unlikely to fall onto the vehicle's path. In such cases, this could cause discomfort to the driver.

[0012] The object of the present invention is to provide a vehicle driver assistance system that performs driving assistance control to avoid collisions with objects recognized based on surrounding environment information acquired using an in-vehicle camera device, etc., and that can always perform smooth driving control without causing discomfort to the driver. [Means for solving the problem]

[0013] To achieve the above objective, a vehicle driving assistance device according to one aspect of the present invention is a vehicle driving assistance device capable of performing control to avoid collision with a recognized object, comprising: a recognition unit for recognizing the surrounding environment of the vehicle; and a feature information acquisition unit for acquiring feature information of three-dimensional objects in the recognized surrounding environment; beforeA driving path setting unit sets the driving path of the vehicle based on the surrounding environment recognized by the recognition unit, and a feature information acquisition unit acquires the feature information of the air. air Along with recognizing the object, the recognized The aforementioned The vehicle comprises a driving control unit that controls the driving of the vehicle, having a determination unit that determines the type of object based on the characteristic information of the airborne object, and a determination unit that determines whether or not the airborne object may obstruct the driving of the vehicle based on the determination result by the determination unit, and a steering control unit that performs predetermined steering control based on a control signal from the driving control unit. The driving control unit continues normal driving control if it determines that the airborne object may obstruct the driving of the vehicle, and if it determines that the airborne object may obstruct the driving of the vehicle, it estimates the landing position of the airborne object, and if the estimated landing position is on the vehicle's driving path, and there are fewer than two lanes on one side with an oncoming lane, and no oncoming vehicle is detected in the oncoming lane, it sets a new driving path to avoid the landing position, outputs a corresponding control signal to the steering control unit, and performs driving control along the new driving path by steering. [Effects of the Invention]

[0014] According to the present invention, a vehicle driver assistance system is provided that performs driving assistance control to avoid collisions with objects recognized based on surrounding environment information acquired using an in-vehicle camera device, and can always perform smooth driving control without causing discomfort to the driver. [Brief explanation of the drawing]

[0015] [Figure 1] A block diagram showing the schematic configuration of a driver assistance device according to one embodiment of the present invention. [Figure 2] A conceptual diagram showing, from the side, how a vehicle equipped with a driver assistance device according to one embodiment of the present invention recognizes an object in front of it. [Figure 3] Conceptual diagram when looking from the top surface at the state when a host vehicle equipped with a driving support device according to an embodiment of the present invention recognizes a forward object [Figure 4] Diagram showing an example of display of a forward image acquired by an in-vehicle camera device of a host vehicle equipped with a driving support device according to an embodiment of the present invention (in the case of an object in the process of falling, etc.) [Figure 5] Diagram showing another example of display of a forward image acquired by an in-vehicle camera device of a host vehicle equipped with a driving support device according to an embodiment of the present invention (in the case of an airborne floating object, etc.) [Figure 6] Flowchart showing the flow of operation of a driving support device according to an embodiment of the present invention [Figure 7] Flowchart showing a subroutine of the process of step S12 in FIG. 6 (airborne object behavior recognition process)

Mode for Carrying Out the Invention

[0016] Hereinafter, the present invention will be described with reference to the illustrated embodiments. Each of the drawings used in the following description is schematically shown, and in order to show each component in a size that can be recognized on the drawing, the dimensional relationships, scales, etc. of each member may be shown differently for each component. Therefore, the present invention is not limited only to the forms shown in the drawings with respect to the quantity of each component, the shape of each component, the ratio of the size of each component, the relative positional relationship of each component, etc. described in each drawing.

[0017] First, the schematic configuration of a driving support device according to an embodiment of the present invention will be described below using FIG. 1. FIG. 1 is a block configuration diagram showing the schematic configuration of a driving support device according to an embodiment of the present invention. As shown in FIG. 1, the basic configuration of the driving support device 1 of the present embodiment has a configuration substantially the same as that of a conventional driving support device of this type. Therefore, the following description will be limited only to the schematic configuration of the driving support device 1 of the present embodiment.

[0018] The driving support device 1 of the present embodiment has a camera unit 10 which is an in-vehicle camera device fixed to the upper central part near the front in the vehicle interior of the vehicle (hereinafter referred to as the host vehicle) on which the driving support device 1 is mounted.

[0019] The camera unit 10 is configured to include a stereo camera 11, an image processing unit (IPU) 12, an image recognition unit (image recognition_ECU) 13, and a driving control unit (driving_ECU) 14.

[0020] The stereo camera 11 is a sensing device that functions as a recognition unit for recognizing the surrounding environment of the vehicle. The stereo camera 11 has a main camera 11a and a sub camera 11b. The main camera 11a and the sub camera 11b are arranged, for example, symmetrically on the left and right with respect to the center in the vehicle width direction in the vehicle interior of the host vehicle, facing forward (in the traveling direction). The main camera 11a and the sub camera 11b are each constituted by, for example, a CMOS image sensor or the like, and acquire two images of the surrounding environment of a predetermined area in front of the vehicle from different viewpoints at a predetermined imaging cycle synchronized with each other to generate a stereo image.

[0021] The IPU 12 performs predetermined image processing on the surrounding environment image data (image data representing the surrounding environment during the traveling of the host vehicle) captured by the stereo camera 11, and detects the edges of various objects such as objects represented on the image and lane lines marked on the road surface. Thereby, the IPU 12 recognizes three-dimensional objects, lane lines, etc. around the vehicle. Then, the IPU 12 acquires distance information from the amount of positional deviation of corresponding edges in the left and right images, and generates image information (distance image information) including the distance information.

[0022] The image recognition ECU13 determines the road curvature [1 / m] of the lane markings that demarcate the road on the left and right sides of the road the vehicle is traveling on (the vehicle's road) and the width between the left and right lane markings (lane width) based on distance image information received from the IPU12. Various methods are known for determining this road curvature and lane width. For example, the image recognition ECU13 recognizes the left and right lane markings by binarization processing based on brightness differences using surrounding environment information, and determines the curvature of the left and right lane markings for each predetermined section using a curve approximation formula based on the least squares method. Furthermore, the image recognition ECU13 calculates the lane width from the difference in curvature between the left and right lane markings.

[0023] The image recognition ECU13 then calculates the lane center, the distance from the lane center to the vehicle's lateral position deviation (the distance in the vehicle's width direction), etc., based on the curvature of the left and right lane lines and the lane width.

[0024] Furthermore, the image recognition ECU13 performs predetermined pattern matching on distance image information to recognize three-dimensional objects such as guardrails, curbs, and surrounding vehicles that extend along the road. In the recognition of three-dimensional objects by the image recognition ECU13, for example, the type of object, the height of the object, the distance to the object, the speed of the object, the relative speed between the object and the vehicle, and the relative distance between objects (for example, the lateral distance between a curb at the edge of the road and a nearby lane marking).

[0025] The various pieces of information recognized by the image recognition ECU13 are output to the driving ECU14 as the first ambient environment information.

[0026] Thus, in the driver assistance device 1 of this embodiment, the image recognition ECU 13, together with the stereo camera 11 and the IPU 12, realizes the function of an ambient environment recognition device that recognizes a first ambient environment around the vehicle.

[0027] The Driving ECU 14 is a control unit for the overall control of the driver assistance system 1. Various control units, including the cockpit control unit (CP_ECU) 21, the engine control unit (E / G_ECU) 22, the transmission control unit (T / M_ECU) 23, the brake control unit (BK_ECU) 24, and the power steering control unit (PS_ECU) 25, are connected to this Driving ECU 14 via an in-vehicle communication line such as CAN (Controller Area Network).

[0028] Furthermore, the driving ECU14 is connected to various sensors, including a locator unit 36, an on-board radar device 37 (left front side sensor 37lf, right front side sensor 37rf, left rear side sensor 37lr, right rear side sensor 37rr), and a rear sensor 38.

[0029] The CP_ECU21 is connected to a Human-Machine Interface (HMI)31 located around the driver's seat. The HMI31 consists of, for example, switches for instructing the execution of various driving assistance controls, a mode switch for switching driving modes, a steering touch sensor for detecting the driver's steering state, a driver monitoring system (DMS) for detecting the driver's facial recognition and gaze, a touch panel display, a combination meter, a speaker, and the like.

[0030] When CP_ECU21 receives control signals from Driving_ECU14, it appropriately notifies the driver of various warnings regarding the vehicle ahead, the status of driver assistance control implementation, and various information regarding the surrounding environment of the vehicle, etc., via display and voice through HMI31. In addition, CP_ECU21 outputs various input information, such as the on / off operation status of various driver assistance controls, input by the driver via HMI31, to Driving_ECU14.

[0031] The output side of the E / G_ECU22 is connected to the throttle actuator 32 of the electronically controlled throttle, etc. Various sensors, such as an accelerator sensor (not shown), are connected to the input side of the E / G_ECU22.

[0032] The E / G_ECU22 controls the throttle actuator 32 based on control signals from the Driving_ECU14 or detection signals from various sensors. This allows the E / G_ECU22 to adjust the amount of intake air for the engine and generate the desired engine output. The E / G_ECU22 also outputs signals such as the accelerator opening angle detected by the various sensors to the Driving_ECU14.

[0033] The output side of T / M_ECU23 is connected to the hydraulic control circuit 33. Various sensors, such as a shift position sensor (not shown), are connected to the input side of T / M_ECU23. Based on the engine torque signal estimated by E / G_ECU22 and the detection signals from the various sensors, T / M_ECU23 performs hydraulic control on the hydraulic control circuit 33. As a result, T / M_ECU23 operates the friction engagement elements and pulleys provided in the automatic transmission to shift the engine output to the desired gear ratio. T / M_ECU23 also outputs signals such as the shift position detected by the various sensors to the driving_ECU14.

[0034] The output side of the BK_ECU24 is connected to brake actuators 34, which adjust the brake fluid pressure output to the brake wheel cylinders located on each wheel. The input side of the BK_ECU24 is connected to various sensors, including a brake pedal sensor, yaw rate sensor, longitudinal acceleration sensor, and vehicle speed sensor (not shown).

[0035] The BK_ECU24 controls the brake actuator 34 based on control signals from the Driving_ECU14 or detection signals from various sensors. This allows the BK_ECU24 to appropriately generate braking force on each wheel for forced braking control and yaw rate control of the vehicle. The BK_ECU24 also outputs signals such as brake operation status, yaw rate, longitudinal acceleration, and vehicle speed (vehicle speed) detected by various sensors to the Driving_ECU14.

[0036] The output side of the PS_ECU25 is connected to an electric power steering motor 35, which applies steering torque to the steering mechanism through the motor's rotational force. Various sensors, such as a steering torque sensor and a steering angle sensor, are connected to the input side of the PS_ECU25.

[0037] The PS_ECU25 controls the electric power steering motor 35 based on control signals from the driving_ECU14 or detection signals from various sensors. This causes the PS_ECU25 to generate steering torque for the steering mechanism. The PS_ECU25 also outputs signals such as steering torque and steering angle detected by the various sensors to the driving_ECU14.

[0038] The locator unit 36 ​​is comprised of a GNSS sensor 36a and a high-precision road map database (road map DB) 36b.

[0039] The GNSS sensor 36a determines the vehicle's position (latitude, longitude, altitude, etc.) by receiving positioning signals transmitted from multiple positioning satellites.

[0040] The road map DB36b is a large-capacity storage medium such as an HDD or SSD, and stores highly accurate road map information (dynamic map). This road map DB36b holds lane data necessary for autonomous driving, such as lane width data, lane center position coordinate data, lane direction angle data, and speed limit data. This lane data is stored at intervals of several meters for each lane on the road map. The road map DB also holds information on various facilities and parking lots. For example, based on a request signal from the driving_ECU14, the road map DB36b outputs road map information for a set range based on the vehicle's position determined by the GNSS sensor 36a as third surrounding environment information to the driving_ECU14.

[0041] Thus, in the driving support device 1 of this embodiment, the road map DB36b, together with the GNSS sensor 36a, realizes the function of an ambient environment recognition device that recognizes a third ambient environment around the vehicle.

[0042] The left front side sensor 37lf, the right front side sensor 37rf, the left rear side sensor 37lr, and the right rear side sensor 37rr are multiple sensors that constitute the in-vehicle radar system 37, and are composed of, for example, millimeter-wave radar.

[0043] Here, each millimeter-wave radar detects three-dimensional objects, primarily pedestrians and vehicles traveling alongside the road, as well as structures located at the edge of the road (e.g., the shoulder), such as curbs, guardrails, building walls, and planted trees, by analyzing the reflected waves from objects in response to the emitted radio waves. Furthermore, each millimeter-wave radar also detects various three-dimensional objects present on the road or in the air (at a distance from the road surface). In this case, each radar detects specific information about the three-dimensional object, such as its width, the position of a representative point of the object (relative position and relative distance from the vehicle), and its relative velocity.

[0044] The left front side sensor 37lf and the right front side sensor 37rf are, for example, located on the left and right sides of the front bumper, respectively. The left front side sensor 37lf and the right front side sensor 37rf detect three-dimensional objects in the diagonally forward and lateral areas of the vehicle that are difficult to recognize with the image from the stereo camera 11, as second surrounding environment information.

[0045] Furthermore, the left rear side sensor 37lr and the right rear side sensor 37rr are, for example, located on the left and right sides of the rear bumper, respectively. The left rear side sensor 37lr and the right rear side sensor 37rr detect three-dimensional objects in the diagonal lateral and rear areas of the vehicle that are difficult to recognize with the left front side sensor 37lf and the right front side sensor 37rf, as second surrounding environment information.

[0046] Thus, in the driver assistance device 1 of this embodiment, the on-board radar device 37 (front side sensor 37lf, right front side sensor 37rf, left rear side sensor 37lr, right rear side sensor 37rr) functions as an ambient environment recognition device that recognizes a second ambient environment around the vehicle. The information acquired by these sensors 37lf, 37rf, 37lr, and 37rr is sent to the image recognition ECU 13.

[0047] The rear sensor 38 is composed of, for example, a sonar device. This rear sensor 38 is, for example, installed on the rear bumper. The rear sensor 38 detects three-dimensional objects in the area behind the vehicle that are difficult to recognize with the left rear side sensor 37lr and the right rear side sensor 37rr, as a fourth type of surrounding environment information. The rear sensor 38 then functions as a surrounding environment recognition device that recognizes the fourth type of surrounding environment around the vehicle.

[0048] Furthermore, the coordinates of each external object included in the first ambient environment information recognized by the camera unit 10 including the image recognition ECU 13, the third ambient environment information recognized by the locator unit 36, the second ambient environment information recognized by the on-board radar device 37 (left front side sensor 37lf, right front side sensor 37rf, left rear side sensor 37lr, right rear side sensor 37rr), and the fourth ambient environment information recognized by the rear sensor 38 are all converted into coordinates in a three-dimensional coordinate system with the center of the vehicle as the origin by the driving ECU 14.

[0049] The driving ECU14 has several driving modes: a manual driving mode, a first driving control mode and a second driving control mode, and a stow mode. These driving modes can be selectively switched in the driving ECU14 based on, for example, the operation status of the mode switching switch provided on the HMI31.

[0050] Here, manual driving mode refers to a driving mode that requires the driver to maintain steering, and is a driving mode in which the vehicle is driven according to driving operations such as steering, accelerating, and braking performed by the driver.

[0051] Similarly, the first driving control mode is a driving mode that requires the driver to maintain steering. In other words, the first driving control mode reflects the driver's driving operations and, for example, through the control of E / G_ECU22, BK_ECU24, PS_ECU25, etc., primarily sets a target driving path and drives the vehicle along that target driving path by appropriately combining adaptive cruise control (ACC), active lane keep centering (ALKC), and active lane departure prevention (Active Lane Keep Bouncing) to set a target driving path and drive the vehicle along that target driving path. This is a semi-autonomous driving mode, or a driving assistance mode.

[0052] Here, adaptive cruise control (ACC) is basically performed based on first ambient environment information input from the image recognition ECU13. In other words, adaptive cruise control (ACC) is performed based on, for example, information about the preceding vehicle included in the first ambient environment information from the image recognition ECU13.

[0053] Furthermore, lane centering control and lane departure prevention control are basically performed based on first and third surrounding environment information input from at least one of the image recognition ECU 13 or the locator unit 36. In other words, lane centering control and lane departure prevention control are performed based, for example, on lane marking information included in the first and third surrounding environment information from the image recognition ECU 13 or the locator unit 36.

[0054] Furthermore, the second driving control mode is an automated driving mode that realizes a so-called hands-off function, which allows the vehicle to travel according to a target route (route map information) by appropriately combining preceding vehicle following control, lane centering control, and lane departure prevention control, for example, through the control of E / G_ECU22, BK_ECU24, PS_ECU25, etc., without requiring the driver to operate the steering, accelerator, or brakes.

[0055] The escape mode is a mode that automatically stops the vehicle on the roadside or elsewhere in the event that, for example, while driving in the second driving control mode, it becomes impossible to continue driving in that mode and it is not possible to take over driving operations to the driver (i.e., it is not possible to switch to manual driving mode or the first driving control mode).

[0056] Furthermore, in each of the above-mentioned driving modes, if the driving ECU14 detects an object such as a preceding vehicle or a three-dimensional object such as a fallen object on the road that has a high probability of colliding with the vehicle, it will determine whether or not to perform obstacle avoidance control accompanied by emergency braking (AEB (Autonomous Emergency Braking): collision damage mitigation brake) control or emergency steering control, and will perform the predetermined control as appropriate.

[0057] Furthermore, all or part of the locator unit 36, image recognition ECU 13, driving ECU 14, CP ECU 21, E / G ECU 22, T / M ECU 23, BK ECU 24, PS ECU 25, etc., are composed of a processor that includes hardware.

[0058] Here, the processor is composed of a well-known configuration including, for example, a central processing unit (CPU), RAM (Random Access Memory), ROM (Read Only Memory), non-volatile memory, non-volatile storage, and non-transitory computer-readable medium, as well as its peripheral devices.

[0059] ROM, non-volatile memory, and non-volatile storage devices pre-store software programs executed by the CPU, as well as fixed data such as data tables. The CPU reads the software programs stored in ROM, loads them into RAM, and executes them. The software programs then refer to various data as appropriate, thereby realizing the functions of each of the above-mentioned components and units (13, 14, 21-25, 36).

[0060] Furthermore, the processor may be composed of semiconductor chips such as FPGAs (Field Programmable Gate Arrays). Also, each of the above components and components (13, 14, 21~25, 36), etc., may be composed of electronic circuits.

[0061] Furthermore, the software program may be recorded in whole or in part as a computer program product on portable disc media such as flexible disks, CD-ROMs, and DVD-ROMs, or on non-transitory computer-readable media such as card-type memory, HDDs (Hard Disk Drives), and SSDs (Solid State Drives).

[0062] Furthermore, instead of (or in addition to) the stereo camera 11 included in the camera unit 10, a monocular camera may be used as the surrounding environment recognition device. Also, instead of (or in addition to) the on-board radar device 37, a LiDAR (Light Detection and Ranging) or the like may be used.

[0063] The operation of the driver assistance device 1 of this embodiment, configured in this manner, will be explained below with reference to Figures 2 to 7.

[0064] As described above, the driver assistance device 1 of this embodiment has functions to assist the driver's driving operations by performing so-called adaptive cruise control (ACC), lane centering control (ALKC), and lane departure control (ALKB).

[0065] Furthermore, the driver assistance device 1 of this embodiment recognizes whether an object detected on the road or in the air while the vehicle is in motion is an object that could potentially obstruct the vehicle's movement. If the driver assistance device 1 of this embodiment determines that the object is an object that could potentially obstruct the vehicle's movement, it has the function of assisting driving by executing obstacle avoidance control, including braking control and steering control. If the driver assistance device 1 determines that the object is not an object that could potentially obstruct the vehicle's movement, it suppresses the execution of obstacle avoidance control.

[0066] Objects that can be recognized by the driver assistance system 1 include, for example, various road markings such as lane markings and planar structures such as manholes. These planar structures, i.e., lane markings, are treated as not having the potential to obstruct the vehicle's movement and are therefore treated as such in conventional driver assistance systems as well.

[0067] Furthermore, roads contain both permanent, stationary objects such as curbs, guardrails, utility poles, road signs, road markings, and commercial billboards, as well as three-dimensional objects such as pedestrians, bicycles, other vehicles, and animals. In addition, there are various other forms of three-dimensional objects on roads, including temporarily placed stationary objects such as road cones and billboards near construction sites, and objects that have fallen from the backs of other vehicles.

[0068] These three-dimensional objects may, in some cases, obstruct the movement of the vehicle. Examples of situations in which the aforementioned three-dimensional objects may be recognized as potentially obstructing the movement of the vehicle include when such objects are located in the vehicle's path, or when an object among these objects is suspended in the air and is presumed to fall onto the vehicle's path.

[0069] On the other hand, so-called plastic bags and vinyl bags, for example, are extremely light in mass when they are empty. Because of this, these objects (plastic bags, vinyl bags, etc.) may exist as floating objects in the air, influenced by wind, etc. (hereinafter, these types of objects will be referred to as "floating objects, etc."). These types of floating objects, etc., are three-dimensional objects, but can be considered to have little potential to obstruct the movement of the vehicle. The driving assistance device 1 of this embodiment can recognize these types of floating objects, etc., by acquiring their characteristic information.

[0070] Here, for example, characteristic information of an aerial object that is unlikely to obstruct the movement of the vehicle itself could include, It is located in the air, that is, away from the road surface. Its apparent size (the area it occupies in the recognition image) is extremely small (miniature). Because it is lightweight, its falling speed is extremely low (it falls at a slow speed). Furthermore, the behavior during a fall is not solely in the direction of gravity (it can fluctuate up, down, left, right, forward, and backward due to wind, etc. Therefore, the falling speed may also change). Its apparent size and shape change during freefall (its shape changes due to being buffeted by wind, etc.). Some have high transparency (such as colorless and transparent plastic bags). These are some of the characteristics that can be mentioned.

[0071] On the other hand, in the case of falling objects, if the estimated landing position is on the vehicle's travel path, there is a possibility of obstructing the vehicle's movement. However, for example, a cardboard box assembled into a box shape, when empty (a three-dimensional object), is itself light in mass, similar to a bag-shaped object. As such, such cardboard boxes may be carried through the air onto the road by strong winds, etc. In this case, the cardboard box that has been carried through the air does not necessarily fall on the vehicle's travel path. Therefore, for falling objects, if the estimated landing position is not on the vehicle's travel path, it can be considered an object that is less likely to obstruct the vehicle's movement. The driving support device 1 of this embodiment can recognize various falling objects by acquiring their characteristic information and estimate their landing position.

[0072] Here, for example, characteristic information of a falling object that is unlikely to obstruct the vehicle's movement (specifically, an empty cardboard box, etc.) could be, for example, It is located in the air, that is, away from the road surface. The apparent size (the area it occupies in the recognition image) is small. Because it is lightweight, its falling speed is low (it falls slowly). Furthermore, the behavior during a fall is not limited to the direction of gravity (it may fluctuate up, down, left, right, forward, and backward due to strong winds, etc. The falling speed may also change). During a fall, its apparent size and shape change (it rotates due to strong winds, etc., changing the angle from which it is seen by the vehicle). These are some of the characteristics that can be mentioned.

[0073] These characteristic information of airborne objects, falling objects, etc., can be obtained, for example, from the results of predetermined image processing performed based on images acquired by the camera unit 10. In this case, the camera unit 10, which functions as a surrounding environment recognition device, also functions as a characteristic information acquisition unit that acquires characteristic information of target objects from the recognized surrounding environment. At this time, the characteristic information acquired by the characteristic information acquisition unit is sent to the driving_ECU 14. Upon receiving this, the driving_ECU 14 functions as a determination unit that determines the type of target object (for example, the type of stationary object in the air (details described later), airborne object, falling object, etc.) based on the received characteristic information, and at the same time functions as a determination unit that determines whether the determined target object has the potential to obstruct the vehicle's movement.

[0074] For example, Figures 2 to 5 conceptually illustrate the situation in which an object located in front of a vehicle equipped with the driver assistance device of this embodiment is recognized while the vehicle is traveling on a road. Specifically, Figure 2 is a conceptual diagram of the situation when a vehicle equipped with the driver assistance device of one embodiment of the present invention recognizes an object in front of it, viewed from the side. Figure 3 is a conceptual diagram of the situation in Figure 2 viewed from above. Figures 4 and 5 are diagrams illustrating examples of forward-facing images acquired by the onboard camera device of a vehicle equipped with the driver assistance device of one embodiment of the present invention. Of these, Figure 4 is an example of the forward-facing image display in the situation of Figures 2 and 3, in the case of a falling object, etc. Figure 5 is an example of the forward-facing image display in the case of an airborne object, etc.

[0075] In Figure 2, reference numeral 100 indicates the road on which the vehicle is traveling. Also, reference numeral M in Figure 2 indicates the vehicle itself. The vehicle itself M is equipped with a camera unit 10, which is a surrounding environment recognition device and an on-board camera device that constitutes part of the driver assistance system 1. This camera unit 10 is fixed in a predetermined position inside the vehicle's interior. In Figure 2, reference numeral V1 conceptually indicates the field of view (imaging range) of the stereo camera 11 of the camera unit 10. This field of view V1 corresponds to the angle of view of the image acquired by the camera unit 10. Note that in Figure 2, the field of view indicated by reference numeral V1 only shows the range in the height direction. Furthermore, reference numeral M1 in Figure 2 indicates another vehicle ahead. Reference numeral 110A indicates an object that has fallen from the cargo bed of the other vehicle M1. Here, the fallen object is still in the air and refers to an object that is falling, etc.

[0076] In Figure 3, reference numeral 101 indicates the lane in which the vehicle M and the other vehicle M1 are traveling (hereinafter referred to as the vehicle's lane). Reference numeral 102 in Figure 3 indicates the lane adjacent to the vehicle's lane 101 (hereinafter referred to as the adjacent lane). If the road 100 has two lanes in each direction, this adjacent lane 102 corresponds to what is called the overtaking lane. Note that the road 100 shown in Figure 3 may also be considered as a road with one lane in each direction. In this case, the adjacent lane 102 can be replaced with the oncoming lane.

[0077] In Figure 3, reference numeral 103 indicates the lane boundary line on the vehicle's own lane 101 side. Reference numeral 104 indicates a curb or other roadside feature or lane marking. Reference numeral 105 indicates the planned route of the vehicle M (hereinafter referred to as the vehicle's route).

[0078] In Figure 3, the symbol V2 conceptually represents the field of view (imaging range) of the stereo camera 11 of the camera unit 10. This field of view V2 corresponds to the angle of view of the image acquired by the camera unit 10. Note that in Figure 3, the field of view indicated by the symbol V2 only shows the range in the horizontal (width) direction.

[0079] Note that in Figure 3, only the side including the vehicle's lane 101 on a two-lane road is shown, and the opposing lane is omitted. Also, the same reference numerals are used for the same elements in Figures 2 to 5.

[0080] In Figure 3, reference numeral 105A indicates an example of a new driving path that the vehicle M generates to avoid a falling object 110A when the falling object falls onto the vehicle's driving path 105. Details of this new driving path 105A will be described later (see the processing flow in Figure 6).

[0081] Under these circumstances, the vehicle M equipped with the driver assistance device 1 is assumed to be traveling on its own lane 101 on road 100. At this time, the driver assistance device 1 continuously acquires images of the surrounding environment (mainly in front of the direction of travel) at predetermined intervals using the camera unit 10, and recognizes the surrounding environment by performing predetermined image processing based on the acquired image data.

[0082] At this time, it is assumed that a falling object 110A exists within the range defined by the field of view V1 and V2 in front of the vehicle M in the direction of travel, at a position away from the surface of the road 100 (at a height H from the road surface; see Figure 2). At this time, the driver assistance device 1 of the vehicle M recognizes this falling object 110A. Figures 2 and 3 conceptually illustrate this situation.

[0083] Figures 4 and 5 show examples of images acquired by the camera unit 10 of the driver assistance system 1 of the vehicle M under the conditions shown in Figures 2 and 3.

[0084] In Figure 4, reference numeral 10A indicates an example of an image acquired by the camera unit 10. In Figure 5, reference numeral 10B indicates another example of an image acquired by the camera unit 10. Within the range of these images 10A and 10B, images of the surroundings of the vehicle M (mainly in front of the direction of travel) are acquired.

[0085] In Figure 4, reference numeral 110A indicates an object in the air that is falling, similar to Figures 2 and 3. In Figure 5, reference numeral 110B indicates an airborne object that is floating in the air. In Figure 4, reference numeral 110Aa indicates the shadow cast on the road surface by the falling object 110A. Similarly, in Figure 5, reference numeral 110Ba indicates the shadow cast on the road surface by the airborne object 110B.

[0086] Furthermore, in Figures 4 and 5, reference numeral 106 indicates the median strip of the road 100. Reference numeral 107 indicates boundary walls, etc., installed along the left and right edges of the road 100. Reference numeral 108 indicates a landscape such as mountains visible in the distance. Reference numeral 109 indicates a landscape such as clouds floating in the sky. Reference numeral 111 (shown only in Figure 4) indicates plantings, etc., provided on the outside of the road 100. Reference numeral 112 indicates road signs, etc., installed on the side edges of the road 100.

[0087] Note that reference numeral 113 indicates the opposing lane included in road 100. In other words, in the display examples shown in Figures 4 and 5, road 100 is shown as a road with two lanes in each direction.

[0088] Furthermore, the arrows X, Y, and Z shown in Figures 2 to 5 represent the coordinate axes that indicate the space in which the vehicle M is located. In this case, arrow X is the coordinate axis along the width direction (horizontal direction) of the vehicle M. The positive (+) direction of arrow X is defined as the left direction when viewed from the front of the vehicle M. Arrow Y is the coordinate axis along the height direction (vertical direction) of the vehicle M. The positive (+) direction of arrow Y is defined as the upward direction when viewed from the front of the vehicle M. Arrow Z is the coordinate axis along the direction of travel of the vehicle M. The positive (+) direction of arrow Z is defined as the direction coming towards the vehicle M from the front of the vehicle M.

[0089] Based on the image data representing images 10A and 10B, the driver assistance device 1 recognizes various objects present in the surrounding environment of its own vehicle M.

[0090] In addition, the symbols 108, 109, and 111 shown as part of the scenery in Figures 4 and 5 are recognized as part of the surrounding environment, but are treated as having no potential to affect the vehicle's operation, and are therefore also treated as such in conventional driver assistance systems.

[0091] Next, the operation of the driver assistance device 1 of the vehicle M in this situation will be explained below with reference to the flowcharts in Figures 6 and 7. Here, Figure 6 is a flowchart showing the operation flow of the driver assistance device according to one embodiment of the present invention. Figure 7 is a flowchart showing the subroutine for the process in step S12 of Figure 6 (aerial object behavior recognition process). Note that the processing flow described below shows only the characteristic processes unique to the present invention among the various driver assistance controls performed by the driver assistance device 1 of this embodiment.

[0092] First, assume that the vehicle M is traveling along a pre-set driving path 105 on the road 100 in its own driving lane 101, with the driving assistance control of the driving assistance device 1 turned on (for example, in the first driving control mode). At this time, assume that the driving assistance device 1 is continuously recognizing the surrounding environment using the surrounding environment recognition device (mainly the camera unit 10, etc.).

[0093] First, in step S11 of Figure 6, the driver assistance device 1 checks whether or not an object in the air (hereinafter referred to as "airborne object, etc.") has been recognized among the various objects that have been recognized. Here, the determination of whether or not a recognized object is an airborne object, etc. is made as follows.

[0094] For example, as shown in Figure 2, the relative positional relationship between the height of the sensor (camera unit 10) installed on the vehicle M and the height of the target object is estimated based on image data. This allows the height H of the target object from the road surface to be calculated. If the height H is above a predetermined threshold, the target object can be estimated to be an airborne object or the like.

[0095] Furthermore, as shown in Figures 4 and 5, for example, it is checked whether a shadow (110Aa, 110Ba) corresponding to the target object (110A, 110B) is projected below the target object (on the road surface of road 100) in images 10A and 10B. If a shadow of the target object is confirmed, it can be estimated that the target object is an aerial object or the like (110A, 110B).

[0096] In step S11 in Figure 6, if an aerial object is recognized, the process proceeds to the next step S12. In step S12, the driving ECU 14 recognizes the aerial object recognized in step S11 as the target object and performs a process to recognize its behavior (aerial object behavior recognition process; see Figure 7).

[0097] First, in step S31 of Figure 7, the driving ECU 14 of the driving support device 1 checks whether the amount of movement per unit time in the positive (+) direction of the aerial object, etc., which has been recognized in the process of step S12 described above, is less than a predetermined threshold. If the amount of movement per unit time in the positive (+) direction of the target aerial object, etc., in the X direction is less than the predetermined threshold, the process proceeds to step S32. If the amount of movement per unit time in the positive (+) direction of the target aerial object, etc., in the X direction is equal to or greater than the predetermined threshold, the process proceeds to step S35.

[0098] In step S35, the driving ECU14 checks whether there is any movement in the negative (-) direction in the X direction of the target aerial object. If negative (-) movement in the X direction of the target aerial object is confirmed, the process proceeds to step S38. If negative (-) movement in the X direction of the target aerial object is not confirmed, the process proceeds to step S40.

[0099] In step S40, the driving_ECU14 checks whether the amount of movement per unit time in the positive (+) direction of the target aerial object in the Y direction is less than a predetermined threshold. If the amount of movement per unit time in the positive (+) direction of the target aerial object in the Y direction is less than the predetermined threshold, the process proceeds to step S39. If the amount of movement per unit time in the positive (+) direction of the target aerial object in the Y direction is greater than or equal to the predetermined threshold, the process proceeds to step S38. In step S38, the driving_ECU14 determines that the target aerial object is an aerial floating object. After that, the process proceeds to step S13 in Figure 6 (return).

[0100] In step S39, the driving ECU14 determines that the target aerial object is a falling object, but is relatively lightweight enough to move in the air due to the influence of strong winds, etc. Then, it proceeds to the process in step S13 of Figure 6 (return).

[0101] On the other hand, if the process in step S35 described above proceeds to step S38, in step S38 the driving_ECU14 determines that the target aerial object is an aerial floating object. After that, the process proceeds to step S13 in Figure 6 (return).

[0102] On the other hand, if the process in step S31 described above proceeds to step S32, in step S32, the driving_ECU14 checks whether the amount of movement per unit time in the positive (+) direction of the movement of the target aerial object in the Y direction is less than a predetermined threshold. If the amount of movement per unit time in the positive (+) direction of the target aerial object in the Y direction is less than the predetermined threshold, the process proceeds to step S33. If the amount of movement per unit time in the positive (+) direction of the target aerial object in the Y direction is greater than or equal to the predetermined threshold, the process proceeds to step S38. In step S38, the driving_ECU14 determines that the target aerial object is an aerial floating object. After that, the process proceeds to step S13 in Figure 6 (return).

[0103] On the other hand, if the process in step S32 described above proceeds to step S33, in step S33, the driving_ECU14 checks whether the amount of movement per unit time in the positive (+) direction of the behavior of the target aerial object in the Z direction is less than a predetermined threshold. If the amount of movement per unit time in the positive (+) direction of the target aerial object in the Z direction is less than the predetermined threshold, the process proceeds to step S34. In step S34, the driving_ECU14 determines that the target aerial object is stationary in the air. After that, the process proceeds to step S13 in Figure 6 (return).

[0104] Here, "objects stationary in the air" refers to stationary objects permanently installed at a location away from the road surface, such as road signs, road markings, and commercial billboards (hereinafter referred to as "stationary objects in the air"). In images 10A and 10B shown in Figures 4 and 5, this corresponds to the road markings indicated by reference numeral 112.

[0105] Furthermore, if, in step S33, the amount of movement per unit time in the positive (+) direction in the Z direction of the target aerial object is greater than or equal to a predetermined threshold, the process proceeds to step S36.

[0106] In step S36, the driving ECU 14 checks whether there is any movement in the negative (-) direction in the Z-direction of the target aerial object. If negative (-) movement in the Z-direction of the target aerial object is confirmed, the process proceeds to step S38. In step S38, the driving ECU 14 determines that the target aerial object is an aerial floating object. After that, the process proceeds to step S13 in Figure 6 (return).

[0107] Furthermore, if no movement in the negative (-) direction in the Z direction of the target aerial object is detected during the process in step S36 described above, the process proceeds to step S37.

[0108] In step S37, the driving ECU14 determines that the target aerial object is a falling object, and is a relatively heavy object that is moving towards the vehicle M with a predetermined acceleration, unaffected by wind or other factors. After that, the process proceeds to step S13 in Figure 6 (return).

[0109] As described above, the driving support device 1 of this embodiment recognizes airborne objects and determines the type of the recognized airborne object. The types of airborne objects include stationary objects in the air, falling objects (light and heavy, two types), and floating objects.

[0110] Returning to Figure 6, in step S13, the driving ECU 14 checks the result of the process in step S12 (aerial object behavior recognition process; see Figure 7). If the result is "falling object, etc.", the process proceeds to step S14. If the result is anything other than "falling object, etc.", the normal driving control currently being executed continues. Then, the series of processes ends and the process returns to the original processing step (return). Here, "anything other than a falling object, etc." refers to "floating object, etc." or "stationary object in the air".

[0111] In other words, in the driving support device 1 of this embodiment, if the recognized aerial object is an "aerial floating object" or "stationary object in the air" other than an "object in the fall," the execution of obstacle avoidance control is suppressed because these aerial objects are considered not to have the potential to obstruct the driving of the vehicle M.

[0112] In step S14, the Driving_ECU14 estimates the landing point of the target object, i.e., the "falling object, etc." The estimated landing point of the target object can be calculated based, for example, on the characteristic information of the target object (various information such as size, falling speed, direction of movement, amount of movement, etc.) and information such as the vehicle speed of the vehicle M. The characteristic information of the target object is obtained in the processing of step S12 described above (subroutine in Figure 7). In addition, information about the vehicle M is obtained by the Driving_ECU14 as it is driving.

[0113] Next, in step S15, the driving ECU 14 checks whether the estimated fall location estimated in step S14 overlaps with the area on the vehicle's driving path 105 (see Figure 3) that the vehicle M is currently traveling on. If it is confirmed that the estimated fall location of the "falling object, etc." is on the vehicle's driving path 105, the process proceeds to step S16. If it is confirmed that the estimated fall location of the "falling object, etc." is outside the vehicle's driving path 105, the normal driving control currently being executed continues. Then, the series of processes ends and the process returns to the original processing step.

[0114] In this context, specific examples of "falling objects, etc." that are likely to fall outside the vehicle's travel path 105 include, for example, relatively lightweight objects that may be carried by strong winds and float in the air for a predetermined period of time. In other words, examples include assembled cardboard boxes that do not contain any contents, and empty containers made of various lightweight materials (polymers, plastics, etc.).

[0115] Next, in step S16, the driving ECU 14 checks whether the road 100 on which the vehicle M is currently traveling has fewer than two lanes in each direction. If the road 100 on which the vehicle M is traveling has fewer than two lanes in each direction, the process proceeds to step S17. If the road 100 on which the vehicle M is traveling has two or more lanes in each direction, the process proceeds to step S20.

[0116] In step S17, the driving ECU14 checks whether there are any oncoming vehicles around the vehicle M. If there are oncoming vehicles around the vehicle M, the process proceeds to step S18. If there are no oncoming vehicles around the vehicle M, the process proceeds to step S23.

[0117] In step S18, the driving ECU14 checks whether there are any following vehicles around the vehicle M. If there are following vehicles around the vehicle M, the process proceeds to step S19. If there are no following vehicles around the vehicle M, the process proceeds to step S22.

[0118] In step S19, the driving ECU 14 checks whether there is sufficient time or distance to allow the vehicle M to reach the estimated drop point. If there is sufficient time, the process proceeds to step S20. If there is not sufficient time, the process proceeds to step S22.

[0119] In step S20, the driving ECU14 displays a warning to the driver, such as "There is a falling object ahead" or "There is a possibility that the falling object will fall onto the driving path ahead." This warning can be a visual warning, such as an image displayed on the HMI31 (display panel, etc.), or an auditory warning, such as an audio warning using the HMI31 (speaker, etc.). In this case, the display panel and speaker included in the HMI31 function as notification devices that inform the driver of predetermined information. After that, the series of processes ends and the process returns to the original processing step.

[0120] On the other hand, if the process in step S16 described above proceeds to step S21, in step S21, the driving ECU 14 checks whether there are other vehicles such as following vehicles, oncoming vehicles, or vehicles driving alongside the vehicle M. If there are other vehicles around the vehicle M, the process proceeds to step S24. If there are no other vehicles around the vehicle M, the process proceeds to step S23.

[0121] Then, if it is confirmed in step S17 above that there are no oncoming vehicles, and if it is confirmed in step S21 above that there are no other vehicles, the process proceeds to step S23. In step S23, the driving_ECU14 mainly performs obstacle avoidance control by steering control. In this case, the driving_ECU14 first generates a new driving path 105A (see dotted line in Figure 3) to avoid the estimated drop point. Next, the driving_ECU14 sets the newly generated driving path 105A in place of the current vehicle driving path 105. Then, the driving_ECU14 performs driving control to drive the vehicle M along the newly generated driving path 105A. After that, the series of processes ends and the process returns to the original processing step (return). Note that the generation of the new driving path 105A can be done using the same means as those used in conventional obstacle avoidance control.

[0122] On the other hand, if it is confirmed in step S18 that there are no following vehicles, and if, in step S19, there is insufficient time or distance for the vehicle M to reach the estimated point of impact, the process proceeds to step S22. In step S22, the driving ECU 14 primarily performs obstacle avoidance control through braking control. After that, the series of processes is completed, and the process returns to the original processing step.

[0123] Furthermore, if, in step S21 described above, another vehicle is present around the vehicle M, the process proceeds to step S24. In step S24, the driving ECU 14 performs normal obstacle avoidance control, including braking control and steering control, according to the surrounding conditions. After that, the series of processes is completed, and the process returns to the original processing step.

[0124] In this case, the situation is determined to be two or more lanes on one side in the process of step S16 described above. At this time, for example, if there are no oncoming vehicles and the presence of other vehicles such as following vehicles or vehicles running alongside is confirmed, the control will differ depending on whether the other vehicles are only following vehicles in the vehicle's own lane, following vehicles or vehicles running alongside are in adjacent lanes, or other vehicles are present in both the vehicle's own lane and adjacent lanes.

[0125] In this case, if there are other vehicles (in this case, following vehicles) only in the vehicle's own lane, and the vehicle brakes suddenly to avoid a fallen object, it may affect the following vehicle in the same lane. Therefore, in such cases, steering control should be applied to an adjacent lane where there are no following vehicles.

[0126] Furthermore, if other vehicles (in this case, at least one or both of the following vehicle and the vehicle traveling alongside) are present only in the adjacent lane, if your vehicle veers into the adjacent lane or changes lanes due to steering control to avoid a fallen object, it may affect the following vehicle in the adjacent lane. Therefore, in such cases, braking control should be performed within your own lane where there are no following vehicles. After the following vehicle in the adjacent lane has passed, steering control should be performed to veer into the adjacent lane or change lanes to avoid the obstacle.

[0127] Furthermore, if other vehicles are present in either the vehicle's own lane or an adjacent lane, the driver should take into consideration the impact of changing the vehicle's route on following vehicles and perform the necessary driving control as appropriate.

[0128] Furthermore, if there is an oncoming vehicle and the vehicle is traveling in a lane close to the center line, steering control should be applied to change the vehicle's path to the lane opposite the oncoming vehicle.

[0129] As described above, according to the above embodiment, when the vehicle M equipped with the driver assistance device 1 is traveling on a road while recognizing the surrounding environment in front of the direction of travel using the surrounding environment recognition device (10, 37), it recognizes objects in the air from among the recognized surrounding environment. The driving control unit 14 then determines the type of the recognized airborne object based on the object's characteristic information. Based on the determination result, it determines whether the target object has the potential to obstruct the driving of the vehicle M. If it is determined that the target object does not have the potential to obstruct the driving of the vehicle M, the normal driving control of the vehicle M is continued. If it is determined that the target object has the potential to obstruct the driving of the vehicle M, the estimated fall location of the target object is estimated. If the estimated fall location is on the vehicle's driving path 105, a new driving path that avoids the estimated fall location is set, and driving control along the new driving path is executed.

[0130] With this configuration, if the recognized aerial object does not pose a risk of obstructing the vehicle M's movement (for example, if it is a light object such as a floating object or a falling object), unnecessary obstacle avoidance control can be suppressed. Furthermore, if the recognized aerial object does pose a risk of obstructing the vehicle M's movement (for example, if it is a heavy falling object), a new driving path can be set to avoid the estimated landing position of the object, and driving control (obstacle avoidance control) along this new driving path can be reliably performed. Therefore, smooth driving assistance control can always be performed without causing discomfort to the driver.

[0131] The present invention is not limited to the embodiments described above, and various modifications and applications can be implemented without departing from the spirit of the invention. Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple constituent elements disclosed. For example, if the problem that the invention aims to solve can be solved and the effects of the invention can be obtained even if some constituent elements are deleted from all the constituent elements shown in one embodiment, then the configuration with these deleted constituent elements can be extracted as an invention. Furthermore, constituent elements from different embodiments may be combined as appropriate. This invention is not limited by any particular embodiment other than being limited by the appended claims. [Explanation of Symbols]

[0132] 1…Driving assistance system 10…Camera unit 11…Stereo camera (environmental awareness device) 12…Image Processing Unit (IPU) 13…Image Recognition Unit (Image Recognition ECU) 14…Driving control unit (Driving ECU) 21…Cockpit Control Unit (CP_ECU) 22…Engine control unit (E / G_ECU) 23…Transmission Control Unit (T / M_ECU) 24…Brake control unit (BK_ECU) 25…Power steering control unit (PS_ECU) 31…Human-Machine Interface (HMI) 32… Throttle Actuator 33... Hydraulic control circuit 34…Brake actuator 35…Electric power steering motor 36...Locator Unit 36a...GNSS sensor 36b…High-precision road map database (road map DB) 37… Vehicle-mounted radar system (environmental awareness system) 37lf... Left front side sensor 37rf...Right front side sensor 37lr…Left rear side sensor 37rr...Right rear side sensor 38…Rear sensor 100...Road 101... Floating objects, etc. M... My vehicle M1...Other vehicles 105... My own driving route 105A... New route 110A... Falling objects, etc. 110B... Floating objects, etc.

Claims

1. A vehicle driving assistance system capable of performing control to avoid collisions with recognized objects, An ambient environment recognition device having a recognition unit that recognizes the surrounding environment of the vehicle, and a feature information acquisition unit that acquires feature information of three-dimensional objects in the recognized ambient environment, A driving path setting unit sets the driving path of the vehicle based on the surrounding environment recognized by the recognition unit, A determination unit recognizes an aerial object in the air based on the feature information acquired by the feature information acquisition unit, and determines the type of object based on the feature information of the recognized aerial object. A determination unit that determines whether the aerial object may obstruct the movement of the vehicle based on the determination result from the determination unit, A driving control unit that has and controls the driving of the vehicle, A steering control unit that performs predetermined steering control based on a signal from the aforementioned driving control unit, It is equipped with, The aforementioned driving control unit is If it is determined that the aerial object is unlikely to obstruct the vehicle's movement, the normal driving control will be continued. If it is determined that the aerial object may obstruct the vehicle's movement, the system estimates the object's landing position. If the estimated landing position lies on the vehicle's travel path, and the road has fewer than two lanes on one side with an oncoming lane, and no oncoming vehicles are detected in the oncoming lane, the system sets a new travel path to avoid the landing position and outputs a corresponding control signal to the steering control unit to perform steering control along the new travel path. A vehicle driving assistance device characterized by the following features.

2. The vehicle driving assistance device according to claim 1, characterized in that the surrounding environment recognition device is a stereo camera.

3. The system further comprises a brake control unit that performs predetermined braking control based on a control signal from the aforementioned driving control unit, If the aforementioned driving control unit determines that the aerial object may obstruct the vehicle's movement, and that there is insufficient time and distance to allow the vehicle to travel along the new driving path, it will output a predetermined control signal to the brake control unit and execute a predetermined braking control. A vehicle driving assistance device according to claim 1 or 2.

4. It is further equipped with a notification device that notifies the driver of predetermined information, The vehicle driving assistance device according to any one of claims 1 to 3, characterized in that the notification device notifies the driver of predetermined warning information without setting a new driving route if it is determined that there is sufficient time or distance before the vehicle reaches the point where the airborne object falls and that there is no possibility of hindering the vehicle's movement.

5. The vehicle driving assistance device according to claim 4, characterized in that the notification device includes at least one of a display panel or a speaker.

Images