Driving assistance device for vehicle

Abstract

The present invention comprises: an image acquisition device 10 for acquiring image data about a front area which is in front of a host vehicle and is to be imaged; a luminance distribution detection unit 14a for detecting a luminance distribution on the basis of the image data; and a control unit 14 for performing travel control of the host vehicle. The control unit recognizes an intersection in the front area on the basis of the detected luminance distribution data, and executes deceleration control when recognizing another vehicle approaching the intersection through an intersection road extending left and right from the intersection.

Description

Vehicle driving support device

[0001] This invention relates to a vehicle driving support device that executes driving control to avoid collisions at intersections and the like between vehicles such as automobiles.

[0002] In recent years, in vehicles such as automobiles, for example, autonomous sensors such as cameras and radars are installed, and the surrounding environment of the host vehicle (the situation in the visual field range mainly including the forward area in the traveling direction) is recognized using the autonomous sensor. Based on the recognized surrounding environment information, other vehicles such as the preceding vehicle, oncoming vehicle, or a vehicle traveling on an intersecting road are recognized, and various driving support devices that use the recognized other vehicle information as information for driving support control have been proposed and are generally in practical use.

[0003] In recent years, among accidents between vehicles, as a means to avoid so-called head-on collision accidents that are particularly likely to occur at intersections with poor visibility, etc., various driving support devices with functions aiming at collision avoidance by using autonomous sensors have been proposed.

[0004] For example, in Japanese Patent Laid-Open No. 2021-175,630 and the like, the surrounding environment is recognized using an autonomous sensor such as a camera, and when entering an intersection with poor visibility, a driving control is performed to temporarily stop the vehicle step by step, and when it is confirmed that the vehicle is in a situation where it can travel safely, the vehicle is made to enter the road on which it is traveling or the intersecting road, and then an acceleration driving control is executed. A vehicle driving support device is disclosed.

[0005] Japanese Patent Laid-Open No. 2021-175,630

[0006] These conventional driving support devices, for example, start operating immediately before the host vehicle enters an intersection, and perform a driving control to temporarily stop the vehicle step by step in the area in front of the intersection.

[0007] Therefore, for example, at intersections with poor visibility where there are obstacles such as buildings near the intersection, the driving control to temporarily stop the host vehicle in front of the intersection and confirm the situation where the vehicle can travel safely can be said to be a very effective means.

[0008] On the other hand, for example, in an environment where there are no legal regulations requiring a stop at an intersection with good visibility, a driving control system that forces drivers to come to a complete stop just before the intersection, while they are still aware of their surroundings, can be annoying for drivers.

[0009] Furthermore, a phenomenon known as the collision course phenomenon is known to occur at intersections with good visibility, where it is difficult to notice other vehicles approaching the intersection at a similar speed to one's own vehicle. This collision course phenomenon is mainly noticeable during the daytime, but it has been found to occur at night as well.

[0010] The object of the present invention is to provide a vehicle driving assistance device that can avoid collision accidents at intersections with other vehicles traveling from either the left or right direction on an intersecting road, particularly when the vehicle is driving at night or other times.

[0011] To achieve the above objective, a vehicle driving assistance device according to one aspect of the present invention comprises an image acquisition device that acquires image data with the area in front of the vehicle as the target of imaging, a brightness distribution detection unit that detects a brightness distribution based on the image data, and a control unit that performs driving control of the vehicle, wherein the control unit recognizes an intersection in the area in front based on the detected brightness distribution data, and if it recognizes another vehicle approaching the intersection on an intersecting road extending to the left or right from the intersection, it performs deceleration control.

[0012] Another embodiment of the present invention provides a vehicle driving assistance system comprising an image acquisition device that acquires image data with the area in front of the vehicle as the target of imaging, and a processor, wherein the processor detects a brightness distribution based on the image data, recognizes an intersection in the area in front based on the detected brightness distribution data, and, if it recognizes another vehicle approaching the intersection on an intersecting road extending to the left or right from the intersection, it performs deceleration control.

[0013] According to the present invention, in particular, it is possible to provide a vehicle driving assistance device that can avoid collision accidents at intersections with other vehicles traveling from either the left or right direction on an intersecting road while the vehicle is driving at night.

[0014] A block diagram showing the schematic configuration of a vehicle driver assistance device according to one embodiment of the present invention; a schematic diagram showing an example of displaying image data acquired by an image acquisition device while the vehicle equipped with the driver assistance device of Figure 1 is in motion; a conceptual diagram showing a simplified view of the situation in Figure 2 and an overhead view; a schematic diagram corresponding to the image display example of Figure 2, and showing an example of displaying the situation (image data) in Figure 2, especially at night; a conceptual diagram corresponding to the conceptual diagram of Figure 3, and showing the situation in Figure 3, especially at night; a graph conceptually showing the brightness distribution of the road surface in the intersection under the situation in Figure 5; a conceptual diagram showing a situation that is substantially the same as the situation in Figure 5, but where another vehicle is traveling from the left side (opposite side to the situation in Figure 5); and a flowchart showing the operation of the vehicle driver assistance device according to one embodiment of the present invention.

[0015] The present invention will be described below with reference to the illustrated embodiments. The drawings used in the following description are schematic. Therefore, in these drawings, each component is shown at a size that is recognizable on the drawing. For this reason, the dimensional relationships and scales of each component may differ for each component shown on the drawing. The present invention is not limited to the illustrated forms with respect to the quantity, shape, size ratio, relative positional relationships, etc., of each component shown in each drawing.

[0016] First, the schematic configuration of the vehicle driver assistance system of one embodiment of the present invention will be described below with reference to Figure 1. Figure 1 is a block diagram showing the schematic configuration of the vehicle driver assistance system of one embodiment of the present invention.

[0017] The basic configuration of the driver assistance device shown in Figure 1 is substantially the same as that of conventional driver assistance devices of the same type. Therefore, in the following description, the illustration and detailed explanation of the general configuration of the driver assistance device will be omitted, and only the configuration directly related to the driver assistance device of this embodiment will be described below.

[0018] In this embodiment, the vehicle equipped with the driver assistance system of this embodiment is referred to as "the vehicle," and vehicles traveling in the surrounding area (oncoming lanes, intersecting roads, etc.) are referred to as "other vehicles."

[0019] The vehicle driving assistance device 1 of this embodiment has a camera unit 10, which is an on-board camera device and image acquisition device, fixed to the upper central part of the front of the passenger compartment of the vehicle (the vehicle) on which the driving assistance device 1 is installed.

[0020] The camera unit 10 is comprised of a stereo camera 11, an image processing unit (IPU) 12, an image recognition unit 13, a control unit 14, and the like.

[0021] The stereo camera 11 is formed by having two cameras: a main camera 11a and a sub-camera 11b. The main camera 11a and the sub-camera 11b are, for example, positioned symmetrically on either side of the center in the width direction of the vehicle's interior, facing forward (in the direction of travel) of the vehicle. That is, the stereo camera 11 included in the camera unit 10 functions as an image acquisition device that acquires image data with the area in front of the vehicle as the target of imaging.

[0022] In the stereo camera 11, the main camera 11a and the sub-camera 11b are imaging devices each composed of, for example, an imaging optical system, an image sensor such as a CMOS image sensor, and a processing circuit for processing imaging signals, etc.

[0023] With this configuration, the stereo camera 11, using the main camera 11a and the sub-camera 11b, acquires two image data from two different viewpoints, covering a predetermined range of the surrounding environment outside the vehicle, at predetermined imaging cycles synchronized with each other.

[0024] Then, stereo image data is generated based on the two image data acquired in this way. This stereo image data is treated as ambient environment information representing the surrounding environment while the vehicle is in motion. The ambient environment information (image data) generated by the stereo camera 11 is output to the image processing unit (IPU) 12.

[0025] The image processing unit (IPU) 12 is a component or circuit unit that performs predetermined image processing on ambient environment information (image data representing the surrounding environment while the vehicle is in motion) acquired by the stereo camera 11. For example, the image processing unit (IPU) 12 performs processing such as detecting the edges of various objects (objects or lane markings, etc.) displayed on the image.

[0026] Furthermore, the image processing unit (IPU) 12 acquires distance information based on the amount of positional displacement of corresponding edges on the left and right images based on stereo image data, and generates image information (distance image information) that includes said distance information.

[0027] The distance image information generated by the image processing unit (IPU) 12 is output to the image recognition unit 13. In this case, the camera unit 10, which includes the stereo camera 11 and the image processing unit (IPU) 12, functions as a distance measuring device.

[0028] The image recognition unit 13 calculates the road curvature [1 / m] of the left and right lane markings on the road the vehicle is traveling on (the vehicle's road) and the width between the left and right lane markings (lane width) based on distance image information and the like input from the image processing unit (IPU) 12. Various well-known methods are used to determine this road curvature and lane width.

[0029] Furthermore, the image recognition unit 13 performs predetermined pattern matching and other operations based on image data acquired by the stereo camera 11 to recognize objects such as three-dimensional objects extending along the road (e.g., guardrails, curbs, and other vehicles in the vicinity), as well as the condition of the road surface around the vehicle (hereinafter referred to as road surface conditions, etc.).

[0030] In the image recognition unit 13, object recognition of three-dimensional objects is performed, for example, on the type of three-dimensional object, the height of the three-dimensional object, the width of the three-dimensional object, the distance from the vehicle to the three-dimensional object, the movement speed of the three-dimensional object, the relative speed between the three-dimensional object and the vehicle, and the relative distance between three-dimensional objects (for example, the lateral distance between a curb at the edge of the road and a nearby lane marking).

[0031] Furthermore, the road surface conditions recognized by the image recognition unit 13 include, for example, situations where the road surface is wet due to rain or melting snow, as well as rainfall conditions, snow accumulation conditions, compacted snow conditions, and icy road conditions, in addition to unstable ground conditions such as snow-covered areas, muddy areas, and sandy areas. These road surface conditions are estimated, for example, based on the brightness difference of the image detected by the image processing unit (IPU) 12.

[0032] The various types of information recognized by the image recognition unit 13 are output to the control unit 14 as ambient environment information. In this case, the camera unit 10, including the image recognition unit 13, functions as an ambient environment recognition device that recognizes the surrounding environment of the vehicle.

[0033] The control unit 14 controls the camera unit 10 and is a component unit or circuit unit that controls the entire driving support device 1 of this embodiment.

[0034] Various control units, such as an engine control unit (E / G_ECU) 22, a transmission control unit (T / M_ECU) 23, a brake control unit (BK_ECU) 24, and a power steering control unit (PS_ECU) 25, are connected to this control unit 14 via an in-vehicle communication line, such as a CAN (Controller Area Network; not shown).

[0035] The output side of the E / G_ECU22 is connected to the throttle actuator 32 of the electronically controlled throttle, etc. Various sensors, such as an accelerator sensor (not shown), are connected to the input side of the E / G_ECU22.

[0036] The E / G_ECU 22 performs drive control to the throttle actuator 32 based on control signals from the control unit 14 or detection signals from various sensors. As a result, the E / G_ECU 22 functions as a drive force control device that generates or suppresses the driving force of the vehicle's drive source (engine, etc.).

[0037] The E / G_ECU 22 then adjusts the amount of intake air for the engine to generate the desired engine output. The E / G_ECU 22 also outputs signals such as the amount of accelerator operation detected by various sensors to the control unit 14.

[0038] A hydraulic control circuit 33 is connected to the output side of the T / M_ECU 23. Various sensors, such as a shift position sensor (not shown), are connected to the input side of the T / M_ECU 23.

[0039] The T / M_ECU 23 performs hydraulic control on the hydraulic control circuit 33 based on the engine torque signal estimated by the E / G_ECU 22 and detection signals from various sensors. As a result, the T / M_ECU 23 operates friction engagement elements and pulleys provided in the automatic transmission to shift the engine output to the desired gear ratio. The T / M_ECU 23 also outputs signals such as the shift position detected by various sensors to the control unit 14.

[0040] The output side of the BK_ECU24 is connected to a brake actuator 34, which adjusts the brake fluid pressure output to the brake wheel cylinders provided on each wheel. The input side of the BK_ECU24 is connected to various sensors, including a brake pedal sensor, yaw rate sensor, longitudinal acceleration sensor, and vehicle speed sensor (not shown).

[0041] The BK_ECU24 is a braking device that controls the braking of the vehicle by performing drive control to the brake actuator 34 based on control signals from the control unit 14 or detection signals from various sensors. As a result, the BK_ECU24 appropriately generates braking force on each wheel for forced braking control and yaw rate control of the vehicle. The BK_ECU24 also outputs signals such as brake operation status, yaw rate, longitudinal acceleration, and vehicle speed (vehicle speed) detected by various sensors to the control unit 14.

[0042] On the output side of the PS_ECU 25, an electric power steering motor 35 that applies a steering torque by the rotational force of a motor to the steering mechanism is connected. On the input side of the PS_ECU 25, various sensors (not shown) such as a steering torque sensor and a steering angle sensor are connected.

[0043] The PS_ECU 25 is a steering device that performs drive control on the electric power steering motor 35 based on a control signal from the control unit 14 or a detection signal from various sensors, and performs steering control of the vehicle. Thereby, the PS_ECU 25 generates a steering torque for the steering mechanism. Also, the PS_ECU 25 outputs signals such as the steering torque and steering angle detected by various sensors to the control unit 14.

[0044] Further, to the control unit 14, an input device 26 as an information input unit, an alarm device 27 as an information output unit, an in-vehicle radar device 28 as various sensors, etc., are connected.

[0045] The input device 26 is an information input unit that receives operations by the driver or passengers of the vehicle and inputs various information to the control unit 14. As the input device 26, for example, in addition to a touch panel type display device (visual display device; so-called touch panel, etc.), it includes various switches for instructing the execution of various driving support controls and a mode change switch for changing the driving mode, etc.

[0046] The alarm device 27 is an information output device that receives an instruction signal from the control unit 14 and performs warning display or alarm display of various information to the driver or passengers of the vehicle. As the alarm device 27, for example, it includes an audio output device (auditory display device) such as a speaker, etc., a visual information display device (display monitor, etc.), or various forms of indicators, etc.

[0047] Although not shown in the figure, the in-vehicle radar device 28 is, for example, composed of a plurality of sensors (for example, a plurality of millimeter wave radars, etc.). Here, the plurality of millimeter wave radars detect three-dimensional obstacles, etc. existing around the host vehicle by receiving and analyzing the reflected waves from the object with respect to the radiated waves output.

[0048] As the three-dimensional objects that can be detected by the plurality of millimeter-wave radars, for example, in addition to pedestrians, accompanying vehicles, following vehicles, etc., there are also structures provided at the roadside (such as curbs, guardrails, walls of buildings, etc., three-dimensional objects such as vegetation).

[0049] Here, the plurality of millimeter-wave radars detect, as specific information regarding the three-dimensional object, the lateral width dimension of the three-dimensional object, the position of the representative point of the three-dimensional object, the relative position and relative distance from the host vehicle, etc., and the relative speed, etc. In this case, the in-vehicle radar device 28 functions as a distance measuring device.

[0050] The plurality of sensors (such as a plurality of millimeter-wave radars) included in the in-vehicle radar device 28 are disposed, for example, on the left and right sides of the front bumper (referred to as front-side left and right side sensors) and on the left and right sides of the rear bumper (referred to as rear-side left and right side sensors). The front-side left and right side sensors detect three-dimensional objects existing in the regions of the left and right obliquely front and side of the host vehicle, which are difficult to recognize in the image of the stereo camera 11, as surrounding environment information. Further, the rear-side left and right side sensors detect three-dimensional objects existing in the regions of the left and right obliquely side and rear of the host vehicle, which are difficult to recognize by the front-side left and right side sensors, as surrounding environment information.

[0051] Thus, in the present embodiment, the in-vehicle radar device 28 has a function as a surrounding environment recognition device for recognizing the surrounding environment of the vehicle. And the information acquired by the in-vehicle radar device 28 is sent to the image recognition unit 13 through the control unit 14.

[0052] The coordinates of each object outside the vehicle included in the surrounding environment information recognized by the image recognition unit 13 and the surrounding environment information recognized by the in-vehicle radar device 28 are all converted in the control unit 14 into coordinates in a three-dimensional coordinate system with the center of the host vehicle as the origin.

[0053] For example, the information acquired by the in-vehicle radar device 28 is sent to the control unit 14, and in the control unit 14, the running control of the vehicle is executed. In this case, the running control performs, for example, engine output control by the E / G_ECU 22 and torque distribution control of each drive wheel.

[0054] In addition, the T / M_ECU23 controls the transmission to control the direction of travel (forward or reverse), and the BK_ECU24 performs individual braking control (brake control) for each wheel, thereby performing the necessary vehicle driving control as needed.

[0055] In addition to those mentioned above, various sensors can be used to acquire information about the surrounding environment, such as sonar devices that use ultrasound to measure the distance to an object, and LiDAR (Light Detection and Ranging) devices that use laser light to measure the distance and shape of an object.

[0056] Furthermore, the control unit 14 includes a luminance distribution detection unit 14a, a luminance comparison unit 14b, and a temporary storage unit 14c.

[0057] The luminance distribution detection unit 14a is a component unit or circuit unit that performs processing to detect the luminance distribution within an image based on image data acquired by the stereo camera 11.

[0058] The luminance distribution detection unit 14a detects the luminance caused by reflected light from the vehicle's headlights, particularly at night, which is reflected by objects on the road surface or in the surrounding area, and acquires luminance distribution data within the image.

[0059] The luminance detection process performed by the luminance distribution detection unit 14a is carried out continuously at predetermined time intervals while the vehicle is in motion. The multiple luminance distribution data acquired over time are temporarily stored in the temporary storage unit 14c.

[0060] The multiple luminance distribution data temporarily stored in the temporary storage unit 14c are multiple pieces of information acquired over time. In this case, the information to be recorded only needs to include at least the most recent information for a predetermined period of time.

[0061] The luminance comparison unit 14b is a component unit or circuit unit that performs processing to detect biases in luminance within a predetermined area of ​​an image and changes in luminance over time within that predetermined area, based on a plurality of luminance distribution data detected at predetermined time intervals by the luminance distribution detection unit 14a and temporarily stored in the temporary storage unit 14c.

[0062] Furthermore, all or part of the image processing unit (IPU) 12, image recognition unit 13, control unit 14, E / G_ECU 22, T / M_ECU 23, BK_ECU 24, PS_ECU 25, etc., are composed of a processor including hardware.

[0063] Here, the processor is comprised of a well-known configuration including, for example, a central processing unit (CPU), RAM (Random Access Memory), ROM (Read Only Memory), non-volatile memory, non-volatile storage, and a non-transitory computer-readable medium, as well as peripheral devices.

[0064] ROM, non-volatile memory, and non-volatile storage devices pre-store software programs executed by the CPU, as well as fixed data such as data tables. The CPU then reads the software programs stored in the ROM, loads them into RAM, and executes them. The software programs then refer to various data as appropriate, thereby realizing the functions of each of the above-mentioned components and units (12, 13, 14, 22, 23, 24, 25).

[0065] Furthermore, the processor may be composed of semiconductor chips such as FPGAs (Field Programmable Gate Arrays). Also, each of the above components and components (12, 13, 14, 22, 23, 24, 25), etc., may be composed of electronic circuits.

[0066] Furthermore, the software program may be in a form in which all or part of it is recorded as a computer program product on a portable disc medium such as a flexible disk, CD-ROM, or DVD-ROM, or on a non-transitory computer-readable medium such as a card-type memory, HDD (Hard Disk Drive) device, or SSD (Solid State Drive) device. Such a program can be read by a computer and all or part of its operation can be executed. Alternatively, all or part of the program can be distributed or provided via a communication network. Users can easily implement the driving assistance device of the present invention by downloading and installing the program on their computer via a communication network, or by installing it on their computer from a recording medium.

[0067] The operation of the driver assistance device of this embodiment, configured as described above, will be explained below with reference to Figures 2 to 8. Figure 2 is a schematic diagram showing an example of displaying image data acquired by the image acquisition device while the vehicle equipped with the driver assistance device of this embodiment is in motion. Figure 3 is a conceptual diagram that simplifies the situation in Figure 2 and shows an overview view.

[0068] First, it is assumed that the vehicle M1 equipped with the driving assistance device 1 of this embodiment is traveling normally on a road 101M. In this case, "normal travel" refers to the travel of a vehicle at a general speed range while complying with traffic laws and regulations.

[0069] Here, road 101M refers to the road extending in the longitudinal direction from intersection 101 located in the area in front of the vehicle M1. Here, the longitudinal direction refers to the direction along the direction of travel of the vehicle M1 (see arrow A in Figures 2 and 3).

[0070] Furthermore, roads extending to the left and right from intersection 101 are referred to as intersecting roads. These intersecting roads are indicated by right-hand intersecting road 101R, which extends to the right, and left-hand intersecting road 101L, which extends to the left.

[0071] Along the road 101M, a roadway outer line 102 and a sidewalk 103 are provided on both sides, extending from the center of the road towards the outer edge. The sidewalk 103 is formed at a higher level than the road surface of the road 101M. As a result, a curb 103a is formed on the roadside edge of the sidewalk 103.

[0072] In this case, the area to the right of road 101M consists of flat land such as fields, and a clear view is available. Here, the area to the right of road 101M has good visibility.

[0073] On the other hand, the left side of road 101M is continuously covered by obstacles 104, such as high embankments (which may also be buildings). As a result, the left side of road 101M is obstructed and visibility is poor.

[0074] It is assumed that, for example, there is a four-way intersection 101 in the area in front of the vehicle M1 that does not have traffic lights or stop signs.

[0075] Road 101M is connected to intersection 101. The left and right intersecting roads 101L and 101R that extend to the left and right of intersection 101 are, for example, roads that are approximately perpendicular to road 101M.

[0076] In this case, other vehicles M2 and M3 are traveling on the left and right intersecting roads 101L and 101R, as shown in Figures 2 and 3. Vehicle M2 is traveling on the right intersecting road 101R in the direction of approaching intersection 101 (see the direction along arrow R in Figure 3). Vehicle M3 is traveling on the left intersecting road 101L in the direction of approaching intersection 101 (see the direction along arrow L in Figure 3).

[0077] In this situation, the driver of vehicle M1 has a clear view of vehicle M2. However, due to a phenomenon that makes it difficult for the driver of vehicle M1 to see vehicle M2 (the so-called collision course phenomenon), there is a possibility that vehicle M1 and vehicle M2 may collide.

[0078] On the other hand, the driver of vehicle M1 cannot see the other vehicle M3 due to the obstruction 104. In this case, it is extremely difficult for the driver of vehicle M1 to recognize the other vehicle M3 in advance.

[0079] The situations illustrated in Figures 2 and 3 change to those shown in Figures 4 and 5, for example, at night. Figure 4 shows an example of how the situation (image data) in Figure 2 is displayed at night.

[0080] Generally, when a vehicle is driven at night, its headlights are turned on. The light from the headlights illuminates a predetermined area in front of the vehicle. In a typical vehicle, there is one headlight at each end of the vehicle in the width direction. The light from these two headlights illuminates the left and right areas of the vehicle in the width direction, with an overlapping area approximately in the center.

[0081] Therefore, the brightness distribution of an image will be high in areas where the illumination from two headlights overlaps, or in areas where the light is reflected from the road surface or objects at close range. Conversely, in areas in front of the vehicle where the headlight illumination does not reach (for example, distant areas or high-altitude areas such as the sky), the brightness will be low.

[0082] For example, in the situation shown in Figure 4, a high-luminosity area [B1] exists within a predetermined range on the road surface illuminated by the headlights in the area in front of the vehicle M1. In this case, the example in Figure 4 assumes that the vehicle M1 is traveling just before the intersection 101. Therefore, the headlights of the vehicle M1 have not yet reached the interior of the intersection 101.

[0083] Furthermore, for example, in the situation shown in Figure 4, there is an obstruction 104 in the left-hand region of the road 101M. In this case, when the light from the headlights illuminates the surface of the obstruction 104, this region becomes a high-luminosity region. In Figure 4, the high-luminosity region indicated by the symbol [B2] is represented by hatching. In this case, the width of the hatching is varied according to the level of luminosity within the high-luminosity region [B2]. For example, a narrower hatching width represents higher luminosity.

[0084] Furthermore, the outer edge line 102 of the roadway is designed to be clearly visible even in low light conditions. Therefore, the area around the outer edge line 102 of the roadway has high brightness.

[0085] Furthermore, if other vehicles M2 and M3 are traveling on the intersecting roads 101L and 101R, a linear high-luminance region is formed along the intersecting roads 101L and 101R, for example (see reference numeral [B4] in Figure 4).

[0086] In this case, even if the left-hand intersecting road 101L is obscured by the obstruction 104, the headlights of another vehicle M3 (not shown in Figure 4) traveling along the left-hand intersecting road 101L toward intersection 101 illuminate the road surface above intersection 101 even before the other vehicle M3 reaches intersection 101. At this time, within intersection 101, the illumination light from the headlights of the other vehicle M2 illuminates, for example, a part of the curb (see reference numeral [B3] in Figure 4). As a result, a part [B3] within intersection 101 becomes highly luminous.

[0087] Thus, the driver assistance system 1 of the vehicle M1 can detect the brightness distribution from the image of the area in front of the vehicle when driving at night, and estimate the surrounding environment based on that brightness distribution data. Here, the surrounding environment can recognize, for example, the presence of other vehicles M2, M3, etc., that are traveling on the left and right intersecting roads 101L and 101R towards intersection 101.

[0088] Specifically, for example, as shown in Figure 4, when a linear high-luminance area is detected along the right-hand intersecting road 101R, the presence of another vehicle M2 traveling along the right-hand intersecting road 101R towards intersection 101 can be recognized.

[0089] Similarly, as shown in Figure 4, even if an obstruction 104 is present on the left side, if, for example, a linear high-luminance area along the left-hand intersecting road 101L is detected within the intersection 101, the presence of another vehicle M3 traveling along the left-hand intersecting road 101L towards the intersection 101 can be recognized.

[0090] Furthermore, if an object within the intersection 101 (for example, part of a curb; see reference numeral [B3] in Figure 4) is detected as a high-luminance area, the presence of another vehicle M2 traveling towards the intersection 101 on the right-hand intersecting road 101R can be recognized.

[0091] Figure 5 shows the situation in Figure 3 at night. In the example in Figure 5, vehicle M1 is traveling just before intersection 101, and the light from vehicle M1's headlights reaches inside intersection 101.

[0092] Figure 6 is a conceptual graph showing the brightness distribution of the road surface within the intersection under the conditions shown in Figure 5. In Figure 6, the X-axis represents the width direction of the vehicle M1. The Y-axis in Figure 6 represents the brightness level, with brightness increasing towards the top. In Figure 6, the symbol L represents the left-hand region within the intersection (within a predetermined range in the image). The symbol R in Figure 6 represents the right-hand region within the intersection (within a predetermined range in the image).

[0093] Figure 5 shows the situation where vehicle M1 is traveling on road 101M and approaching intersection 101. At this time, the illumination light from vehicle M1's headlight HL1 is illuminating the road surface in the area ahead of road 101M (including within intersection 101).

[0094] In Figure 5, the symbol V indicates the field of view of the stereo camera 11 included in the camera unit 10. Of this, the region indicated by the symbol Vb in Figure 5 shows that it is obscured by the obstruction 104. Therefore, the field of view of the vehicle M1 is the total field of view V minus the obstruction region Vb.

[0095] In this situation, suppose, for example, that another vehicle M2 is traveling on the right-hand intersecting road 101R towards intersection 101. When the light from the other vehicle M2's headlight HL2 reaches intersection 101, the light from headlight HL2 is added to the light from the vehicle M1's headlight HL1. As a result, in the region where the light from both (HL1 and HL2) overlaps, the brightness becomes even higher (see, for example, the symbol BRh in Figure 5).

[0096] In this case, when viewing the road surface of intersection 101 from the vehicle M1 (within a predetermined range in the image), the brightness differs between the left region (code L in Figures 5 and 6) and the right region (code R in Figures 5 and 6). When such a brightness distribution is detected, it can be estimated that the region is an intersection.

[0097] Here, for example, if the right-hand region R has higher brightness, it can be determined that another vehicle M2 is approaching intersection 101 on the right-hand intersecting road 101R.

[0098] The brightness of the high-brightness region BRh changes over time as both the own vehicle M1 and the other vehicle M2 approach the intersection 101.

[0099] Furthermore, this effect can be similarly applied to other vehicles M3 approaching intersection 101 from the left-hand intersecting road 101L, even in an intersection where, for example, there is an obstruction 104 on the left side. Figure 7 shows a situation that is substantially the same as the situation in Figure 5, but when another vehicle is traveling from the left.

[0100] The situation in Figure 7 can be similarly explained by swapping the left and right sides of the situation in Figure 5.

[0101] In the situation shown in Figure 7, the other vehicle M3 traveling on the left-hand intersecting road 101L is in the shielded area Vb and therefore cannot be directly seen from the vehicle M1.

[0102] However, even in this case, as shown in Figure 7, the illumination light from the headlight HL3 of the other vehicle M3 illuminates the area within the intersection 101 and also extends to the area in front of the other vehicle M3. At this time, the illumination light from the area in front of the headlight HL3 of the other vehicle M3 can be detected from the vehicle M1.

[0103] Therefore, even if the other vehicle M3 cannot be directly seen from the vehicle M1 due to the obstruction 104, the other vehicle M3 can be recognized at night (when the headlights are on) by detecting the illumination light from the headlights.

[0104] In this way, the driving assistance device 1 of this embodiment detects the luminance distribution and the change in luminance over time based on image data continuously acquired during nighttime driving. This makes it possible to recognize the presence of other vehicles M2 and M3 approaching the intersection 101 in the area in front of the vehicle M1, on the left and right intersecting roads 101L and 101R of the intersection 101.

[0105] The operation of the driving support device 1 of this embodiment will be briefly explained below using the flowchart in Figure 8.

[0106] First, the vehicle M1 equipped with the driver assistance device 1 of this embodiment is assumed to be in a normal driving state. At this time, the driver assistance device 1 is assumed to be in an activated state and sequentially performing imaging operations by the camera unit 10 and predetermined image processing (including brightness distribution detection processing) on ​​the image data acquired by said imaging operations. The various data acquired in this way are temporarily stored in, for example, a temporary storage unit 14c for a predetermined amount of time and are sequentially updated.

[0107] When in this state, first, in step S11, the control unit 14 of the driver assistance device 1 checks whether the headlight HL1 of the vehicle M1 is in the ON state. If the headlight HL1 is in the ON state, the process proceeds to the next step S12. If the headlight HL1 is in the OFF state, the same check process is repeated. Note that if the headlight HL1 remains in the OFF state for a predetermined time and no change to the ON state is confirmed, the process sequence may be exited.

[0108] In step S12, the control unit 14 assumes a predetermined range frame (see reference numeral 100 in Figures 2 and 4) within the range of the image, and checks whether or not there is a difference in brightness between the left and right regions within the frame 100.

[0109] The predetermined range frame 100 envisioned here refers, for example, to a predetermined range within the entire image displayed by image data captured from the area in front of the vehicle M1. Specifically, for example, it is assumed to be a frame within a predetermined range that includes approximately the central area of ​​the displayed image from the said image data.

[0110] Note that the reference numeral 100a shown in Figures 3, 5, and 7 is a conceptual frame that represents the region corresponding to the reference numeral 100 in Figures 2 and 4. Furthermore, the left and right regions within the frame 100 refer to, for example, the left and right regions when the frame 100 is divided into two halves.

[0111] Then, if it is confirmed in step S12 that there is a difference in brightness between the left and right regions within the frame 100, the process proceeds to the next step, S13. If there is no difference in brightness between the left and right regions, the same confirmation process is repeated.

[0112] Next, in step S13, the control unit 14 checks whether the brightness of the left-side region L (see Figures 5 and 7) within the frame 100 is changing in a direction that makes it brighter over time. If it is confirmed that the brightness of the left-side region L is changing in a direction that makes it brighter over time, the process proceeds to the next step S14. If the brightness of the left-side region L is not changing over time, or is not changing in a direction that makes it brighter, the process proceeds to step S15.

[0113] In step S14, the control unit 14 determines that another vehicle is approaching the intersection from the left on the intersecting road in the area ahead. Then, the process proceeds to step S19.

[0114] Furthermore, in step S15, the control unit 14 checks whether the brightness of the right-side region R (see Figures 5 and 7) within the frame 100 is changing in a direction that makes it brighter over time. If it is confirmed that the brightness of the right-side region R is changing in a direction that makes it brighter over time, the process proceeds to the next step S16. If the brightness of the right-side region R is not changing over time, or is not changing in a direction that makes it brighter, the process proceeds to step S17.

[0115] In step S16, the control unit 14 determines that another vehicle is approaching the intersection from the right on the intersecting road in the area ahead. The process then proceeds to step S19.

[0116] Furthermore, in step S17, the control unit 14 checks whether the brightness of the left and right side regions L and R (see Figures 5 and 7) within the frame 100 is changing in a direction that makes them brighter over time. If it is confirmed that the brightness of the left and right side regions L and R is changing in a direction that makes them brighter over time, the process proceeds to the next step S18. If the brightness of the left and right side regions L and R is not changing over time, or is not changing in a direction that makes them brighter, the process returns to step S11 (return).

[0117] In step S18, the control unit 14 determines that other vehicles are approaching the intersection from both the left and right sides on the intersecting roads in the area ahead. The process then proceeds to step S19.

[0118] In step S19, the control unit 14 drives the alarm device 27 to display a predetermined warning or alarm. The warning or alarm displayed here may indicate, for example, that another vehicle is approaching the intersection from the left (in step S14), from the right (in step S16), or from both sides (in step S18). The process then proceeds to step S20.

[0119] In step S20, the control unit 14 performs predetermined driving control (for example, deceleration control or steering control) to avoid a collision with another vehicle at the intersection. After that, it returns to the process of step S11.

[0120] As described above, according to the above embodiment, while the vehicle is driving at night, the image acquisition device continuously acquires image data of the area in front of the vehicle at predetermined time intervals. Based on the multiple image data thus acquired, the brightness distribution is detected for each image and brightness distribution data within the image is acquired.

[0121] Here, for example, at an intersection, the area where the reflected light from the headlights of a vehicle traveling straight and the reflected light from the headlights of other vehicles approaching the intersection from the left or right intersecting roads overlaps results in high brightness.

[0122] Therefore, for example, if a difference in brightness between the left and right sides within a predetermined range in the area in front of the vehicle is detected based on brightness distribution data, it can be estimated that another vehicle is approaching from the side of the high-brightness area.

[0123] Furthermore, when a change in brightness is detected in the high-brightness region within the left and right regions, if the brightness of that high-brightness region changes in a direction that becomes brighter over time, the approach of another vehicle can be estimated more reliably.

[0124] In this way, by detecting brightness distribution data from image data acquired by the image acquisition device, it is possible to easily estimate from which direction (left or right) another vehicle is approaching from the left or right intersection.

[0125] Therefore, if the approach of other vehicles can be estimated, collision avoidance controls such as deceleration control and steering control of the vehicle can be performed in advance. Thus, head-on collisions at intersections and other locations can be prevented.

[0126] Furthermore, other vehicles approaching the intersection from intersecting roads may be difficult to notice, even at night. In such cases, if another vehicle is detected, the warning device 27 will display a warning or alarm in advance. This allows the driver assistance device 1 to recognize other vehicles that the driver is unaware of, and when the other vehicle is detected, it will display a warning or alarm to notify the driver. Thus, the driver's safety can be ensured.

[0127] In this embodiment, the brightness distribution of images from the headlights of the own vehicle and other vehicles is utilized. Therefore, even if visibility is poor near an intersection in the area in front of the own vehicle due to the presence of obstructions, other vehicles can be recognized more quickly and reliably. Thus, collision accidents at intersections can be prevented.

[0128] The present invention is not limited to the embodiments described above, and various modifications and applications can be implemented without departing from the spirit of the invention. Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple components disclosed. For example, if the problem that the invention aims to solve can be solved and the effects of the invention can be obtained even if some components are deleted from all the components shown in one embodiment, then the configuration with these components deleted can be extracted as an invention. Furthermore, components from different embodiments may be combined as appropriate. This invention is not limited by any particular embodiment other than being limited by the appended claims.

[0129] 1...Driving assistance device 10...Camera unit 11...Stereo camera 11a...Main camera 11b...Sub camera 13...Image recognition unit 14...Control unit 14a...Brightness distribution detection unit 14b...Brightness comparison unit 14c...Temporary storage unit 22...Engine control unit (E / G_ECU) 23...Transmission control unit (T / M_ECU) 24...Brake control unit (BK_ECU) 25...Power steering control unit (PS_ECU) 26...Input device 27...Warning device 28...On-board radar device 32...Throttle actuator 33...Hydraulic control circuit 34...Brake actuator 35...Electric power steering motor 100 (100a)...Range frame 101...Intersection 101L, 101R...Left and right intersecting roads 101M...Road M1...Own vehicle M2, M3...Other vehicles

Claims

1. A vehicle driving assistance device comprising: an image acquisition device that acquires image data with the area in front of the vehicle as the target of imaging; a brightness distribution detection unit that detects a brightness distribution based on the image data; and a control unit that performs driving control of the vehicle, wherein the control unit recognizes an intersection in the area in front based on the detected brightness distribution data, and if it recognizes another vehicle approaching the intersection on an intersecting road extending to the left or right from the intersection, it performs deceleration control.

2. The vehicle driving assistance device according to claim 1, characterized in that the control unit recognizes the other vehicle when it detects a difference in brightness between the left and right sides within a predetermined area in the image.

3. The vehicle driving assistance device according to claim 1, further comprising: a luminance distribution detection unit that continuously acquires luminance distribution data at predetermined time intervals while the vehicle is in motion; a temporary storage unit that sequentially stores a plurality of luminance distribution data acquired sequentially at predetermined time intervals; and a luminance comparison unit that compares the plurality of luminance distribution data temporarily stored in the temporary storage unit to detect changes in luminance over time.

4. The vehicle driving assistance device according to claim 3, characterized in that when the luminance comparison unit detects a change in luminance over time in a high-luminance region within the predetermined region, the control unit recognizes the presence of another vehicle.

5. The vehicle driving assistance device according to claim 3, characterized in that, if the luminance comparison unit detects a change in luminance over time in the direction in which the reflected light from the road surface in the intersection or the reflected light from objects around the intersection, due to the illumination light from the headlights of the own vehicle and the other vehicle, the control unit recognizes the presence of the other vehicle.

6. The vehicle driving assistance device according to claim 3, characterized in that when the luminance comparison unit detects a linear high-luminance region in the luminance distribution that extends in the left-right direction of the forward region and moves and changes over time, the control unit recognizes the presence of another vehicle.

7. The vehicle driving assistance device according to claim 1, characterized in that the control unit further performs steering control when executing the deceleration control to avoid collision with other vehicles.

8. The vehicle driving assistance device according to claim 1, further comprising a warning device that provides visual or auditory warning displays, wherein the control unit, upon detecting the presence of another vehicle approaching the intersection, provides the warning display before executing the deceleration control.

9. A vehicle driving assistance device comprising: an image acquisition device that acquires image data with the area in front of the vehicle as the imaging target; and a processor, wherein the processor detects a brightness distribution based on the image data, recognizes an intersection in the area in front based on the detected brightness distribution data, and, when it recognizes another vehicle approaching the intersection on an intersecting road extending to the left or right from the intersection, it performs deceleration control.

Images