Headlight control device and headlight control method

Abstract

The present invention includes: a visual recognition object detection unit (11) that detects a visual recognition object on the basis of first outside-of-vehicle object detection information; a candidate object detection unit (12) that detects a candidate object on the basis of second outside-of-vehicle object detection information; a line-of-sight detection unit (13) that detects a direction of line of sight of a driver on the basis of in-vehicle captured images; target determination units (14, 14a) that perform first target determination processing for determining whether or not an object corresponding to the detected visual recognition object has been detected as a candidate object in a case where the visual recognition object has been detected; overlooking estimation units (15, 15a) that estimate, on the basis of the result of the first target determination processing performed by the target determination units (14, 14a) and the direction of the line of sight of the driver, whether or not there has been an overlooking of the visual recognition object by the driver; and a light distribution control unit (17) that controls headlights (5) so as to perform emphasis illumination for causing the driver to notice the visual recognition object in a case in which estimation is made that there has been an overlooking of the visual recognition object by the driver.

Description

Headlamp control device and headlamp control method

[0001] The present disclosure relates to a headlamp control device and a headlamp control method.

[0002] For example, while driving a vehicle at night, a driver is likely to overlook an object that should be visually recognized outside the vehicle (hereinafter referred to as a "visual object"). Therefore, a conventional technology is known that controls headlights, in other words, headlights, to illuminate the periphery of the visual object to prevent the driver from overlooking the object. Meanwhile, for example, Patent Document 1 discloses a technology that determines an area where the driver should gaze based on a surrounding situation detected by a periphery monitoring sensor, determines whether the driver has gazed at the area where the driver should gaze based on the driver's visual area detected by the driver monitoring sensor, and displays a gaze indication on a display device such as an in-vehicle display if it is determined that the driver has not gazed at the area where the driver should gaze.

[0003] Japanese Patent Application Laid-Open No. 2015-125686

[0004] In the prior art, there was a problem that lighting controlled to prevent the driver from overlooking the visual object may be irritating to the driver. Note that the technology disclosed in Patent Document 1 is a technology that attempts to guide the driver's gaze to the periphery of the visual object by displaying a gaze indication on a display device, but is not a technology that attempts to make the visual object easier for the driver by having the headlights irradiate the periphery of the visual object.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a headlight control device that can control the illumination of headlights so that the driver does not overlook visual objects, while taking into consideration the inconvenience caused to the driver.

[0006] The headlamp control device according to the present disclosure includes a visual object detection unit that detects a visual object that the driver should view in a visual target area that the driver should view, based on first external object detection information in which a position type detection device capable of detecting the position and type of an object that exists in the vicinity of the vehicle has detected an object that exists in the vicinity of the vehicle; a candidate object detection unit that detects a candidate object that is a visual object in the visual target area, based on second external object detection information in which a position detection device capable of detecting the position of an object that exists in the vicinity of the vehicle has detected an object that exists in the vicinity of the vehicle; and a gaze detection unit that detects the direction of the driver's gaze, based on an in-vehicle image captured by an in-vehicle camera of the driver. The system includes a detection unit, a target determination unit that performs a first target determination process that, when a visual target is detected by the visual target detection unit, determines whether or not an object corresponding to the detected visual target has been detected as a candidate object by the candidate object detection unit, an oversight estimation unit that estimates whether or not the driver has overlooked the visual target based on a result of the first target determination process performed by the target determination unit and the direction of the driver's line of sight detected by the line of sight detection unit, and a light distribution control unit that, when the oversight estimation unit estimates that the driver has overlooked the visual target, controls the headlights to perform accentuated illumination to alert the driver to the visual target.

[0007] According to the present disclosure, headlight illumination control can be performed so that the driver does not overlook an object to be viewed, while taking into consideration the inconvenience to the driver.

[0008] 10 is a diagram illustrating an example of a configuration of a headlight control device according to Embodiment 1. FIG. 2A and FIG. 2B are diagrams for explaining an overview of an example of headlight control by the headlight control device according to Embodiment 1 when it is estimated that the driver has overlooked a pedestrian. FIG. 2B is a flowchart for explaining the operation of the headlight control device according to Embodiment 1. FIG. 3 is a flowchart for explaining in detail an example of oversight estimation processing by the oversight estimation unit in step ST5 of FIG. 3. FIG. 6A and FIG. 6B are diagrams for explaining in detail an example of illumination mode determination processing by the illumination mode determination unit in step ST6 of FIG. 10. FIG. 6A and FIG. 6B are diagrams for explaining an example of a hardware configuration of the headlight control device according to Embodiment 1. FIG. 6B is a diagram for explaining an example of a configuration of a headlight control device according to Embodiment 2. FIG. 6B is a diagram for explaining an overview of an example of headlight control by the headlight control device according to Embodiment 2. FIG. 6C is another diagram for explaining an overview of an example of headlight control by the headlight control device according to Embodiment 2. FIG. 6D is a flowchart for explaining the operation of the headlight control device according to Embodiment 2. FIG. 6E is a flowchart for explaining in detail an example of oversight estimation processing by the oversight estimation unit in step ST5a of FIG. 11 is a flowchart for explaining in detail an example of an irradiation mode determination process by an irradiation mode determination unit in step ST6a of FIG. 10 .

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Embodiment 1. Fig. 1 is a diagram showing an example of the configuration of a headlamp control device 1 according to embodiment 1. In embodiment 1, it is assumed that the headlamp control device 1 is mounted on a vehicle (not shown).

[0010] When the headlight control device 1 estimates that the driver has overlooked an object that should be visually recognized outside the vehicle (hereinafter referred to as a "visual target area") and that is present in the visually recognized area outside the vehicle, the headlight control device 1 controls the headlights 5 to perform illumination (hereinafter referred to as "emphasis illumination") to alert the driver to the visual target. In the first embodiment, the control of the headlights 5 performed by the headlight control device 1 is specifically assumed to be control of the illumination range, illumination intensity, or illumination method of light emitted by the headlights 5. Furthermore, in the first embodiment, it is assumed that the control of the headlights 5 by the headlight control device 1 is performed in a state in which the headlights 5 are turned on, such as with automatic lights, in a dark location such as an urban area at night. In the following first embodiment, as an example, the visual target area is assumed to be a predetermined area in front of the vehicle. The visual target area is predetermined by a developer or the like as an area where a visual target may exist, and is specified by a depth and width based on the position of the vehicle. Developers and the like set the visual target area so that, for example, the width increases as the area becomes farther from the vehicle. Note that the visual target area is not fixed, but may be changed according to the vehicle speed or road information (such as the curvature of a curve in a road or the shape of an intersection) acquired using vehicle position information and map information from a navigation system or the like. In the following first embodiment, the visual target is assumed to be a pedestrian as an example. Note that this is merely an example, and the visual target may also be, for example, an animal. The type of object to be the visual target is determined in advance by developers and the like.

[0011] Specifically, when the headlight control device 1 detects a pedestrian based on an image of the vehicle's surroundings captured by the exterior camera 2 (hereinafter referred to as the "exterior captured image"), the headlight control device 1 determines whether an object corresponding to the pedestrian has been detected to date based on information on objects detected by the exterior radar 3 around the vehicle (hereinafter referred to as the "radar detection information").The headlight control device 1 then estimates whether the driver has overlooked the pedestrian based on the result of the determination and the driver's gaze direction detected from an image of the driver inside the vehicle captured by the interior camera 2 (hereinafter referred to as the "interior captured image").If it determines that the driver has overlooked the pedestrian, the headlight control device 1 controls the headlights 5 to perform accentuated illumination.In the first embodiment, the "driver's gaze direction" is represented by the driver's facial orientation or the driver's gaze direction.The exterior camera 2, exterior radar 3, and interior camera 4 will be described later.

[0012] More specifically, when the headlight control device 1 detects a pedestrian based on an exterior-of-vehicle captured image, if it determines that an object corresponding to the pedestrian has not been detected in radar detection information to date or that the driver has not visually observed the pedestrian, it presumes that the driver has overlooked the pedestrian and controls the headlights 5 to perform accentuated illumination. If the headlight control device 1 determines that an object corresponding to the pedestrian has been detected in radar detection information to date and that the driver has already visually observed the pedestrian based on the radar detection information and the driver's line of sight, it presumes that the driver has not overlooked the pedestrian. In this case, the headlight control device 1 does not cause the headlights 5 to perform accentuated illumination. In addition, when the headlight control device 1 detects a pedestrian based on an image taken outside the vehicle, even if an object corresponding to the pedestrian has not been detected in the radar detection information to date, if it can determine that the driver has already seen the pedestrian, for example, if the driver was looking at the pedestrian at the time the pedestrian's position was detected based on the image taken outside the vehicle, it will assume that the driver has not overlooked the pedestrian and will not cause the headlights 5 to perform accentuated illumination.

[0013] Here, an overview of the control by the headlight control device 1 according to embodiment 1 to cause the headlights 5 to perform accentuated illumination when it is estimated that the driver has overlooked a pedestrian, and the significance of using the exterior camera 2 and the exterior radar 3 to perform this control, will be described with reference to the drawings. Figures 2A and 2B are diagrams for explaining an overview of an example of control of the headlights 5 by the headlight control device 1 according to embodiment 1 when it is estimated that the driver has overlooked a pedestrian. In Figures 2A and 2B, the vehicle is indicated by "C" and the pedestrian is indicated by "P." For convenience, the exterior camera 2, the exterior radar 3, the interior camera 4, and the headlights 5 are not shown in Figures 2A and 2B.

[0014] 2A and 2B respectively show an example of the situation ahead of the vehicle and the driver while the vehicle is traveling at night. Assume that after the situation shown in FIG. 2A , the vehicle travels further ahead and reaches the situation shown in FIG. 2B . FIG. 2A illustrates a situation in which an object presumed to be a pedestrian is detected by exterior radar 3 and the driver visually observes the object presumed to be a pedestrian ahead of the vehicle before the pedestrian is captured by exterior camera 2. The vehicle then travels forward and approaches the pedestrian to a position where exterior camera 2 can or easily captures the pedestrian, as shown in FIG. 2B . The headlight control device 1 according to the first embodiment determines that the driver has not visually observed the pedestrian at the time the situation shown in FIG. 2B occurs, and therefore does not cause the headlights 5 to perform accentuated illumination.

[0015] In this way, if the headlight control device 1 determines that a distant pedestrian who cannot be detected by the exterior camera 2, more specifically, who cannot be detected from the exterior image captured by the exterior camera 2, has been detected by the exterior radar 3 at that time, more specifically, that an object that can be considered to be the pedestrian has been detected from the radar detection information, and that the driver has visually observed the object, it presumes that the driver has not overlooked the pedestrian and does not control the headlights 5 to provide accentuated illumination. In this way, the headlight control device 1 can, for example, provide accentuated illumination to pedestrians that the driver has already visually recognized, thereby preventing the accentuated illumination from becoming an annoyance to the driver.

[0016] For example, when a pedestrian is present in front of a vehicle while the vehicle is traveling at night, the farther the pedestrian is in front of the vehicle, the darker the area around the pedestrian will appear to the vehicle. If the headlight control device 1 attempts to detect a pedestrian at night from an exterior-captured image captured by the exterior-captured camera 2, it will have difficulty detecting a pedestrian at a distance. However, even if a pedestrian is present at a distance that is difficult for the exterior-captured camera 2 to detect, the pedestrian may be visible to the driver of the vehicle. Therefore, if the headlight control device 1 were to control the headlights 5 to perform accentuated illumination to alert the driver to a pedestrian that the driver has already seen, this would result in accentuated illumination being unnecessary for the driver or could be annoying to the driver.

[0017] In contrast to the exterior camera 2, the exterior radar 3 can detect pedestrians at a distance, even at night. However, in order to detect pedestrians with low radio wave reflectivity, the sensitivity of the exterior radar 3 must be increased. Therefore, if the headlight control device 1 attempts to detect pedestrians at night from radar detection information detected by the exterior radar 3, there is a possibility that it will overdetect objects other than pedestrians. It is generally more difficult to detect a type of object (e.g., a pedestrian) using a radar than it is to detect a type of object (e.g., a pedestrian) using a camera. Therefore, if the headlight control device 1 were to control the headlights 5 to perform accentuated illumination to alert the driver to a pedestrian every time it detects a pedestrian from radar detection information using only the exterior radar 3, there is a possibility that accentuated illumination will be performed on objects other than pedestrians that the driver does not need to see. This could result in unnecessary accentuated illumination for the driver or cause annoyance to the driver.

[0018] In this way, if the headlight control device 1 uses only the exterior camera 2 or only the exterior radar 3 to estimate whether the driver has overlooked a pedestrian and control the headlights 5, there is a risk that unnecessary highlighted illumination will be performed on the driver or that the driver will be annoyed.

[0019] Therefore, as described above, when an object presumed to be a pedestrian is detected by exterior radar 3 at a stage before the pedestrian ahead of the vehicle is imaged by exterior camera 2, and when it is determined that the driver has visually observed the object presumed to be a pedestrian, the headlight control device 1 according to embodiment 1 determines that the driver has not overlooked the pedestrian, and does not cause accentuated illumination of the headlights 5. In this way, the headlight control device 1 realizes illumination control of the headlights 5 that prevents the driver from overlooking a pedestrian, while taking into consideration the inconvenience to the driver.

[0020] As described above, there is a difference between the distance to a pedestrian in front of the vehicle that can or can easily be captured by exterior camera 2 and the distance to a pedestrian in front of the vehicle that can or can easily be captured by exterior radar 3, and exterior radar 3 can capture pedestrians that are farther away than exterior camera 2. There is a possibility that the driver of the vehicle can visually see a pedestrian in front of the vehicle before exterior camera 2 captures the pedestrian, but in this case, there is a high possibility that exterior radar 3 will capture a pedestrian that the driver can visually see.

[0021] 1 , an example configuration of the headlight control device 1 according to embodiment 1 will be described. The headlight control device 1 is connected to an exterior camera 2, an exterior radar 3, an interior camera 4, and a headlight 5. The exterior camera 2, the exterior radar 3, the interior camera 4, and the headlight 5 are provided in the vehicle.

[0022] The exterior camera 2 acquires exterior images of the area around the vehicle. The exterior camera 2 is installed so that it can capture images of at least the visual target area. Here, the visual target area is the area in front of the vehicle, so the exterior camera 2 is, for example, a front camera, and acquires exterior images of the area in front of the vehicle. The exterior camera 2 is, for example, a visible light camera. The exterior camera 2 outputs the exterior images to the headlamp control device 1.

[0023] The exterior radar 3 is, for example, a millimeter-wave radar. The exterior radar 3 detects objects present around the vehicle and acquires information about the detected objects as radar detection information. The exterior radar 3 transmits radio waves, such as millimeter waves, around the vehicle and receives the reflected waves from the objects. The exterior radar 3 can detect the presence, distance, shape, and speed of an object associated with the reflected waves based on the time from transmitting the radio waves to receiving the reflected waves. The exterior radar 3 can also detect the direction of movement of an object by tracking the trajectory of the object, for example. The radar detection information includes information about the presence or absence of an object, the distance to the object if present, the shape of the object, the speed of the object, and the direction of movement of the object. The radio waves are irradiated to multiple areas within a predetermined range around the vehicle, and the radar detection information includes information about the presence, distance, shape, speed, and direction of movement of the object for each area irradiated with the radio waves. The exterior radar 3 is installed so as to be able to detect at least objects present in the visual target area. Here, the visual target area is the area in front of the vehicle, so the exterior radar 3 is installed, for example, in front of the vehicle so as to be able to detect objects present in front of the vehicle. The exterior radar 3 outputs radar detection information to the headlamp control device 1.

[0024] The interior camera 4 is a camera or the like installed in the vehicle for the purpose of monitoring the interior of the vehicle, and is installed so as to be able to capture at least an image of the driver's face. The interior camera 4 is, for example, an infrared camera or a visible light camera. The interior camera 4 outputs the captured interior image to the headlight control device 1. In the first embodiment, the interior camera 4 may be, for example, a camera shared with an imaging device included in a so-called "driver monitoring system (DMS)" that is installed in the vehicle to monitor the state of the driver inside the vehicle.

[0025] The headlamp 5 is a lighting fixture that illuminates the area ahead of the vehicle. The headlamp 5 is, for example, a general headlamp capable of emitting a high beam, a low beam, and an auxiliary light, in other words, a headlight, and therefore a detailed configuration example will not be described here. The headlamp 5 includes a left light (not shown) mounted on the left side of the vehicle relative to the vehicle's traveling direction, and a right light (not shown) mounted on the right side of the vehicle relative to the vehicle's traveling direction. The left light, for example, is composed of a high beam unit (not shown) that illuminates distant objects and a low beam unit (not shown) that illuminates nearby objects. The right light, for example, is composed of a high beam unit (not shown) that illuminates distant objects and a low beam unit (not shown) that illuminates nearby objects. The high beam unit and the low beam unit are each composed of, for example, a plurality of light sources (not shown), such as LED light sources, arranged in an array, and each light source can be turned on individually. Note that, in the first embodiment, "arranged in an array" refers to the light sources being arranged in a line in the width direction of the vehicle. When each light source is turned on, a high beam or a low beam is irradiated onto the area in front of the vehicle. Note that, here, the headlamp 5 is configured with a plurality of light sources such as LED light sources arranged in an array, but this is merely an example, and the headlamp 5 may have a different configuration. For example, the headlamp 5 may have a configuration in which a laser excitation light source is used as a light source.

[0026] In the first embodiment, the area in front of the vehicle where the high beam unit can irradiate a high beam is referred to as the "high beam irradiable area." How far forward of the vehicle the high beam irradiable area is and how large the area is is determined in advance according to the specifications of the high beam unit, etc. In the first embodiment, the area in front of the vehicle where the low beam unit can irradiate a low beam is referred to as the "low beam irradiable area." How far forward of the vehicle the low beam irradiable area is and how large the area is is determined in advance according to the specifications of the low beam unit, etc.

[0027] The headlamp control device 1 according to the first embodiment can, for example, control the illumination or blocking of high beams or low beams by turning on or off each light source. This allows the headlamp control device 1 to control the illumination range of the light from the headlamp 5. The headlamp control device 1 can not only turn on or off each light source, but also control the amount of light when turned on or the lighting method.

[0028] As shown in FIG. 1 , the headlamp control device 1 includes a visual target detection unit 11, a candidate object detection unit 12, a line of sight detection unit 13, a target determination unit 14, an oversight estimation unit 15, an illumination mode determination unit 16, and a light distribution control unit 17.

[0029] The visual object detection unit 11 acquires an exterior image from the exterior camera 2 and detects a pedestrian in front of the vehicle based on the acquired exterior image. More specifically, the visual object detection unit 11 detects a pedestrian in front of the vehicle based on the exterior image. When a pedestrian is detected, the visual object detection unit 11 detects the pedestrian's position, movement direction, etc. In the first embodiment, the position and movement direction of the pedestrian detected by the visual object detection unit 11 are the pedestrian's position in real space and movement direction in real space. The visual object detection unit 11 may detect the pedestrian by, for example, performing image recognition processing on the exterior image using a known image recognition technology. The visual object detection unit 11 may also detect the pedestrian using, for example, a known object detection technology based on machine learning. When the visual object detection unit 11 detects a pedestrian in the exterior image, it calculates the pedestrian's position in real space (hereinafter referred to as the "pedestrian's three-dimensional position"). The three-dimensional position of a pedestrian is a point in real space, and is expressed, for example, by coordinate values ​​that can be mapped onto a space with the center of the host vehicle as the origin, the forward direction of the host vehicle as the X axis, the left-right direction as the Y axis, and the up-down direction as the Z axis. The installation position and angle of view of the exterior camera 2 are known in advance. In addition, the positional relationship between the center position of the host vehicle and the installation position of the exterior camera 2 is known in advance. The visual object detection unit 11 can calculate the three-dimensional position of the pedestrian, for example, from the position and size of the pedestrian on the captured image (hereinafter referred to as the "two-dimensional position of the pedestrian").

[0030] The visual object detection unit 11 detects the movement direction of the pedestrian from the calculated three-dimensional position of the pedestrian, for example. For example, the visual object detection unit 11 stores information indicating the calculated three-dimensional position of the pedestrian in an internal buffer in chronological order, and detects the movement direction of the pedestrian from the stored information indicating the three-dimensional position of the pedestrian. When the visual object detection unit 11 newly detects a pedestrian, it assigns a unique ID to the pedestrian. The visual object detection unit 11 then tracks the pedestrian detected across multiple frames of the exterior-of-vehicle images, and assigns the same ID to pedestrians detected across multiple frames of the exterior-of-vehicle images. The visual object detection unit 11 can associate the same pedestrian detected across multiple exterior-of-vehicle images by using a known tracking method. By including the pedestrian's ID in the information indicating the calculated three-dimensional position of the pedestrian, the visual object detection unit 11 can determine which pedestrian is moving in which direction. Furthermore, for example, the visual object detection unit 11 may store the exterior-of-vehicle captured images acquired from the exterior camera 2 in an internal buffer in chronological order, and detect the movement direction of the pedestrian from the exterior-of-vehicle captured images stored in the internal buffer. The visual object detection unit 11 may detect the movement direction of the pedestrian from, for example, a change in the position of the pedestrian in the exterior-of-vehicle captured images or a change in the size of the pedestrian in the exterior-of-vehicle captured images. Furthermore, the visual object detection unit 11 may detect the movement direction of the pedestrian using a known machine learning technique.

[0031] The visual object detection unit 11 outputs information that can identify a detected pedestrian, the two-dimensional position of the detected pedestrian, the moving direction of the detected pedestrian, the calculated three-dimensional position of the pedestrian, and information that associates the pedestrian detection time (hereinafter referred to as "visual object detection information") to the target determination unit 14. The visual object detection unit 11 also stores the visual object detection information in a storage unit (not shown) in chronological order. The visual object detection information may include an image captured outside the vehicle. The information that can identify a detected pedestrian is, for example, an ID.

[0032] The candidate object detection unit 12 acquires radar detection information from the exterior radar 3 and detects candidate objects in the visual target area based on the acquired radar detection information. In the first embodiment, a candidate object refers to an object that is likely to be a pedestrian and has the potential to be a pedestrian. More specifically, if an object is detected in the visual target area by the exterior radar 3 based on the radar detection information, the candidate object detection unit 12 detects the object as a candidate object. When a candidate object is detected, the candidate object detection unit 12 detects the position and movement direction of the candidate object based on the radar detection information. The position of the candidate object is a point in real space and is represented by coordinate values ​​that can be mapped in a space with the center of the host vehicle as the origin, the forward direction of the host vehicle as the X axis, the left-right direction as the Y axis, and the up-down direction as the Z axis. The installation position of the exterior radar 3 and the radio wave transmission range of the exterior radar 3 are known in advance. Furthermore, the positional relationship between the center of the host vehicle and the installation position of the exterior radar 3 is known in advance. The candidate object detection unit 12 can calculate the position of the candidate object based on the radar detection information.

[0033] The candidate object detection unit 12 outputs information capable of identifying a candidate object, information associating the position of the detected candidate object, the moving direction of the candidate object, and the detection time of the candidate object (hereinafter referred to as "candidate object detection information") to the target determination unit 14. The candidate object detection unit 12 also stores the candidate object detection information in chronological order in a storage unit (not shown). The candidate object detection information may include radar detection information. Note that the information capable of identifying a candidate object is, for example, an ID. When the candidate object detection unit 12 detects a new candidate object, it assigns a unique ID to the candidate object. The candidate object detection unit 12 then tracks candidate objects detected across multiple pieces of radar detection information and assigns the same ID to candidate objects detected across multiple pieces of radar detection information. The candidate object detection unit 12 can associate the same candidate object detected across multiple pieces of radar detection information by using a known tracking method.

[0034] The gaze detection unit 13 acquires an in-vehicle image captured by the in-vehicle camera 4 and detects the direction of the driver's gaze based on the acquired in-vehicle image. The gaze detection unit 13 detects the direction of the driver's gaze, in other words, the driver's facial orientation or gaze direction, using a known image recognition technology. The image recognition technology for detecting the facial orientation of a person from an image capturing the person's face and the image recognition technology for detecting the gaze direction of a person from an image capturing the person's face are known technologies, and therefore detailed description thereof will be omitted.

[0035] In the first embodiment, the direction of the driver's line of sight is expressed, for example, by an angle with respect to a reference direction. In the first embodiment, the reference direction is, for example, a straight line passing through the center of the driver's head and a point in front of the center of the head. Since the installation position and angle of view of the interior camera 4 are known in advance, the line of sight detection unit 13 can calculate the position of the center of the driver's head. The position of the driver's head center is a point in real space, and is expressed, for example, by coordinate values ​​that can be mapped in a space with the center of the vehicle as the origin, the forward direction of the vehicle as the X axis, the left-right direction as the Y axis, and the up-down direction as the Z axis. The driver's direction is expressed, for example, by horizontal and vertical angles with respect to the reference direction. In detail, the orientation of the driver is, for example, based on the state (0 degrees) when the driver faces forward with respect to the traveling direction of the vehicle, and the left-right orientation of the driver is expressed as an angle that increases as the driver faces to the right with respect to the traveling direction of the vehicle from the state facing forward, and an angle that decreases as the driver faces to the left with respect to the traveling direction of the vehicle from the state facing forward.Furthermore, the orientation of the driver's up-down direction is expressed as an angle that increases as the driver faces upward with respect to the traveling direction of the vehicle from the state facing forward, and an angle that decreases as the driver faces downward with respect to the traveling direction of the vehicle from the state facing forward.In addition, in the first embodiment, "forward" is not limited to being strictly directly forward, but also includes an angle that is approximately forward.

[0036] Furthermore, upon detecting the direction of the driver's gaze, the gaze detection unit 13 calculates an area that is estimated to be viewed by the driver (hereinafter referred to as an "estimated visual area") based on the detected direction of the driver's gaze. For example, the estimated visual area is expressed as an angle as seen from the position of the vehicle (particularly the driver). The installation position of the interior camera 4, the angle of view of the interior camera 4, and the positional relationship between the driver's head position, which serves as a reference for the driver's gaze direction, and the installation position of the interior camera 4 are known in advance. In addition, the positional relationship between the vehicle position and the installation position of the interior camera 4 is known in advance. The gaze detection unit 13 can calculate the estimated visual area of ​​the driver from the direction of the driver's gaze detected based on the interior captured image.

[0037] The gaze detection unit 13 outputs information (hereinafter referred to as "gaze information") in which the detected direction of the driver's gaze, the calculated estimated visual recognition area of ​​the driver, and the detection time of the driver's gaze direction are associated with each other to the oversight estimation unit 15. The gaze detection unit 13 also stores the gaze information in a storage unit (not shown) in chronological order. The gaze information may include an image captured inside the vehicle.

[0038] When a pedestrian is detected by the viewed object detection unit 11, the target determination unit 14 performs a process (hereinafter referred to as a "first target determination process") to determine whether or not an object corresponding to the detected pedestrian has been detected as a candidate object by the candidate object detection unit 12. For example, the viewed object detection unit 11 outputs information indicating whether or not a pedestrian has been detected (hereinafter referred to as "viewed object presence / absence information") to the target determination unit 14. When the viewed object detection unit 11 outputs viewed object presence / absence information indicating "pedestrian present," the target determination unit 14 determines that a pedestrian has been detected by the viewed object detection unit 11.

[0039] In the first target determination process, the target determination unit 14 compares the viewed target detection information output from the viewed target detection unit 11 with the candidate object detection information stored in a storage unit (not shown), and determines whether a candidate object that can be considered to be the same object as the pedestrian detected by the viewed target detection unit 11 has been previously detected by the candidate object detection unit 12, depending on whether a candidate object corresponding to the pedestrian detected by the viewed target detection unit 11 has been identified. As described above, the viewed target detection information includes information indicating the three-dimensional position and movement direction of the pedestrian, and the candidate object detection information includes information indicating the position and movement direction of the candidate object. The target determination unit 14 can determine whether a candidate object that can be considered to be the same object as the pedestrian detected by the viewed target detection unit 11 has been previously detected by the candidate object detection unit 12, by comparing the three-dimensional position and movement direction of the pedestrian included in the viewed target detection information with the position and movement direction of the candidate object included in the candidate object detection information.

[0040] For example, if a candidate object has been detected in the past at a position that can be considered the same as the position of a pedestrian and moving in the same direction, the target determination unit 14 determines that a candidate object corresponding to the pedestrian detected by the viewed object detection unit 11 has been identified; in other words, a candidate object that can be considered the same as the pedestrian detected by the viewed object detection unit 11 has been detected in the past by the candidate object detection unit 12. In other words, the target determination unit 14 determines that an object corresponding to the detected pedestrian has been detected as a candidate object by the candidate object detection unit 12. A "position that can be considered the same as the position of a pedestrian" is, for example, a position within a predetermined range (hereinafter referred to as a "candidate object detection range") centered on the position of the pedestrian. The size of the candidate object detection range is determined in advance by developers, etc. On the other hand, if a candidate object has not been detected in the past at a position that can be considered the same as the position of the pedestrian and in the same moving direction, the target determination unit 14 determines that a candidate object corresponding to the pedestrian detected by the viewed object detection unit 11 cannot be identified, in other words, that a candidate object that can be considered to be the same object as the pedestrian detected by the viewed object detection unit 11 has not been detected in the past by the candidate object detection unit 12. In other words, the target determination unit 14 determines that an object corresponding to the detected pedestrian has not been detected as a candidate object by the candidate object detection unit 12.

[0041] The target object determination unit 14 outputs information indicating the result of the first target object determination process (hereinafter referred to as "first target object determination information") to the oversight estimation unit 15. The first target object determination information includes information indicating whether a candidate object that can be considered to be the same object as the pedestrian detected by the viewed object detection unit 11 has been detected in the past by the candidate object detection unit 12, in other words, whether a candidate object that can be considered to be the same object as the pedestrian detected based on the outside-of-vehicle image has been detected in the past based on radar detection information, and the viewed object detection information output from the viewed object detection unit 11. The target object determination unit 14 also outputs viewed object presence / absence information indicating "pedestrian present" output from the viewed object detection unit 11 to the oversight estimation unit 15 together with the first target object determination information.

[0042] In addition, if a pedestrian is not detected by the visual object detection unit 11, that is, if the visual object detection unit 11 outputs visual object presence / absence information indicating “no pedestrian,” the target determination unit 14 does not perform the first target determination process, but simply outputs the visual object presence / absence information to the oversight estimation unit 15 as is.

[0043] When a pedestrian is detected by the visual object detection unit 11, in other words, when visual object presence / absence information indicating “pedestrian present” is output from the target object determination unit 14, the oversight estimation unit 15 estimates whether or not the driver has overlooked the pedestrian, based on the result of the first target object determination process performed by the target object determination unit 14 and the direction of the driver's gaze detected by the gaze detection unit 13. In the first embodiment, the oversight estimation process performed by the oversight estimation unit 15 to estimate whether or not the driver has overlooked a pedestrian is referred to as the “first oversight estimation process.”

[0044] More specifically, when a pedestrian is detected by the visual object detection unit 11, the oversight estimation unit 15 estimates whether or not the driver has overlooked the pedestrian based on the pedestrian oversight estimation conditions. The pedestrian oversight estimation conditions are conditions that define under what circumstances the driver is assumed to have overlooked the pedestrian detected based on the outside-of-vehicle image and the candidate object detected based on the radar detection information. The pedestrian oversight estimation conditions are set in advance by a developer or the like, and information indicating the pedestrian oversight estimation conditions (hereinafter referred to as "pedestrian oversight estimation condition information") is stored in an internal buffer or the like of the oversight estimation unit 15. Note that the pedestrian oversight estimation condition information may be updated as appropriate. The pedestrian oversight estimation conditions include, for example, the following conditions (Condition (1-1)) to (Condition (1-3)).

[0045] <Condition (1-1)> If the pedestrian detected from the exterior-of-vehicle image is a newly detected pedestrian, a candidate object corresponding to the pedestrian has been identified in the first target object determination process, and the driver has already visually observed the candidate object, it is presumed that the driver did not overlook the pedestrian. <Condition (1-2)> Even if the above <Condition (1-1)> is not met, if the driver has visually observed the pedestrian at present or in the past, it is presumed that the driver did not overlook the pedestrian. <Condition (1-3)> If the above <Condition (1-1)> is not met, and the driver has not visually observed the pedestrian at present or in the past, it is presumed that the driver overlooked the pedestrian. Note that with regard to <Condition (1-1)>, since the candidate object is determined to be the same object as the pedestrian in the first target object determination process, if the driver has visually observed the candidate object at present, it can be considered that the driver has visually observed the pedestrian at present.

[0046] For example, the target object determination unit 14 determines whether a pedestrian detected by the viewed object detection unit 11 based on the image captured outside the vehicle is a newly detected pedestrian, and if the pedestrian is not a newly detected pedestrian, outputs information indicating that the pedestrian is not a new pedestrian (hereinafter referred to as "viewed object previously detected information") to the oversight estimation unit 15 together with the viewed object detection information output from the viewed object detection unit 11. The oversight estimation unit 15 can determine whether a pedestrian detected from the image captured outside the vehicle is a newly detected pedestrian depending on whether the viewed object previously detected information has been output from the target object determination unit 14. For example, the target object determination unit 14 may include the viewed object previously detected information in the viewed object presence / absence information.

[0047] Furthermore, the oversight estimation unit 15 can determine, from the first target object determination information output from the target object determination unit 14, whether or not a candidate object corresponding to a pedestrian has been identified in the first target object determination process.

[0048] The oversight estimation unit 15 may determine whether the driver has visually recognized the candidate object by, for example, the following method. First, the oversight estimation unit 15 compares the line of sight information stored in a storage unit (not shown) with the candidate object detection information to determine whether, at a certain point in the past, the driver's estimated visual recognition area was within a predetermined range (hereinafter referred to as the "object-based visual detection range") centered on the position of the candidate object. The size of the object-based visual detection range is determined in advance by a developer or the like. Note that the oversight estimation unit 15 can identify the driver's estimated visual recognition area and the position of the candidate object at a certain point in the past by, for example, comparing the line of sight information with the candidate object detection information at the detection time. In the first embodiment, the process performed by the oversight estimation unit 15 to determine whether the driver's estimated visual recognition area was within the object-based visual detection range by the above-described method is referred to as the "first object-based determination process."

[0049] If the oversight estimation unit 15 determines, as a result of the first object-based determination process, that the driver's estimated visual recognition area was within the object-based visual detection range at some point in the past, it determines that the driver has visually recognized the candidate object.On the other hand, if the oversight estimation unit 15 determines, as a result of the first object-based determination process, that the driver's estimated visual recognition area was not within the object-based visual detection range at any point in the past, it determines that the driver has not visually recognized the candidate object.

[0050] The oversight estimation unit 15 may determine whether the driver has visually identified a pedestrian at present or up to now, for example, by the following method. First, the oversight estimation unit 15 compares the line of sight information stored in a storage unit (not shown) with the visual object detection information to determine whether the driver's estimated visual position at present or up to now is within a predetermined range (hereinafter referred to as an "object-based visual detection range") centered on the three-dimensional position of the pedestrian, or whether the driver's estimated visual position has been within the object-based visual detection range. Note that the oversight estimation unit 15 may identify the current three-dimensional position of the pedestrian from the visual object detection information output from the visual object detection unit 11 via the target determination unit 14. For example, the oversight estimation unit 15 can compare the line of sight information with the visual object detection information at the detection time, thereby identifying the driver's estimated visual area and the three-dimensional position of the pedestrian at present or at a certain point up to now. In embodiment 1, the process performed by the oversight estimation unit 15 to determine whether the driver's estimated visual recognition area is currently or has been currently within the object-based visual detection range is referred to as the "object-based determination process."

[0051] Then, if the oversight estimation unit 15 determines, as a result of the object-based determination process, that the driver's estimated visual recognition area is within the object-based visual detection range or that the driver's estimated visual recognition area was within the object-based visual detection range at the present time or at any time up to the present time, it determines that the driver has visually recognized the pedestrian.On the other hand, if the oversight estimation unit 15 determines, as a result of the object-based determination process, that the driver's estimated visual recognition area is not within the object-based visual detection range or that the driver's estimated visual recognition area was not within the object-based visual detection range at the present time or at any time up to the present time, it determines that the driver has not visually recognized the pedestrian.

[0052] As described above, after the oversight estimation process, in other words, the first oversight estimation process, is performed, the oversight estimation unit 15 outputs information indicating the result of the oversight estimation process (hereinafter referred to as “oversight estimation information”) to the illumination mode determination unit 16. The oversight estimation information includes first oversight estimation information. The first oversight estimation information is information indicating the result of the first oversight estimation process performed by the oversight estimation unit 15. The first oversight estimation information includes information indicating whether the driver overlooked a pedestrian, the three-dimensional position of the pedestrian, the direction of the driver's gaze, and information indicating an estimated visual area. The oversight estimation unit 15 can identify the three-dimensional position of the pedestrian, for example, from the visual object detection information acquired from the visual object detection unit 11 via the target determination unit 14. The oversight estimation unit 15 can also identify the direction of the driver's gaze and the estimated visual area, for example, from gaze information acquired from the gaze detection unit 13.

[0053] When the target object determination unit 14 outputs visual object presence / absence information indicating "no pedestrian," that is, when the visual object detection unit 11 has not detected a pedestrian in the outside-of-vehicle image, the oversight estimation unit 15 determines that driver oversight estimation is unnecessary and does not perform the first oversight estimation process. When the oversight estimation unit 15 does not perform the first oversight estimation process, it outputs the visual object presence / absence information indicating "no pedestrian," which is output from the target object determination unit 14, to the illumination mode determination unit 16.

[0054] The illumination mode determination unit 16 determines the illumination mode to be performed by the headlights 5. In the first embodiment, the illumination mode may be, for example, normal light distribution, accentuated illumination, or additional illumination. In the first embodiment, the normal light distribution refers to, for example, a light distribution that radiates light forward with uniform brightness. This refers to, for example, low beam or high beam. Furthermore, as described above, accentuated illumination refers to illumination intended to alert the driver to pedestrians. For example, accentuated illumination may involve increasing the amount of light only on the pedestrian like a spotlight, illuminating the pedestrian so as to surround the pedestrian, flashing the light distribution near the pedestrian, or displaying an icon around the pedestrian using road surface light distribution. Furthermore, additional illumination refers to illumination intended to improve the visibility of a visually target area, in this case, the area ahead of the vehicle. For example, additional illumination may involve moving the brightest point of the high beam near the pedestrian, or increasing the light intensity of the high beam segment that illuminates only the pedestrian or the area surrounding the pedestrian. Since this illumination is used when there is little chance of insufficient light at a distance, the additional illumination subtly increases the light intensity, making it easier for drivers to distinguish pedestrians when they turn their eyes toward it, and it is desirable for the light distribution to be light that does not bother drivers by directing their gaze.

[0055] The illumination mode determination unit 16 determines the illumination mode to be performed by the headlights 5 in accordance with the illumination mode determination conditions. The illumination mode determination conditions are conditions that define in what mode the headlights 5 should be controlled to emit light, and include a first illumination mode determination condition. The first illumination mode determination condition is a condition that defines in what mode the headlights 5 should be controlled to emit light in what situation depending on whether or not the driver has overlooked a pedestrian. The illumination mode determination conditions are set in advance by a developer or the like, and information indicating the illumination mode determination conditions (hereinafter referred to as "illumination mode determination condition information") is stored in an internal buffer or the like of the illumination mode determination unit 16. The illumination mode determination conditions include, for example, first illumination mode determination conditions such as the following <Condition (2-1)> to <Condition (2-3)>.

[0056] <Condition (2-1)> If the result of the first oversight estimation process shows that the driver has not overlooked a pedestrian, normal light distribution is performed. <Condition (2-2)> If the result of the first oversight estimation process shows that the driver has overlooked a pedestrian and the distance from the vehicle to the pedestrian is less than a preset threshold (hereinafter referred to as the "distance determination threshold"), emphasized illumination is performed. <Condition (2-3)> If the result of the first oversight estimation process shows that the driver has overlooked a pedestrian and the distance from the vehicle to the pedestrian is equal to or greater than the distance determination threshold, emphasized illumination is not performed and additional illumination is performed.

[0057] The distance determination threshold is set to a value that assumes a distance from the vehicle, such as the distance that the low beam reaches as seen from the vehicle, close enough that the exterior camera 2 can capture an object at that distance.

[0058] Regarding the above <Condition (2-3)>, for example, even if it is estimated that the driver has overlooked a pedestrian, if the distance to the pedestrian is sufficient, it can be said that the urgency of making the driver notice the pedestrian is not so high. For example, the developer or the like sets the first illumination mode determination condition to perform additional illumination that improves visibility so as to help the driver notice the pedestrian more naturally.

[0059] The first illumination mode determination condition described above is merely an example. The first illumination mode determination condition may define a condition for determining in what situation and in what mode the headlamp 5 should be controlled to emit light, depending on whether or not the driver has overlooked a pedestrian.

[0060] For example, when the oversight estimation unit 15 estimates that the driver has overlooked the pedestrian and the distance from the vehicle to the pedestrian is less than the distance determination threshold, the illumination mode determination unit 16 determines to cause the headlights 5 to perform accentuated illumination. The illumination mode determination unit 16 can determine whether the oversight estimation unit 15 estimates that the driver has overlooked the pedestrian and the distance from the vehicle to the pedestrian from the oversight estimation information output from the oversight estimation unit 15. Furthermore, for example, when the oversight estimation unit 15 estimates that the driver has overlooked the pedestrian and the distance from the vehicle to the pedestrian is equal to or greater than the distance determination threshold, the illumination mode determination unit 16 determines to cause the headlights 5 to perform additional illumination. Furthermore, for example, when the oversight estimation unit 15 estimates that the driver has not overlooked the pedestrian, the illumination mode determination unit 16 determines to cause the headlights 5 to perform normal light distribution. In addition, if the oversight estimation unit 15 outputs visible object presence information indicating "no pedestrian," in other words, if the oversight estimation unit 15 does not perform the first oversight estimation process, the illumination mode determination unit 16 may determine that the oversight estimation unit 15 has estimated that the driver has not overlooked a pedestrian.

[0061] The illumination mode determination unit 16 outputs information on the determined illumination mode (hereinafter referred to as “illumination mode information”) to the light distribution control unit 17. The illumination mode information includes information indicating the illumination mode determined by the illumination mode determination unit 16 and oversight estimation information output from the oversight estimation unit 15.

[0062] Based on the illumination mode information output from the illumination mode determination unit 16, the light distribution control unit 17 controls the headlamp 5 so that light is emitted in the illumination mode determined by the illumination mode determination unit 16. More specifically, the light distribution control unit 17 determines an illumination range, an illumination light amount, or an illumination method according to the illumination mode determined by the illumination mode determination unit 16, and causes light to be emitted in the determined illumination range, illumination light amount, or illumination method.

[0063] For example, when the illumination mode determination unit 16 determines that the headlights 5 should perform accentuated illumination, the light distribution control unit 17 controls the headlights 5 to perform accentuated illumination. For example, the light distribution control unit 17 determines the range of light currently being emitted by the headlights 5 in the left-right or up-down direction that the headlights 5 should accentuate and illuminate, based on the three-dimensional position of the pedestrian. The left-right or up-down range of light that the headlights 5 should accentuate and illuminate is expressed, for example, by an angle from the installation position of the headlights 5, with the installation position of the headlights 5 set as the reference (0 degrees). The range of the left-right or up-down range that the headlights 5 should accentuate and illuminate, based on the three-dimensional position of the pedestrian, is determined in advance. Note that the light distribution control unit 17 can identify the three-dimensional position of the pedestrian based on the oversight estimation information. Furthermore, since the positional relationship between the position of the vehicle and the installation positions of the headlights 5 is known in advance, the light distribution control unit 17 can associate the three-dimensional position of the pedestrian with the illumination range of the light that the headlights 5 will illuminate with emphasis. The light distribution control unit 17 determines the range of light in the left-right or up-down direction that the headlights 5 will illuminate with emphasis, as described above, for each of the left and right lights. Note that this is merely an example, and the light distribution control unit 17 may, for example, illuminate either the left light or the right light with emphasis. In this case, the light distribution control unit 17 determines the range of light in the left-right or up-down direction that the headlights 5 will illuminate with emphasis, for the left light or the right light that will be illuminated with emphasis.

[0064] After determining the range in the left-right or up-down direction of the light to be emitted by the headlight 5 for emphasis, the light distribution control unit 17 outputs information for causing the headlight 5 to perform emphasis illumination (hereinafter referred to as "emphasis illumination control information") to the headlight 5. The emphasis illumination control information includes information indicating an angle representing the range in the left-right or up-down direction of the light to be emitted by the headlight 5 for emphasis, and information indicating an instruction to emit light in a blinking manner into that range.

[0065] The headlight 5 performs emphasis illumination in accordance with the emphasis illumination control information output from the light distribution control unit 17. In the above example, the headlight 5 blinks the light that is irradiated to the range set by the light distribution control unit 17.

[0066] In the above example, the light distribution control unit 17 causes the headlights 5 to flash light within a set range to provide emphasized illumination, but this is merely one example. For example, the light distribution control unit 17 may cause the headlights 5 to provide emphasized illumination by increasing the amount of light within the set range, or may cause the headlights 5 to provide emphasized illumination by changing the color of light within the set range. The emphasized illumination performed by the light distribution control unit 17 in the headlights 5 may be illumination that alerts the driver to pedestrians.

[0067] Furthermore, for example, when the illumination mode determination unit 16 determines to cause the headlights 5 to perform additional illumination, the light distribution control unit 17 controls the headlights 5 to perform additional illumination. For example, the light distribution control unit 17 determines the horizontal or vertical range of light to be additionally illuminated by the headlights 5, in addition to the range of light currently illuminated by the headlights 5. The horizontal or vertical range of the light to be additionally illuminated by the headlights 5 is determined in advance. The light distribution control unit 17 determines the horizontal or vertical range of the light to be additionally illuminated by the headlights 5, as described above, for each of the left and right lights. Note that this is merely an example, and the light distribution control unit 17 may, for example, cause either the left or right light to perform additional illumination. In this case, the light distribution control unit 17 determines the horizontal or vertical range of the light to be additionally illuminated for the left or right light to be additionally illuminated.

[0068] After determining the range in the left-right or up-down direction of the light to be additionally irradiated by the headlight 5, the light distribution control unit 17 outputs information for causing the headlight 5 to perform additional irradiation (hereinafter referred to as "additional irradiation control information") to the headlight 5. The additional irradiation control information includes information indicating an angle representing the range in the left-right or up-down direction of the light to be additionally irradiated by the headlight 5, and information indicating an instruction to irradiate the range with light.

[0069] The headlamp 5 performs additional illumination in accordance with the additional illumination control information output from the light distribution control unit 17. In the example described above, the headlamp 5 widens the illumination range of light to the range set by the light distribution control unit 17.

[0070] In the above example, the light distribution control unit 17 sets the range of additional light to be irradiated by the headlights 5 to a range in the left-right or up-down direction that is added to the range of light currently irradiated by the headlights 5. However, this is merely an example. For example, the light distribution control unit 17 may set the range of additional light to be irradiated by the headlights 5 to a range in the depth direction that is added to the range of light currently irradiated by the headlights 5. Furthermore, for example, if the headlights 5 are not currently irradiating high beams, the light distribution control unit 17 may set the range of additional light to be irradiated by the headlights 5 to a high beam irradiable area, and cause the headlights 5 to additionally irradiate high beams. The additional illumination that the light distribution control unit 17 causes the headlights 5 to perform may be intended to improve visibility ahead of the vehicle; in other words, it may be intended to illuminate the area ahead of the vehicle so that pedestrians in front of the vehicle can be more easily captured by the exterior camera 2.

[0071] Furthermore, when causing the headlights 5 to perform additional illumination, the light distribution control unit 17 may control the headlights 5 to gradually increase the intensity of the additional illumination. Specifically, the light distribution control unit 17 may include, in the additional illumination control information, an instruction to gradually increase the intensity of the additional illumination. The light distribution control unit 17 can appropriately set the degree to which the additional illumination is to be increased. However, the light distribution control unit 17 increases the intensity of the additional illumination to an extent that does not cause annoyance to the driver, in other words, increases the amount of light for the additional illumination. Furthermore, when causing the headlights 5 to perform additional illumination, the light distribution control unit 17 may control the headlights 5 to gradually increase the range of the additional illumination. In this case as well, the light distribution control unit 17 increases the range of the additional illumination to an extent that does not cause annoyance to the driver.

[0072] For example, when the illumination mode determination unit 16 determines that normal light distribution should be used, the light distribution control unit 17 outputs, for example, information to the headlights 5 to cause them to illuminate light forward with uniform brightness (hereinafter referred to as "normal light distribution control information") to the headlights 5. The normal light distribution control information includes information indicating an angle that represents the range of light over which the headlights 5 are to perform normal light distribution, and information indicating an instruction to illuminate that range with light. Note that the angle that represents the range of light over which normal light distribution is to be performed, and the amount of light to be illuminated into that range, are determined in advance.

[0073] The headlight 5 performs normal light distribution in accordance with the normal light distribution control information output from the light distribution control unit 17. In the above example, the headlight 5 irradiates the range set by the light distribution control unit 17 with a uniform light amount set by the light distribution control unit 17.

[0074] The operation of the headlamp control device 1 according to embodiment 1 will now be described. Fig. 3 is a flowchart for explaining the operation of the headlamp control device 1 according to embodiment 1. For example, when the automatic lights are turned on, the headlamp control device 1 starts the operation shown in the flowchart of Fig. 3, and repeats the operation shown in the flowchart of Fig. 3 until the automatic lights are turned off.

[0075] The visual object detection unit 11 acquires an exterior image from the exterior camera 2 and detects a pedestrian in front of the vehicle based on the acquired exterior image (step ST1). The visual object detection unit 11 outputs visual object detection information to the target determination unit 14. The visual object detection unit 11 also stores the visual object detection information in a storage unit (not shown) in chronological order.

[0076] The candidate object detection unit 12 acquires radar detection information from the exterior radar 3 and detects candidate objects in the visual target area based on the acquired radar detection information (step ST2). The candidate object detection unit 12 outputs the candidate object detection information to the target determination unit 14. The candidate object detection unit 12 also stores the candidate object detection information in a chronological order in a storage unit (not shown).

[0077] The gaze detection unit 13 acquires an in-vehicle image from the in-vehicle camera 4 and detects the direction of the driver's gaze based on the acquired in-vehicle image (step ST3). Furthermore, upon detecting the direction of the driver's gaze, the gaze detection unit 13 calculates an estimated visual recognition area of ​​the driver. The gaze detection unit 13 outputs gaze information to the oversight estimation unit 15. Furthermore, the gaze detection unit 13 stores the gaze information in a chronological order in a storage unit (not shown).

[0078] When a pedestrian is detected by the viewed object detection unit 11, the target determination unit 14 performs a first target determination process (step ST4). The target determination unit 14 outputs the first target determination information to the oversight estimation unit 15. When a pedestrian is not detected by the viewed object detection unit 11, that is, when the viewed object detection unit 11 outputs viewed object presence / absence information indicating “no pedestrian,” the target determination unit 14 does not perform the first target determination process and simply outputs the viewed object presence / absence information to the oversight estimation unit 15.

[0079] If a pedestrian is detected by the visual object detection unit 11 in step ST1, in other words, if visual object presence / absence information indicating "pedestrian present" is output from the target determination unit 14, the oversight estimation unit 15 performs an oversight estimation process (first oversight estimation process) to estimate whether the driver has overlooked a pedestrian based on the result of the first target determination process performed by the target determination unit 14 in step ST4 and the direction of the driver's gaze detected by the gaze detection unit 13 in step ST3 (step ST5). The oversight estimation unit 15 outputs the oversight estimation information to the illumination mode determination unit 16. If the first oversight estimation process was not performed, the oversight estimation unit 15 outputs the visual object presence / absence information indicating "pedestrian absent" output from the target determination unit 14 in step ST4 to the illumination mode determination unit 16.

[0080] The illumination mode determination unit 16 performs an illumination mode determination process to determine the illumination mode to be performed by the headlamp 5 (step ST6). The illumination mode determination unit 16 outputs illumination mode information to the light distribution control unit 17.

[0081] The light distribution control unit 17 controls the headlights 5 so that light is emitted in the illumination mode determined by the illumination mode determination unit 16 based on the illumination mode information output from the illumination mode determination unit 16 in step ST6 (step ST7).

[0082] Fig. 4 is a flowchart for explaining in detail an example of the oversight estimation process by the oversight estimation unit 15 in step ST5 of Fig. 3. Note that, here, the above-mentioned <Condition (1-1)> to <Condition (1-3)> are set as the conditions for pedestrian oversight estimation, and an example of the oversight estimation process by the oversight estimation unit 15 will be explained in detail.

[0083] The oversight estimation unit 15 determines whether or not a pedestrian has been detected by the visual object detection unit 11 in step ST1 of FIG. 3 (step ST11).

[0084] If a pedestrian is detected by the visual object detection unit 11 in step ST1 of FIG. 3 (if "YES" in step ST11), the oversight estimation unit 15 determines whether the pedestrian detected by the visual object detection unit 11 is a newly detected pedestrian (step ST12).

[0085] If it is determined in step ST12 that the pedestrian detected by the visual object detection unit 11 is a newly detected pedestrian ("YES" in step ST12), the oversight estimation unit 15 determines whether or not the target determination unit 14 has determined, as a result of performing the first target determination process in step ST4 of FIG. 3, that a candidate object corresponding to the pedestrian detected by the visual object detection unit 11 has been identified, in other words, whether or not it has determined that a candidate object that can be considered to be the same object as the pedestrian detected by the visual object detection unit 11 has been detected in the past by the candidate object detection unit 12 (step ST13).

[0086] If the target object determination unit 14 determines in step ST13 that a candidate object corresponding to the pedestrian detected by the visual object detection unit 11 has been identified as a result of the first target object determination process (if "YES" in step ST13), the oversight estimation unit 15 determines whether the driver has visually observed the candidate object up to now (step ST14). Specifically, the oversight estimation unit 15 performs the first object-based determination process and determines whether the driver's estimated visual recognition area was within the object-based visual detection range at a certain point in the past as a result of the first object-based determination process. If it is determined that the driver's estimated visual recognition area was within the object-based visual detection range at a certain point in the past, the oversight estimation unit 15 determines that the driver has visually observed the candidate object. If it is determined that the driver has visually observed the candidate object (if "YES" in step ST14), the oversight estimation unit 15 estimates that the driver has not overlooked the pedestrian (step ST15).

[0087] In other words, regardless of whether the driver is currently visually observing the pedestrian when the pedestrian is detected based on the outside-of-vehicle captured image, if it is determined that the driver has visually observed a candidate object that can be regarded as the same object as the pedestrian up to now, the oversight estimation unit 15 estimates that the driver has not overlooked the pedestrian.The oversight estimation unit 15 then outputs oversight estimation information including information indicating that the driver has not overlooked the pedestrian to the illumination mode determination unit 16.

[0088] If it is determined in step ST12 that the pedestrian detected by the visual object detection unit 11 is not a newly detected pedestrian (if "NO" in step ST12), if it is determined in step ST13 that the target object determination unit 14 has performed the first target object determination process and as a result has not been able to identify a candidate object corresponding to the pedestrian detected by the visual object detection unit 11 (if "NO" in step ST13), or if it is determined in step ST14 that the driver has not visually observed the candidate object up to now (if "NO" in step ST14), the oversight estimation unit 15 determines whether or not the driver has visually observed the pedestrian now or up to now (step ST16).

[0089] Specifically, the oversight estimation unit 15 performs an object-based determination process and determines, as a result of the process, whether the driver's estimated visual recognition area is within the object-based visual detection range or whether the driver's estimated visual recognition area was within the object-based visual detection range at the present time or at a certain point in time up to the present time. If it is determined that the driver's estimated visual recognition area is within the object-based visual detection range or was within the object-based visual detection range at the present time or at a certain point in time up to the present time, the oversight estimation unit 15 determines that the driver has visually recognized the pedestrian. On the other hand, if it is determined that the driver's estimated visual recognition area is not within the object-based visual detection range or was not within the object-based visual detection range at the present time or at any point in time up to the present time, the oversight estimation unit 15 determines that the driver has not visually recognized the pedestrian.

[0090] If it is determined in step ST16 that the driver has visually observed the pedestrian at present or up to now (if "YES" in step ST16), the oversight estimation unit 15 estimates that the driver has not overlooked the pedestrian (step ST15). Then, the oversight estimation unit 15 outputs oversight estimation information including information indicating that the driver has not overlooked the pedestrian to the illumination mode determination unit 16.

[0091] On the other hand, if it is determined in step ST16 that the driver has not visually observed the pedestrian at present or up to now (if "NO" in step ST16), the oversight estimation unit 15 estimates that the driver has overlooked the pedestrian (step ST17). Then, the oversight estimation unit 15 outputs oversight estimation information including information indicating that the driver has overlooked the pedestrian to the illumination mode determination unit 16.

[0092] For example, if the visual object detection unit 11 has already detected a pedestrian based on the exterior-of-vehicle captured image, the driver should have already visually observed the pedestrian when the pedestrian was newly detected. If the driver had not visually observed the pedestrian and it was estimated that the driver had overlooked the pedestrian, the headlights 5 should have been controlled to provide enhanced illumination. However, even if the headlights 5 provide enhanced illumination of the pedestrian, it cannot be said with certainty that the driver would visually observe the pedestrian. Therefore, if the visual object detection unit 11 detected a pedestrian based on the exterior-of-vehicle captured image but the pedestrian was not newly detected ("NO" in step ST12), the oversight estimation unit 15 performs the object-based determination process (step ST16). If the oversight estimation unit 15 determines that the driver had visually observed the pedestrian detected based on the exterior-of-vehicle captured image at the present time or at a point up to the present time, it estimates that the driver did not overlook the pedestrian. On the other hand, if the oversight estimation unit 15 determines that the driver has not visually observed the pedestrian detected based on the image taken outside the vehicle at the present time or at some point up to the present time, it estimates that the driver has overlooked the pedestrian.

[0093] Furthermore, even if a candidate object that can be identified as a pedestrian has not been detected based on radar detection information up to now, the driver may have visually observed the pedestrian detected based on the exterior image at the present time or at some point up to now. For example, the exterior radar 3 may detect a candidate object located in front of the candidate object that can be identified as a pedestrian, and thus may not output radar detection information indicating the detection of the candidate object that can be identified as a pedestrian. In this case, even if the driver has visually observed the pedestrian up to now, the oversight estimation unit 15 cannot infer that the pedestrian has been visually observed based on the candidate object detection information. Therefore, if the oversight estimation unit 15 determines that a candidate object corresponding to the pedestrian detected by the visual object detection unit 11 has not been identified ("NO" in step ST13), it performs the object-based determination process (step ST16). If the oversight estimation unit 15 determines that the driver has visually observed a pedestrian detected based on an image taken outside the vehicle at the present time or at some point up to the present time, it estimates that the driver has not overlooked the pedestrian.

[0094] Furthermore, even if a candidate object is determined to be the same object as a pedestrian, it does not necessarily mean that the candidate object is the same object as a pedestrian, i.e., that the candidate object is a pedestrian. Therefore, if the driver has not visually observed a candidate object that can be considered the same object as a pedestrian and that was detected based on radar detection information up to now (if "NO" in step ST14), the oversight estimation unit 15 performs the object-based determination process (step ST16). If the oversight estimation unit 15 determines that the driver has visually observed a pedestrian detected based on an outside-vehicle image at the present time or at some point up to the present time, it estimates that the driver has not overlooked the pedestrian. If the driver visually observed a pedestrian detected based on an outside-vehicle image, it can be safely assumed that the driver has visually observed the pedestrian.

[0095] Returning to the description of the flowchart in Fig. 4 , if the visual object detection unit 11 does not detect a pedestrian in step ST1 of Fig. 3 (in the case of "NO" in step ST11), it is determined that driver oversight estimation is unnecessary, and the first oversight estimation process is not performed (step ST18). Then, the oversight estimation unit 15 outputs visual object presence / absence information indicating "no pedestrian" to the illumination mode determination unit 16.

[0096] 5 is a flowchart for explaining in detail an example of the irradiation mode determination process by the irradiation mode determination unit 16 in step ST6 in FIG. 3. Note that, here, the irradiation mode determination conditions are set to the first irradiation mode determination conditions such as the above-mentioned <Condition (2-1)> to <Condition (2-3)>, and an example of the irradiation mode determination process by the irradiation mode determination unit 16 will be explained in detail.

[0097] The illumination mode determination unit 16 determines whether the oversight estimation unit 15 has performed the oversight estimation process (first oversight estimation process) (step ST21). If the illumination mode determination unit 16 determines that the oversight estimation unit 15 has performed the oversight estimation process (YES in step ST21), the illumination mode determination unit 16 determines whether the oversight estimation unit 15 has estimated that the driver overlooked a pedestrian (step ST22).

[0098] If the oversight estimation unit 15 estimates that the driver has overlooked a pedestrian (if "YES" in step ST22), the illumination mode determination unit 16 determines whether the distance from the vehicle to the pedestrian is less than the distance determination threshold (step ST23).

[0099] If it is determined in step ST23 that the distance from the vehicle to the pedestrian is less than the distance determination threshold (if "YES" in step ST23), the illumination mode determination unit 16 determines to cause the headlights 5 to perform accentuated illumination (step ST24).

[0100] If it is determined in step ST23 that the distance from the vehicle to the pedestrian is greater than or equal to the distance determination threshold (if "NO" in step ST23), the illumination mode determination unit 16 determines to cause the headlight 5 to perform additional illumination (step ST25).

[0101] For example, even if the driver visually observes a candidate object that can be considered the same as a pedestrian, detected based on radar detection information, the candidate object may be an object that the driver does not need to focus on (such as a hedge or a telephone pole). Therefore, the headlight control device 1 preferably does not cause the driver to perform accentuated illumination until the driver determines that the candidate object is a visual target, in this case, a pedestrian. This is because additional illumination of an object that the driver does not need to focus on could be annoying to the driver. Therefore, if the illumination mode determination unit 16 determines that the distance from the vehicle to the pedestrian is less than the distance determination threshold ("YES" in step ST23), it determines to cause the headlights 5 to perform accentuated illumination (step ST24). As described above, the distance determination threshold is set to a value that assumes a distance from the vehicle close enough that the exterior camera 2 can capture an object at that distance.

[0102] If the oversight estimation unit 15 determines that the oversight estimation process was not performed (if "NO" in step ST21), or if the oversight estimation unit 15 estimates that the driver did not overlook a pedestrian (if "NO" in step ST22), the illumination mode determination unit 16 determines to have the headlights 5 perform normal light distribution (step ST26).

[0103] 5 is merely an example. The illumination mode determination unit 16 determines the illumination mode to be performed by the headlights 5 in accordance with the illumination mode determination conditions. For example, if a first illumination mode determination condition other than the above-described <Condition (2-1)> to <Condition (2-3)> is set in the illumination mode determination conditions, the illumination mode determination unit 16 determines the illumination mode to be performed by the headlights 5 in accordance with the first illumination mode determination condition. For example, if the first illumination mode determination condition is set to "perform accentuated illumination when the driver has overlooked a pedestrian as a result of performing the first oversight estimation process," the illumination mode determination unit 16 determines to cause the headlights 5 to perform accentuated illumination without performing the process of step ST23 in the flowchart of FIG. 5.

[0104] In this way, the headlight control device 1 detects a pedestrian in front of the vehicle based on an exterior image of the vehicle's surroundings captured by the exterior camera 2. The headlight control device 1 also detects a candidate object in front of the vehicle based on radar detection information indicating that the exterior radar 3 has detected an object in the vehicle's surroundings. The headlight control device 1 also detects the driver's gaze direction based on an interior image of the driver captured by the interior camera 4. When a pedestrian is detected, the headlight control device 1 performs a first target object determination process to determine whether an object corresponding to the detected pedestrian has been detected as a candidate object. Based on the result of the first target object determination process and the detected driver's gaze direction, the headlight control device 1 estimates whether the driver has overlooked the pedestrian. If it estimates that the driver has overlooked the pedestrian, the headlight control device 1 controls the headlights 5 to perform accentuated illumination to alert the driver to the pedestrian. Therefore, the headlight control device 1 can control the illumination of the headlights 5 to prevent the driver from overlooking the pedestrian while taking into consideration the inconvenience to the driver.

[0105] In the first embodiment described above, when the headlight control device 1 detects a pedestrian based on an exterior image captured by the exterior camera 2 of the area around the vehicle, it determines whether an object corresponding to the pedestrian has been detected to date based on radar detection information obtained by the exterior radar 3 detecting an object around the vehicle. However, this is merely an example. When the headlight control device 1 detects a pedestrian based on information obtained by a device capable of detecting the position and type of an object around the vehicle (hereinafter referred to as a “position type detection device”) detecting an object around the vehicle (hereinafter referred to as “first exterior object detection information”), it is sufficient that the headlight control device 1 determines whether an object corresponding to the pedestrian has been detected to date based on information obtained by a device capable of detecting the position of an object around the vehicle but not the type of object (hereinafter referred to as a “position detection device”) detecting the object around the vehicle (hereinafter referred to as “second exterior object detection information”). The position type detection device is not limited to the exterior camera 2 or the exterior radar 3. In other words, if the headlamp control device 1 determines that there is some kind of object near the vehicle, even if it does not know whether it is a pedestrian, it estimates whether the driver has overlooked it, identifies the type of object, and determines that it is a type of object that requires the driver's attention, such as a pedestrian.If it determines that the driver cannot see the pedestrian, it performs illumination (emphasis illumination) to make the driver aware of the pedestrian.For example, assume that the exterior camera 2 has a function to identify only the frame of an object and a function to identify the type of object within the identified frame.In this case, the exterior camera 2 in a state where the function to identify only the frame of an object is being executed may be used as a position detection device.

[0106] In the first embodiment, the light distribution control unit 17 may have the function of the illumination mode determination unit 16. In this case, the headlamp control device 1 may be configured without the illumination mode determination unit 16.

[0107] In the first embodiment described above, the headlamp control device 1 is an on-board device mounted on a vehicle, and the on-board device is provided with the visual target detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17. This is merely an example, and for example, some of the visual target detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17 may be provided in the on-board device, and the rest may be provided in a server connected to the on-board device via a network. In addition, all of the visual target detection unit 11, candidate object detection unit 12, gaze detection unit 13, target determination unit 14, oversight estimation unit 15, illumination mode determination unit 16, and light distribution control unit 17 may be provided on the server.

[0108] 6A and 6B are diagrams illustrating an example of the hardware configuration of the headlamp control device 1 according to embodiment 1. In embodiment 1, the functions of the visual target detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17 are realized by a processing circuit 1001. That is, the headlamp control device 1 includes a processing circuit 1001 that performs a first target determination process based on visual target detection information related to a visual target detected based on an exterior captured image of the vehicle periphery captured by the exterior camera 2, for example, and candidate object information related to a candidate object detected based on radar detection information of an object detected around the vehicle by the exterior radar 3, estimates whether the driver has overlooked the visual target based on the result of the first target determination process and the direction of the driver's line of sight detected based on an interior captured image of the driver captured by the interior camera 4, and controls the headlamp 5 to perform accentuated illumination when it is estimated that the driver has overlooked the visual target. The processing circuit 1001 may be dedicated hardware as shown in FIG. 6A, or a processor 1004 that executes a program stored in a memory 1005 as shown in FIG. 6B.

[0109] If the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0110] When the processing circuit is the processor 1004, the functions of the visual object detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target object determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 reads and executes the program stored in the memory 1005 to execute the functions of the visual object detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target object determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17. In other words, the headlamp control device 1 includes the memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST1 to ST7 of FIG. 3 described above. In addition, it can also be said that the program stored in memory 1005 causes the computer to execute the processing procedures or methods of the visual target detection unit 11, the candidate object detection unit 12, the gaze detection unit 13, the target determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17. Here, the memory 1005 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

[0111] Note that the functions of the visual target detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17 may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the functions of the visual target detection unit 11, the candidate object detection unit 12, and the line of sight detection unit 13 may be implemented by a processing circuit 1001 as dedicated hardware, and the functions of the target determination unit 14, the oversight estimation unit 15, the illumination mode determination unit 16, and the light distribution control unit 17 may be implemented by a processor 1004 reading and executing programs stored in a memory 1005. The headlight control device 1 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the exterior camera 2, the exterior radar 3, the interior camera 4, or the headlights 5. The storage unit (not shown) is configured by the memory 1005 or a HDD or the like.

[0112] As described above, the headlamp control device 1 according to the first embodiment includes the following components: a visual object detection unit 11 that detects a visual object that the driver should be visually aware of in a visual target area that the driver should be visually aware of, based on first external object detection information (e.g., an external-vehicle captured image) in which an object present in the vicinity of the vehicle is detected by a position type detection device (e.g., the exterior-vehicle camera 2) that can detect the position and type of an object present in the vicinity of the vehicle; a candidate object detection unit 12 that detects a candidate object that is a visual object in the visual target area, based on second external object detection information (e.g., radar detection information) in which an object present in the vicinity of the vehicle is detected by a position detection device (e.g., the exterior-vehicle radar 3) that can detect the position of an object present in the vicinity of the vehicle; and an interior-vehicle captured image in which the driver is captured by the interior-vehicle camera 4. the target object determination unit 14 that performs a first target object determination process that, when a target object is detected by the target object detection unit 11, determines whether an object corresponding to the detected target object has been detected as a candidate object by the candidate object detection unit 12; an oversight estimation unit 15 that estimates whether the driver has overlooked the target object based on the result of the first target object determination process performed by the target object determination unit 14 and the direction of the driver's line of sight detected by the target object detection unit 13; and a light distribution control unit 17 that, when the oversight estimation unit 15 estimates that the driver has overlooked the target object, controls the headlights 5 to perform emphasized illumination to alert the driver to the target object. Therefore, the headlight control device 1 can control the illumination of the headlights 5 so as to prevent the driver from overlooking the target object while taking into consideration the inconvenience caused to the driver.

[0113] Furthermore, in the headlamp control device 1, the oversight estimation unit 15 estimates that the driver has not overlooked the visual object if it is determined that an object corresponding to the visual object has been detected as a candidate object in the first target object determination process and it is determined that the driver has already visually observed the candidate object based on the direction of the driver's line of sight detected by the line of sight detection unit 13 and the candidate object detection information regarding the candidate object detected by the candidate object detection unit 12. This allows the headlamp control device 1 to, for example, highlight visual objects that the driver has already visually recognized, preventing the highlight illumination from being an annoyance to the driver.

[0114] Furthermore, in the headlamp control device 1, the light distribution control unit 17 controls the headlamp 5 to perform accentuated illumination when the oversight estimation unit 15 estimates that the driver has overlooked an object to be viewed and the distance from the vehicle to the object to be viewed is less than the distance determination threshold. For example, the headlamp control device 1 controls the headlamp 5 to perform accentuated illumination when it becomes possible to determine that an object (candidate object) that the driver has overlooked is the object to be viewed, thereby preventing the accentuated illumination from becoming an annoyance to the driver.

[0115] Furthermore, in the headlamp control device 1, even if the oversight estimation unit 15 estimates that the driver has overlooked the visual target, if the distance from the vehicle to the visual target is equal to or greater than the distance determination threshold, the light distribution control unit 17 controls the headlamp 5 not to perform accentuated illumination but to perform additional illumination to improve the visibility of the visual target area. In this way, the headlamp control device 1 can help the driver to more naturally notice the visual target when it can be said that there is not much urgency in making the driver notice the visual target.

[0116] Embodiment 2 In the first embodiment, if the headlight control device does not detect a pedestrian based on the image captured outside the vehicle, the headlight control device does not perform the oversight estimation process (first oversight estimation process) to determine whether the driver has overlooked the pedestrian, and controls the headlights 5 to provide normal light distribution. In the second embodiment, an embodiment will be described in which the headlight control device performs additional illumination when a candidate object is detected based on radar detection information even if a pedestrian is not detected based on the image captured outside the vehicle, thereby assisting in the detection of a pedestrian based on the image captured outside the vehicle.

[0117] FIG. 7 is a diagram showing a configuration example of a headlight control device 1a according to a second embodiment. Regarding the configuration example of the headlight control device 1a according to the second embodiment, components similar to those of the headlight control device 1 according to the first embodiment described using FIG. 1 in the first embodiment are denoted by the same reference numerals, and redundant description will be omitted. In the headlight control device 1a according to the second embodiment, the details of the processing performed by the target object determination unit 14a, the oversight estimation unit 15a, and the illumination mode determination unit 16a are different from the details of the processing performed by the target object determination unit 14, the oversight estimation unit 15, and the illumination mode determination unit 16 in the headlight control device 1 according to the first embodiment. Note that, like the headlight control device 1 according to the first embodiment, the headlight control device 1a according to the second embodiment is assumed to be mounted on a vehicle. Also, in the second embodiment, as in the first embodiment, the visual target area is assumed to be a predetermined area ahead of the vehicle, and the visual target is assumed to be a pedestrian, by way of example.

[0118] Here, an overview of the control of the headlights 5 performed by the headlight control device 1a according to the second embodiment will be described with reference to the drawings. FIGS. 8 and 9 are diagrams for explaining an overview of an example of control of the headlights 5 by the headlight control device 1a according to the second embodiment. In FIGS. 8 and 9, vehicles are indicated by "C," pedestrians are indicated by "P," and candidate objects are indicated by "B." In FIGS. 8 and 9, the candidate objects are, for example, trees. For convenience, the exterior camera 2, exterior radar 3, interior camera 4, and headlights 5 are not shown in FIGS. 8 and 9. The headlight control device 1a controls the headlights 5 as described below in addition to the control of the headlights 5 performed by the headlight control device 1 according to the first embodiment.

[0119] For example, if a candidate object exists at a distance that is difficult for the exterior camera 2 to capture (a distance from the vehicle that is equal to or greater than the first object distance threshold) (see FIG. 8 ), the exterior camera 2 has difficulty capturing the candidate object, but the exterior radar 3 can capture it. The first object distance threshold is, for example, the distance that the low beam reaches as seen from the vehicle. The first object distance threshold may be set to, for example, the same value as the distance determination threshold described in the first embodiment. Even if the exterior radar 3 captures a candidate object that is at a distance that is difficult for the exterior camera 2 to capture, as described above, it is difficult to determine the type of the candidate object from the radar detection information. However, if the candidate object is a pedestrian, it means that the driver has overlooked the pedestrian. Therefore, it is desirable to determine early on whether the candidate object is a pedestrian. When the exterior radar 3 detects a candidate object that is too far away for the exterior camera 2 to detect, but the driver cannot see the candidate object, enabling the exterior camera 2 to detect the candidate object early allows for early determination of whether the candidate object is a pedestrian. Therefore, the headlight control device 1a according to the second embodiment determines whether a candidate object has been detected when a pedestrian is not detected based on the exterior-captured image, and estimates whether the driver has overlooked the candidate object based on the result of the determination and the driver's line of sight. If the headlight control device 1a estimates that the driver has overlooked the candidate object, it controls the headlights 5 to provide additional illumination to improve the visibility of pedestrians (see FIG. 8 ).

[0120] As a result, when a pedestrian is not detected in the outside-of-vehicle captured image but a candidate object is detected from the radar detection information in the visual target area, the headlight control device 1a can assist in pedestrian detection based on the outside-of-vehicle captured image. As a result, the headlight control device 1a can detect a pedestrian based on the outside-of-vehicle captured image early, and can quickly estimate whether the driver has overlooked the pedestrian. Note that even when a pedestrian is not detected in the outside-of-vehicle captured image but a candidate object is detected from the radar detection information in the visual target area, the headlight control device 1a does not control the headlights 5 to perform additional illumination if the driver has already visually observed the candidate object. This allows the headlight control device 1a to prevent the headlights 5 from performing unnecessary additional illumination.

[0121] Furthermore, as described above, even if the headlight control device 1a controls the headlights 5 to provide additional illumination and the exterior camera 2 is able to capture a candidate object early, the candidate object may not be a pedestrian. For example, as shown in FIG. 8 , even if the exterior radar 3 captures a candidate object at a distance equal to or greater than the first object distance threshold, it is difficult to determine whether the candidate object is a pedestrian or a tree. Once the exterior camera 2 is able to capture the candidate object, it becomes possible to determine whether the candidate object is a pedestrian based on the exterior-captured image. In this case, if the candidate object is not a pedestrian, the pedestrian is not detected in the exterior-captured image, or is detected as an object other than a pedestrian. In this case, there is no pedestrian that the driver should be aware of, and assistance by additional illumination is not required. Therefore, if the headlight control device 1a determines that a candidate object detected from radar detection information may be a pedestrian and controls the headlights 5 to provide additional illumination so that the exterior camera 2 can capture the candidate object early, and then, if it estimates that the driver has overlooked the candidate object and the distance to the candidate object becomes such that it can be easily captured by the exterior camera 2 (the distance from the vehicle is less than the second object distance threshold), it controls the headlights 5 to stop providing additional illumination and return to the original light distribution (see Figure 9).

[0122] If the candidate object detected from the radar detection information is not a pedestrian, it can be said that the additional illumination that the headlight 5 has been performing has been a waste of illumination. However, the headlight control device 1a can stop the additional illumination, which can be said to have been a waste of illumination, before the vehicle approaches and even passes the candidate object and the candidate object can no longer be detected from the radar detection information.

[0123] The details of the processes performed by the target object determination unit 14a, the oversight estimation unit 15a, and the illumination mode determination unit 16a in the headlamp control device 1a will be described.

[0124] In addition to the first target determination process, the target determination unit 14a performs a process (hereinafter referred to as "second target determination process") of determining whether or not a candidate object has been detected by the candidate object detection unit 12 when a pedestrian has not been detected by the viewed object detection unit 11. The details of the first target determination process have been explained in the first embodiment, so a detailed explanation will be omitted. When viewed object presence / absence information indicating "no pedestrian" is output from the viewed object detection unit 11, the target determination unit 14a may determine that a pedestrian has not been detected by the viewed object detection unit 11.

[0125] In the second target determination process, the target determination unit 14 a determines, based on the candidate object detection information output from the candidate object detection unit 12 , whether or not a candidate object has been detected by the candidate object detection unit 12 .

[0126] The target determination unit 14a outputs first target determination information when the first target determination process has been performed, or outputs information indicating the result of the second target determination process when the second target determination process has been performed (hereinafter referred to as "second target determination information") to the oversight estimation unit 15a. The second target determination information includes information indicating whether or not a candidate object has been detected by the candidate object detection unit 12, and the candidate object detection information output from the candidate object detection unit 12. The target determination unit 14a outputs the first target determination information or the second target determination information to the oversight estimation unit 15a, together with the viewed object presence / absence information output from the viewed object detection unit 11.

[0127] When a pedestrian is detected by the viewed object detection unit 11, in other words, when viewed object presence / absence information indicating “pedestrian present” is output from the target determination unit 14a, the oversight estimation unit 15a performs a first oversight estimation process to estimate whether or not the driver has overlooked a pedestrian, based on the result of the first target determination process performed by the target determination unit 14a and the direction of the driver's gaze detected by the gaze detection unit 13. When a pedestrian is not detected by the viewed object detection unit 11 and a candidate object is detected by the candidate object detection unit 12, in other words, when the target determination unit 14a outputs second target determination information including information indicating that a candidate object has been detected by the candidate object detection unit 12, the oversight estimation unit 15a estimates whether or not the driver has overlooked the candidate object, based on the result of the second target determination process performed by the target determination unit 14a and the direction of the driver's gaze detected by the gaze detection unit 13.

[0128] In the second embodiment, the oversight estimation process performed by the oversight estimation unit 15a to estimate whether or not the driver has overlooked a candidate object is referred to as a "second oversight estimation process." That is, in the second embodiment, the oversight estimation process performed by the oversight estimation unit 15a includes a first oversight estimation process and a second oversight estimation process. The details of the first oversight estimation process performed by the oversight estimation unit 15a are the same as the details of the first oversight estimation process performed by the oversight estimation unit 15 in the first embodiment, and therefore a duplicated description will be omitted.

[0129] In more detail, regarding the second oversight estimation process, when a pedestrian is not detected by the visual object detection unit 11, the oversight estimation unit 15a estimates whether or not the driver has overlooked a candidate object based on the candidate object oversight estimation conditions. The candidate object oversight estimation conditions are conditions that define under what circumstances a candidate object detected based on radar detection information is estimated to have been overlooked by the driver. The candidate object oversight estimation conditions are set in advance by a developer or the like, and information indicating the candidate object oversight estimation conditions (hereinafter referred to as "candidate object oversight estimation condition information") is stored in an internal buffer or the like of the oversight estimation unit 15a. Note that the candidate object oversight estimation condition information may be updated as appropriate. The candidate object oversight estimation conditions include, for example, the following conditions (3-1) to (3-2).

[0130] <Condition (3-1)> If it is determined that a candidate object has been detected in the second target object determination process, and the driver is currently visually observing the candidate object, it is presumed that the driver has not overlooked the candidate object. <Condition (3-2)> If it is determined that a candidate object has been detected in the second target object determination process, and the driver is not currently visually observing the candidate object, it is presumed that the driver has overlooked the candidate object.

[0131] The oversight estimation unit 15a may determine whether the driver is currently visually observing a candidate object by, for example, the following method. First, the oversight estimation unit 15a compares the line-of-sight information output from the line-of-sight detection unit 13 with the candidate object detection information output from the candidate object detection unit 12, and determines whether the driver's estimated visual recognition area is within the object-based visual detection range. The oversight estimation unit 15a can acquire the candidate object detection information output from the candidate object detection unit 12 from the target determination unit 14a. In the second embodiment, the above-described process performed by the oversight estimation unit 15a to determine whether the driver's estimated visual recognition area is within the object-based visual detection range is referred to as a "second object-based determination process."

[0132] Then, when the oversight estimation unit 15a determines that the driver's estimated visual recognition area is within the object-based visual detection range as a result of the second object-based determination process, it determines that the driver is visually recognizing the candidate object.On the other hand, when the oversight estimation unit 15a determines that the driver's estimated visual recognition area is not within the object-based visual detection range as a result of the second object-based determination process, it determines that the driver is not visually recognizing the candidate object.

[0133] When the oversight estimation process, in other words, the first oversight estimation process or the second oversight estimation process, is performed, the oversight estimation unit 15a outputs oversight estimation information to the illumination mode determination unit 16a. In the second embodiment, the oversight estimation information includes first oversight estimation information indicating the result of the first oversight estimation process performed by the oversight estimation unit 15a, and second oversight estimation information indicating the result of the second oversight estimation process performed by the oversight estimation unit 15a. The first oversight estimation information has already been described in the first embodiment, so a duplicated description will be omitted. The second oversight estimation information includes information indicating whether the driver has overlooked a candidate object, the position of the candidate object, the direction of the driver's line of sight, and information indicating an estimated visual recognition area. The oversight estimation unit 15a can identify the position of the candidate object, for example, from the candidate object detection information acquired from the candidate object detection unit 12 via the target determination unit 14a. Furthermore, the oversight estimation unit 15 a can identify the direction of the driver's line of sight and the estimated visual recognition area from, for example, line of sight information acquired from the line of sight detection unit 13 .

[0134] When the target determination unit 14a outputs second target determination information including information indicating that a candidate object has not been detected by the candidate object detection unit 12, that is, when the visual target detection unit 11 has not detected a pedestrian based on the outside-of-vehicle image and the candidate object detection unit 12 has not detected a candidate object based on the radar detection information, the oversight estimation unit 15a determines that driver oversight estimation is unnecessary and does not perform the second oversight estimation process. When the oversight estimation unit 15a does not perform the oversight estimation process, it outputs the second target determination information including information indicating that a candidate object has not been detected by the candidate object detection unit 12, which is output from the target determination unit 14a, to the illumination mode determination unit 16a.

[0135] The illumination mode determination unit 16a determines the illumination mode to be performed by the headlights 5. In the second embodiment, the illumination mode is, for example, normal light distribution, accentuated illumination, or additional illumination. Similarly to the illumination mode determination unit 16 according to the first embodiment, the illumination mode determination unit 16a determines the illumination mode to be performed by the headlights 5 according to illumination mode determination conditions. However, in the second embodiment, the illumination mode determination conditions include a second illumination mode determination condition in addition to a first illumination mode determination condition. The first illumination mode determination condition has already been described in the first embodiment, so a redundant description will be omitted. The second illumination mode determination condition defines the mode in which the headlights 5 are controlled to emit light in a given situation, depending on whether or not the driver has overlooked a candidate object. In the second embodiment, the illumination target determination conditions include, for example, first illumination mode determination conditions such as the following <Condition (4-1)> to <Condition (4-3)> and second illumination mode determination conditions such as the following <Condition (4-4)> to <Condition (4-7)>. <Condition (4-8)> may be set in the first irradiation mode determination condition or in the second irradiation mode determination condition. Note that <Condition (4-1)> to <Condition (4-3)> are the same as <Condition (2-1)> to <Condition (2-3)> described in the first embodiment, respectively.

[0136] <Condition (4-1)> If the result of the first oversight estimation process shows that the driver did not overlook a pedestrian, normal light distribution is performed. <Condition (4-2)> If the result of the first oversight estimation process shows that the driver overlooked a pedestrian and the distance from the vehicle to the pedestrian is less than the distance determination threshold, enhanced illumination is performed. <Condition (4-3)> If the result of the first oversight estimation process shows that the driver overlooked a pedestrian and the distance from the vehicle to the pedestrian is equal to or greater than the distance determination threshold, enhanced illumination is not performed and additional illumination is performed. <Condition (4-4)> If the result of the second oversight estimation process shows that the driver did not overlook a candidate object, and additional illumination has already been performed on the candidate object, additional illumination is canceled, and if additional illumination has not already been performed on the candidate object, normal light distribution is performed. <Condition (4-5)> If the result of the second oversight estimation process shows that the driver overlooked a candidate object and the distance to the candidate object is equal to or greater than the first object distance threshold, additional illumination is performed. <Condition (4-6)> If the result of performing the second oversight estimation process shows that the driver has overlooked a candidate object, and the distance to the candidate object is less than the second object distance threshold, the additional illumination is terminated. <Condition (4-7)> If the result of performing the second oversight estimation process shows that the driver has overlooked a candidate object, and the distance to the candidate object is less than the first object distance threshold and equal to or greater than the second object distance threshold, the additional illumination is continued. <Condition (4-8)> If neither the first oversight estimation process nor the second oversight estimation process is performed, normal light distribution is performed.

[0137] The first illumination mode determination condition and the second illumination mode determination condition described above are merely examples. The first illumination mode determination condition may define a condition as to what situation and what mode the headlights 5 should be controlled to emit light in, depending on whether or not the driver has overlooked a pedestrian. The second illumination mode determination condition may define a condition as to what situation and what mode the headlights 5 should be controlled to emit light in, depending on whether or not the driver has overlooked a candidate object.

[0138] For example, when the second oversight estimation process performed by the oversight estimation unit 15a estimates that the driver has overlooked a candidate object and the distance to the candidate object is equal to or greater than the first object distance threshold, the illumination mode determination unit 16a determines to cause the headlights 5 to perform additional illumination. The illumination mode determination unit 16a can determine whether the oversight estimation unit 15a estimates that the driver has overlooked a candidate object and the distance from the vehicle to the candidate object from the oversight estimation information output from the oversight estimation unit 15a, more specifically, the second oversight estimation information.

[0139] Furthermore, for example, when the second oversight estimation process is performed by the oversight estimation unit 15a and it is estimated that the driver has overlooked a candidate object, and the distance to the candidate object is less than the second object distance threshold, the illumination mode determination unit 16a determines to stop the additional illumination of the headlights 5. Furthermore, for example, when the second oversight estimation process is performed by the oversight estimation unit 15a and it is estimated that the driver has overlooked a candidate object, and the distance to the candidate object is less than the first object distance threshold and equal to or greater than the second object distance threshold, the illumination mode determination unit 16a determines to continue the additional illumination of the headlights 5.

[0140] Furthermore, for example, when the second oversight estimation unit 15a estimates that the driver has not overlooked a candidate object as a result of the second oversight estimation process, and if the candidate object has already been additionally illuminated, the illumination mode determination unit 16a determines to cancel additional illumination, whereas if the candidate object has not already been additionally illuminated, the illumination mode determination unit 16a determines to perform normal light distribution. For example, the illumination mode determination unit 16a stores information indicating the determined illumination mode in an internal buffer or the like of the illumination mode determination unit 16a, and updates the stored information indicating the illumination mode when the illumination mode is changed. Based on the stored information indicating the illumination mode, the illumination mode determination unit 16a can determine what illumination mode is currently being set to be performed by the headlights 5. In the above example, the illumination mode determination unit 16a can determine whether it has been determined that the headlights 5 should perform additional illumination, i.e., whether the candidate object has already been additionally illuminated.

[0141] In addition, when the oversight estimation unit 15a outputs second target determination information including information indicating that a candidate object has not been detected by the candidate object detection unit 12, in other words, when the oversight estimation unit 15a has not performed either the first oversight estimation process or the second oversight estimation process, the illumination mode determination unit 16a determines to cause the headlights 5 to perform normal light distribution.

[0142] The illumination mode determination unit 16a outputs the illumination mode information to the light distribution control unit 17. In the second embodiment, the illumination mode information includes information indicating the illumination mode determined by the illumination mode determination unit 16a and oversight estimation information output from the oversight estimation unit 15a.

[0143] The light distribution control unit 17 controls the headlights 5 to emit light in the illumination mode determined by the illumination mode determination unit 16a based on the illumination mode information output from the illumination mode determination unit 16a. When canceling the additional illumination performed by the headlights 5, the light distribution control unit 17 may control the headlights 5 to gradually weaken the additional illumination. Specifically, the light distribution control unit 17 may include an instruction to gradually weaken the additional illumination in the additional illumination control information. The light distribution control unit 17 can appropriately set the degree to which the additional illumination is weakened. However, the light distribution control unit 17 weakens the additional illumination to an extent that does not bother the driver, in other words, reduces the amount of light of the additional illumination. Furthermore, when canceling the additional illumination of the headlights 5, the light distribution control unit 17 may control the headlights 5 to gradually narrow the range of the additional illumination. In this case, too, the light distribution control unit 17 narrows the range of the additional illumination to an extent that does not bother the driver.

[0144] The operation of the headlamp control device 1a according to the second embodiment will now be described. FIG. 10 is a flowchart for explaining the operation of the headlamp control device 1a according to the second embodiment. For example, when the automatic lights are turned on, the headlamp control device 1a starts the operation shown in the flowchart of FIG. 10, and repeats the operation shown in the flowchart of FIG. 10 until the automatic lights are turned off. The contents of the processing in steps ST1 to ST3 and step ST7 in FIG. 10 are the same as the contents of the processing in steps ST1 to ST3 and step ST7 in FIG. 3, which have already been described in the first embodiment, and therefore redundant description will be omitted.

[0145] The target determination unit 14a performs a first target determination process when a pedestrian is detected by the viewed object detection unit 11. The target determination unit 14a performs a second target determination process when a pedestrian is not detected by the viewed object detection unit 11 and a candidate object is detected by the candidate object detection unit 12 (step ST4a). The target determination unit 14a outputs the first target determination information or the second target determination information to the oversight estimation unit 15a. When a pedestrian is not detected by the viewed object detection unit 11 and a candidate object is not detected by the candidate object detection unit 12, the target determination unit 14 does not perform the first target determination process or the second target determination process, and simply outputs the viewed object presence / absence information to the oversight estimation unit 15a.

[0146] If a pedestrian is detected by the visual object detection unit 11 in step ST1, in other words, if visual object presence / absence information indicating "pedestrian present" is output from the target determination unit 14a, the oversight estimation unit 15a performs an oversight estimation process (first oversight estimation process) to estimate whether or not the driver has overlooked the pedestrian, based on the result of the first target determination process performed by the target determination unit 14a in step ST4 and the direction of the driver's gaze detected by the gaze detection unit 13 in step ST3. Furthermore, if the visual object detection unit 11 does not detect a pedestrian in step ST1 and the candidate object detection unit 12 detects a candidate object in step ST2, in other words, if the target determination unit 14a outputs second target determination information including information indicating that the candidate object has been detected by the candidate object detection unit 12, the oversight estimation unit 15a performs an oversight estimation process (second oversight estimation process) to estimate whether the driver has overlooked the candidate object based on the result of the second target determination process performed by the target determination unit 14a and the direction of the driver's line of sight detected by the line of sight detection unit 13 (step ST5a). The oversight estimation unit 15a outputs the oversight estimation information to the illumination mode determination unit 16a. If neither the first oversight estimation process nor the second oversight estimation process is performed, the oversight estimation unit 15a outputs to the illumination pattern determination unit 16a the second target determination information output from the target determination unit 14a in step ST4, the second target determination information including information indicating that the candidate object has not been detected by the candidate object detection unit 12.

[0147] The illumination mode determination unit 16a performs an illumination mode determination process to determine the illumination mode to be performed by the headlamp 5 (step ST6a). The illumination mode determination unit 16a outputs illumination mode information to the light distribution control unit 17.

[0148] FIG. 11 is a flowchart for explaining in detail an example of the oversight estimation process performed by the oversight estimation unit 15a in step ST5a of FIG. 10. Here, the pedestrian oversight estimation conditions are set to the above-described <Condition (1-1)> to <Condition (1-3)>, and the candidate object oversight estimation conditions are set to, for example, the above-described <Condition (3-1)> to <Condition (3-2)>. This example of the oversight estimation process performed by the oversight estimation unit 15a will be described in detail. However, the process steps ST11 to ST17 in FIG. 11 are the same as the process steps ST11 to ST17 in FIG. 4, which were already described in the first embodiment, and therefore will not be described again. The process steps ST11 to ST17 in FIG. 11 are the first oversight estimation process performed by the oversight estimation unit 15a.

[0149] If a pedestrian is not detected by the visual object detection unit 11 in step ST1 of FIG. 10 (if "NO" in step ST11), the oversight estimation unit 15a determines whether or not a candidate object has been detected by the candidate object detection unit 12 as a result of the target object determination unit 14a performing the second target object determination process in step ST4a of FIG. 10 (step ST101).

[0150] In step ST101, when the target object determination unit 14a performs the second target object determination process and determines that the candidate object has been detected by the candidate object detection unit 12 ("YES" in step ST101), the oversight estimation unit 15a determines whether the driver is currently visually observing the candidate object (step ST102). Specifically, the oversight estimation unit 15a performs the second object-based determination process and determines whether the driver's estimated visual recognition area is within the object-based visual detection range as a result of the second object-based determination process. If it is determined that the driver's estimated visual recognition area is within the object-based visual detection range, the oversight estimation unit 15a determines that the driver has visually observed the candidate object.

[0151] When it is determined in step ST102 that the driver has visually observed the candidate object ("YES" in step ST102), the oversight estimation unit 15a estimates that the driver has not overlooked the candidate object (step ST104). In other words, when it is determined that the driver has visually observed a candidate object that is currently located farther away than the distance that can be captured by the exterior camera 2, the oversight estimation unit 15a estimates that the driver has not overlooked the candidate object. Then, the oversight estimation unit 15a outputs oversight estimation information (more specifically, second oversight estimation information) including information indicating that the driver has not overlooked the candidate object to the illumination mode determination unit 16a.

[0152] On the other hand, if it is determined in step ST102 that the driver has not visually observed the candidate object ("NO" in step ST102), the oversight estimation unit 15a estimates that the driver has overlooked the candidate object (step ST103). In other words, in a current situation where a candidate object is present farther away than the distance that can be captured by the exterior camera 2, if it is determined that the driver has not visually observed the candidate object, the oversight estimation unit 15a estimates that the driver has overlooked the candidate object. Then, the oversight estimation unit 15a outputs oversight estimation information (more specifically, second oversight estimation information) including information indicating that the driver has overlooked the candidate object to the illumination mode determination unit 16a.

[0153] In step ST101, if the target determination unit 14a determines that the candidate object has not been detected by the candidate object detection unit 12 as a result of performing the second target determination process (if "NO" in step ST101), that is, if the target determination unit 14a outputs second target determination information including information indicating that the candidate object has not been detected by the candidate object detection unit 12, the oversight estimation unit 15a determines that driver oversight estimation, more specifically, the second oversight estimation process, is unnecessary and does not perform the second oversight estimation process (step ST105).The oversight estimation unit 15a then outputs the second target determination information including information indicating that the candidate object has not been detected by the candidate object detection unit 12, which is output from the target determination unit 14a, to the illumination mode determination unit 16a.

[0154] Fig. 12 is a flowchart for explaining in detail an example of the irradiation mode determination process by the irradiation mode determination unit 16a in step ST6a of Fig. 10. Note that here, the first irradiation mode determination condition or the second irradiation mode determination condition such as the above-mentioned <condition (4-1)> to <condition (4-8)> is set as the irradiation mode determination condition, and an example of the irradiation mode determination process by the irradiation mode determination unit 16a will be explained in detail. However, the contents of the process from step ST21 to step ST26 in Fig. 12 are the same as the contents of the process from step ST21 to step ST26 in Fig. 5 that have been explained in embodiment 1, so duplicated explanations will be omitted.

[0155] 10, the illumination mode determination unit 16a determines whether or not a pedestrian has been detected by the visual object detection unit 11 (step ST201). The illumination mode determination unit 16a acquires visual object presence / absence information from the target determination unit 14a via the oversight estimation unit 15a, and determines whether or not a pedestrian has been detected by the visual object detection unit 11 from the visual object presence / absence information.

[0156] 10, when it is determined that the visual object detection unit 11 has not detected a pedestrian (in the case of "NO" in step ST201), the illumination mode determination unit 16a determines whether the oversight estimation unit 15a has performed the oversight estimation process, more specifically, the second oversight estimation process (step ST202).When it is determined that the oversight estimation unit 15a has performed the second oversight estimation process (in the case of "YES" in step ST202), the illumination mode determination unit 16a determines whether the oversight estimation unit 15a has estimated that the driver has overlooked a candidate object (step ST203).

[0157] In step ST203, if the oversight estimation unit 15a estimates that the driver has overlooked the candidate object (if the answer is "YES" in step ST203), the illumination mode determination unit 16a determines whether the distance from the vehicle to the candidate object is greater than or equal to the first object distance threshold (step ST204).

[0158] If it is determined in step ST204 that the distance from the vehicle to the candidate object is equal to or greater than the first object distance threshold (if "YES" in step ST204), the illumination mode determination unit 16a determines to cause the headlights 5 to perform additional illumination (step ST205).

[0159] If it is determined in step ST204 that the distance from the vehicle to the candidate object is less than the first object distance threshold (if "NO" in step ST204), the illumination mode determination unit 16a determines whether the distance from the vehicle to the candidate object is less than the second object distance threshold (step ST206).

[0160] If it is determined in step ST206 that the distance from the vehicle to the candidate object is less than the second object distance threshold (if "YES" in step ST206), the illumination mode determination unit 16a decides to cancel the additional illumination that was being performed by the headlight 5 (step ST207).

[0161] If it is determined in step ST206 that the distance from the vehicle to the candidate object is equal to or greater than the second object distance threshold (if "NO" in step ST206), the illumination mode determination unit 16a determines to continue the additional illumination currently being performed by the headlight 5 (step ST208).

[0162] In step ST203, if the oversight estimation unit 15a estimates that the driver has not overlooked the candidate object (if the answer is "NO" in step ST203), the irradiation mode determination unit 16a determines whether additional irradiation has been performed on the candidate object (step ST209).

[0163] If it is determined in step ST209 that additional illumination has been performed on the candidate object ("YES" in step ST209), the illumination mode determination unit 16a decides to cancel the additional illumination performed by the headlight 5 (step ST210).

[0164] If the oversight estimation unit 15a determines that the oversight estimation process, more specifically, the second oversight estimation process, has not been performed (if "NO" in step ST202), or if it determines in step ST209 that additional illumination has not been applied to the candidate object (if "NO" in step ST209), the illumination mode determination unit 16a determines that the headlight 5 will perform normal light distribution (step ST211).

[0165] The illumination mode determination process by the illumination mode determination unit 16a described using the flowchart of Fig. 12 is merely an example. The illumination mode determination unit 16a determines the illumination mode to be performed by the headlights 5 in accordance with the illumination mode determination conditions. For example, if a first illumination mode determination condition or a second illumination mode determination condition other than the above-mentioned <Condition (4-1)> to <Condition (4-8)> is set in the illumination mode determination conditions, the illumination mode determination unit 16a determines the illumination mode to be performed by the headlights 5 in accordance with the first illumination mode determination condition or the second illumination mode determination condition. For example, if the second illumination mode determination condition is set to "If the driver has overlooked a candidate object as a result of performing the second oversight estimation process, perform emphasized illumination," the illumination mode determination unit 16a determines to cause the headlights 5 to perform emphasized illumination without performing the process of step ST204 in the flowchart of Fig. 12. Furthermore, for example, if the second illumination mode determination condition is set as follows: "If, as a result of performing the second oversight estimation process, it is found that the driver has overlooked the candidate object and the distance to the candidate object is equal to or greater than the first object distance threshold, additional illumination is performed; and if the driver has overlooked the candidate object and the distance to the candidate object is less than the first object distance threshold, additional illumination is canceled," the illumination mode determination unit 16a determines to cancel additional illumination without performing the processing of step ST206 in the flowchart of Figure 12.

[0166] In this way, when a pedestrian is not detected based on the exterior image, the headlamp control device 1a performs a second target object determination process to determine whether a candidate object has been detected, and estimates whether the driver has overlooked the candidate object based on the result of the second target object determination process and the direction of the driver's line of sight detected based on the interior image. When the headlamp control device 1a estimates that the driver has overlooked the candidate object, it controls the headlamp 5 to provide additional illumination to improve the visibility of the visual target area.

[0167] As a result, when a pedestrian is not detected in the image captured outside the vehicle but a candidate object is detected from the radar detection information in the visual target area, the headlight control device 1a can assist in detecting the pedestrian based on the image captured outside the vehicle. As a result, the headlight control device 1a can detect the pedestrian based on the image captured outside the vehicle early and can estimate early whether the driver has overlooked the pedestrian. Furthermore, even when a pedestrian is not detected in the image captured outside the vehicle but a candidate object is detected from the radar detection information in the visual target area, the headlight control device 1a does not control the headlights 5 to perform additional illumination if the driver has already visually confirmed the candidate object, thereby preventing the headlights 5 from performing unnecessary additional illumination.

[0168] In addition, if the headlight control device 1a estimates that the driver has overlooked a candidate object after controlling the headlight 5 to perform additional illumination and the distance to the candidate object becomes less than a second object distance threshold that is smaller than the first object distance threshold, the headlight control device 1a may control the headlight 5 to cancel the additional illumination that the headlight 5 is performing.

[0169] This allows the headlamp control device 1a to stop the additional illumination, which could be considered a waste of illumination, before the vehicle approaches and passes the candidate object and the candidate object is no longer detected from the radar detection information.

[0170] In the second embodiment, as in the first embodiment, the exterior camera 2 and the exterior radar 3 are merely examples. When the headlight control device 1a detects a pedestrian based on first exterior object detection information in which the position type detection device detects an object present around the vehicle, the headlight control device 1a may determine whether an object corresponding to the pedestrian has been detected to date based on second exterior object detection information in which the position detection device detects an object present around the vehicle. When a pedestrian is not detected based on the first exterior object detection information, the headlight control device 1a may perform a second target object determination process to determine whether a candidate object has been detected. Based on the result of the second target object determination process and the driver's line of sight detected based on the interior-captured image, the headlight control device 1a may estimate whether the driver has overlooked the candidate object. If it estimates that the driver has overlooked the candidate object, the headlight control device 1a may control the headlights 5 to provide additional illumination to improve the visibility of the visual target area.

[0171] In the second embodiment, the light distribution control unit 17 may have the function of the illumination mode determination unit 16a. In this case, the headlamp control device 1a may be configured without the illumination mode determination unit 16a.

[0172] In the second embodiment described above, the headlamp control device 1a is an on-board device mounted on a vehicle, and the on-board device is provided with the visual target detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target determination unit 14a, the oversight estimation unit 15a, the illumination mode determination unit 16a, and the light distribution control unit 17. This is merely an example, and for example, some of the visual target detection unit 11, the candidate object detection unit 12, the line of sight detection unit 13, the target determination unit 14a, the oversight estimation unit 15a, the illumination mode determination unit 16a, and the light distribution control unit 17 may be provided in the on-board device, and the rest may be provided in a server connected to the on-board device via a network. In addition, all of the visual target detection unit 11, candidate object detection unit 12, gaze detection unit 13, target determination unit 14a, oversight estimation unit 15a, illumination mode determination unit 16a, and light distribution control unit 17 may be provided on the server.

[0173] The hardware configuration of the headlamp control device 1a according to the second embodiment is the same as the configuration shown in Figures 6A and 6B in the first embodiment, and therefore a duplicated description will be omitted. In the second embodiment, the headlamp control device 1a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST1 to ST7 in Figure 10 described above.

[0174] As described above, in the headlight control device 1a according to the second embodiment, when the visual object detection unit 11 has not detected a visual object, the target object determination unit 14a performs a second target object determination process to determine whether the candidate object detection unit 12 has detected a candidate object. The oversight estimation unit 15a estimates whether the driver has overlooked a candidate object based on the result of the second target object determination process performed by the target object determination unit 14a and the direction of the driver's line of sight detected by the line of sight detection unit 13. When the oversight estimation unit 15a estimates that the driver has overlooked a candidate object, the light distribution control unit 17 controls the headlights 5 to perform additional illumination to improve the visibility of the visual target area. Therefore, when a pedestrian is not detected in the image captured outside the vehicle but a candidate object is detected from the radar detection information in the visual target area, the headlight control device 1a can assist in the detection of the visual target object based on the image captured outside the vehicle. As a result, the headlight control device 1a can quickly detect a visual target based on an outside-vehicle captured image and can quickly estimate whether the driver has overlooked the visual target. Furthermore, even if a visual target is not detected in the visual target area from the first outside-vehicle object detection information (e.g., an outside-vehicle captured image) but a candidate object is detected from the second outside-vehicle object detection information (e.g., radar detection information), if the driver has already visually observed the candidate object, the headlight control device 1a does not control the headlights 5 to perform additional illumination, thereby preventing the headlights 5 from performing unnecessary additional illumination.

[0175] More specifically, in the headlamp control device 1a, the light distribution control unit 17 controls the headlamp 5 to perform additional illumination when the oversight estimation unit 15a estimates that the driver has overlooked a candidate object and the distance to the candidate object is equal to or greater than a first object distance threshold. Therefore, when a candidate object is not detected in the visual target area from the first external object detection information (e.g., an external image captured by the vehicle) but is detected from the second external object detection information (e.g., radar detection information), the headlamp control device 1a can assist in the detection of the visual target based on the first external object detection information (e.g., an external image captured by the vehicle). As a result, the headlamp control device 1a can quickly detect the visual target based on the first external object detection information (e.g., an external image captured by the vehicle) and quickly estimate whether the driver has overlooked the visual target. Furthermore, even if no visual object is detected in the visual target area from the first external object detection information (e.g., an external image captured by the vehicle) but a candidate object is detected from the second external object detection information (e.g., radar detection information), if the driver has already visually observed the candidate object, the headlight control device 1a does not control the headlight 5 to perform additional illumination, thereby preventing unnecessary additional illumination from the headlight 5.

[0176] Furthermore, in the headlamp control device 1a, when the light distribution control unit 17 controls the headlamp 5 to perform additional illumination, the light distribution control unit 17 controls the headlamp 5 to gradually increase the additional illumination, thereby enabling the headlamp control device 1a to perform additional illumination while taking into consideration the inconvenience it causes to the driver.

[0177] Furthermore, in the headlamp control device 1a according to the second embodiment, after controlling the headlamp 5 to perform additional illumination, if the oversight estimation unit 15a estimates that the driver has overlooked a candidate object and the distance to the candidate object is less than a second object distance threshold that is smaller than the first object distance threshold, the light distribution control unit 17 controls the headlamp 5 to cancel the additional illumination. Therefore, the headlamp control device 1a can stop the additional illumination, which could be considered unnecessary illumination, before the vehicle approaches and passes the candidate object and the candidate object is no longer detected from the radar detection information.

[0178] Furthermore, in the headlamp control device 1a according to the second embodiment, when the light distribution control unit 17 controls the headlamp 5 to cancel the additional illumination, the light distribution control unit 17 controls the headlamp 5 to gradually weaken the additional illumination. This allows the headlamp control device 1a to cancel the additional illumination while taking into consideration the inconvenience it causes to the driver.

[0179] It should be noted that the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0180] The headlamp control device according to the present disclosure can control the illumination of the headlamp so that the driver does not overlook an object to be viewed, while taking into consideration the inconvenience to the driver.

[0181] 1, 1a Headlight control device, 11 Visual object detection unit, 12 Candidate object detection unit, 13 Line of sight detection unit, 14, 14a Target determination unit, 15, 15a Missed area estimation unit, 16, 16a Illumination mode determination unit, 17 Light distribution control unit, 2 Exterior camera, 3 Exterior radar, 4 Interior camera, 5 Headlight, 1001 Processing circuit, 1002 Input interface device, 1003 Output interface device, 1004 Processor, 1005 Memory.

Claims

1. A visual object detection unit that detects a visual object that the driver should view in a visual object area that the driver should view, based on first external object detection information in which a position type detection device capable of detecting the position and type of an object that exists in the vicinity of the vehicle has detected the object that exists in the vicinity of the vehicle; a candidate object detection unit that detects a candidate object that is a candidate for the visual object in the visual object area, based on second external object detection information in which a position detection device capable of detecting the position of the object that exists in the vicinity of the vehicle has detected the object that exists in the vicinity of the vehicle; a gaze detection unit that detects the direction of the driver's gaze, based on an in-vehicle image of the driver captured by an in-vehicle camera; and a target determination unit that, when the visual object is detected by the visual object detection unit, performs a first target determination process to determine whether an object corresponding to the detected visual object has been detected as the candidate object by the candidate object detection unit. a headlamp control device comprising: an oversight estimation unit that estimates whether or not the driver has overlooked the visual object based on a result of the first target object determination process performed by the target object determination unit and the direction of the driver's line of sight detected by the line of sight detection unit; and a light distribution control unit that controls the headlamp to provide emphasized illumination to make the driver aware of the visual object when the oversight estimation unit estimates that the driver has overlooked the visual object.

2. The headlamp control device described in claim 1, characterized in that the oversight estimation unit estimates that the driver has not overlooked the visual object if it is determined in the first target determination process that an object corresponding to the visual object has been detected as the candidate object, and if it is determined that the driver has already visually observed the candidate object based on the direction of the driver's gaze detected by the gaze detection unit and the candidate object detection information regarding the candidate object detected by the candidate object detection unit.

3. A headlight control device as described in claim 1 or claim 2, characterized in that the light distribution control unit controls the headlight to perform the accentuated illumination when the oversight estimation unit estimates that the driver has overlooked the visual object and when the distance from the vehicle to the visual object is less than a distance determination threshold.

4. The headlight control device according to claim 3, characterized in that even if the oversight estimation unit estimates that the driver has overlooked the visual object, if the distance from the vehicle to the visual object is equal to or greater than the distance determination threshold, the light distribution control unit controls the headlight not to perform the accentuated illumination, but to perform additional illumination to improve the visibility of the visual object area.

5. A headlamp control device as described in claim 1, characterized in that the target object determination unit performs a second target object determination process to determine whether or not the candidate object has been detected by the candidate object detection unit when the visual target object has not been detected by the visual target detection unit, the oversight estimation unit estimates whether or not the driver has overlooked the candidate object based on the result of the second target object determination process performed by the target object determination unit and the direction of the driver's line of sight detected by the line of sight detection unit, and the light distribution control unit controls the headlamp to provide additional illumination to improve visibility of the visual target area when the oversight estimation unit estimates that the driver has overlooked the candidate object.

6. The headlight control device according to claim 5, characterized in that the light distribution control unit controls the headlight to perform the additional illumination when the oversight estimation unit estimates that the driver has overlooked the candidate object and the distance to the candidate object is equal to or greater than a first object distance threshold.

7. A headlamp control device according to claim 5 or 6, characterized in that, when controlling the headlamp to provide the additional illumination, the light distribution control unit controls the headlamp to gradually increase the additional illumination.

8. The headlight control device of claim 6, characterized in that, after controlling the headlights to perform the additional illumination, if the oversight estimation unit estimates that the driver has overlooked the candidate object and the distance to the candidate object becomes less than a second object distance threshold that is smaller than the first object distance threshold, the light distribution control unit controls the headlights to cancel the additional illumination.

9. The headlamp control device according to claim 8, characterized in that, when controlling the headlamp to cancel the additional illumination, the light distribution control unit controls the headlamp to gradually weaken the additional illumination.

10. A headlamp control device as described in claim 1, characterized in that the position type detection device is an exterior camera, the first exterior object detection information is an exterior image of the periphery of the vehicle captured by the exterior camera, the visual object detection unit detects the visual object in the visual target area based on the exterior image, the position detection device is an exterior radar, the second exterior object detection information is radar detection information of the object in the periphery of the vehicle detected by the exterior radar, and the candidate object detection unit detects the candidate object in the visual target area based on the radar detection information.

11. A step in which a visual object detection unit detects a visual object that the driver should view in a visual object area that the driver should view, based on first external object detection information in which a position type detection device capable of detecting the position and type of an object present in the vicinity of the vehicle has detected the object present in the vicinity of the vehicle; a step in which a candidate object detection unit detects a candidate object that is a candidate for the visual object in the visual object area, based on second external object detection information in which a position detection device capable of detecting the position of the object present in the vicinity of the vehicle has detected the object present in the vicinity of the vehicle; a step in which a gaze detection unit detects the direction of the driver's gaze based on an in-vehicle image of the driver captured by an in-vehicle camera; and a step in which a target determination unit performs a first target determination process in which, when the visual object is detected by the visual object detection unit, a target determination unit determines whether an object corresponding to the detected visual object has been detected as the candidate object by the candidate object detection unit. a step in which an oversight estimation unit estimates whether or not the driver has overlooked the visual object based on a result of the first target object determination process performed by the target object determination unit and the direction of the driver's line of sight detected by the line of sight detection unit; and a step in which a light distribution control unit controls the headlights to perform accentuated illumination to make the driver aware of the visual object when the oversight estimation unit estimates that the driver has overlooked the visual object.

Images