Driving support device and driving support method

Abstract

To suppress an erroneous alarm.SOLUTION: A driving support device 3 comprises: an acquisition section 321 for acquiring an infrared image picked up by irradiating with an infrared ray an imaging region including a driver seat where a driver 5 of a vehicle is seated; a determination section 322 for determining whether or not white-out occurs in the infrared image based on whether or not a pixel value of a pixel in a determination portion where a determination region exposed to sunlight is imaged in the imaging region in the infrared image is equal to or more than a predetermined value in a case where the driver cannot be recognized from the infrared image; and a setting section 323 by which, in a case where it is determined that white-out occurs in the infrared image, a standby time from a state where the driver cannot be recognized from the infrared image to a notification of an alarm is set longer than a reference time.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a driving support device and a driving support method for supporting the driving of a vehicle.

Background Art

[0002] Techniques for determining whether a vehicle driver is abnormal are known. Patent Document 1 discloses a technique for determining whether the state of a driver recognized from an infrared image captured by irradiating an infrared ray within a shooting range of a camera is abnormal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when sunlight hits the driver's seat, white spots may occur in the infrared image, making it impossible to recognize the driver. When the driver cannot be recognized, even if the driver is driving normally, the driver may be determined to be abnormal and an alarm may be notified.

[0005] Therefore, the present invention has been made in view of these points, and an object thereof is to suppress false alarms.

Means for Solving the Problems

[0006] In a first aspect of the present invention, there is provided a driving support device including: an acquisition unit that acquires an infrared image obtained by irradiating infrared rays and imaging in a first area including a driver's seat where a driver of a vehicle is seated; and a determination unit that determines whether whiteout has occurred in the infrared image based on whether a pixel value of a pixel in a determination portion of the first area irradiated with sunlight in the infrared image is equal to or greater than a predetermined value when the driver cannot be recognized from the infrared image, and a setting unit that, when it is determined that whiteout has occurred in the infrared image, makes a waiting time from when the driver cannot be recognized from the infrared image until an alarm is notified longer than a reference time.

[0007] When the determination unit determines that the pixel value of the pixel in the determination portion is equal to or greater than the predetermined value, the acquisition unit acquires an imaging image obtained by imaging without irradiating the infrared rays. The determination unit may determine that whiteout has occurred in the infrared image when the pixel value of the pixel in the determination portion is equal to or greater than the predetermined value and the driver can be recognized from the imaging image.

[0008] The determination unit may determine whether whiteout has occurred in the infrared image when a time at which the infrared image is captured is included in a time zone when sunlight hits the second area.

[0009] The acquisition unit may acquire the infrared image captured after an exposure correction in which an imaging device that captures the infrared image changes at least one of an aperture value and an exposure time.

[0010] When the determination unit determines that the pixel value of the pixel in the determination portion of the infrared image is less than the predetermined value, the determination unit determines that whiteout has not occurred in the infrared image. When the determination unit determines that whiteout has not occurred in the infrared image, the setting unit may set the waiting time to the reference time.

[0011] When the statistic of the pixel values of a plurality of pixels included in the determination part is greater than or equal to the predetermined value, the determination part may determine that white noise has occurred in the infrared image, and when the statistic is less than the predetermined value, the determination part may determine that no white noise has occurred in the infrared image.

[0012] The waiting time may be an abnormality determination time from when the driver cannot be recognized from the infrared image until an abnormality in which the driver deviates from a normal driving posture occurs.

[0013] It has a notification control part for notifying the warning, the acquisition part acquires operation information related to the driving operation of the driver, the determination part determines whether the driving operation of the operation information is abnormal, and the notification control part determines that the driving operation is abnormal, and when the time elapsed since the driver cannot be recognized from the infrared image is equal to or longer than the waiting time set by the setting part, the warning may be notified.

[0014] In a second aspect of the present invention, there is provided a driving support method including: a step of acquiring an infrared image obtained by irradiating infrared rays and imaging a first area including a driver's seat on which the driver of the vehicle is seated, which is executed by a computer of the vehicle; a step of determining whether white noise has occurred in the infrared image based on whether the pixel value of a pixel of a determination part in which a second area irradiated with sunlight in the first area in the infrared image is imaged is greater than or equal to a predetermined value when the driver cannot be recognized from the infrared image; and a step of making a waiting time from when the driver cannot be recognized from the infrared image until a warning is notified longer than a reference time when it is determined that the white noise has occurred in the infrared image.

Advantages of the Invention

[0015] According to the present invention, there is an effect that false alarms can be suppressed.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0017] [Configuration of Driving Support System S] The driving support system S supports the driving of a driver 5 who drives a vehicle. The configuration of the driving support system S will be described with reference to FIGS. 1 and 2. FIG. 1 is a diagram for explaining the configuration of the driving support system S. FIG. 2 is a diagram for explaining the imaging area 21 of the imaging device 2. The driving support system S includes an infrared light 1, an imaging device 2, a driving support device 3, and an output unit 4.

[0018] The infrared light 1 irradiates infrared rays 11 on a first area that is the imaging area 21 including the driver's seat where the driver 5 of the vehicle is seated. The infrared light 1 irradiates the infrared rays 11 on an area that is wider than the imaging area 21 and includes the entire imaging area 21 so that the entire imaging area 21 can be irradiated with the infrared rays 11. The infrared light 1 is provided at a position where it can irradiate the imaging area 21 with the infrared rays 11. For example, the infrared light 1 is provided on the dashboard of the vehicle, but the installation position of the infrared light 1 may be any position as long as it can irradiate the imaging area 21 with the infrared rays 11.

[0019] The imaging device 2 is provided at a position where it can image the imaging area 21 inside the vehicle compartment of the vehicle. The imaging device 2 is provided, for example, on the dashboard of the vehicle in front of the driver's seat and images the imaging area 21. Taking a specific example, the imaging device 2 is provided between the windshield and the steering wheel on the dashboard. Note that the installation position of the imaging device 2 is not limited to this and may be any position as long as it can image the imaging area 21.

[0020] The imaging device 2 captures an imaging area 21 to generate an image. For example, the imaging device 2 captures the imaging area 21 while the infrared light 1 irradiates the infrared rays 11 onto the imaging area 21 to generate an infrared image 71. FIG. 3 is a schematic diagram of the infrared image 71 captured by the imaging device 2. Specifically, the imaging device 2 captures the imaging area 21 to generate an infrared image 71 after performing exposure correction in a state where the infrared rays 11 are irradiated. More specifically, the imaging device 2 performs exposure correction to change at least one of the aperture value and the exposure time so that the gradation of the infrared image captured of the imaging area 21 becomes appropriate, and then captures the imaging area 21 to generate an infrared image 71.

[0021] The driving assistance device 3 analyzes the infrared image 71 to extract features such as the contour of the face of the driver 5, the positions of the eyes, nose, and mouth, and recognizes the driver 5 from the infrared image 71. However, when the posture of the driver 5 with an abnormality collapses and is no longer in a normal driving posture, the driver 5 deviates from the imaging area 21. In the following description, an abnormality in which the driver 5 deviates from the normal driving posture is referred to as a posture collapse.

[0022] Posture collapses include, for example, lying prone, bowing the head, falling backward, arching the back, tilting only the neck to the side, tilting to the side, and slumping to the side. Lying prone is a state where the driver falls forward and the driver's face is near the steering wheel. Bowing the head is a state where the driver's face is facing downward. Falling backward is a state where the upper body of the driver is tilted backward and the face is facing upward. Arching the back is a state where the upper body of the driver arches upward and the face is facing upward. Tilting only the neck to the side is a state where the driver's head is tilted to the left or right. Tilting to the side is a state where the upper body of the driver is tilted to the left or right and the face is tilted in the same direction as the tilt of the upper body. Slumping to the side is a state where the upper body of the driver is tilted to the left or right.

[0023] When the driving support device 3 cannot extract the features of the face of the driver 5 from the infrared image 71 because the driver 5 has a slouch as exemplified above, the driver 5 cannot be recognized from the infrared image 71. When the driving support device 3 cannot recognize the driver 5 from the infrared image 71, it determines that an abnormality of slouch has occurred to the driver 5. Specifically, when the state in which the abnormality of slouch in which the driver 5 cannot be recognized from the infrared image 71 continues, the driving support device 3 determines that a slouch has occurred and causes the output unit 4 to issue an alarm. The output unit 4 includes a speaker or a buzzer that issues an alarm. When there is a slouch in the driver 5, the driving support device 3 controls the output unit 4 to cause a message notifying of the abnormality to be notified.

[0024] By the way, sunlight 6 may hit a part of the imaging area 21 (see FIG. 2). The area of the imaging area 21 that sunlight 6 hits becomes brighter than the area of the imaging area 21 that sunlight 6 does not hit. For example, when the sun exists in the front or the left front of the vehicle, the sunlight 6 from the windshield or the window of the door of the vehicle hits the determination area 22 which is the second area, so the determination area 22 becomes brighter. Since the distance between the determination area 22 and the imaging device 2 is shorter than the distance between the imaging device 2 and the area other than the determination area 22 in the imaging area 21, when sunlight 6 hits the determination area 22, the determination part where the determination area 22 in the infrared image is imaged becomes too bright. In this case, even if the imaging device 2 performs exposure correction so that the gradation of the determination part is not lost, the gradation of the infrared image cannot be made appropriate, and clipping may occur in the infrared image.

[0025] Clipping means white bleeding in which the pixel value of the pixel of the infrared image is saturated and the gradation of the infrared image is lost, and black crush in which the pixel value of the pixel is too small due to insufficient exposure amount and the gradation of the infrared image is lost. For example, when the determination area 22 is too bright, even if the aperture value and the exposure time are reduced, the pixel value of the pixel of the captured infrared image is saturated and white bleeding occurs. Further, as a result of reducing the aperture value and the exposure time so that the gradation of the determination area 22 is not lost, black crush may occur due to insufficient exposure amount of the part of the imaging area 21 excluding the determination area 22.

[0026] FIG. 4 is a diagram for explaining an infrared image 72 in which white blooming has occurred. A determination portion 73 of the infrared image 72 is a portion where the determination area 22 has been imaged. In the determination portion 73 of the infrared image 72 in FIG. 4, white blooming has occurred and the gradation of the determination portion 73 has been lost. Also, in the portion of the infrared image 72 excluding the determination portion 73, the exposure amount is insufficient and the gradation has been lost, resulting in black crush. In this case, since features such as the contour line of the driver 5's chin cannot be extracted, the driver 5 cannot be recognized from the infrared image 72. When the driver 5 cannot be recognized from the infrared image 72, there is a possibility that an alarm may be notified even though the driver 5 is driving in a normal driving posture, but it is determined that the driver 5 has lost his / her posture.

[0027] As described above, when white blooming occurs in the infrared image, there is a risk that an alarm may be notified even though the driver 5 is driving in a normal driving posture. However, whether or not the sunlight 6 hits the determination area 22 changes due to the influence of the position of the sun and surrounding buildings relative to the vehicle. And if the position of the sun and surrounding buildings relative to the vehicle changes before the alarm is notified and the sunlight 6 no longer hits the determination area 22, the imaging device 2 can perform exposure correction so that the gradation of the infrared image is not lost, and thus the white blooming of the infrared image is eliminated. In this case, since the driver 5 can be recognized from the infrared image, the alarm is not notified.

[0028] Therefore, when the driver 5 cannot be recognized and white blooming or black crush has occurred in the infrared image, the driving support device 3 lengthens the time until an alarm is notified. When the position of the sun and surrounding buildings relative to the vehicle changes before the alarm is notified and the sunlight 6 no longer hits the determination area 22, the white blooming of the infrared image is eliminated. When the white blooming of the infrared image is eliminated, the driver 5 can be recognized, so the posture collapse alarm is not notified. That is, the driving support device 3 suppresses false alarms by lengthening the time until an alarm is notified when white blooming occurs in the infrared image. Hereinafter, the processing executed by the driving support device 3 will be specifically described.

[0029] The driving support device 3 includes a storage unit 31 and a control unit 32. The storage unit 31 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk, and the like. The storage unit 31 stores programs executed by the control unit 32.

[0030] The control unit 32 is a computing resource including a processor such as a CPU (Central Processing Unit). By executing the program stored in the storage unit 31, the control unit 32 realizes functions as an acquisition unit 321, a determination unit 322, a setting unit 323, and a notification control unit 324.

[0031] The acquisition unit 321 acquires an infrared image captured in a state where the imaging area 21 is irradiated with infrared rays 11. Specifically, the acquisition unit 321 acquires an infrared image captured after the imaging device 2 performs exposure correction. Further, the acquisition unit 321 acquires the imaging time at which the infrared image is captured from the imaging device 2. Specifically, the acquisition unit 321 acquires image information associated with the imaging time in the infrared image from the imaging device 2.

[0032] The determination unit 322 determines whether the driver 5 can be recognized from the infrared image. When the determination unit 322 can extract the features of the face of the driver 5 by analyzing the infrared image, the determination unit 322 determines that the driver 5 can be recognized from the infrared image. When the determination unit 322 cannot extract the features of the face of the driver 5 even by analyzing the infrared image, the determination unit 322 determines that the driver 5 cannot be recognized from the infrared image. When the driver 5 cannot be recognized, the determination unit 322 determines whether there is blooming in the infrared image.

[0033] Incidentally, the time zones when sunlight 6 hits the determination area 22 and those when it does not are fixed. The sunlight 6 in the first time zone from sunrise until a predetermined time has elapsed and in the second time zone from a predetermined time before sunset until sunset hits the determination area 22. However, the sunlight 6 in the time zone including the time when the sun is at its zenith does not hit the determination area 22 because it is blocked by the ceiling of the passenger compartment. Also, in the time zone from after sunset until before sunrise, the sunlight 6 does not hit the determination area 22. Thus, in the time zones when the sunlight 6 does not hit the determination area 22, since the probability of white streaks occurring in the infrared image is low, it is not necessary to determine whether white streaks have occurred.

[0034] Therefore, the determination unit 322 determines whether the imaging time when the infrared image was captured is included in the time zone when the sunlight 6 hits the determination area 22. The time zones when the sunlight 6 hits are, for example, the first time zone from sunrise until a predetermined time has elapsed and the second time zone from a predetermined time before sunset until sunset. The shorter the time from sunrise to sunset on the day when the infrared image was captured, the longer the determination unit 322 sets the predetermined times for the first and second time zones. The specific value of the predetermined time is, for example, 2 hours or 3 hours, but is not limited thereto. By way of example, when the sunrise time is 6:45 and the sunset time is 16:30, the determination unit 322 sets the first time zone from 6:45 to 9:45 and the second time zone from 14:30 to 16:30.

[0035] When the imaging time is included in either the first time period or the second time period, the determination unit 322 determines whether there is white blooming in the infrared image. When the imaging time is not included in either the first time period or the second time period, the determination unit 322 does not determine whether there is white blooming in the infrared image. In other words, when the imaging time is included in the third time period, which is all time periods other than the first time period and the second time period in a day, the determination unit 322 does not determine whether there is white blooming in the infrared image. Thus, the determination unit 322 determines whether there is white blooming in a time period when the probability of white blooming in the infrared image is high, and does not determine whether there is white blooming when the probability of white blooming is low. As a result, the determination unit 322 can suppress unnecessary determinations, and thus can suppress waste of computing resources.

[0036] The determination unit 322 determines whether there is white blooming in the infrared image based on whether the pixel value of the pixel in the determination portion 73 is greater than or equal to a predetermined value. The predetermined value is the maximum value of a processable signal, but may also be a value lower than the maximum value. For example, the predetermined value is a value that is 90% of the maximum value of a processable signal. The determination unit 322 determines that there is white blooming in the infrared image when the pixel value of the pixel in the determination portion 73 is greater than or equal to the predetermined value, and determines that there is no white blooming in the infrared image when the pixel value of the pixel in the determination portion 73 is less than the predetermined value.

[0037] The determination unit 322 may also determine whether there is white blooming in the infrared image based on whether the statistic of the pixel values of a plurality of pixels included in the determination portion 73 is greater than or equal to a predetermined value. The statistic is any one of the average value, median value, sum, maximum value, and minimum value of the pixel values of a plurality of pixels included in the determination portion 73. The statistic is preferably the average value, median value, or maximum value. Taking a specific example, when the maximum value of the pixel values of the pixels in the determination portion 73 is less than the predetermined value, it can be said that the pixel values of the pixels are not saturated and the gradation of the determination portion 73 is not lost, so the determination unit 322 determines that there is no white blooming in the infrared image.

[0038] When the statistic of the pixel values of the pixels in the determination part 73 is greater than or equal to a predetermined value, the determination unit 322 determines that white flashing has occurred in the infrared image. For example, when the average value, median value, or minimum value of the pixel values of the pixels in the determination part 73 is greater than or equal to a predetermined value, the determination unit 322 determines that white flashing has occurred in the infrared image. Specifically, when the minimum value of the pixel values of the pixels in the determination part 73 is greater than or equal to a predetermined value, it is considered that the pixel values of the pixels are saturated and the gradation of the determination part 73 is lost, so the determination unit 322 determines that white flashing has occurred in the infrared image.

[0039] When the determination unit 322 determines that white flashing has occurred in the infrared image, the accuracy of the determination can be improved by re-determining whether white flashing has occurred in the infrared image based on the captured image captured in a state where no infrared ray is irradiated. When it is determined that white flashing has occurred in the infrared image, the acquisition unit 321 acquires a captured image captured without irradiating the infrared ray 11. Specifically, when it is determined that white flashing has occurred in the infrared image, after the irradiation of the infrared ray 11 to the infrared light 1 is stopped, the imaging device 2 is caused to generate a captured image, and the captured image generated by the imaging device 2 is acquired. Thereby, the acquisition unit 321 can acquire a captured image captured in a state where the infrared ray 11 is not irradiated to the imaging region 21 and the sunlight 6 is irradiated to the imaging region 21.

[0040] The determination unit 322 determines whether the driver 5 can be recognized from the captured image captured by the imaging device 2 when the infrared light has stopped irradiating the infrared ray 11. By using the captured image in which the infrared ray 11 is not irradiated and the sunlight 6 is irradiated, the determination unit 322 can determine whether the driver 5 can be recognized in a state where there is no influence of the infrared ray 11 on the imaging area 21 and there is an influence of the sunlight 6. When the determination unit 322 can recognize the driver 5 from the captured image, it re-determines that there is white blooming in the infrared image. Specifically, when the determination unit 322 determines that there is white blooming in the infrared image and can recognize the driver 5 from the captured image, the determination that there is white blooming in the infrared image continues. In this way, when there is white blooming in the infrared image, the determination unit 322 recognizes the driver 5 from the captured image without the irradiation of the infrared ray, thereby increasing the accuracy of the determination that white blooming has occurred in the infrared image when the driver 5 is in a normal driving posture.

[0041] When the determination unit 322 cannot recognize the driver 5 from the captured image, it determines that there is an abnormality in the posture collapse of the driver 5. Specifically, the determination unit 322 cancels the determination that there is white blooming in the infrared image and determines that there is an abnormality in the posture collapse of the driver 5. When there is white blooming in the infrared image, the determination unit 322 can appropriately determine whether there is an abnormality in the posture collapse of the driver 5 based on whether the driver 5 can be recognized from the captured image.

[0042] The setting unit 323 sets the waiting time from the state where the driver 5 cannot be recognized until the start of a predetermined control. The predetermined control is, for example, a notification control for notifying an alarm or an automatic stop control for stopping the vehicle. The setting unit 323 sets the waiting time from the state where the driver 5 cannot be recognized from the infrared image until the start of the notification control. Further, the setting unit 323 may set the waiting time from the state where the driver 5 cannot be recognized until the start of the automatic stop control. Specifically, the setting unit 323 sets the waiting time from the moment when the notification control unit 324 is caused to notify an alarm after the state where the driver 5 cannot be recognized until the start of the automatic stop control.

[0043] When it is determined that white noise has occurred in the infrared image, the setting unit 323 makes the standby time longer than the reference time. The standby time is the abnormal determination time from when the driver 5 cannot be recognized in the infrared image until it is determined that a slouch has occurred. The reference time is the reference time for the abnormal determination time, for example, 10 seconds, but is not limited thereto. For a specific example, when white noise has occurred in the infrared image, the setting unit 323 sets the standby time to 15 seconds, which is longer than the reference time of 10 seconds.

[0044] When no white noise has occurred in the infrared image, the setting unit 323 sets the standby time to the reference time. After it is determined that white noise has occurred in the infrared image, when it is determined that no white noise has occurred in the infrared image, that is, when the white noise in the infrared image has disappeared, the standby time set longer than the reference time is returned to the reference time. Thereby, the setting unit 323 can appropriately set the standby time according to whether white noise has occurred in the infrared image.

[0045] Also, when the determination unit 322 cancels the determination that white noise has occurred in the infrared image and determines that there is an abnormality in the slouch of the driver 5, the setting unit 323 may make the standby time shorter than the reference time. In this case, the setting unit 323 sets the standby time to 5 seconds, which is shorter than the reference time of 10 seconds. By doing so, the setting unit 323 can cause the alarm to be notified earlier when there is an abnormality in the driver 5.

[0046] When the elapsed time since the driver 5 cannot be recognized from the infrared image is equal to or longer than the standby time set by the setting unit 323, the notification control unit 324 causes the output unit 4 to notify an alarm. Specifically, when the elapsed time becomes equal to or longer than the standby time, the notification control unit 324 notifies a message indicating an abnormality as an alarm. The message indicating an abnormality is, for example, a message such as "Abnormal driving posture. Please drive in a correct driving posture.", but is not limited thereto. When the driver 5 performs an operation to stop the alarm, the notification control unit 324 stops the alarm.

[0047] If the notification control unit 324 can recognize the driver 5 from the infrared image before the elapsed time reaches or exceeds the standby time, it will not cause the output unit 4 to issue an alarm. Specifically, if the determination unit 322 determines that there is no whiteout in the infrared image before the elapsed time reaches or exceeds the standby time, the notification control unit 324 will set the elapsed time to zero and will not cause the output unit 4 to issue an alarm again until the elapsed time reaches or exceeds the standby time.

[0048] As described above, the notification control unit 324 will issue an alarm when the elapsed time since the driver 5 cannot be recognized from the infrared image reaches or exceeds the standby time set by the setting unit 323. That is, when there is whiteout in the infrared image, since the standby time is set longer than the reference time, the time from when the driver 5 cannot be recognized from the infrared image until the alarm is issued becomes longer. If the position of the sun relative to the vehicle or the buildings around the vehicle changes during the standby time until the alarm is issued and the sunlight 6 no longer hits the determination area 22, the whiteout in the infrared image will be eliminated. When the whiteout in the infrared image is eliminated, the driver 5 can be recognized, so the notification control unit 324 will not cause the output unit 4 to issue an alarm. In this way, the driving support device 3 can suppress false alarms by increasing the time until the alarm is issued when there is whiteout in the infrared image.

[0049] Even if there is whiteout in the infrared image, if something abnormal has occurred to the driver 5, it is desirable to issue an alarm promptly. Therefore, the driving support device 3 will issue an alarm if there is an abnormality in the driving operation of the driver 5. Hereinafter, the process of issuing an alarm when there is an abnormality in the driving operation will be described.

[0050] The acquisition unit 321 acquires operation information regarding the driving operation of the driver 5. The driving operation is, for example, at least one of a steering operation, an accelerator operation, and a brake operation. The acquisition unit 321 sequentially acquires the detection values detected by each of a sensor that detects the rotation angle of the steering wheel, a sensor that detects the depression amount of the accelerator pedal, and a sensor that detects the depression amount of the brake pedal as operation information.

[0051] Next, the determination unit 322 determines whether the driving operation of the operation information is abnormal. For example, if the driving operation of the operation information acquired from a time point a predetermined time before the current time to the current time is different from the normal driving operation of the driver 5, the determination unit 322 determines that the driving operation of the acquired operation information is abnormal. Specifically, the determination unit 322 determines a similarity indicating the degree of similarity between the driving operation of the acquired operation information and the normal driving operation of the driver 5, and if the determined similarity is outside the allowable range that can be tolerated as the normal driving operation of the driver 5, the determination unit 322 determines that the driving operation of the acquired operation information is abnormal.

[0052] When the driving operation is abnormal, the notification control unit 324 causes the output unit 4 to notify an alarm. Specifically, the setting unit 323 causes the output unit 4 to notify an alarm when it is determined that the driving operation is abnormal and the time elapsed since the driver 5 cannot be recognized from the infrared image is equal to or longer than the standby time. Thereby, even if the driver 5 cannot be recognized from the infrared image, the notification control unit 324 can notify an alarm when the probability that an abnormality has occurred to the driver 5 is high.

[0053] [Processing for setting standby time] FIG. 5 is a flowchart showing an example of the processing for setting the standby time. The processing for setting the standby time is executed at a predetermined interval while the vehicle engine is operating. The predetermined interval is, for example, 100 milliseconds, but is not limited thereto.

[0054] The acquisition unit 321 acquires an infrared image (step S1). Specifically, the acquisition unit 321 acquires the infrared image captured after exposure correction and the imaging time at which the captured image was captured from the imaging device 2.

[0055] The determination unit 322 determines whether an infrared image has been captured during the time period when the sunlight 6 hits the determination area 22 (step S2). For example, when the imaging time at which the infrared rays were captured is included in a predetermined time period, the determination unit 322 determines that an infrared image has been captured during the time period when the sunlight 6 hits the determination area 22. The determination unit 322 determines that an infrared image has been captured during the time period when the sunlight 6 hits the determination area 22 when the imaging time is included in the first time period from the time after a predetermined time has elapsed since sunrise to the time of sunset, or when the imaging time is included in the second time period from the time a predetermined time before sunset to sunset. When the imaging time is included in a time period that is not the first time period or the second time period, the determination unit 322 determines that an infrared image has been captured during the time period when the sunlight 6 does not hit the determination area 22.

[0056] When the determination unit 322 determines that an infrared image has been captured during the time period when the sunlight 6 hits the determination area 22 (Yes in step S2), the determination unit 322 determines whether the pixel values of the pixels of the determination part 73 that captured the determination area 22 are equal to or greater than a predetermined value (step S3). For example, the determination unit 322 determines whether the statistical quantity of the pixel values of a plurality of pixels included in the determination part 73 is equal to or greater than a predetermined value. Specifically, the determination unit 322 determines whether the average value of the pixel values of the plurality of pixels is equal to or greater than a predetermined value.

[0057] When it is determined that the pixel value of a pixel is equal to or greater than a predetermined value (Yes in step S3), the acquisition unit 321 acquires the captured image captured without irradiating the infrared rays 11 (step S4). Specifically, the acquisition unit 321 stops the irradiation of the infrared rays 11 on the infrared light 1 and causes the imaging device 2 to image the imaging area 21 to generate a captured image.

[0058] The determination unit 322 determines whether the driver 5 can be recognized from the captured image (step S5). When the determination unit 322 cannot extract the facial features of the driver 5 from the captured image, the determination unit 322 determines that the driver 5 cannot be recognized from the captured image. When the determination unit 322 can extract the facial features of the driver 5 from the captured image, the determination unit 322 determines that the driver 5 can be recognized from the captured image.

[0059] When the setting unit 323 can recognize the driver 5 from the captured image (Yes in step S5), it makes the standby time longer than the reference time (step S6). Specifically, when there is white noise in the infrared image and the driver 5 can be recognized from the captured image, the setting unit 323 also makes the standby time longer than the reference time.

[0060] When it is determined that the infrared image has been captured during a time period when the sunlight 6 does not hit the determination area 22 (No in step S2), the setting unit 323 sets the standby time to the reference time (step S7). Also, when the pixel value of the pixels in the determination part 73 of the infrared image is less than a predetermined value (No in step S3), or when the driver 5 cannot be recognized from the captured image (No in step S5), the setting unit 323 sets the standby time to the reference time (step S7).

[0061] Note that the clipping of the infrared image may occur not only when the sunlight 6 hits the determination area 22 but also when the hand of the driver 5 approaches the imaging device 2. When the hand of the driver 5 approaches the imaging device 2, the light reflected by the hand of the driver 5 from the infrared light 11 of the infrared light source 1 enters the imaging device 2. In this case, as a result of the imaging device 2 performing exposure correction so that the gradation of the hand of the driver 5 is not lost, the exposure amount of the area other than the hand of the driver 5 becomes small and blackout where the gradation is lost occurs. Even in such a case, the driving support device 3 according to the present embodiment can suppress false alarms by making the standby time longer.

[0062] [Effect of the driving support device 3] As described above, when the driving support device 3 cannot recognize the driver 5 from the infrared image captured by irradiating the infrared light 11 on the imaging area 21 including the driver's seat where the driver 5 of the vehicle is seated, it determines whether there is white noise in the infrared image based on whether the pixel value of the pixels in the determination part 73 of the imaging area 21 in the infrared image where the sunlight 6 hits is greater than or equal to a predetermined value. Then, when the driving support device 3 determines that there is white noise in the infrared image, it makes the standby time from the state where the driver 5 cannot be recognized from the infrared image until the alarm is notified longer than the reference time.

[0063] When there is white blooming in the infrared image, since the standby time is set longer than the reference time, the time from when the driver 5 cannot be recognized from the infrared image until the warning is notified becomes longer. During the standby time until the warning is notified, if the position of the sun with respect to the vehicle or the influence of buildings existing around the vehicle changes and the sunlight 6 no longer hits the determination area 22, the white blooming in the infrared image is eliminated. When the white blooming in the infrared image is eliminated, the driver 5 can be recognized, so the warning is not notified. Thus, the driving support device 3 can suppress false warnings by increasing the time until the warning is notified when there is white blooming in the infrared image.

[0064] As described above, the present invention has been described using the embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist. For example, all or part of the device can be configured by being functionally or physically dispersed and integrated in an arbitrary unit. Also, new embodiments generated by any combination of a plurality of embodiments are included in the embodiments of the present invention. The effects of the new embodiments generated by the combination have the effects of the original embodiments combined.

Explanation of Reference Numerals

[0065] S Driving support system 1 Infrared light 2 Imaging device 3 Driving support device 4 Output unit 5 Driver 31 Storage unit 32 Control unit 321 Acquisition unit 322 Determination unit 323 Setting unit 324 Notification control unit

Claims

1. An acquisition unit that acquires an infrared image obtained by irradiating and imaging an infrared ray in a first area including a driver's seat where a driver of a vehicle is seated; When the driver cannot be recognized from the infrared image, based on whether or not the pixel value of the pixel of the determination part where sunlight hits in the first area in the infrared image is equal to or more than a predetermined value, a determination unit that determines whether or not there is a whiteout in the infrared image; A setting unit that, when it is determined that the whiteout has occurred in the infrared image, makes a waiting time from when the driver cannot be recognized from the infrared image until an alarm is notified longer than a reference time; A driving support device having the above.

2. When the determination unit determines that the pixel value of the pixel of the determination part is equal to or more than the predetermined value, the acquisition unit acquires an imaging image obtained by imaging without irradiating the infrared ray, When the pixel value of the pixel of the determination part is equal to or more than the predetermined value and the driver can be recognized from the imaging image, the determination unit determines that there is a whiteout in the infrared image. The driving support device according to claim 1.

3. When the time when the infrared image is captured is included in a time zone when sunlight hits the second area, the determination unit determines whether or not there is a whiteout in the infrared image. The driving support device according to claim 1.

4. The acquisition unit acquires the infrared image captured after performing exposure correction in which an imaging device that captures the infrared image changes at least one of an aperture value and an exposure time. The driving support device according to claim 1.

5. When the pixel value of the pixel of the determination part of the infrared image is less than the predetermined value, the determination unit determines that there is no whiteout in the infrared image, When the determination unit determines that there is no whiteout in the infrared image, the setting unit sets the waiting time to the reference time. The driving support device according to claim 4.

6. When the statistic of the pixel values of a plurality of pixels included in the determination part is equal to or more than the predetermined value, the determination unit determines that there is a whiteout in the infrared image, and when the statistic is less than the predetermined value, the determination unit determines that there is no whiteout in the infrared image. The driving support device according to claim 1.

7. The waiting time is an abnormality determination time from when the driver cannot be recognized from the infrared image until it is determined that an abnormality has occurred in which the driver deviates from a normal driving posture. The driving support device according to claim 1.

8. having a notification control unit that notifies the warning; the acquisition unit acquires operation information regarding the driving operation of the driver; the determination unit determines whether the driving operation of the operation information is abnormal; when the driving operation is abnormal and the time elapsed since the driver cannot be recognized from the infrared image is equal to or longer than the standby time set by the setting unit, the notification control unit notifies the warning; The driving support device according to any one of claims 1 to 7.

9. executed by a vehicle computer, acquiring an infrared image obtained by irradiating infrared rays and imaging a first area including a driver's seat where the driver of the vehicle is seated; when the driver cannot be recognized from the infrared image, determining whether there is a white jump in the infrared image based on whether the pixel value of the pixel of the determination part where the second area irradiated with sunlight in the first area in the infrared image is imaged is equal to or greater than a predetermined value; when it is determined that the white jump has occurred in the infrared image, lengthening a standby time from when the driver cannot be recognized from the infrared image until a warning is notified to be longer than a reference time; A driving support method having.

Images