Driving assistance device

Abstract

A processor (152) of an ECU (150) is provided with: a line-of-sight estimation unit (152a) that estimates the direction of the line of sight of a driver of a vehicle; a first side view determination unit (152b) that determines whether or not the driver looks sideward on the basis of a predetermined determination logic when the driver's line of sight faces a first side view region deviating from the vehicle traveling direction; and a second side view determination unit (152c) that determines whether or not the driver is in side view on the basis of a predetermined determination logic different from the first side view determination unit (152b) when the driver's line of sight faces a second side view region deviating from the vehicle traveling direction.

Description

Technical Field

[0001] The present invention relates to a driving assistance device. Background Art

[0002] In the past, it is well known that the user's line of sight is detected, and based on the time the line of sight is directed in a direction other than the direction that should be observed while driving, it is determined whether the user is in a state of side-gaze (looking away, not observing the front carefully, visually distracted, and not concentrating) (for example, refer to Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-108784 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, when determining the driver's sideways gaze based on the direction of the line of sight, there is a problem: the driver's line of sight due to driving behavior may be determined as sideways gaze. For example, the driver's behavior of visually recognizing (visually identifying) the interior rearview mirror (central rearview mirror) or the door rearview mirror is caused by the driving action to confirm the rear, but these behaviors will be determined as sideways gaze. Moreover, if warnings are issued for these behaviors, the driver may feel distrustful or uncomfortable.

[0008] In addition, even if the driver is looking at the direction he should observe while driving, staring at the interior rearview mirror or the door mirror without any relation to the driving action is not a normal driving action, but should be determined as sideways viewing, and the user should be warned. However, in the technology described in the above patent document, as long as the user is looking at the direction he should observe while driving, such behavior will not be determined as sideways viewing, and therefore there is a problem that necessary warnings will not be issued.

[0009] In view of the above, for an area where the driver's line of sight may be directed due to the driver's driving action, unlike other areas, it is preferable to determine sideways viewing in consideration of whether the driver's line of sight is directed due to the driving action.

[0010] Therefore, an object of the present invention is to provide a driving assistance device capable of assisting a driver's driving with high accuracy by determining the driver's sideways looking based on the characteristics of the area to which the driver's line of sight is directed.

[0011] Technical solutions to solve problems

[0012] The gist of the present disclosure is as follows.

[0013] (1) A driving assistance device comprising: a line of sight estimating unit for estimating the direction of a line of sight of a driver of a vehicle; a first side-viewing determination unit for determining whether the driver is looking sideways (distracted) based on a predetermined determination logic when the driver's line of sight is directed toward a first side-viewing area deviating from the direction of travel of the vehicle; and a second side-viewing determination unit for determining whether the driver is looking sideways based on a predetermined determination logic different from that of the first side-viewing determination unit when the driver's line of sight is directed toward a second side-viewing area deviating from the direction of travel of the vehicle.

[0014] (2) According to the driving assistance device described in (1) above, the first side-looking determination unit determines whether the driver is looking aside based on both the first determination logic for determining whether the driver is looking aside continuously and the second logic for determining whether the driver is looking aside intermittently, and the second side-looking determination unit determines whether the driver is looking aside only based on the first determination logic for determining whether the driver is looking aside continuously.

[0015] (3) According to the driving assistance device described in (1) above, the first side-viewing determination unit determines that the driver is looking sideways when the driver's line of sight is directed toward the first side-viewing area for more than a first predetermined time, and determines that the driver is looking sideways when the driver's line of sight is directed toward the first side-viewing area for more than a predetermined cumulative time or a predetermined proportion within a second predetermined time, and the second side-viewing determination unit determines that the driver is looking sideways when the driver's line of sight is directed toward the second side-viewing area for more than the first predetermined time.

[0016] (4) The driving assistance device according to any one of (1) to (3) above, wherein the second side view area is an area toward which the driver's line of sight is directed due to the driver's driving action.

[0017] (5) The driving assistance device according to any one of (1) to (4) above, further comprising a warning processing unit that performs processing for warning the driver when the first lateral looking determination unit or the second lateral looking determination unit determines that the driver is looking sideways.

[0018] Effects of the Invention

[0019] According to the present invention, there is provided an effect of being able to provide a driving assistance device capable of assisting the driver's driving with high accuracy by determining the driver's sideways looking based on the characteristics of the area toward which the driver's line of sight is directed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the configuration of a driving assistance system according to one embodiment.

[0021] Figure 21 is a diagram showing a state of the traveling direction of the vehicle as viewed from the interior of the vehicle, and is a schematic diagram showing a visual area of ​​the driver.

[0022] Figure 3 It is a schematic diagram showing the functional blocks of the processor of the ECU.

[0023] Figure 4 This is a flowchart showing the processing performed by the processor of the ECU in each predetermined control cycle.

[0024] Figure 5 Yes means Figure 4 Flowchart of the processing of the long-term gaze (looking away for a long time) determination logic in step S16.

[0025] Figure 6 Yes means Figure 4 Flow chart of the processing of the occasional glance (occasionally glancing away) determination logic in step S14. DETAILED DESCRIPTION

[0026] Hereinafter, several embodiments of the present invention will be described with reference to the drawings. However, these descriptions are intended only to illustrate preferred embodiments of the present invention, and are not intended to limit the present invention to such specific embodiments.

[0027] Figure 1 1 is a schematic diagram of a driving assistance system 1000 according to an embodiment. The driving assistance system 1000 is mounted on a vehicle such as a car, and includes a driver monitoring camera 120, a display device 130, a speaker 140, and an electronic control unit (ECU: Electronic Control Unit, hereinafter referred to as ECU) 150. The driver monitoring camera 120, the display device 130, the speaker 140, and the ECU 150 are each connected via an in-vehicle network that complies with a standard such as a controller area network (CAN).

[0028] The driver monitoring camera 120 has a two-dimensional detector composed of an array of photoelectric conversion elements such as CCD or C-MOS that are sensitive to visible light, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The driver monitoring camera 120 is set near the steering column or dashboard, the front windshield, etc. inside the vehicle, facing the assumed position of the driver, and shoots the driver's face to generate an image reflecting the driver's face. In this embodiment, the driver monitoring camera 120 is set on the steering column. The driver monitoring camera 120 shoots at a predetermined shooting cycle (for example, 1 / 30 second to 1 / 10 second). Whenever the driver monitoring camera 120 generates an image, it outputs the generated image to the ECU 150 via the in-vehicle network.

[0029] The display device 130 is constituted by a liquid crystal display (LCD), for example, and is provided near a dashboard or an instrument panel, etc., to display various information. The speaker 140 outputs various information by sound.

[0030] ECU150 is a component that controls the entire driving assistance system 1000 and is a form of a driving assistance device. ECU150 has a processor 152, a memory 154 and a communication interface (I / F) 156. The processor 152 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 152 may also have other computing circuits such as a logic operation unit, a numerical operation unit or a graphics processing unit. The memory 154 has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory, and stores data associated with the processing involved in this embodiment as needed. The memory 154 stores three-dimensional position information representing the boundaries of the side-viewing area, the front area, and the area except for occasional glances at the side, which will be described later. The communication interface 156 has an interface circuit for connecting the ECU150 to the in-vehicle network.

[0031] The driving assistance system 1000 detects the driver's line of sight through the driver monitoring camera 120 and issues a warning when the driver looks to the side, thereby suppressing the driver's side-looking. More specifically, the driving assistance system 1000 sets a side-looking area in the driver's visual recognition area, and when the driver's line of sight is directed toward the side-looking area, the driver's side-looking is classified into long-term side-looking and occasional side-looking (short-term and intermittent non-frontal gaze) according to the time of the driver's side-looking.

[0032] Long-term sideways looking and occasional sideways looking are determined by different judgment logics. Specifically, when the driver continues to look aside for more than a predetermined time (for example, more than 3 seconds, more than 5 seconds, etc.), it is determined to be long-term sideways looking, and when the driver looks aside for a total of more than a predetermined cumulative time (for example, more than 5 seconds) or a predetermined proportion (for example, more than 50%) within a predetermined time (for example, within 10 seconds), it is determined to be occasional sideways looking. In addition, long-term sideways looking is not limited to the situation where the driver looks at the same place while looking aside, but can also include the situation where the line of sight moves to different places while looking aside. The same is true for occasional sideways looking.

[0033] Furthermore, the driving assistance system 1000 warns the driver when the driver looks to the side for a long time or occasionally glances to the side. Thus, the driving assistance system 1000 prompts the driver to turn his / her sight forward and assists safe driving.

[0034] In this way, by determining both long-term sideways looking and occasional sideways looking, for example, in two cases, the driver is warned when the driver looks out of the window for a long time and when the driver intermittently glances at the portable terminal at hand. In particular, by determining occasional sideways looking, a warning is issued even when the condition does not meet the condition of long-term sideways looking, thereby more reliably assisting the driver in safe driving.

[0035] Long-term sideways glances are a state where the driver looks sideways for a long time. Therefore, the driving assistance system 1000 must issue a warning when the driver looks sideways for a long time. On the other hand, regarding occasional sideways glances, if a warning is issued for all short-term and intermittent non-frontal gazes, the driver may feel uncomfortable and distrustful when the occasional sideways glances are caused by the driver's driving behavior. For example, when the driver occasionally glances at the door mirror, since occasionally glancing at the door mirror is a normal driving behavior for safety confirmation, if a warning is issued in this case, the driver may feel uncomfortable and distrustful.

[0036] Therefore, the driving assistance system 1000 according to the present embodiment is configured to, even when the driver is glancing sideways, not to issue a warning to the driver as an exception when the glancing sideways is caused by the driving action.

[0037] Figure 2 1 is a diagram showing the state of the vehicle's traveling direction as viewed from the interior of the vehicle, and is a schematic diagram showing the driver's visual area. Figure 2 As shown, a front windshield 10 is provided in the traveling direction of the vehicle, and an interior rearview mirror 12 is provided in a part of the area of ​​the front windshield 10. In addition, door mirrors 14 are provided on the left and right of the vehicle.

[0038] In order to determine whether a person is looking away for a long time or looking away from time to time, Figure 2 As shown, the driver's visual area is classified into a side view area 20 and a front area (non-side view area) 30. The side view area 20 is an area deviating from the vehicle's traveling direction, including all areas except the area of ​​the front windshield 10 and the area of ​​the interior rearview mirror 12. The front area 30 is basically the area of ​​the front windshield 10, but is the area except the area of ​​the interior rearview mirror 12.

[0039] In the side-viewing area 20, an occasional side-viewing exclusion area 40 is provided overlapping with the side-viewing area 20. Figure 2 As shown, the area 40 except for occasional glances includes areas of the interior rearview mirror 12 and the door mirrors 14 . The area 40 except for occasional glances is an area toward which the driver's line of sight is directed due to the driver's normal driving behavior.

[0040] Even when the driver's line of sight is directed toward the side-viewing area 20, the driving assistance system 1000 does not exceptionally determine the driver's side-viewing when the driver's line of sight is directed toward the side-viewing exclusion area 40. Thus, for example, when the driver occasionally glances at the interior rearview mirror 12 or the door mirror 14, it is not determined as the side-viewing, and the driver is not warned, thereby preventing the driver from feeling uncomfortable or distrustful.

[0041] On the other hand, the side view area 20 other than the occasional side glance exclusion area 40 is an area to which the driver's line of sight is not directed due to normal driving behavior. When the driver's line of sight is directed toward the side view area 20 other than the occasional side glance exclusion area 40, the driving assistance system 1000 determines that the driver is occasionally glancing aside, and warns the driver if it is determined that the driver is glancing aside. Thus, in a case where the driver occasionally glances at a portable terminal at hand, etc., the driver is warned of the occasional side glance that is not caused by driving behavior.

[0042] As described above, when the driver's line of sight is directed toward the side-looking area 20, the driving assistance system 1000 changes the logic for determining side-looking according to whether the line of sight is directed toward the occasionally-glancing-aside-excepted area 40 or toward an area other than the occasionally-glancing-aside-excepted area 40. Furthermore, when the line of sight is directed toward the occasionally-glancing-aside-excepted area 40, the occasionally-glancing-aside-looking determination is not performed, thereby not providing an unnecessary warning to the driver's normal driving behavior, and thus preventing the driver from feeling uncomfortable or distrustful. On the other hand, when the driver's line of sight is directed toward the side-looking area 20 other than the occasionally-glancing-aside-excepted area 40, the occasionally-glancing-aside-looking determination is performed, thereby providing a warning to the driver for behaviors that are not caused by the driver's normal driving behavior, such as occasionally glancing at a portable terminal. Therefore, the warning to the driver in the case of side-looking can be achieved with high accuracy.

[0043] Furthermore, even when the driver's line of sight is directed to occasionally glance at the side-gaze exclusion area 40, a long-term side-gaze determination is performed, and thus, for example, a warning is issued when the driver continues to look at the door mirror 14 for a long time. Therefore, even when the driver's line of sight is directed to occasionally glance at the side-gaze exclusion area 40, a warning is issued when it is not caused by normal driving behavior, thereby ensuring safety.

[0044] Figure 3 Schematic diagram showing the functional blocks of processor a of ECU 150 for implementing the above processing. Processor 152 of ECU 150 has a line of sight estimation unit 152a, a first lateral view determination unit 152b, a second lateral view determination unit 152c and a warning processing unit 152d. These parts of processor 152 are, for example, functional modules implemented by a computer program running on processor 152. In other words, the functional blocks of processor 152 are composed of processor 152 and a program (software) for making it function. In addition, the program can also be recorded in the memory 154 of ECU 150 or a recording medium connected from the outside. Alternatively, these parts of processor 152 can also be dedicated operation circuits provided in processor 152.

[0045] The sight line estimation unit 152a of the processor 152 detects the sight line of the driver by a known sight line detection method based on the image representing the driver's face generated by the driver monitoring camera. For example, the sight line estimation unit 152a detects the sight line using a corneal reflection method, which uses a reflection image (Purkinje image) of a near-infrared light source on the surface of the cornea to calculate the sight line angle according to the distance between the center of the pupil and the Purkinje image. Then, the sight line estimation unit 152a estimates the direction of the driver's sight line based on the detected sight line direction and the three-dimensional position information of the boundary of the side-viewing area 20, the front area 30 and the area 40 except for occasional glances stored in the memory 154 in advance. Specifically, the sight line estimation unit 152a estimates which of the side-viewing area 20 and the front area the driver's sight line is toward. In addition, when the sight line estimation unit 152a estimates that the driver's sight line is toward the side-viewing area 20, it estimates whether the sight line is toward the area 40 except for occasional glances. In addition, the above-mentioned method of detecting the line of sight direction is just an example, and the line of sight estimating unit 152a may detect the line of sight direction using other methods.

[0046] In addition, the sight line estimation unit 152a detects the starting point of the driver's sight line, that is, the position of the eyes, or the position and contour of various components of the driver's face by inputting the image into a pre-learned identifier. For example, as such an identifier, the sight line estimation unit 152a can use a segmentation identifier that is pre-learned to output the certainty of the object represented in the pixel according to various types of objects that the pixel may represent, and recognizes the object represented as having the greatest certainty. As such an identifier, the sight line estimation unit 152a can use a deep neural network (DNN) having an architecture of a convolutional neural network type (CNN) for segmentation, such as a fully convolutional network (FCN). Alternatively, the sight line estimation unit 152a can also use a segmentation identifier according to other machine learning methods such as random forests or support vector machines. In this case, the sight line estimation unit 152a determines the pixels in which various components are reflected in the image by inputting the image into the segmentation identifier. Then, the sight line estimating unit 152a regards a set of images in which elements of the same type are reflected as a region indicating the element.

[0047] Instead of detecting the direction of the driver's gaze, the sight line estimation unit 152a may detect the driver's facial orientation angle based on the information obtained from the above-mentioned identifier, and estimate which of the sideways viewing area 20, the frontal viewing area 30, and the sideways glancing exclusion area 40 the driver's gaze is directed to based on the facial orientation angle and three-dimensional position information indicating the boundaries of the sideways viewing area 20, the frontal viewing area 30, and the sideways glancing exclusion area 40 pre-stored in the memory 154. For example, the sight line estimation unit 152a may calculate the left and right facial orientation angles (yaw angles) based on the distance between the left and right eyes, and calculate the upper and lower facial orientation angles (pitch angles) based on the distance between the nose position and the straight line connecting the left and right eye positions.

[0048] The first side-view determination unit 152b of the processor 152 determines whether the driver is looking sideways based on a predetermined determination logic when the driver's line of sight deviates from the first side-view area in the vehicle's traveling direction, based on the line of sight direction estimated by the line of sight estimation unit 152a.

[0049] More specifically, the first side-viewing area is the area remaining after removing the occasional side-viewing exclusion area 40 from the side-viewing area 20. The first side-viewing determination unit determines whether the driver is looking to the side based on both the first determination logic (long-term side-viewing determination logic) for determining whether the driver is continuously looking to the side and the second logic (occasional side-viewing determination logic) for determining whether the driver is intermittently looking to the side. The first side-viewing determination unit 152b may also determine that the driver is looking to the side for a long time based on the long-term side-viewing determination logic when the line of sight is directed toward the first side-viewing area for more than the first predetermined time. In addition, the first side-viewing determination unit 152b may also determine that the driver is looking to the side from time to time based on the occasional side-viewing determination logic when the line of sight is directed toward the first side-viewing area for more than a predetermined cumulative time or more than a predetermined proportion within the second predetermined time.

[0050] The second lateral viewing determination unit 152c of the processor 152 determines whether the driver is looking to the side based on a predetermined determination logic different from that of the first lateral viewing determination unit 152b, based on the direction of the driver's line of sight estimated by the line of sight estimation unit 152a and in the case of a second lateral viewing area where the line of sight deviates from the direction of vehicle travel.

[0051] More specifically, the second side-looking area is the occasional side-looking exclusion area 40. The second side-looking determination unit 152c determines whether the driver is looking aside based only on the first determination logic (long-term side-looking determination logic) for determining whether the driver is continuously looking aside. The second side-looking determination unit 152c may also determine that the driver is looking aside for a long time based on the long-term side-looking determination logic when the line of sight is directed toward the second area for more than the first predetermined time. In this way, the second side-looking determination unit 152c does not make a determination based on the occasional side-looking determination logic, but only determines whether the driver is looking aside based on the long-term side-looking determination logic.

[0052] In addition, the long-term side-looking determination logic and the occasional side-looking determination logic are not limited to the above. For example, the occasional side-looking determination logic may also determine that the driver is occasionally glancing aside when the line of sight continues to be directed toward the first side-looking area for more than the third predetermined time in the same way as the long-term side-looking determination logic. In this case, the third predetermined time may also be set to be shorter than the first predetermined time of the long-term side-looking determination logic.

[0053] The warning processing unit 152d of the processor 152 performs a process for warning the driver when the first sideways looking determination unit 152b or the second sideways looking determination unit 152c determines that the driver is looking sideways. Specifically, the warning processing unit 152d performs a process for displaying a warning on the display device 130 or outputting a warning by sound from the speaker 140.

[0054] Next, the processing performed by processor 152 of ECU 150 will be described. Figure 4 1 is a flowchart showing the processing performed by the processor 152 of the ECU 150 in each predetermined control cycle. First, the sight line estimation unit 152a estimates whether the driver's sight line is toward the front area 30 (step S10). When the driver's sight line is not toward the front area 30, it is estimated that the driver's sight line is toward the side view area 20. In this case, the sight line estimation unit 152a further estimates whether the driver's sight line is toward the occasional side view exclusion area 40 (step S12).

[0055] In addition, since the driver monitoring camera 120 is disposed on the steering column, the field of view of the driver monitoring camera 120 may be blocked by the spokes of the steering wheel depending on the angular position of the steering wheel. In step S10, when the field of view of the driver monitoring camera 120 is blocked by the spokes of the steering wheel, the line of sight estimating unit 152a may also estimate that the driver's line of sight is directed toward the front area 30.

[0056] In step S12, if the driver's line of sight is not directed toward the occasional glance-away exclusion area 40, it is estimated that the driver's line of sight is directed toward the area remaining from the side-viewing area 20 excluding the occasional glance-away exclusion area 40, that is, the first side-viewing area. Therefore, the first side-viewing determination unit 152b executes the occasional glance-away determination logic (step S14). Next, the first side-viewing determination unit 152b executes the long-term side-viewing determination logic (step S16).

[0057] On the other hand, when it is estimated in step S12 that the driver's line of sight is directed toward the occasional glance at the sideways exclusion area 40, the driver's line of sight is directed toward the second sideways area. Therefore, the second sideways determination unit 152c executes the long-term sideways determination logic (step S16). In this way, when the driver's line of sight is directed toward the occasional glance at the sideways exclusion area 40, the occasional glance at the sideways determination logic of step S14 is not executed, and only the long-term sideways determination logic of step S16 is executed.

[0058] Thus, when the driver's line of sight is directed toward the glance-away exclusion area 40, the glance-away determination logic is not executed, thereby suppressing the driver from being warned due to normal driving behavior. Therefore, it is possible to suppress the driver from feeling uncomfortable or distrustful.

[0059] On the other hand, even when the driver's line of sight is directed to occasionally glance at the side-gaze exclusion area 40, by executing the long-term side-gaze determination logic in step S16, a warning is issued, for example, when the driver continues to look at the door mirror 14 for a long time. Therefore, even when the driver's line of sight is directed to occasionally glance at the side-gaze exclusion area 40, a warning is issued when it is not caused by normal driving behavior, thereby ensuring safety.

[0060] After step S16, it is determined whether the duration of the vehicle speed being 0 [km / h] exceeds a predetermined time T_1 (step S18). If the duration of the vehicle speed being 0 [km / h] exceeds T_1, the accumulated time of glances is reset to 0 (step S20). As described later, the accumulated time of glances is used in the glance determination logic of step S14. When the accumulated time of glances reaches a predetermined accumulated time or more, a warning is issued to the driver. By resetting the accumulated time of glances to 0 in step S20, it is possible to suppress the warning of glances to the driver when the driver is not driving the vehicle.

[0061] In addition, as described above, depending on the angular position of the steering wheel, the field of view of the driver monitoring camera 120 may be blocked by the spokes of the steering wheel. Therefore, it is determined whether the duration of the state in which the field of view of the driver monitoring camera 120 is blocked by the steering wheel exceeds a predetermined time T_2 (step S22). If the duration exceeds T_2, the accumulated time of occasional glances to the side is reset to 0 (step S24). In this way, it is possible to suppress the warning to the driver of occasional glances to the side in a state in which the driver's line of sight cannot be accurately estimated. After step S24, the processing in this control cycle ends.

[0062] In addition, when it is estimated in step S10 that the driver's line of sight is directed toward the front area 30, the time (front duration) that the driver's line of sight continues to be directed toward the front area 30 is counted (step S26), and when the front duration exceeds the predetermined time T_3 (step S28: Yes), the occasional glance aside cumulative time is reset to 0 (step S30). After step S30, the process proceeds to step S18.

[0063] On the other hand, if the front-facing duration does not exceed the predetermined time T3 in step S28, the process of step S30 is not performed, and the process proceeds to step S18.

[0064] Figure 5 Yes means Figure 4 First, the time (side-view duration) during which the driver's line of sight continues to be directed toward the side-viewing area 20 is counted (step S40), and the front-view duration is reset to 0 (step S42).

[0065] Next, it is determined whether the side-looking duration is longer than the first predetermined time T_4 (step S44). If the side-looking duration is longer than T_4, it is determined to be long-term side-looking. In this case, the warning processing unit 152d performs processing for warning the driver (step S48). On the other hand, in step S44, if the side-looking duration is shorter than T_4, the processing of this control cycle ends.

[0066] As described above, in the long-term side-looking determination logic, when the driver's line of sight is not directed toward the front area 30, the side-looking duration of the driver's line of sight continuously directed toward the side-looking area 20 is counted, and when the side-looking duration becomes longer than the first predetermined time T_4, it is determined as long-term side-looking. Since the side-looking duration is the time during which the driver's line of sight continuously directs toward the side-looking area 20, when the driver's line of sight directs toward the front area 30 in the subsequent control cycle, the counting of the side-looking duration is stopped, and the side-looking duration is reset to 0.

[0067] Figure 6Yes means Figure 4 Flowchart of the processing of the occasional glance-away judgment logic in step S14. First, the occasional glance-away cumulative time within the past second predetermined time T_5 [sec] is counted (step S50), and the front-facing duration is reset to 0 (step S52).

[0068] Next, it is determined whether the accumulated time of occasional glances is greater than or equal to a predetermined accumulated time T_6 (step S54). If the accumulated time of occasional glances is greater than or equal to T_6, it is determined to be occasional glances. In this case, the warning processing unit 152d performs processing for warning the driver (step S56). On the other hand, in step S54, if the accumulated time of occasional glances is less than T_6, the processing of this control cycle ends.

[0069] As described above, in the occasional glance judgment logic, when the driver's line of sight is not toward the front area 30 and the occasional glance exclusion area 40, the occasional glance accumulation time is counted, and when the occasional glance accumulation time becomes greater than the predetermined accumulation time T_6, it is determined to be an occasional glance. In the subsequent control cycle, when the driver's line of sight is toward the front area 30, or when the driver's line of sight is toward the occasional glance exclusion area 40, the counting of the occasional glance accumulation time is stopped. Although the occasional glance accumulation time is maintained in the memory 154, it is not counted when entering the control cycle. Figure 4 In any of steps S20, S24, and S30, the accumulated time of occasional glances is reset to 0.

[0070] In addition, the warning processing unit 152d can Figure 5 Step S48 and Figure 6 The intensity of the warning may be changed in step S56, and the intensity of the warning (step S48) in the case of determining that the user is looking away for a long time may be higher than the intensity of the warning (step S56) in the case of determining that the user is looking away from time to time. For example, the warning processing unit 152d may increase the size of the warning displayed on the display device 130 when the user is looking away for a long time, or may increase the volume of the sound output from the speaker 140, compared to when the user is looking away from time to time.

[0071] Furthermore, after the driver is warned of the problem of occasionally looking to the side, if it is estimated that the driver's line of sight is directed toward the front area 30 in the next control cycle (in the case of Figure 4In the case of "yes" in step S10 of step S28), T_3 of step S28 may also be set to a larger value. Thus, after it is determined that the driver is glancing aside from time to time, it takes a longer time before the accumulated time of glancing aside from time to time is reset to 0. Therefore, for example, in the case where the driver occasionally glances at the portable terminal, even if the driver intentionally returns the line of sight to the front area 30 in order to avoid the warning, the accumulated time of glancing aside from time to time is difficult to be reset to 0 in step S30. Therefore, even if the driver performs such an action, the possibility of being determined as glancing aside from time to time is higher, and the intentional behavior used to avoid the warning can be prevented before it happens.

[0072] As described above, according to the present embodiment, when the driver's line of sight is directed toward the side-looking area 20, the driving assistance system 1000 changes the logic for determining side-looking according to whether the line of sight is directed toward the occasional side-glancing exclusion area 40 or toward an area other than the occasional side-glancing exclusion area 40. Furthermore, even when the driver's line of sight is directed toward the side-looking area 20, the driving assistance system 1000 does not exceptionally determine the occasional side-glancing when the line of sight is directed toward the occasional side-glancing exclusion area 40. Thus, when the driver's line of sight occasionally glances at the interior rearview mirror 12 or the door rearview mirror 14 in the occasional side-glancing exclusion area 40 due to normal driving behavior, it is not determined as occasional side-glancing, and the driver is not warned, thereby preventing the driver from feeling uncomfortable or distrustful.

[0073] Description of Reference Numerals

[0074] 10 front windshield; 12 interior rearview mirror; 14 door mirror; 20 side-view area; 30 front area (non-side-view area); 40 area except for occasional side-view glances; 120 driver monitoring camera; 130 display device; 140 speaker; 150 electronic control unit (ECU); 152 processor; 152a line of sight estimation unit; 152b first side-view determination unit; 152c second side-view determination unit; 152d warning processing unit; 154 memory; 156 communication interface; 1000 driving assistance system.

Claims

1. A driving assistance device comprising: a sight line estimating unit for estimating a direction of a sight line of a driver of the vehicle; a first side-view determination unit that determines whether the driver is looking sideways based on a predetermined determination logic when the driver's line of sight is directed toward a first side-view area that deviates from the vehicle's traveling direction; and The second side-view determination unit determines whether the driver is looking sideways based on a predetermined determination logic different from that of the first side-view determination unit when the driver's line of sight is directed toward a second side-view area deviating from the vehicle's traveling direction.

2. The driving assistance device according to claim 1, The first side-looking determination unit determines whether the driver is looking aside based on both a first determination logic for determining whether the driver is looking aside continuously and a second determination logic for determining whether the driver is looking aside intermittently. The second side-looking determination unit determines whether the driver is looking sideways based only on the first determination logic for determining whether the driver is continuously looking sideways.

3. The driving assistance device according to claim 1, The first side-viewing determination unit determines that the driver is looking aside when the driver's sight is directed toward the first side-viewing area for more than a first predetermined time, and determines that the driver is looking aside when the driver's sight is directed toward the first side-viewing area for more than a predetermined cumulative time or a predetermined proportion within a second predetermined time. The second side-viewing determination unit determines that the driver is looking sideways when the driver's line of sight is directed toward the second side-viewing area for more than the first predetermined time.

4. The driving assistance device according to any one of claims 1 to 3, The second side-viewing area is an area toward which the driver's line of sight is directed due to the driver's driving action.

5. The driving assistance device according to any one of claims 1 to 3, The system further includes a warning processing unit configured to perform a process for warning the driver when the first sideways looking determination unit or the second sideways looking determination unit determines that the driver is looking sideways.

Images