Inattentive state determining device, inattentive state determining method, and computer program

The inattentiveness determination device adjusts the visual judgment range based on target proximity to prevent misjudgment of in-vehicle equipment viewing, ensuring accurate detection of driver attention.

JP2024162113AActive Publication Date: 2024-11-21TOYOTA JIDOSHA KK +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023077338
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-09
Publication Date
2024-11-21
Estimated Expiration
2043-05-09

AI Technical Summary

Technical Problem

Existing driver monitoring systems struggle to accurately differentiate between a driver looking at in-vehicle equipment and a target outside the vehicle, particularly when the target is near, leading to potential misjudgment of inattentiveness.

Method used

An inattentiveness determination device that adjusts the visual judgment range based on the positional relationship with the target, changing the upper end position of the range when the target is nearby to prevent misjudgment by setting it lower, and using driver data to detect the driver's viewpoint and line of sight direction.

Benefits of technology

Prevents erroneous determination of inattentiveness by adjusting the visual judgment range to account for the proximity of the target, ensuring accurate detection of in-vehicle equipment viewing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162113000001_ABST
    Figure 2024162113000001_ABST
Patent Text Reader

Abstract

To suppress that it is erroneously determined to visually observe an in-vehicle device, despite the fact that a front target outside a vehicle is visually observed.SOLUTION: An inattentive state determining device 4 detects a target present in front of a vehicle, based on peripheral data indicating a situation around the vehicle; detects a viewpoint position and a visual line direction of a driver, based on driver data indicating a state of the driver; changes a visual observation determination range in a vertical direction in order to determine whether or not the driver visually observes an in-vehicle device arranged in front of a cabin thereof, according to a positional relation to the target; determines that the in-vehicle device is inattentively observed if the visual line direction of the driver is continuously kept within the visual observation determination range for a predetermined inattentive state determination time or more; and changes the visual observation determination range in such a manner that an upper end position in the visual observation determination range is made to be at a lower side by comparing when the target is located at a nearby place with when the target is located at a distance.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an inattentive driving determination device, an inattentive driving determination method, and a computer program. [Background technology]

[0002] Patent Document 1 discloses a conventional driver monitoring device that is configured to expand the permissible left-right range for the driver's line of sight as the distance to a preceding vehicle traveling in another lane (adjacent lane) becomes shorter, and to determine that the driver is looking away if the driver's line of sight deviates from the range. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-87143 A Summary of the Invention [Problem to be solved by the invention]

[0004] A windshield is disposed above and in front of the driver's seat and passenger seat of a vehicle, and a dashboard is disposed below and in front of the driver's seat. Devices that are visually observed by the driver, such as a monitor, a car navigation system, and instruments, are attached to the dashboard (hereinafter referred to as "in-vehicle devices"). Therefore, when the driver is looking at the in-vehicle devices (i.e., while looking aside), the driver's line of sight is directed downward in the vertical direction (vehicle height direction) compared to when the driver is looking at an external target in front of the vehicle, such as another vehicle, through the windshield (hereinafter referred to as "external forward target").

[0005] Similarly, when a driver is looking at an external object ahead of the vehicle through the windshield, if the object ahead of the vehicle is close to the vehicle, the driver's line of sight tends to be directed downward in the vertical direction compared to when the object is far away.

[0006] Therefore, when an external target in front of the vehicle is located near the vehicle, it is more difficult to determine whether the driver is looking at the external target in front of the vehicle through the windshield or at in-vehicle equipment, compared to when the target is located far away. Even if the left-right range permitted for the driver's line of sight is changed depending on the positional relationship with the vehicle ahead traveling in another lane (adjacent lane), as in Patent Document 1 mentioned above, when the driver is looking at an external target in front of the vehicle through the windshield and the distance to the external target in front of the vehicle gradually becomes smaller so that the driver's line of sight turns downward in the vertical direction, there is a risk that the driver will erroneously determine that the driver is looking at in-vehicle equipment even though he or she is actually looking at the external target in front of the vehicle.

[0007] The present invention has been made in response to such problems, and aims to prevent the driver from erroneously determining that he or she is looking at in-vehicle equipment when in fact he or she is looking at a target ahead outside the vehicle. [Means for solving the problem]

[0008] The gist of the present disclosure is as follows.

[0009] (1) An inattentiveness judgment device that judges whether a vehicle driver is looking away, which detects targets present in front of the vehicle based on surrounding data that represents the situation around the vehicle, detects the driver's viewpoint position and line of sight based on driver data that represents the driver's state, and changes the vertical visual judgment range to judge whether the driver is looking at in-vehicle equipment located at the front of the vehicle depending on the positional relationship with the target. If the driver's line of sight remains within the visual judgment range for a predetermined inattentiveness judgment time or more, it is judged that the driver is looking away from the in-vehicle equipment. When the target is located nearby, the visual judgment range is changed so that the upper end position of the visual judgment range is lower than when the target is located far away.

[0010] (2) The inattentive driving judgment device described in (1) above, in which the upper end position of the visual judgment range is set to any position between the upper end position of the in-vehicle equipment and the lower end position of the vehicle windshield when the target is located far away.

[0011] (3) An inattentive judgment device as described in (1) or (2) above, in which when the driver's line of sight falls within the visual judgment range and the positional relationship with the target changes from a distant state to a nearby state between the time the driver's line of sight falls within the visual judgment range and the elapse of the inattentive judgment time, the visual judgment range is not changed.

[0012] (4) An inattentive behavior judgment device as described in any of (1) to (3) above, which grasps the positional relationship with a target by detecting the distance to the target, and judges that the target is located far away if the distance to the target is equal to or greater than a predetermined threshold, and judges that the target is located nearby if the distance to the target is less than the predetermined threshold.

[0013] (5) An inattentive driving judgment device as described in any of (1) to (3) above, which grasps the positional relationship with a target by detecting the driver's gaze angle, the gaze angle being the angle between a horizon extending horizontally from the driver's viewpoint and a gaze direction line extending from the driver's viewpoint in a direction downward from the horizon, and judges the target to be located far away if the gaze angle is equal to or smaller than a predetermined angle, and judges the target to be located nearby if the gaze angle is greater than the predetermined angle.

[0014] (6) An inattentive driving judgment device as described in any one of (1) to (5) above, wherein the inattentive driving judgment time is made longer when the target is located nearby compared to when the target is located far away.

[0015] (7) An inattentive driving judgment device as described in any one of (1) to (6) above, which issues a warning to the driver regarding inattentive driving when it is determined that the driver is inattentive to an in-vehicle device.

[0016] (8) An inattentiveness judgment device that judges whether a driver of a vehicle is looking away, the device detects targets present in front of the vehicle based on surrounding data that represents the situation around the vehicle, detects the driver's viewpoint position, line of sight direction and line of sight angle based on driver data that represents the driver's state, changes the inattentiveness judgment angle to judge whether the driver is looking at in-vehicle equipment located at the front of the vehicle according to a positional relationship with the target, judges that the driver is looking away from the in-vehicle equipment if the driver's line of sight angle remains equal to or greater than the inattentiveness judgment angle for a predetermined inattentiveness judgment time or more, and when the target is located nearby, the inattentiveness judgment angle is made larger than when the target is located far away, the line of sight angle is the angle between a horizon line extending horizontally from the driver's viewpoint and a line of sight line extending from the driver's viewpoint in a direction of sight downward below the horizon, and the inattentiveness judgment angle is the angle between the horizon and the inattentiveness judgment line extending from the driver's viewpoint toward the top position of the in-vehicle equipment.

[0017] (9) An inattentiveness judgment method using an inattentiveness judgment device that judges whether a vehicle driver is looking away, comprising the steps of: detecting a target object in front of the vehicle based on surrounding data that represents the situation around the vehicle; detecting the driver's viewpoint position and line of sight based on driver data that represents the driver's state; changing the vertical visual judgment range to determine whether the driver is looking at in-vehicle equipment located at the front of the vehicle according to the positional relationship with the target object; determining that the driver is looking at the in-vehicle equipment if the driver's line of sight remains within the visual judgment range for a predetermined inattentiveness judgment time or more; and, when the target object is located nearby, changing the visual judgment range so that the upper end position of the visual judgment range is lower than when the target object is located far away.

[0018] (10) A computer program for an inattentive judgment device that judges whether a vehicle driver is looking away, the computer program causing the inattentive judgment device to execute the following: detect targets in front of the vehicle based on surrounding data that represents the situation around the vehicle; detect the driver's viewpoint position and line of sight based on driver data that represents the driver's state; change the vertical visual judgment range to determine whether the driver is looking at in-vehicle equipment located at the front of the vehicle according to the positional relationship with the target; judge the driver to be looking away at in-vehicle equipment if the driver's line of sight remains within the visual judgment range for a predetermined inattentive judgment time or more; and, when the target is located nearby, change the visual judgment range so that the upper end position of the visual judgment range is lower than when the target is located far away. Effect of the Invention

[0019] According to these aspects of the present invention, when an object in front of the vehicle outside is located nearby, the upper end position of the vertical visual judgment range for determining whether or not the vehicle is viewing in-vehicle equipment is shifted downward compared to when the object in front of the vehicle is located far away, thereby preventing the driver from erroneously determining that the vehicle is viewing in-vehicle equipment when the vehicle is actually viewing the object in front of the vehicle outside. [Brief description of the drawings]

[0020] [Figure 1] 1 is a schematic system configuration diagram of a vehicle according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of a control device. [Diagram 3] 5 is a flowchart illustrating details of an inattentive driving determination process of an in-vehicle device according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating a visual determination range of an in-vehicle device. [Diagram 5] FIG. 13 is a diagram illustrating that when an external forward target is present near the vehicle, the driver's line of sight tends to be directed downward in the vertical direction compared to when the target is present far away. [Figure 6]10 is a flowchart illustrating details of an inattentive driving determination process of an in-vehicle device according to a second embodiment of the present invention. [Figure 7A] 1 is a diagram illustrating a line-of-sight angle Θ1 and an aside-looking determination angle Θ2. FIG. [Figure 7B] 1 is a diagram illustrating a line-of-sight angle Θ1 and an aside-looking determination angle Θ2. FIG. [Figure 8] 13 is a flowchart illustrating details of an inattentive driving determination process of an in-vehicle device according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, like components are designated by like reference numerals.

[0022] (First embodiment) FIG. 1 is a system schematic diagram of a vehicle 100 according to a first embodiment of the present invention.

[0023] The vehicle 100 includes a surrounding sensor 1, a driver sensor 2, an output device 3, and a control device 4. The surrounding sensor 1, the driver sensor 2, the output device 3, and the control device 4 are communicatively connected via an in-vehicle network 5 that complies with a standard such as a controller area network. The vehicle 100 may be a vehicle with an automatic driving function or a driving assistance function, or may be a manually driven vehicle.

[0024] The surrounding sensor 1 is a sensor for generating surrounding data that indicates the situation around the vehicle 100. In this embodiment, the surrounding sensor 1 includes one or more external cameras for capturing images of the surroundings of the vehicle 100. The external cameras capture images of the surroundings of the vehicle 100 at a predetermined frame rate (for example, 10 [Hz] to 40 [Hz]) and generate surrounding images showing the surroundings of the vehicle 100. Each time the external cameras generate a surrounding image, they transmit the generated surrounding image to the control device 4 as surrounding data.

[0025] Instead of or in addition to the external camera, a distance measuring sensor that measures the distance to targets such as other vehicles, pedestrians, and bicycles present around the vehicle 100 may be provided as the surrounding sensor 1. Examples of distance measuring sensors include a LiDAR (Light Detection And Ranging) that irradiates radar light and measures distance based on the reflected light, and a millimeter wave radar sensor that irradiates radio waves and measures distance based on the reflected waves.

[0026] The driver sensor 2 is a sensor for generating driver data representing the state of the driver. In this embodiment, the driver sensor 2 includes a driver monitor camera for capturing an image of the driver's appearance including the driver's face. The driver monitor camera captures the driver's appearance at a predetermined frame rate (for example, 10 [Hz] to 40 [Hz]) and generates an appearance image showing the driver's appearance. Every time the driver monitor camera generates an appearance image of the driver, it transmits the generated appearance image to the control device 4 as driver data.

[0027] The output device 3 is a device for notifying the driver of the vehicle 100 through the driver's bodily senses (for example, vision, hearing, touch, etc.). In this embodiment, the output device 3 includes a display (for example, a meter display, a center display, or a head-up display) arranged at a position visible to the driver, and a speaker. The display displays information (for example, text information or image information) according to a display signal output from the control device 4. The speaker outputs sound according to an audio signal output from the control device 4.

[0028] The control device 4 is an ECU (Electronic Control Unit) including a communication unit 41, a storage unit 42, and a processing unit 43 (see FIG. 2), and is configured to at least determine whether the driver of the vehicle 100 is looking aside at in-vehicle devices (devices that are visually targeted by the driver, such as a monitor attached to the dashboard, a car navigation system, and instruments) and to issue a warning to the driver according to the determination result. In addition to the above-mentioned surrounding data and driver data, various types of data, such as vehicle position data acquired by a positioning sensor and vehicle speed data acquired by a vehicle speed sensor, can be input to the control device 4 as necessary.

[0029] FIG. 2 is a diagram showing a hardware configuration of the control device 4. As shown in FIG.

[0030] The control device 4 includes a communication unit 41, a storage unit 42, and a processing unit 43.

[0031] The communication unit 41 includes an interface circuit for connecting the control device 4 to the in-vehicle network 5. The communication unit 41 supplies data received from the outside (peripheral data, driver data, etc.) to the processing unit 43. The communication unit 41 also transmits the display signal and audio signal output from the processing unit 43 to the output device 3.

[0032] The storage unit 42 has a storage medium such as a hard disk drive (HDD), a solid disk drive (SSD), or a semiconductor memory, and stores various computer programs and data used for processing by the processing unit 43.

[0033] The processing unit 43 has one or more CPUs (Central Processing Units) and their peripheral circuits, and is, for example, a processor, which executes various computer programs stored in the storage unit 42. Hereinafter, an example of inattentive driving determination processing for in-vehicle devices, which is one of the processes executed by the processing unit 43 and, in turn, the control device 4, will be described with reference to FIG.

[0034] 3 is a flowchart for explaining details of the inattentive driving determination process of the in-vehicle device according to this embodiment. The control device 4 repeatedly executes this routine at a predetermined calculation cycle.

[0035] In step S1, the control device 4 detects a target (external forward target) existing in front of the vehicle 100 based on surrounding data received from the surrounding sensor 1, specifically, a surrounding image received from an external camera.

[0036] In this embodiment, the control device 4 sequentially inputs the surrounding images received from the external camera to the classifier, thereby detecting the area in which the outside front target is shown in the surrounding image and the type of the outside front target shown in the area. The classifier can be, for example, a convolutional neural network (CNN) having a plurality of convolution layers connected in series from the input side to the output side. The control device 4 estimates the distance (inter-vehicle distance) from the vehicle 100 to the outside front target using the standard size of the target stored in the storage unit 42 for each type of target and the size of the outside front target detected in the surrounding image, and calculates the position and speed of the outside front target by tracking the outside front target detected in the surrounding image in a time series. Note that the method of detecting the outside front target is not limited to this method, and various known methods may be used for detection.

[0037] In step S2, the control device 4 detects the driver's line of sight based on the driver data received from the driver sensor 2, specifically, on an external image of the driver received from the driver monitor camera.

[0038] In this embodiment, the control device 4 sequentially applies image processing to the facial image received from the driver monitor camera 21 to detect the position of the pupil center, which is the moving point (i.e., the driver's viewpoint position), and the position of the Purkinje image (corneal reflection image), which is the reference point, and detects the driver's line of sight based on the positional relationship between the Purkinje image and the pupil center (the position of the moving point relative to the reference point).

[0039] In step S3, the control device 4 judges whether or not a vehicle external forward target has been detected. If a vehicle external forward target has been detected, the control device 4 proceeds to processing in step S4. On the other hand, if a vehicle external forward target has not been detected, the control device 4 proceeds to processing in S5.

[0040] In step S4, the control device 4 judges whether the distance between the vehicle and the external forward object is equal to or greater than a predetermined threshold. If the distance between the vehicle and the external forward object is equal to or greater than the predetermined threshold, the control device 4 judges that the external forward object is far away and proceeds to processing in step S5. On the other hand, if the distance between the vehicle and the external forward object is less than the predetermined threshold, the control device 4 judges that the external forward object is nearby and proceeds to processing in step S6.

[0041] In step S5, the control device 4 sets the visual determination range of the in-vehicle device to a first range. As shown in FIG. 4, the visual determination range of the in-vehicle device is a certain range in the vertical direction for determining whether or not the driver is visually checking the in-vehicle device attached to the dashboard (the car navigation monitor in the example shown in FIG. 4). The upper end position of the first range is set to any position between the upper end position of the in-vehicle device (the upper end position of the car navigation monitor in the example shown in FIG. 4) and the lower end position of the windshield (the upper end position of the dashboard). The lower end position of the first range can be, for example, the lower end position of the in-vehicle device (the lower end position of the car navigation monitor in the example shown in FIG. 4), but may be lower than that position or may be somewhat higher than that position.

[0042] In step S6, the control device 4 sets the visual determination range of the in-vehicle device to a second range different from the first range. The upper end position of the second range is set lower than the upper end position of the first range. The reason for this will be described below.

[0043] As shown in Fig. 4, if the driver's line of sight is directed above the bottom edge of the windshield, it can be determined that the driver is looking at an external target ahead of the vehicle, such as another vehicle, through the windshield. Conversely, if the driver's line of sight is directed below the bottom edge of the windshield, it can be determined that the driver is not looking at an external target ahead of the vehicle through the windshield, but is looking at an in-vehicle device.

[0044] Here, a method for sensitively detecting whether the driver is looking at the in-vehicle device is to set the upper limit position of the visual judgment range of the in-vehicle device as high as possible, for example, to the bottom edge of the windshield. This makes it possible to quickly detect whether the driver is looking at the in-vehicle device when the driver's line of sight is directed downward from the bottom edge of the windshield.

[0045] However, as shown in FIG. 5, when a driver is looking at an external object ahead of the vehicle through the windshield, if the external object ahead of the vehicle is close to the vehicle, the driver's line of sight tends to be directed downward in the vertical direction (height direction) compared to when the object is far away.

[0046] Therefore, if the upper limit position of the visual judgment range of the in-vehicle equipment is set to the upper side in order to sensitively detect whether the driver is looking at the in-vehicle equipment, when the driver is looking at an object ahead of the vehicle outside the vehicle through the windshield and the distance between the vehicle and the object ahead of the vehicle outside the vehicle gradually becomes smaller and the driver's gaze direction turns downward, the gaze direction may temporarily fall within the visual judgment range of the in-vehicle equipment, and as a result, it may be determined that the driver is looking at the in-vehicle equipment while looking at the object ahead of the vehicle outside the vehicle through the windshield.

[0047] Therefore, in this embodiment, when the distance between the vehicle and the external forward object is less than a predetermined threshold, that is, when it can be determined that the external forward object is nearby, the visual determination range of the in-vehicle device is set to a second range whose upper end position is set lower than the first range. As a result, even if the distance between the vehicle and the external forward object gradually decreases and the driver's line of sight turns downward, the line of sight is unlikely to fall within the visual determination range of the in-vehicle device, so that it is possible to suppress the in-vehicle device from determining that the driver is looking aside.

[0048] In step S7, the control device 4 judges whether the driver is looking aside at the in-vehicle device, specifically, whether the driver's line of sight is within the visual judgment range of the in-vehicle device for a predetermined inattentive judgment time (e.g., 2 seconds) or more. If the driver's line of sight is within the visual judgment range of the in-vehicle device for a predetermined inattentive judgment time or more, the control device 4 judges that the driver is not looking at the object ahead outside the vehicle and is looking aside, and proceeds to the process of step S8. On the other hand, if the driver's line of sight is not within the visual judgment range of the in-vehicle device for a predetermined inattentive judgment time or more, the control device 4 ends the current process.

[0049] In step S8, the control device 4 issues a warning to the driver regarding inattentive driving via the output device 3. The method and content of the inattentive driving warning are not particularly limited, and for example, the warning may be issued by making a voice announcement that the vehicle is unable to recognize a target ahead of the vehicle, by issuing a warning sound, by displaying image information or text information on a display, or by a combination of these.

[0050] According to the present embodiment described above, the control device 4 (inattentive judgment device) that judges whether the driver of the vehicle 100 is looking away detects targets (external front targets) present in front of the vehicle 100 based on surrounding data that represents the situation around the vehicle 100, detects the driver's viewpoint position and line of sight based on driver data that represents the driver's state, changes the vertical visual judgment range to judge whether the driver is looking at in-vehicle equipment located at the front of the vehicle according to the positional relationship with the external front target, judges that the driver is looking away from the in-vehicle equipment if the driver's line of sight direction remains within the visual judgment range for a predetermined inattentive judgment time or more, and is configured to change the visual judgment range so that the upper end position of the visual judgment range is lower when the external front target is located nearby compared to when the external front target is located far away.

[0051] As a result, even if the distance between the vehicle and the external forward target gradually decreases and the driver's line of sight turns downward, the line of sight is unlikely to fall within the visual judgment range of the in-vehicle device, so that it is possible to suppress erroneous judgment that the driver is visually checking the in-vehicle device even though the driver is visually checking the external forward target.

[0052] In this embodiment, the control device 4 is configured to grasp the positional relationship with the external front object by detecting the distance between the vehicle and the external front object, and to determine that the external front object is located far away if the distance between the vehicle and the external front object is equal to or greater than a predetermined threshold, and to determine that the external front object is located nearby if the distance between the vehicle and the external front object is less than the predetermined threshold.

[0053] This makes it possible to appropriately change the upper end position of the visual determination range in accordance with the inter-vehicle distance to the external forward target.

[0054] In addition, when the external forward target is located far away, the upper end position of the visual determination range can be set to any position between the upper end position of the in-vehicle equipment and the lower end position of the windshield of the vehicle 100, for example.

[0055] As a result, when the external forward target is located far away, it is possible to detect with high sensitivity that the driver is looking at the in-vehicle equipment when the driver's line of sight is directed downward from the bottom edge of the windshield. Conversely, when the external forward target is located close to the vehicle, the upper edge position of the visual judgment range is changed to the lower side compared to when the external forward target is located far away, so that the sensitivity of the visual judgment of the in-vehicle equipment can be reduced.

[0056] Second embodiment Next, a second embodiment of the present invention will be described. This embodiment differs from the first embodiment in that, when a target ahead of the vehicle is present far away and visual inspection (looking aside) of an in-vehicle device has already been confirmed, even if the target ahead of the vehicle approaches and the inter-vehicle distance becomes less than a predetermined threshold value, the visual inspection determination range is not changed to the second range but is maintained as the first range. The difference will be described below.

[0057] In the first embodiment described above, when an external object ahead of the vehicle is already being visually observed through the windshield and the external object ahead of the vehicle approaches, the visual judgment range of the in-vehicle equipment is changed from the first range to the second range, and the sensitivity of the visual judgment of the in-vehicle equipment is suppressed, thereby preventing the in-vehicle equipment from determining that the driver is looking away despite the driver being visually observed at the external object ahead of the vehicle.

[0058] However, in the first embodiment, when a distant external target ahead of the vehicle is not viewed through the windshield but the driver is viewing the in-vehicle device, if the external target ahead of the vehicle approaches suddenly, the viewing determination range of the in-vehicle device may be set to the second range before a predetermined time has elapsed since the driver started viewing the in-vehicle device. In this case, if the driver's line of sight is within a region that is included in the first range but not included in the second range, it becomes impossible to issue a warning to the driver about inattentive driving.

[0059] In this embodiment, if the distance between the vehicle and the vehicle ahead becomes less than a predetermined distance during the period from when the driver starts to visually check the in-vehicle device until the predetermined inattentiveness determination time has elapsed, the visual determination range of the in-vehicle device is not changed to the second range but is maintained at the first range. As a result, even if a distant external target ahead of the vehicle suddenly approaches while the driver is not visually checking the in-vehicle device, the visual determination range of the in-vehicle device is not changed from the first range, so that a warning about inattentiveness can be given to the driver.

[0060] Fig. 6 is a flowchart for explaining the details of the inattentive driving determination process of the in-vehicle device according to this embodiment. The control device 4 repeatedly executes this routine at a predetermined calculation period. In Fig. 6, the contents of the process from step S1 to S8 are the same as those in the first embodiment, so the description will be omitted here.

[0061] In step S21, the control device 4 judges whether the driver is looking aside at the in-vehicle device. If the driver's line of sight is within the visual judgment range of the in-vehicle device, the control device 4 judges that the driver is looking aside at the in-vehicle device and proceeds to the process of step S22. On the other hand, if the driver's line of sight is not within the visual judgment range of the in-vehicle device, the control device 4 judges that the driver is not looking aside at the in-vehicle device and proceeds to the process of step S24.

[0062] In step S22, the control device 4 judges whether the time elapsed since the driver started looking inattentively at the in-vehicle device is equal to or greater than the inattentiveness judgment time. If the time elapsed since the driver started looking inattentively is equal to or greater than the inattentiveness judgment time, the control device 4 proceeds to processing in step S23 to warn the driver about looking inattentively at the in-vehicle device. On the other hand, if the time elapsed since the driver started looking inattentively is less than the inattentiveness judgment time, the control device 4 judges that it is too early to issue a warning and proceeds to processing in step S25.

[0063] In step S23, the control device 4 issues a warning to the driver regarding inattentive driving via the output device 3, similar to the process in step S8 of FIG.

[0064] In step S24, the control device 4 sets the flag F to 0. The flag F is a flag that is set to 1 during a period from when the driver starts to look inattentively at the in-vehicle device until the inattentiveness determination time has elapsed, and is a flag that is initially set to 0.

[0065] In step S25, the control device 4 sets a flag F to 1.

[0066] In step S26, the control device 4 judges whether or not the flag F is set to 1. When the flag F is set to 1, that is, when the driver is not visually checking a distant external object in front of the vehicle through the windshield and is looking inattentively at an in-vehicle device, and the inter-vehicle distance from the external object in front of the vehicle becomes less than a predetermined threshold before the inattentiveness judgment time has elapsed since the driver started looking inattentively at the in-vehicle device, the control device 4 proceeds to processing in step S5 and maintains the visual judgment range of the in-vehicle device as the first range without changing it to the second range. On the other hand, when the flag F is set to 0, the control device 4 proceeds to processing in step S6.

[0067] In this way, the control device 4 (inattentive judgment device) according to this embodiment is configured not to change the visual judgment range when the positional relationship with an object ahead of the vehicle changes from a distant state to a close state between the time when the driver's line of sight falls within the visual judgment range and the time when the inattentive judgment time has elapsed.

[0068] As a result, if the inter-vehicle distance from the vehicle ahead outside the vehicle falls below a predetermined distance during the period from when the driver starts to visually check the in-vehicle device until the inattentiveness determination time has elapsed, the visual determination range of the in-vehicle device can be maintained at the first range without being changed to the second range. Therefore, even if a distant external target ahead of the vehicle suddenly approaches while the driver is not visually checking the inattentiveness of the in-vehicle device and the target suddenly approaches, the visual determination range of the in-vehicle device is not changed from the first range, so that the driver can be sensitively detected as being visually checking the in-vehicle device and a warning about inattentiveness can be given to the driver.

[0069] Third embodiment Next, a third embodiment of the present invention will be described. This embodiment differs from the first embodiment in the method of determining whether or not the driver is looking aside at an in-vehicle device. The difference will be described below.

[0070] As shown in Figures 7A and 7B, the line extending horizontally from the driver's viewpoint is the horizon line A, the line extending from the driver's viewpoint toward the line of sight downward from the horizon is the line of sight direction line B, the line extending from the driver's viewpoint toward the upper end position of the visual judgment range of the in-vehicle device (in this example, the upper end position of the car navigation monitor) is the inattentive judgment line C, the angle formed by the horizontal line A and the line of sight parallel line B is the line of sight angle Θ1, and the angle formed by the horizontal line A and the inattentive judgment line C is the inattentive judgment angle Θ2, when the line of sight angle Θ1 is equal to or larger than the inattentive judgment angle Θ2, it can be determined that the driver is inattentive to the in-vehicle device. Note that Figure 7A shows an example where an external forward target is present in the distance, and Figure 7B shows an example where an external forward target is present in the vicinity.

[0071] When the driver is looking at an external target ahead of the vehicle, if the external target ahead of the vehicle is nearby as shown in Fig. 7B, the line of sight is directed downward, and the line of sight angle Θ1 tends to be larger, compared to when the external target ahead of the vehicle is far away as shown in Fig. 7A. Therefore, when the external target ahead of the vehicle is nearby, even if the driver is looking at the external target ahead of the vehicle, the angle difference between the line of sight angle Θ1 and the inattentiveness judgment angle Θ2 tends to be smaller, and there is a risk of erroneously determining that the driver is looking at an in-vehicle device despite looking at the external target ahead of the vehicle.

[0072] Therefore, in this embodiment, when the external forward target is nearby, the inattentive judgment angle Θ2 is set to be larger than when the external forward target is far away. Even in this way, the same effects as those of the first embodiment can be obtained.

[0073] Fig. 8 is a flowchart for explaining details of the inattentive driving determination process of the in-vehicle device according to this embodiment. The control device 4 repeatedly executes this routine at a predetermined calculation period. In Fig. 6, the contents of the processes from steps S1 to S4, S7, and S8 are the same as those in the first embodiment, so the explanation will be omitted here.

[0074] In step S31, the control device 4 detects the driver's viewpoint position and line-of-sight direction, as well as the line-of-sight angle Θ1, in the same manner as in the process of step S2 in FIG.

[0075] In step S32, the control device 4 sets the inattentive behavior judgment angle Θ2 to a first angle. The first angle can be, for example, the angle between the inattentive behavior judgment line C and the horizontal line A when the upper end position of the visual judgment range of the in-vehicle device is set to the upper end position of the first range described above in the first embodiment.

[0076] In step S33, the control device 4 sets the inattentive behavior judgment angle Θ2 to a second angle greater than the first angle. The second angle can be set to, for example, the angle between the inattentive behavior judgment line C and the horizontal line A when the upper end position of the visual judgment range of the in-vehicle device is set to the upper end position of the second range described above in the first embodiment.

[0077] In step S34, the control device 4 judges whether the driver is looking aside at the in-vehicle device, specifically, whether the line of sight angle Θ1 is equal to or greater than the inattentiveness judgment angle Θ2 for a predetermined inattentiveness judgment time or more. If the line of sight angle Θ1 is equal to or greater than the inattentiveness judgment angle Θ2 for a predetermined inattentiveness judgment time or more, the control device 4 judges that the driver is looking aside at the in-vehicle device and proceeds to the process of step S8. On the other hand, if the line of sight angle Θ1 is equal to or greater than the inattentiveness judgment angle Θ2 for a predetermined inattentiveness judgment time or more, the control device 4 ends the current process.

[0078] The control device 4 (inattentiveness determination device) according to the present embodiment described above is configured to detect targets (external forward targets) present in front of the vehicle 100 based on surrounding data that indicates the situation around the vehicle 100, detect the driver's viewpoint position, line of sight direction, and line of sight angle based on driver data that indicates the driver's state, change the inattentiveness determination angle for determining whether the driver is looking at an in-vehicle device arranged in the front of the vehicle according to the positional relationship with the external forward target, determine that the driver is looking at the inattentive device if the driver's line of sight angle is equal to or greater than the inattentiveness determination angle for a predetermined inattentiveness determination time or more, and increase the inattentiveness determination angle when the target is located nearby compared to when it is located far away. The line of sight angle is the angle between the horizon line extending horizontally from the driver's viewpoint and the line of sight line extending from the driver's viewpoint toward the line of sight pointing downward below the horizon, and the inattentiveness determination angle is the angle between the horizon line and the inattentiveness determination line extending from the driver's viewpoint toward the upper end position of the in-vehicle device.

[0079] As a result, when the distance between the vehicle and the object ahead outside the vehicle gradually becomes smaller and the driver's line of sight turns downward, the line of sight angle is less likely to become equal to or greater than the inattentive viewing judgment angle, which prevents the driver from erroneously determining that he or she is looking at an in-vehicle device when in fact he or she is actually looking at an object ahead outside the vehicle, as in the first embodiment.

[0080] Although the embodiments of the present invention have been described above, the above-mentioned embodiments merely show some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above-mentioned embodiments.

[0081] For example, in each of the above-described embodiments, whether the external forward target is far away or nearby is determined according to the distance between the vehicle and the external forward target, and the visual determination range of the in-vehicle device is changed. Specifically, if the distance between the vehicle and the external forward target is equal to or greater than a predetermined threshold, the visual determination range of the in-vehicle device is determined to be far away and the visual determination range of the in-vehicle device is set to a first range, whereas if the distance between the vehicle and the external forward target is less than the predetermined threshold, the visual determination range of the in-vehicle device is determined to be nearby and the visual determination range of the in-vehicle device is set to a second range.

[0082] However, the present invention is not limited to this. For example, as described above with reference to Fig. 7A and Fig. 7B, when the driver is looking at the vehicle external front target, the line of sight is directed downward when the vehicle external front target is nearby, and the line of sight angle Θ1 tends to be larger than when the vehicle external front target is far away. Therefore, it may be determined whether the vehicle external front target is far away or nearby according to the line of sight angle Θ1. Specifically, if the line of sight angle Θ1 is equal to or smaller than a predetermined angle, it may be determined that the vehicle external front target is far away, and if the line of sight angle Θ1 is larger than the predetermined angle, it may be determined that the vehicle external front target is nearby.

[0083] In other words, the control device 4 may be configured to grasp the positional relationship with an object ahead of the vehicle by detecting the driver's line of sight angle Θ1, and determine that the object ahead of the vehicle is located far away if the line of sight angle Θ1 is equal to or smaller than a predetermined angle, and determine that the object ahead of the vehicle is located nearby if the line of sight angle Θ1 is greater than the predetermined angle.

[0084] In addition, for example, in each of the above embodiments, when the external forward target is located nearby, the inattentiveness judgment time may be set to be longer than when the external forward target is located far away. This makes it possible to suppress the sensitivity of the visual judgment of the in-vehicle device when the external forward target is located nearby compared to when the external forward target is located far away, thereby further suppressing the erroneous judgment that the in-vehicle device is being visually judged when the external forward target is being visually judged.

[0085] For example, in the above embodiment, the computer program executed in the control device 4 (the inattentive driving judgment process for in-vehicle equipment) may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium. [Explanation of symbols]

[0086] 4. Control device (inattentive driving detection device) 100 vehicles

Claims

1. An inattentive driving determination device for determining whether a driver of a vehicle is inattentive, comprising: Detecting a target present in front of the vehicle based on surrounding data representing a situation around the vehicle; Detecting a viewpoint position and a line of sight direction of the driver based on driver data representing a state of the driver; changing a vertical visual determination range for determining whether the driver is visually checking an in-vehicle device disposed at a front portion of the vehicle in accordance with a positional relationship with the target; If the direction of the driver's line of sight is continuously within the visual determination range for a predetermined inattentiveness determination time or more, it is determined that the driver is inattentively looking at the in-vehicle device, When the target is located nearby, the visual determination range is changed so that the upper end position of the visual determination range is lower than when the target is located far away. Inattentive detection device.

2. When the target is located far away, the upper end position of the visual determination range is set to an arbitrary position between the upper end position of the in-vehicle device and the lower end position of the windshield of the vehicle. The inattentive driving detection device according to claim 1 .

3. When the positional relationship between the driver's line of sight and the target changes from a distant state to a nearby state during the period from when the driver's line of sight direction falls within the visual judgment range to when the inattentive judgment time has elapsed, the visual judgment range is not changed. The inattentive driving detection device according to claim 1 or 2.

4. A positional relationship with the target is grasped by detecting a distance to the target, If the distance to the target is equal to or greater than a predetermined threshold, it is determined that the target is located far away; If the distance to the target is less than a predetermined threshold, it is determined that the target is located nearby. The inattentive driving detection device according to claim 1 or 2.

5. The positional relationship with the target is grasped by detecting the line of sight angle of the driver, The line of sight angle is an angle between a horizontal line extending horizontally from the driver's viewpoint and a line of sight direction extending from the driver's viewpoint in a direction downward from the horizontal line, If the line of sight angle is equal to or smaller than a predetermined angle, it is determined that the target is located far away, If the line of sight angle is greater than a predetermined angle, it is determined that the target is located nearby. The inattentive driving detection device according to claim 1 or 2.

6. When the target is located nearby, the inattentive judgment time is made longer than when the target is located far away. The inattentive driving detection device according to claim 1 or 2.

7. When it is determined that the driver is looking away from the vehicle, a warning is given to the driver regarding the driver looking away from the vehicle. The inattentive driving detection device according to claim 1 or 2.

8. An inattentive driving determination device for determining whether a driver of a vehicle is inattentive, comprising: Detecting a target present in front of the vehicle based on surrounding data representing a situation around the vehicle; Detecting a viewpoint position, a line of sight direction, and a line of sight angle of the driver based on driver data representing a state of the driver; changing an inattentiveness determination angle for determining whether the driver is looking at an in-vehicle device disposed at a front portion of the vehicle in accordance with a positional relationship with the target; If the driver's line of sight angle is equal to or greater than the inattentiveness determination angle for a predetermined inattentiveness determination time or more, it is determined that the driver is inattentively looking at the in-vehicle device, When the target is located nearby, the inattentive judgment angle is made larger than when the target is located far away; The line of sight angle is an angle between a horizontal line extending horizontally from the viewpoint of the driver and a line of sight direction extending from the viewpoint of the driver in a direction downward from the horizontal line, The inattentive driving judgment angle is an angle between the horizontal line and an inattentive driving judgment line extending from the driver's viewpoint toward the upper end position of the in-vehicle device. Inattentive detection device.

9. An inattentive driving judgment method by an inattentive driving judgment device for judging an inattentive driving of a vehicle driver, comprising: Detecting a target present in front of the vehicle based on surrounding data representing a situation around the vehicle; Detecting a viewpoint position and a line of sight direction of the driver based on driver data representing a state of the driver; changing a vertical visual determination range for determining whether the driver is visually checking an in-vehicle device disposed at a front portion of the vehicle in accordance with a positional relationship with the target; If the direction of the driver's line of sight is continuously within the visual determination range for a predetermined inattentiveness determination time or more, it is determined that the driver is inattentively looking at the in-vehicle device, When the target is located nearby, the visual determination range is changed so that the upper end position of the visual determination range is lower than when the target is located far away. A method for determining inattentiveness.

10. A computer program for an inattentive driver detection device that detects an inattentive driver of a vehicle, comprising: Detecting a target present in front of the vehicle based on surrounding data representing a situation around the vehicle; Detecting a viewpoint position and a line of sight direction of the driver based on driver data representing a state of the driver; changing a vertical visual determination range for determining whether the driver is visually checking an in-vehicle device disposed at a front portion of the vehicle in accordance with a positional relationship with the target; If the direction of the driver's line of sight is continuously within the visual determination range for a predetermined inattentiveness determination time or more, it is determined that the driver is inattentively looking at the in-vehicle device, When the target is located nearby, the visual determination range is changed so that the upper end position of the visual determination range is lower than when the target is located far away. A computer program for causing the inattentive driving judgment device to execute the above.

Citation Information

Patent Citations

  • Apparatus for detecting looking aside

    JP2004114931A

  • Looking-aside detection device

    JP2004347464A

  • Look-away detecting device, method and program

    JP2009157736A

  • Inattentive driving determination device, inattentive driving determination method, and program

    JP2019091268A

  • Visual notification of distracted driving

    US20230066670A1