Device for estimating visual field defect of driver

Abstract

To estimate a state of a visual field defect of a driver.SOLUTION: A control section 3 of a device for estimating a visual field defect of a driver includes: a first identification section (S1, S6) for analyzing a photographing image of an exterior camera 1 so as to identify an object to be an index for safety driving operation behavior; a second identification section (S2-S5) for selecting driving operation information related to the object identified by the first identification section (S1, S6) from the driving operation information to be acquired by in-vehicle CAN communication, and comparing the selected driving operation information with a predetermined threshold, so as to identify the driving operation behavior; and an estimation section (table 1) for collating the driving operation behavior identified by the second identification section (S2-S5) with state identification information stored in a non-volatile storage device of the control section 3, so as to estimate the state of the visual field defect of a driver.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a driver's visual field defect estimation device. [Background technology]

[0002] For example, Patent Document 1 describes that "a driver state estimation device (1) that estimates the state of a driver driving a vehicle includes a steering sensor (6) that detects steering operations by the driver, a driver camera (8) that captures an image of the driver driving the vehicle, an image analysis unit (16) that detects the direction of the driver's line of sight based on the image captured by the driver camera, and a driver abnormality determination unit (18) that determines whether there is an abnormality in the driver based on the steering operation detected by the steering sensor (6) and the line of sight detected by the image analysis unit, and the driver abnormality determination unit (18) determines that there is a suspicion of visual field loss in the driver when an expansion of the distribution of the driver's line of sight direction relative to the normal state is detected and a change in steering operation relative to the normal state is detected." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-198842 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned Patent Document 1, a suspected visual field defect of the driver is determined based on the steering operation and the direction of the line of sight, but the technical idea of ​​estimating the state of the visual field defect is not apparent.

[0005] In view of the above circumstances, an object of the present invention is to provide a driver's visual field defect estimation device that can estimate the state of a driver's visual field defect. [Means for solving the problem]

[0006] The driver's visual field defect estimation device of the present invention comprises an exterior camera that captures still images and videos of the driving environment as seen by the driver of the vehicle; a memory unit that stores state identification information that correlates the driver's driving operation behavior with the driver's state of visual field defect; and a control unit that acquires operation information (driving operation information) of each functional element that operates in response to the driver's driving operation behavior via in-vehicle CAN communication and estimates the driver's state of visual field defect.The control unit is characterized by including a first identification unit that identifies a target object that serves as an indicator for performing safe driving operation behavior by analyzing the image captured by the exterior camera; a second identification unit that selects driving operation information related to the target object identified by the first identification unit from the driving operation information acquired via the in-vehicle CAN communication and identifies the driving operation behavior by comparing the selected driving operation information with a predetermined threshold; and an estimation unit that estimates the driver's state of visual field defect by comparing the driving operation behavior identified by the second identification unit with the state identification information.

[0007] This configuration estimates the state of visual field loss without taking pictures of the driver's eyes with a camera. As a result, the present invention makes it possible to easily estimate the state of visual field loss with a relatively simple configuration. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a driver's visual field defect estimation device that can estimate the state of a driver's visual field defect. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram schematically illustrating the configuration of an embodiment of a driver visual field defect estimation device according to the present invention. [Figure 2] 10 is a flowchart showing a first requirement for estimating the state of a driver's visual field defect. [Figure 3] 10 is a flowchart showing a second requirement for estimating the state of a driver's visual field defect. [Figure 4]10 is a flowchart showing a third requirement for estimating the state of a driver's visual field defect. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the accompanying drawings.

[0011] An embodiment of the present invention is shown in Figures 1 to 4. The driver's visual field defect estimation device shown in the figures is mounted on a vehicle (not shown), and includes an outside camera 1, a notification unit 2, and a control unit 3.

[0012] The vehicle exterior camera 1 is mounted, for example, on a rearview mirror inside the vehicle so as to face the front of the vehicle, and captures still and moving images of the driving environment as seen by the driver of the vehicle.

[0013] The vehicle exterior camera 1 is configured to incorporate, for example, an imaging element (CCD: Charge-Coupled Device, CMOS: Complementary Metal-Oxide-Semiconductor) or the like.

[0014] The notification unit 2 can be, for example, at least one of a video display device for outputting text regarding the estimated result of the driver's visual field defect, and an audio output device (a speaker that is standard equipment in the vehicle or an aftermarket speaker) for outputting audio regarding the estimated result of the visual field defect.

[0015] The image display device may be a display such as a meter that is standard equipment on the vehicle, or a retrofitted display, but if the vehicle is already equipped with a car navigation system, it is also possible to use the display of that car navigation system.

[0016] The control unit 3 is a known electronic control unit (ECU) and includes a central processing unit (CPU), a non-volatile memory (Read Only Memory: ROM), and a temporary memory (Random Access Memory: RAM), as well as an input interface circuit for inputting the detection signals and an output interface circuit for outputting the control signals. The ROM stores various control programs and maps referenced when executing the various control programs. The CPU performs arithmetic processing based on the various control programs and maps stored in the ROM. The RAM is a memory that temporarily stores the results of calculations by the CPU and data input from each sensor.

[0017] Specifically, the control unit 3 is configured to receive each detection signal from the outside camera 1 and output a control signal to the notification unit 2.

[0018] The control unit 3 also acquires operation information (driving operation information) of each functional element that operates in response to a driving operation action by the driver through in-vehicle CAN (Controller Area Network) communication.

[0019] The functional elements correspond to an accelerator pedal, brake pedal, steering wheel, drive shaft, etc., which are not shown. The driving operation information corresponds to detection signals from an accelerator position sensor, brake stroke sensor, steering sensor, vehicle speed sensor, acceleration sensor, etc., which are not shown. The accelerator position sensor detects the depression depth of the accelerator pedal. The brake stroke sensor detects the depression force of the brake pedal. The steering sensor is a sensor that detects the steering angle by the driver, and is, for example, a rotary encoder that detects the rotation angle of a steering shaft, not shown. The vehicle speed sensor detects the traveling speed of the vehicle (vehicle speed). The acceleration sensor detects the acceleration of the vehicle (longitudinal acceleration, lateral acceleration, vertical acceleration, etc.).

[0020] Furthermore, the nonvolatile storage device (not shown) of the control unit 3 stores state specification information (see Table 1) that associates the driver's driving operation behavior with the state of the driver's visual field defect. The nonvolatile storage device corresponds to the storage unit described in the claims.

[0021] The driving operation behaviors refer to actions performed when braking, accelerating, turning, and the like.

[0022] [Table 1] Then, while driving the vehicle, the control unit 3 performs the following processes: identifying a target object by analyzing the image (detection signal) captured by the external camera 1; selecting driving operation information related to the identified target object from the driving operation information acquired through in-vehicle CAN communication; partially or comprehensively accumulating analysis information of the captured image and the driving operation information; identifying a driving operation behavior by comparing the selected driving operation information with a predetermined threshold; and estimating the driver's visual field defect state by matching the identified driving operation behavior with the state identification information.

[0023] The captured image can be analyzed by a known technique. The target object is an indicator for enabling the driver to perform safer driving operations, such as pedestrians, traffic lights, road signs, the lane to be driven in, and other signage. Furthermore, in this embodiment, the control unit 3 is configured to output a command to the notification unit 2 to visually or audibly notify the driver of the estimated visual field defect.

[0024] Next, the operation of the control unit 3 will be described with reference to FIGS.

[0025] The flowcharts shown in FIGS. 2 to 4 start when the start switch of the vehicle is turned on or when the vehicle starts to travel.

[0026] 2, in step S1, it is determined whether a "red light" has appeared as a target object that serves as an indicator of safe driving. Here, the appearance of a "red light" as a target object is determined by analyzing the captured image (detection signal) input from the outside camera 1.

[0027] If the determination in step S1 is affirmative, the process proceeds to steps S2-S5, whereas if the determination is negative, the process proceeds to steps S6-S10.

[0028] First, in step S2, the peak value of longitudinal acceleration (also called longitudinal G) due to braking at one intersection is selected. The braking occurs due to tire friction resistance caused by the driver's application of the brake or engine braking caused by releasing the accelerator pedal. Here, longitudinal acceleration due to braking is acquired via in-vehicle CAN communication, and the peak value of this acquired longitudinal acceleration is selected.

[0029] In the next step S3, the variance of the peak value of the longitudinal acceleration during the period from the departure of the vehicle to the arrival is determined.

[0030] Thereafter, in step S4, it is determined whether or not "N number > A" and "variance > B" are established. Note that "A" and "B" are parameters set to enable separation between healthy subjects with no visual problems and diseased subjects with visual field defects, and are set based on the results of experiments using the healthy subjects and the diseased subjects as subjects.

[0031] In general, the affected person frequently brakes suddenly due to a delay in noticing a red light or a stop sign, resulting in a larger variance in the peak value of longitudinal acceleration compared to the healthy person. The N number is a parameter for ensuring the reliability of the judgment. For example, there are cases where the driver delays noticing a red light or a stop sign and brakes suddenly, resulting in the vehicle passing through the intersection, or where the driver does not notice the red light or stop sign and enters the intersection without braking.

[0032] If the determination in step S4 is negative, the process ends; if the determination is positive, the state of visual field defect is recognized as "upper defect" in step S5, the "red light emergency braking flag" is turned ON, and the process ends.

[0033] Next, in step S6, it is determined whether a "stop sign" has appeared as a target object that serves as an indicator of safe driving. Here, the appearance of a "stop sign" as a target object is determined by analyzing the captured image (detection signal) input from the outside camera 1.

[0034] If the determination in step S6 is negative, the process returns to step S1, whereas if the determination is positive, the process proceeds to step S7.

[0035] In step S7, the peak value of the longitudinal acceleration due to braking at one intersection is selected. Here, the longitudinal acceleration due to braking is acquired via in-vehicle CAN communication, and the peak value of the acquired longitudinal acceleration is selected.

[0036] In the next step S8, the variance of the peak value of the longitudinal acceleration during the period from the departure of the vehicle to the arrival is determined.

[0037] Thereafter, in step S9, it is determined whether or not "N number > C" and "variance > D" are established. Note that "C" and "D" are parameters set to enable separation of the healthy subjects and the diseased subjects, and are set based on the results of an experiment using the healthy subjects and the diseased subjects as subjects.

[0038] If the determination in step S9 is negative, the process ends; if the determination is positive, the process recognizes the visual field defect as "upper left defect" in the following step S10, turns on the "temporary stop emergency braking flag," and ends.

[0039] 3, in step S11, it is determined whether or not a "red light" has appeared as a target object that serves as an indicator of safe driving. Here, the appearance of a "red light" as a target object is determined by analyzing the captured image (detection signal) input from the outside camera 1.

[0040] If the determination in step S11 is affirmative, the process proceeds to steps S12-S14, whereas if the determination is negative, the process proceeds to steps S15-S18.

[0041] First, in step S12, the minimum speed for one intersection is selected. Here, the speed is acquired through in-vehicle CAN communication, and the minimum speed among the acquired speeds is selected.

[0042] In the next step S13, it is determined whether or not "minimum speed > E" is established. If the determination in step S13 is negative, the process ends. However, if the determination is positive, the process in the next step S14 recognizes that the visual field defect state is "upper visual field defect," turns on the "red light running flag," and ends.

[0043] Incidentally, serious accidents can occur when a driver is late in noticing a red light or a stop sign and applies sudden braking, but still passes through the intersection, or when a driver does not notice a red light or a stop sign and passes through the intersection without braking. The minimum speed differs in these two cases. In other words, the minimum speed is a parameter that captures these two cases.

[0044] Next, in step S15, it is determined whether a "stop sign" has appeared as a target object that serves as an indicator of safe driving. Here, the appearance of a "stop sign" as a target object is determined by analyzing the captured image (detection signal) input from the outside camera 1.

[0045] If the determination in step S15 is negative, the process returns to step S11, whereas if the determination is positive, the process proceeds to step S16, where the minimum speed for one intersection is selected. Here, the speed is acquired via in-vehicle CAN communication, and the minimum speed among the acquired speeds is selected.

[0046] Then, in step S17, it is determined whether or not "minimum speed > F" is established. If the determination in step S17 is negative, the process ends, whereas if the determination is affirmative, the process in the following step S18 recognizes that the visual field defect state is "upper left defect," turns on the "pause ignore flag," and ends.

[0047] In the flowchart shown in FIG. 4, in step S21, it is determined whether the lateral acceleration (also referred to as lateral G) acquired through in-vehicle CAN communication is less than a predetermined threshold G.

[0048] If the determination in step S21 is negative, the process ends, whereas if the determination is positive, the process proceeds to step S22.

[0049] In step S22, the average value X of the lateral position in one straight line segment is calculated. The "one straight line segment" refers to one of the multiple straight sections in the driving lane of a highway that the vehicle actually traveled through. The straight line segment includes not only straight lines but also sections that can be considered straight because of low lateral acceleration. The "lateral position" refers to the lateral position of the vehicle relative to the center of the driving lane.

[0050] In step S23, the average value Y of the lateral position for the main lane travel is calculated. The "main lane travel" refers to all of the straight sections that the vehicle has actually traveled among the multiple straight sections in the travel lane of the expressway. The straight sections include not only straight sections, but also sections that can be considered straight because of low lateral acceleration. The "lateral position" refers to the lateral position of the vehicle relative to the center of the travel lane.

[0051] In steps S22 and S23, vehicle driving information is acquired through in-vehicle CAN communication, and calculations can be made based on this acquired vehicle driving information.

[0052] In step S24, it is determined whether or not "average value X>H" is established. If the determination in step S24 is affirmative, the state of the visual field defect is recognized as "right-side defect" in step S25, the "straight line left deviation flag" is turned ON, and the process ends.

[0053] On the other hand, if a negative determination is made in step S24, it is determined in step S26 whether or not "average value Y>I" is established. Note that "H" and "I" are parameters set so as to be able to separate the healthy subjects from the diseased subjects, and are set based on the results of an experiment using the healthy subjects and the diseased subjects as subjects.

[0054] If the determination in step S26 is negative, the process ends; if the determination is positive, the state of the visual field defect is recognized as "left-side defect" in step S27, the "straight line right deviation flag" is turned ON, and the process ends.

[0055] In the flowchart of Fig. 2, steps S1 and S6 correspond to the first specification section set forth in the claims, and steps S2-S5 and S7-S10 correspond to the second specification section set forth in the claims. In the flowchart of Fig. 3, steps S11 and S15 correspond to the first specification section set forth in the claims, and steps S12-S14 and S16-S18 correspond to the second specification section set forth in the claims. Furthermore, in the flowchart of Fig. 4, step S21 corresponds to the first specification section set forth in the claims, and steps S22-S27 correspond to the second specification section set forth in the claims.

[0056] Next, the control unit 3 estimates the state of the driver's visual field defect by comparing the flag that has been turned ON according to the flowcharts of Figures 2 to 4 with state specifying information (see Table 1) previously stored in the nonvolatile storage device of the control unit 3. This process corresponds to the estimation unit described in the claims.

[0057] (Case 1) When the "sudden braking at red light flag" and the "sudden braking at stop flag" are ON, the state of visual field defect is estimated to be "upper visual field defect."

[0058] (Case 2) If the "Sudden braking at temporary stop flag" and the "Ignoring temporary stop flag" are ON, the visual field defect state is estimated to be "upper left defect."

[0059] (Case 3) If the "stop sign sudden braking flag" and the "straight line right deviation flag" are ON, it is estimated that the visual field defect state is likely to be "upper left defect" and "left side defect."

[0060] (Case 4) If the "Sudden braking at a stop flag", "Ignoring a stop flag", and "Straight line right deviation flag" are ON, the visual field defect state is estimated to be "upper left defect" and "left side defect".

[0061] (Case 5) If the "sudden braking at red light flag," "sudden braking at stop sign flag," "ignoring red light flag," and "ignoring stop sign flag" are ON, the visual field defect is estimated to be "upper visual field defect" and "upper left visual field defect."

[0062] (Case 6) If the "sudden braking at red light flag," "sudden braking at stop sign flag," "ignoring red light flag," "ignoring stop sign flag," and "straight line right deviation flag" are ON, the visual field defect is estimated to be "upper visual field defect," "upper left visual field defect," and "left visual field defect."

[0063] As described above, in the embodiment of the present invention, for example, without capturing images of both of the driver's eyes with a camera, the driving behavior of the driver responding to an object in the image captured by the exterior camera 1 is identified, and the state of visual field defect is estimated by comparing this driving behavior with state identification information (see Table 1) stored in advance in the non-volatile memory device of the control unit 3.

[0064] This makes it possible to easily estimate the state of visual field defect with a relatively simple configuration. Moreover, in this embodiment, the result of the visual field defect state estimation can be quickly notified to the driver by the notification unit 2, which is advantageous in ensuring safe driving of the vehicle.

[0065] The present invention is not limited to the above-described embodiments, but can be modified as appropriate within the scope of the claims and the equivalents thereof.

[0066] (1) For example, although not shown, it is possible to equip a vehicle with a driving assistance device and a communication device, and configure the control unit 3 to output a command signal to the driving assistance device via the communication device based on the visual field defect estimation result to cause the vehicle to avoid danger, and such a configuration is also included in the present invention.

[0067] The driving assistance device includes, for example, a steering actuator, a brake actuator, an accelerator actuator, etc. (all not shown), as well as various sensors that detect the operation of each of the actuators. The driving assistance device is configured to perform collision avoidance operations, lane keep assist operations, etc., by controlling each of the actuators based on detection signals from the various sensors in response to commands sent from the control unit 3.

[0068] (2) For example, in the above embodiment, the control unit 3 causes the notification unit 2 to notify the result of the visual field defect state estimation, but the present invention is not limited to this.

[0069] For example, although not shown, it is possible to configure the above embodiment without the notification unit 2, and such a configuration is also included in the present invention. [Industrial Applicability]

[0070] The present invention can be suitably used in a driver's visual field defect estimation device. [Explanation of symbols]

[0071] 1. Exterior camera 2. Information Department 3. Control Unit

Claims

[Claim 1] An external camera that captures still images and videos of the driving environment as seen by the driver of the vehicle; a storage unit that stores state specification information that associates the driver's driving operation behavior with the driver's visual field defect state; a control unit that acquires operation information (driving operation information) of each functional element that operates in response to a driving operation action by a driver via in-vehicle CAN communication and estimates the state of visual field defect of the driver; The control unit a first identification unit that identifies a target object serving as an indicator for a safe driving operation behavior by analyzing an image captured by the exterior camera; a second identification unit that selects driving operation information related to the target object identified by the first identification unit from the driving operation information acquired through the in-vehicle CAN communication, and identifies a driving operation behavior by comparing the selected driving operation information with a predetermined threshold; and an estimation unit that estimates the driver's visual field defect state by comparing the driving operation behavior identified by the second identification unit with the state identification information.

Images