Driver's visual field defect estimation device

Abstract

To enable estimation of a region of a driver's visual field defect.SOLUTION: A driver's visual field defect estimation device includes a control unit 4 for acquiring operation information (driving operation information) on each function element that is operated in response to a driving operation action by a driver from in-vehicle CAN communication and estimating the driver's visual field defect. The control part 4 includes: a first specification part S1 and S2 for specifying a target object that serves as an index for a safe driving operation action by analyzing an image captured by an exterior camera 1; a second specification part S3 for calculating a visual action by analyzing an image captured by an in-vehicle camera 2, and specifying the driving operation action for the target object on the basis of the visual action; a third specification part S4 for specifying a visual field region divided into a plurality of regions on the basis of a difference between the target object and the visual action; a storage part S5 for storing the target object and the driving operation action for each visual field region; and a calculation part S7 for calculating a visual field defect estimation value for each visual field region.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a driver's visual field defect estimation device. [Background technology]

[0002] For example, Patent Document 1 describes that "a driver state estimation device (1) that estimates the state of a driver driving a vehicle includes a steering sensor (6) that detects steering operations by the driver, a driver camera (8) that captures an image of the driver driving the vehicle, an image analysis unit (16) that detects the direction of the driver's line of sight based on the image captured by the driver camera, and a driver abnormality determination unit (18) that determines whether there is an abnormality in the driver based on the steering operation detected by the steering sensor (6) and the line of sight detected by the image analysis unit, and the driver abnormality determination unit (18) determines that there is a suspicion of visual field loss in the driver when an expansion of the distribution of the driver's line of sight direction relative to the normal state is detected and a change in steering operation relative to the normal state is detected." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-198842 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above Patent Document 1, the driver's suspected visual field defect is determined based on the steering operation and the direction of the line of sight, but the technical idea of ​​recognizing the area of ​​the visual field defect cannot be seen.

[0005] In view of the above circumstances, an object of the present invention is to provide a driver's visual field defect estimation device that can estimate the area of ​​a driver's visual field defect. [Means for solving the problem]

[0006] The present invention comprises an exterior camera that captures still images and videos of the driving environment as seen by the driver of the vehicle, an interior camera that captures still images and videos of both of the driver's eyes, and a control unit that analyzes the captured images input from the exterior camera and the interior camera, and acquires operation information (driving operation information) of each functional element that operates in response to the driver's driving operation behavior via in-vehicle CAN communication, and estimates the driver's visual field defect.The control unit is characterized by including a first identification unit that identifies a target object that serves as an indicator for safe driving operation behavior by analyzing the images captured by the exterior camera, a second identification unit that calculates visual behavior by analyzing the images captured by the interior camera and identifies a driving operation behavior for the target object based on the visual behavior, a third identification unit that identifies multiple visual field areas based on the difference between the target object identified by the first identification unit and the visual behavior calculated by the second identification unit, a storage unit that stores the target object and driving operation behavior for each visual field area, and a calculation unit that calculates an estimated visual field defect for each visual field area.

[0007] According to this configuration, it is possible to estimate whether or not the driver has a visual field defect, and if it is estimated that there is a visual field defect, it is possible to represent the estimated visual field defect area on an XY coordinate diagram.

[0008] This makes it possible to quickly notify the driver of the presence of a visual field defect when it is estimated that the driver has a visual field defect, which is advantageous in ensuring safe driving of the vehicle. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a driver's visual field defect estimation device that can estimate the area of ​​a driver's visual field defect. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram schematically illustrating the configuration of an embodiment of a driver visual field defect estimation device according to the present invention. [Figure 2]4 is a flowchart for explaining processing by a driver visual field defect estimation device. [Figure 3] FIG. 2 is a diagram showing a schematic view of a field of view in terms of XY coordinates. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0012] An embodiment of the present invention is shown in Figures 1 to 3. The driver's visual field defect estimation device shown in the figures is mounted on a vehicle (not shown), and includes an exterior camera 1, an interior camera 2, a notification unit 3, and a control unit 4.

[0013] The vehicle exterior camera 1 is mounted, for example, on a rearview mirror inside the vehicle so as to face the front of the vehicle, and captures still and moving images of the driving environment as seen by the driver of the vehicle.

[0014] The in-vehicle camera 2 is provided, for example, in the vehicle interior near the steering wheel so as to be directed toward both eyes of the driver, and captures still images and moving images of both eyes of the driver.

[0015] The exterior camera 1 and the interior camera 2 each have a built-in imaging element (CCD: Charge-Coupled Device, CMOS: Complementary Metal-Oxide-Semiconductor), for example.

[0016] The notification unit 3 can be, for example, at least one of a video display device for outputting text regarding the estimated result of the driver's visual field defect, and an audio output device (a speaker that is standard equipment in the vehicle or an aftermarket speaker) for outputting audio regarding the estimated result of the visual field defect.

[0017] The image display device may be a display such as a meter that is standard equipment on the vehicle, or a retrofitted display, but if the vehicle is already equipped with a car navigation system, it is also possible to use the display of that car navigation system.

[0018] The control unit 4 is a known electronic control unit (ECU) and includes a central processing unit (CPU), a non-volatile memory (Read Only Memory: ROM), and a temporary memory (Random Access Memory: RAM), as well as an input interface circuit for inputting the detection signals and an output interface circuit for outputting the control signals. The ROM stores various control programs and maps referenced when executing the various control programs. The CPU performs arithmetic processing based on the various control programs and maps stored in the ROM. The RAM is a memory that temporarily stores the results of calculations by the CPU and data input from each sensor.

[0019] Specifically, the control unit 4 is configured to receive detection signals from the exterior camera 1 and the interior camera 2, and to output a control signal to the notification unit 3.

[0020] Furthermore, the control unit 4 acquires operation information (driving operation information) of each functional element that operates in response to the driving operation action of the driver through in-vehicle CAN (Controller Area Network) communication.

[0021] The functional elements correspond to an accelerator pedal, brake pedal, steering wheel, drive shaft, etc., which are not shown. The driving operation information corresponds to detection signals from an accelerator position sensor, brake stroke sensor, steering sensor, vehicle speed sensor, acceleration sensor, etc., which are not shown. The accelerator position sensor detects the depression depth of the accelerator pedal. The brake stroke sensor detects the depression force of the brake pedal. The steering sensor is a sensor that detects the steering angle by the driver, and is, for example, a rotary encoder that detects the rotation angle of a steering shaft, not shown. The vehicle speed sensor detects the traveling speed of the vehicle (vehicle speed). The acceleration sensor detects the acceleration of the vehicle (longitudinal acceleration, lateral acceleration, vertical acceleration, etc.).

[0022] Furthermore, the control unit 4 determines the driving operation behavior a based on the driving operation information obtained through the in-vehicle CAN communication while driving the vehicle. j By analyzing the image (detection signal) captured by the in-vehicle camera 2, the driver's visual behavior (direction, angle, and stagnant state of the line of sight) is identified, and by analyzing the image (detection signal) captured by the outside-vehicle camera 1, the target object o j The state of the driver's visual field defect is estimated by accumulating the identified information either partially or comprehensively.

[0023] The analysis technique for the captured image can be a known technique. j is a safe driving behavior for the driver. j These are indicators to guide people, such as traffic lights, road signs, pedestrians, and other signs.

[0024] Furthermore, the control unit 4 outputs a command to the notification unit 3 to visually or audibly notify the driver of the estimated result of the visual field defect of the driver.

[0025] Next, the operation of the control unit 4 will be described with reference to FIG.

[0026] When the start switch of the vehicle is turned on or the vehicle starts running, the flowchart starts. In step S1, the target object o j Here, the target object o is detected by analyzing the captured image (detection signal) input from the outside camera 1. j The presence or absence of

[0027] If the determination in step S1 is negative, the process ends, but if the determination is positive, the process proceeds to the following step S2.

[0028] In step S2, the target object o detected in step S1 is j In the next step S3, the target object o identified in step S2 is j Driving behavior for a j Identify.

[0029] In step S3, the image captured by the in-vehicle camera 2 is analyzed to detect visual behavior at predetermined time intervals, and the target object o is detected based on the visual behavior. j The system recognizes the driver's cognitive state and performs driving operations such as sudden steering and sudden braking based on the driving operation information acquired through in-vehicle CAN communication. j The visual behavior includes the direction, angle, and stagnant state of the line of sight. The cognitive state includes no problem, oversight, delayed discovery, etc.

[0030] Next, in step S4, the target object o j Based on the difference between the visual field and the visual behavior, the visual field area r is divided into I as shown in Figure 3. i (r1,r2,r3,...r I ) in which a visual field area suspected of having a visual field defect is identified. j ) in a coordinate system and a visual field coordinate system that represents the image captured by the in-vehicle camera 2 (both eyes of the driver) in a coordinate system from the centers of both eyes. j It is possible to capture the difference between visual behavior and

[0031] Furthermore, in step S5, the visual field area r identified in step S4 is i In contrast, the target object o identified in step S2 j and the driving operation behavior a identified in step S3. j is stored in a temporary storage device or the like of the control unit 4.

[0032] After this, in step S6, all the visual field areas r i (r1,r2,r3,...r I ) for the target object o j and driving behavior a j Here, by repeating steps S2 to S6, the visual field areas r1, r2, r3, . . . r are stored. I For each object, j and driving behavior a j is stored in a temporary storage device or the like of the control unit 4.

[0033] If the determination in step S6 is negative, the process returns to step S2, but if the determination is positive, the process proceeds to the following step S7.

[0034] In step S7, the visual field area r i Estimated visual field loss y i This calculation is performed using the following formula 1. [Formula 1] TIFF2025129851000002.tif19140As shown in Figure 3, in the visual field defect estimation process, the output visual field is divided into I regions r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15, r16, r17, I whereas the output is [y1,y2,y3,...y I ] is a vector.

[0035] Estimated visual field defect y i is the visual field area r iIt represents the state of visual field loss and is expressed as a value ranging from "0" to "1." Note that "0" means no loss (no problem), and the closer it is to "1," the greater the degree of visual field loss. Accordingly, the visual field area r shown in Figure 3 i (r1,r2,r3,...r I ) by color-coding or shading the visual field area suspected of having a visual field defect, the driver can visually recognize it.

[0036] In the flowchart of Figure 2, steps S1 and S2 correspond to the first identification unit described in the claims, step S3 corresponds to the second identification unit described in the claims, step S4 corresponds to the third identification unit described in the claims, step S5 corresponds to the storage unit described in the claims, and step S7 corresponds to the calculation unit described in the claims.

[0037] As described above, according to the embodiment to which the present invention is applied, it is possible to estimate whether or not the driver has a visual field defect, and if it is estimated that there is a visual field defect, it is possible to represent the estimated visual field defect area on an XY coordinate diagram.

[0038] This makes it possible to quickly notify the driver of the presence of a visual field defect when it is estimated that the driver has a visual field defect, which is advantageous in ensuring safe driving of the vehicle.

[0039] The present invention is not limited to the above-described embodiments, but can be modified as appropriate within the scope of the claims and the equivalents thereof.

[0040] For example, although not shown, it is possible to equip the vehicle with a driving assistance device and a communication device, and configure the control unit 4 to output a command signal to the driving assistance device via the communication device based on the visual field defect estimation result to cause the vehicle to avoid danger, and such a configuration is also included in the present invention.

[0041] The driving assistance device includes, for example, a steering actuator, a brake actuator, an accelerator actuator, etc. (all not shown), as well as various sensors that detect the operation of each of the actuators. The driving assistance device is configured to perform collision avoidance operations, lane keep assist operations, etc., by controlling each of the actuators based on detection signals from the various sensors in response to commands sent from the control unit 4. [Industrial Applicability]

[0042] The present invention can be suitably used in a driver's visual field defect estimation device. [Explanation of symbols]

[0043] 1. Exterior camera 2. In-car camera 3. Information Department 4. Control section

Claims

[Claim 1] An external camera that captures still images and videos of the driving environment as seen by the driver of the vehicle; An in-car camera that takes still images and videos of both eyes of the driver, a control unit that analyzes the captured images input from the exterior camera and the interior camera, acquires operation information (driving operation information) of each functional element that operates in response to a driving operation action by the driver via in-vehicle CAN communication, and estimates a visual field defect of the driver; The control unit a first identification unit that identifies a target object serving as an indicator for a safe driving operation behavior by analyzing an image captured by the exterior camera; a second identification unit that calculates a viewing behavior by analyzing an image captured by the in-vehicle camera and identifies a driving operation behavior with respect to the target object based on the viewing behavior; a third identification unit that identifies a plurality of divided visual field areas based on a difference between the target object identified by the first identification unit and the visual behavior calculated by the second identification unit; a storage unit for storing target objects and driving operation actions for each of the visual fields; A driver's visual field defect estimation device comprising: a calculation unit that calculates a visual field defect estimation value for each of the visual field areas.

Images