Driving diagnostic device

Abstract

To perform diagnosis in real time as to whether or not a driver is checking a rear side.SOLUTION: An information processing device comprises: an acquisition unit that acquires vehicle information related to operation of a vehicle, a head turning angle of a driver of the vehicle, and a position of a visual line of the driver for a predetermined acquisition time period; an extraction unit that extracts a checking period before the vehicle moves backward from results acquired by the acquisition unit; and a diagnosis unit that performs a diagnosis of whether or not the driver performs at least one checking operation of a head turning action with a head turning angle being a predetermined angle or larger, and a rear side action with the visual line directed to the rear side of the vehicle in the checking period at each predetermined interval time which is shorter than the acquisition time period.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to a driving diagnosis device.

Background Art

[0002] Patent Document 1 discloses an information recording device. This information recording device includes a vehicle-related information acquisition unit that acquires image information including at least a vehicle peripheral image that is an image of the surroundings of the host vehicle and an image of the occupants of the host vehicle. Further, this information recording device includes a line-of-sight information acquisition unit that detects the line-of-sight position of the occupants of the host vehicle based on the image information and acquires line-of-sight information based on the detected line-of-sight position and object information corresponding to the line-of-sight position, and a recording unit that associates and records the line-of-sight information and the object information at the same time.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technology of Patent Document 1 could not diagnose in real time whether the driver was performing a rear check.

[0005] Therefore, an object of this disclosure is to provide a driving diagnosis device that can diagnose in real time whether the driver is performing a rear check.

Means for Solving the Problems

[0006] The driving diagnosis device according to claim 1 is for a predetermined acquisition time, vehicle information related to the operation of the vehicle, the head swing angle of the driver of the vehicle, and the line-of-sight of the driver directionThe system includes an acquisition unit that acquires data, an extraction unit that extracts the confirmation period before the vehicle moves backward from the acquisition results of the acquisition unit, and a diagnostic unit that diagnoses whether the driver has performed at least one of the following confirmation actions during the confirmation period: a head-turning motion in which the head-turning angle is greater than or equal to a predetermined angle, and a backward movement in which the driver's gaze is directed towards the rear of the vehicle, at predetermined intervals shorter than the acquisition time.

[0007] In the driving diagnostic device according to claim 1, the acquisition unit acquires vehicle information related to the vehicle's operation, the driver's head angle, and the driver's line of sight for a predetermined acquisition time. direction The system acquires the data, and the extraction unit extracts the confirmation period before the vehicle moves backward from the acquisition results. The diagnostic unit then diagnoses whether the driver performed at least one of the following confirmation actions during the confirmation period: a head-turning motion where the head-turning angle is greater than or equal to a predetermined angle, and a rearward movement where the driver's gaze is directed towards the rear of the vehicle, at predetermined intervals shorter than the acquisition time. This allows the driving diagnostic device to diagnose in real time whether the driver is checking behind the vehicle.

[0008] The driving diagnostic device according to claim 2, in claim 1, wherein the extraction unit extracts the period from a predetermined time before the shift range switches to the reverse range in which the vehicle can move backward until the vehicle begins to move backward as the confirmation period.

[0009] In the driving diagnostic device according to claim 2, the extraction unit extracts a confirmation period from a predetermined time before the shift range switches to the reverse range, which allows the vehicle to move backward, until the vehicle begins to move backward. This allows the driving diagnostic device to diagnose in real time whether or not the driver has checked behind them before switching the shift range to the reverse range.

[0010] The driving diagnostic device according to claim 3, in claim 1 or 2, wherein the diagnostic unit diagnoses whether the driver performed the confirmation operation during a combined confirmation period which is a combination of the confirmation period and the previous confirmation period extracted by the extraction unit one period prior to the confirmation period, if the interval between the confirmation period and the previous confirmation period is less than or equal to a predetermined extraction interval.

[0011] In the driving diagnostic device according to claim 3, if the interval between the confirmation period and the previous confirmation period extracted by the extraction unit one period prior to the current confirmation period is less than or equal to a predetermined extraction interval, the diagnostic unit diagnoses whether the driver performed a confirmation action during the combined confirmation period, which is a combination of the confirmation period and the previous confirmation period. This allows the driving diagnostic device to perform diagnoses more efficiently compared to diagnosing the confirmation period and the previous confirmation period separately.

[0012] The driving diagnostic device according to claim 4, in any one of claims 1 to 3, wherein the diagnostic unit does not perform the diagnosis if the vehicle speed during the confirmation period is less than or equal to a predetermined speed.

[0013] In the driving diagnostic device according to claim 4, the diagnostic unit does not perform a diagnosis if the vehicle speed during the confirmation period is below a predetermined speed. As a result, the driving diagnostic device can perform a diagnosis more efficiently compared to a device that performs a diagnosis even when the vehicle speed is below a predetermined speed.

[0014] The driving diagnostic device according to claim 5, in any one of claims 1 to 4, wherein the acquisition unit acquires the vehicle information, the head turn angle, and the line of sight from the present time up to the time before the acquisition time. direction Get all of them.

[0015] In the driving diagnostic device according to claim 5, the acquisition unit acquires vehicle information, head angle, and line of sight from the present time up to the time before acquisition. direction All of the following information is acquired. As a result, the driving diagnostic device acquires vehicle information, head angle, and line of sight from the present time up to the time before acquisition. direction This allows for more accurate diagnosis compared to acquiring only a portion of the data.

Advantages of the Invention

[0016] As described above, in the driving diagnosis device according to the present disclosure, it is possible to diagnose in real time whether the driver is checking the rear.

Brief Description of the Drawings

[0017] [Figure 1] It is an example of a block diagram showing the hardware configuration of a vehicle [Figure 2] It is an example of a block diagram showing an example of the functional configuration of a vehicle. [Figure 3] It is a diagram for explaining conventional driving diagnosis and the driving diagnosis of the present embodiment. [Figure 4] It is an example of a flowchart showing the flow of driving diagnosis processing.

Modes for Carrying Out the Invention

[0018] The vehicle 12 according to the present embodiment will be described. The vehicle 12 may be any of an engine vehicle, a hybrid vehicle, or an electric vehicle. In the present embodiment, as an example, the vehicle 12 is an engine vehicle.

[0019] FIG. 1 is a block diagram showing the hardware configuration of the vehicle 12. As shown in FIG. 1, the vehicle 12 includes an in-vehicle device 20, an ECU (Electronic Control Unit) 22, a sensor group 23, a monitor 24, a speaker 25, a shift lever 26, and a camera 27. The in-vehicle device 20 is an example of the " driving diagnosis device".

[0020] The in-vehicle device 20 is composed of a CPU (Central Processing Unit) 20A, a ROM (Read Only Memory) 20B, a RAM (Random Access Memory) 20C, a storage 20D, a wireless communication I / F (InterFace) 20E, an in-vehicle communication I / F 20F, and an input / output I / F 20G. The CPU 20A, the ROM 20B, the RAM 20C, the storage 20D, the wireless communication I / F 20E, the in-vehicle communication I / F 20F, and the input / output I / F 20G are communicably connected to each other via an internal bus 20I.

[0021] The CPU 20A is a central processing unit that executes various programs and controls each part. That is, the CPU 20A reads a program from the ROM 20B or the storage 20D and executes the program using the RAM 20C as a working area. The CPU 20A performs control of each of the above configurations and various arithmetic processes according to the program recorded in the ROM 20B or the storage 20D.

[0022] The ROM 20B stores various programs and various data. The RAM 20C temporarily stores a program or data as a working area.

[0023] The storage 20D stores various programs and various data. The storage 20D stores a driving diagnosis program for causing the CPU 20A to execute a driving diagnosis process described later.

[0024] The wireless communication I / F 20E is a wireless communication module for communicating with the outside. For example, communication standards such as 5G, LTE, Wi-Fi (registered trademark), etc. are used for the wireless communication module.

[0025] The in-vehicle communication I / F 20F is an interface for connecting to the ECU 22. The communication standard based on the CAN protocol is used for the interface. The in-vehicle communication I / F 20F is connected to an external bus 20H. Although not shown, a plurality of ECUs 22 are provided for each function of the vehicle 12.

[0026] Here, the sensor group 23 is connected to the ECU 22. For example, the sensor group 23 includes sensors for detecting the state of the vehicle 12 and the surrounding conditions, such as a 3D-LiDAR, millimeter-wave sensor, infrared sensor, turn signal sensor, accelerator position sensor, vehicle speed sensor, steering angle sensor, angular velocity sensor, GPS (Global Positioning System) sensor, gyro sensor, and acceleration sensor. The sensor group 23 outputs the detection results of each sensor to the ECU 22.

[0027] Input / Output I / F20G is an interface for communicating with in-vehicle equipment installed in the vehicle 12.

[0028] In-vehicle equipment refers to various devices mounted on the vehicle 12. The vehicle 12 includes, as an example of in-vehicle equipment, a monitor 24, a speaker 25, a shift lever 26, and a camera 27.

[0029] The monitor 24 is a liquid crystal monitor provided on the instrument panel or meter panel, etc., for displaying images and other information related to the operation of the functions of the vehicle 12 and explanations of those functions.

[0030] The speaker 25 is installed on the instrument panel, center console, front pillar, or dashboard, etc., and is a device for outputting audio, etc., that suggests the operation of the functions of the vehicle 12 and explains said functions.

[0031] The shift lever 26 is positioned so that it can be operated by the driver of the vehicle 12 (hereinafter simply referred to as "driver"). In this embodiment, as an example, it is configured to allow switching between P range (parking range), R range (reverse range), N range (neutral range), and D range (drive range).

[0032] Camera 27 is an imaging device that captures images of the driver's face.

[0033] Next, the functional configuration of vehicle 12 will be described. Figure 2 is a block diagram showing an example of the functional configuration of vehicle 12.

[0034] As shown in Figure 2, the CPU 20A of the in-vehicle unit 20 has the following functional configuration: an acquisition unit 200, an extraction unit 210, a diagnostic unit 220, and an output unit 230. Each functional configuration is realized by the CPU 20A reading and executing a driving diagnostic program stored in the storage 20D.

[0035] The acquisition unit 200 has the function of acquiring vehicle information related to the operation of the vehicle 12, the driver's head-turning angle, and the position of the driver's line of sight for a predetermined acquisition time. In this embodiment, the acquisition unit 200 acquires all of the vehicle information, the driver's head-turning angle, and the position of the driver's line of sight from the present moment up to the acquisition time (for example, 10 seconds ago). However, it is not limited to this example. For example, the acquisition unit 200 may acquire only a portion of the vehicle information, the driver's head-turning angle, and the position of the driver's line of sight from the present moment up to the acquisition time. The present moment refers to the time when the acquisition unit 200 acquires the vehicle information, the driver's head-turning angle, and the position of the driver's line of sight for the acquisition time.

[0036] Furthermore, in this embodiment, the vehicle information includes the vehicle speed of the vehicle 12 (hereinafter simply referred to as "vehicle speed") and a reverse signal. The reverse signal is information indicating whether or not the shift range has been switched to the reverse range, which allows the vehicle 12 to move backward.

[0037] Specifically, the acquisition unit 200 acquires all vehicle speeds detected by the vehicle speed sensors of the sensor group 23, and reverse signals input from the shift lever 26, from the present time up to the time before the acquisition. In addition to vehicle speed and reverse signals, the acquisition unit 200 may also acquire vehicle information such as the distance traveled by the vehicle 12.

[0038] Furthermore, the acquisition unit 200 acquires the driver's head-turning angle (hereinafter simply referred to as "head-turning angle") and the driver's line of sight position (hereinafter simply referred to as "line of sight position") from the present time up to the time before acquisition. The head-turning angle is the angle at which the driver's face faces from the front of the vehicle 12 to the rear. For example, the head-turning angle is set to 0 degrees when the direction of the central axis extending in the straight-ahead direction of the vehicle 12 coincides with the direction of the driver's face, and the head-turning angle is set to 180 degrees when the direction of the central axis extending in the straight-ahead rearward direction of the vehicle 12 coincides with the direction of the driver's face. Therefore, the head-turning angle according to this embodiment ranges from 0 degrees to 180 degrees. However, it is not limited to this example. The range of the head-turning angle may be any range.

[0039] Specifically, the acquisition unit 200 acquires a facial image of the driver by detecting the edges of the driver's face from the image data acquired by the camera 27. The acquisition unit 200 then acquires the head-turning angle and the position of the gaze from the facial image of the driver.

[0040] The extraction unit 210 has the function of extracting the confirmation period before the vehicle 12 moves backward from the acquisition results of the acquisition unit 200. In this embodiment, the extraction unit 210 extracts the confirmation period from a predetermined time before the shift range switches to the reverse range (for example, 3 seconds before) until the vehicle 12 starts moving backward. Specifically, the extraction unit 210 extracts the confirmation period from a predetermined time before the reverse signal is input by the shift lever 26 until the vehicle speed exceeds 0 km / h. Note that the confirmation period may be any period before the vehicle 12 moves backward (specifically, before the reverse signal is input).

[0041] The diagnostic unit 220 has a function to diagnose (hereinafter simply referred to as "diagnosis") whether the driver has performed at least one of the following confirmation actions during the confirmation period: a head-turning motion in which the head-turning angle is greater than or equal to a predetermined angle, and a rearward motion in which the driver's gaze is directed towards the rear of the vehicle 12. This diagnosis is performed at predetermined intervals (for example, 1 second). The interval is shorter than the acquisition time.

[0042] In this embodiment, a predetermined angle of 90 degrees or more is applied. Specifically, in this embodiment, the head-turning motion is performed by turning the driver's face until the deviation from the central axis extending in the straight-ahead direction of the vehicle 12 is 90 degrees or more. However, this is not the only example. The predetermined angle may be less than 90 degrees. Alternatively, the head-turning motion may be performed by performing an action such as turning the head at a speed below a predetermined speed until the head-turning angle is greater than or equal to the predetermined angle.

[0043] Furthermore, in this embodiment, the rearward movement may involve the line of sight moving to the rear of the vehicle 12, moving to the rearview mirror installed at the top of the front center of the vehicle 12, or moving to the side mirror installed on the outside of the door closest to the driver's seat. However, this is not the only example. For example, if the image of the area behind the vehicle 12 is displayed on the monitor 24, the rearward movement may involve the line of sight moving to the monitor 24, etc.

[0044] In this embodiment, the diagnostic unit 220 diagnoses whether the driver performed a confirmation action during the combined confirmation period, which is a combination of the confirmation period and the previous confirmation period, if the interval between the confirmation period and the previous confirmation period extracted by the extraction unit 210 immediately prior to the current confirmation period is less than or equal to a predetermined extraction interval (for example, 0.5 seconds). However, the diagnostic unit 220 is not limited to this example. Even if the interval between the confirmation period and the previous confirmation period is less than or equal to the extraction interval, the diagnostic unit 220 may diagnose them separately. Also, in this embodiment, a time shorter than the interval time is applied as the extraction interval. However, the diagnostic unit is not limited to this example. A time longer than the interval time may be applied as the extraction interval.

[0045] Furthermore, in this embodiment, the diagnostic unit 220 does not perform a diagnosis if the vehicle speed during the confirmation period is below a predetermined speed (for example, 1.5 km / h). However, it is not limited to this example. The diagnostic unit 220 may perform a diagnosis regardless of the magnitude of the vehicle speed during the confirmation period.

[0046] The output unit 230 outputs the diagnostic results from the diagnostic unit 220 to at least one of the monitor 24 and the speaker 25. In this embodiment, the output unit 230 outputs the diagnostic results at intervals equal to the intervals at which the diagnostic unit 220 performs diagnostics. However, it is not limited to this example. The output unit 230 may output the diagnostic results at intervals different from those at which the diagnostic unit 220 performs diagnostics. The output unit 230 may also output the diagnostic results from the diagnostic unit 220 to a device other than the monitor 24 and the speaker 25 (for example, a mobile terminal held by the driver). Furthermore, if the diagnostic unit 220 diagnoses that the driver did not perform a confirmation operation during the confirmation period, the output unit 230 may further output the head-turning angle and line-of-sight position acquired by the acquisition unit 200.

[0047] Incidentally, in conventional driving diagnostics, as shown in the upper part of Figure 3, driving diagnostics were performed and reports were created at predetermined intervals, such as 10 minutes. Therefore, when a safety check was actually insufficient, feedback could not be given to the driver, and even if feedback was given, the driver may not remember or may not understand what went wrong.

[0048] Therefore, the in-vehicle device 20 according to this embodiment performs driving diagnostics in real time and provides feedback to the driver.

[0049] To perform a real-time diagnosis, as shown in the lower part of Figure 3, the driving diagnosis is performed by overlapping data from the past 10 seconds or so, with a time interval of 1 second or so. Specifically, the acquisition unit 200 acquires vehicle information, head angle, and line of sight position for the acquisition time. The extraction unit 210 then extracts the confirmation period before the vehicle 12 moves backward from the acquisition results of the acquisition unit 200. The diagnosis unit 220 then diagnoses whether the driver performed the confirmation action during the confirmation period at time intervals shorter than the acquisition time. Note that since the diagnosis may be performed twice due to overlapping diagnoses, the earlier diagnosis result is output first.

[0050] Figure 4 is a flowchart showing the flow of the operation diagnostic process. The operation diagnostic process is performed when the CPU 20A reads the operation diagnostic program from the storage 20D, loads it into the RAM 20C, and executes it.

[0051] In step S100 shown in Figure 4, the CPU 20A acquires vehicle information, head turn angle, and line of sight position for a predetermined acquisition time. Specifically, the CPU 20A acquires all vehicle information, head turn angle, and line of sight position from the present time up to the time before the acquisition.

[0052] In step S102, the CPU 20A extracts a verification period from the results obtained in step S100. Specifically, the CPU 20A extracts the period from a predetermined time before the shift range switches to the reverse range until the vehicle 12 begins to move backward as the verification period.

[0053] In step S104, the CPU 20A determines whether the vehicle speed during the verification period is below a predetermined speed. If the vehicle speed during the verification period is below the predetermined speed (step S104: YES), the CPU 20A proceeds to step S116. On the other hand, if the vehicle speed during the verification period is greater than the predetermined speed (step S104: NO), the CPU 20A proceeds to step S106.

[0054] In step S106, CPU 20A determines whether it extracted a confirmation period before the confirmation period extracted in step S102. If CPU 20A extracted a confirmation period before the confirmation period extracted in step S102 (step S106: YES), it proceeds to step S108. On the other hand, if CPU 20A did not extract a confirmation period before the confirmation period extracted in step S102 (step S106: NO), it proceeds to step S112.

[0055] In step S108, CPU 20A determines whether the interval between the confirmation period and the previous confirmation period extracted immediately before the current confirmation period is less than or equal to a predetermined extraction interval. If the interval between the confirmation period and the previous confirmation period extracted immediately before the current confirmation period is less than or equal to the predetermined extraction interval (step S108: YES), CPU 20A proceeds to step S110. On the other hand, if the interval between the confirmation period and the previous confirmation period extracted immediately before the current confirmation period is greater than the predetermined extraction interval (step S108: NO), CPU 20A proceeds to step S112.

[0056] In step S110, the CPU 20A diagnoses whether the driver performed a verification action during the combined verification period, which is a combination of the verification period and the previous verification period.

[0057] In step S112, the CPU 20A diagnoses whether the driver performed the verification action during the verification period.

[0058] In step S114, the CPU 20A outputs the diagnostic result from step S110 or step S112 to at least one of the monitor 24 and the speaker 25.

[0059] In step S116, the CPU 20A determines whether an interval has elapsed since the previous execution of the process in step S116. If the interval has elapsed since the previous execution of the process in step S116 (step S116: YES), the CPU 20A returns to step S100. On the other hand, if the interval has not elapsed since the previous execution of the process in step S116 (step S116: NO), the CPU 20A proceeds to step S118.

[0060] In step S118, the CPU 20A determines whether the operation of vehicle 12 has ended. For example, the CPU 20A determines whether the ignition switch of vehicle 12 has been turned off. If the CPU 20A determines that the operation of vehicle 12 has ended, it terminates the operation diagnostic process. On the other hand, if the CPU 20A determines that the operation of vehicle 12 has not ended, it returns to step S116.

[0061] (others) In the above embodiment, the in-vehicle unit 20 was used as an example of a driving diagnostic device, but it is not limited to this, and an external terminal located outside the vehicle 12, such as a server, or a mobile terminal such as a smartphone held by the driver may also be used as an example of a driving diagnostic device. Alternatively, a combination of the in-vehicle unit 20, an external terminal, and a mobile terminal may also be used as an example of a driving diagnostic device.

[0062] In addition, the operational diagnostic processing that the CPU 20A reads and executes in the above embodiment may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing, such as ASICs (Application Specific Integrated Circuits). Furthermore, the operational diagnostic processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0063] Furthermore, although the above embodiment describes a configuration in which the operation diagnostic program is pre-stored (installed) in storage 20D, the system is not limited to this configuration. The operation diagnostic program may be provided in the form of a recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the operation diagnostic program may be downloaded from an external device via a network. [Explanation of symbols]

[0064] 10 vehicles 20. On-board devices (driving diagnostic devices) 200 Acquisition Department 210 Extraction part 220 Diagnostic Department

Claims

1. An acquisition unit that acquires vehicle information related to the vehicle's operation, the driver's head-turning angle, and the direction of the driver's gaze for a predetermined acquisition period, From the acquisition results of the acquisition unit, an extraction unit extracts the confirmation period before the vehicle moves backward, A diagnostic unit that performs a diagnosis at predetermined intervals shorter than the acquisition time to determine whether the driver has performed at least one of the following confirmation actions during the confirmation period: a head-turning motion in which the head-turning angle is greater than or equal to a predetermined angle, and a rearward motion in which the driver's gaze is directed towards the rear of the vehicle. A driving diagnostic device equipped with the following features.

2. The driving diagnostic device according to claim 1, wherein the extraction unit extracts data from a predetermined time before the shift range switches to the reverse range, which allows the vehicle to move backward, until the vehicle begins to move backward, with the confirmation period being the period.

3. The driving diagnostic device according to claim 1, wherein the diagnostic unit, if the interval between the confirmation period and the previous confirmation period extracted by the extraction unit one period prior to the confirmation period is less than or equal to a predetermined extraction interval, diagnoses whether the driver performed the confirmation operation during a combined confirmation period which is a combination of the confirmation period and the previous confirmation period.

4. The driving diagnostic device according to claim 1, wherein the diagnostic unit does not perform the diagnosis if the vehicle speed during the confirmation period is below a predetermined speed.

5. The driving diagnostic device according to claim 1, wherein the acquisition unit acquires all of the vehicle information, the head-turn angle, and the direction of the line of sight from the present time up to the time before the acquisition.

Images