Drive diagnostic device
The driving diagnosis device addresses the challenge of real-time driver rear view monitoring by analyzing vehicle and driver data to provide immediate feedback on rear view checks, enhancing safety.
Patent Information
- Application Number
- JP2023090340
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-12
- Estimated Expiration
- 2043-05-31
AI Technical Summary
Existing technologies fail to diagnose in real time whether a driver is checking the rear view during vehicle operation.
A driving diagnosis device that acquires vehicle information, driver swing angle, and line of sight position, extracts a confirmation period before the vehicle moves backward, and diagnoses whether the driver performs a swinging motion or backward gaze at predetermined intervals shorter than the acquisition time.
Enables real-time diagnosis of the driver's rear view check, providing immediate feedback for enhanced safety.
Smart Images

Figure 2024172551000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a driving diagnosis device. [Background technology]
[0002] Patent Document 1 discloses an information recording device. This information recording device includes a vehicle-related information acquisition unit that acquires image information including a vehicle surroundings image, which is an image of at least the surroundings of the vehicle, and an image of an occupant of the vehicle. This information recording device also includes a gaze information acquisition unit that detects the gaze position of the occupant of the vehicle based on the image information, acquires gaze information based on the detected gaze position and object information corresponding to the gaze position, and a recording unit that associates the gaze information and object information at the same time and records them. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2022-018497 Summary of the Invention [Problem to be solved by the invention]
[0004] The technology of Patent Document 1 was not able to diagnose in real time whether the driver was checking behind him or not.
[0005] Therefore, an object of the present disclosure is to provide a driving diagnosis device that can diagnose in real time whether a driver is checking behind him or her. [Means for solving the problem]
[0006] The driving diagnosis device of claim 1 includes an acquisition unit that acquires vehicle information related to the operation of the vehicle, the head turning angle of the driver of the vehicle, and the position of the driver's line of sight for a predetermined acquisition time, an extraction unit that extracts a confirmation period before the vehicle moves backward from the acquisition results of the acquisition unit, and a diagnosis unit that diagnoses whether the driver is performing at least one of the following confirmation actions during the confirmation period: a head turning action in which the head turning angle is equal to or greater than a predetermined angle, and a rearward movement in which the line of sight is directed toward the rear of the vehicle, at every predetermined interval time that is shorter than the acquisition time.
[0007] In the driving diagnosis device according to claim 1, the acquisition unit acquires vehicle information related to the operation of the vehicle, the driver's head turning angle, and the driver's line of sight for a predetermined acquisition time, the extraction unit extracts a confirmation period before the vehicle moves backward from the acquisition result of the acquisition unit, and the diagnosis unit diagnoses whether the driver is performing at least one of the following confirmation actions during the confirmation period: a head turning action in which the head turning angle is equal to or greater than a predetermined angle, and a backward movement in which the driver's line of sight is directed backward of the vehicle, at predetermined intervals shorter than the acquisition time. As a result, the driving diagnosis device can diagnose in real time whether the driver is checking backward.
[0008] The driving diagnosis device according to claim 2 is in claim 1, wherein the extraction unit extracts, as the confirmation period, a period from a predetermined time before the shift range is switched to a reverse range in which the vehicle can move backwards until the vehicle starts to move backwards.
[0009] In the driving diagnosis device according to claim 2, the extraction unit extracts, as the confirmation period, the period from a predetermined time before the shift range is switched to the reverse range in which the vehicle can move backwards until the vehicle starts to move backwards. This allows the driving diagnosis device to diagnose in real time whether the driver is checking backwards before switching the shift range to the reverse range.
[0010] The driving diagnosis device of claim 3 is in claim 1 or 2, and when the interval between the confirmation period and the previous confirmation period extracted by the extraction unit one period before the confirmation period is equal to or less than a predetermined extraction interval, the diagnosis unit diagnoses whether the driver has performed the confirmation action during a consolidated confirmation period that combines the confirmation period and the previous confirmation period.
[0011] In the driving diagnosis device according to claim 3, when the interval between the confirmation period and the previous confirmation period extracted by the extraction unit immediately before the confirmation period is equal to or less than a predetermined extraction interval, the diagnosis unit diagnoses whether the driver has performed a confirmation action during a combined confirmation period that combines the confirmation period and the previous confirmation period. This allows the driving diagnosis device to perform diagnosis more efficiently than when the confirmation period and the previous confirmation period are diagnosed separately.
[0012] A driving diagnosis device according to a fourth aspect of the present invention is any one of the first to third aspects, wherein the diagnosis unit does not perform the diagnosis when the vehicle speed during the confirmation period is equal to or lower than a predetermined speed.
[0013] In the driving diagnosis device according to claim 4, the diagnosis unit does not perform diagnosis when the vehicle speed during the confirmation period is equal to or lower than a predetermined speed. This allows the driving diagnosis device to perform diagnosis more efficiently than when the vehicle speed is equal to or lower than the predetermined speed.
[0014] A driving diagnosis device according to claim 5 is any one of claims 1 to 4, wherein the acquisition unit acquires all of the vehicle information, the head swing angle, and the line of sight position from the present time to a time just before the acquisition time.
[0015] In the driving diagnosis device according to claim 5, the acquisition unit acquires all of the vehicle information, the head swivel angle, and the eye gaze position from the current time to the acquisition time before. This allows the driving diagnosis device to perform diagnosis with high accuracy compared to a case where only a part of the vehicle information, the head swivel angle, and the eye gaze position from the current time to the acquisition time before is acquired. Effect of the Invention
[0016] As described above, the driving diagnosis device according to the present disclosure can diagnose in real time whether the driver is checking behind him or her. [Brief description of the drawings]
[0017] [Figure 1] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a vehicle. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of a vehicle. [Diagram 3] 1 is a diagram for explaining a conventional driving diagnosis and a driving diagnosis according to an embodiment of the present invention; [Figure 4] 4 is an example of a flowchart showing a flow of a driving diagnosis process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0018] The vehicle 12 according to this embodiment will be described below. The vehicle 12 may be any of an engine vehicle, a hybrid vehicle, and an electric vehicle, but in this embodiment, as an example, the vehicle 12 is an engine vehicle.
[0019] Fig. 1 is a block diagram showing a hardware configuration of a vehicle 12. As shown in Fig. 1, the vehicle 12 includes an in-vehicle device 20, an ECU (Electronic Control Unit) 22, a sensor group 23, a monitor 24, a speaker 25, a shift lever 26, and a camera 27. The in-vehicle device 20 is an example of a "driving diagnosis device."
[0020] The vehicle-mounted device 20 includes a central processing unit (CPU) 20A, a read only memory (ROM) 20B, a random access memory (RAM) 20C, a storage 20D, a wireless communication interface (I / F) 20E, an in-vehicle communication I / F 20F, and an input / output I / F 20G. The CPU 20A, the ROM 20B, the RAM 20C, the storage 20D, the wireless communication I / F 20E, the in-vehicle communication I / F 20F, and the input / output I / F 20G are connected to each other via an internal bus 20I so as to be able to communicate with each other.
[0021] The CPU 20A is a central processing unit that executes various programs and controls each part. That is, the CPU 20A reads a program from the ROM 20B or the storage 20D, and executes the program using the RAM 20C as a working area. The CPU 20A controls each of the above components and performs various arithmetic processing according to the program recorded in the ROM 20B or the storage 20D.
[0022] The ROM 20B stores various programs and various data. The RAM 20C temporarily stores programs or data as a working area.
[0023] The storage 20D stores various programs and various data. The storage 20D stores a driving diagnosis program for causing the CPU 20A to execute a driving diagnosis process, which will be described later.
[0024] The wireless communication I / F 20E is a wireless communication module for communicating with the outside. The wireless communication module uses communication standards such as 5G, LTE, and Wi-Fi (registered trademark).
[0025] The in-vehicle communication I / F 20F is an interface for connecting to the ECU 22. The interface uses a communication standard based on the CAN protocol. The in-vehicle communication I / F 20F is connected to the external bus 20H. Although not shown in the figure, a plurality of ECUs 22 are provided for each function of the vehicle 12.
[0026] Here, the sensor group 23 is connected to the ECU 22. For example, the sensor group 23 includes sensors for detecting the state of the vehicle 12 and the surrounding conditions, such as a 3D-LiDAR, a millimeter wave sensor, an infrared sensor, a turn signal sensor, an accelerator position sensor, a vehicle speed sensor, a steering angle sensor, an angular velocity sensor, a GPS (Global Positioning System) sensor, a gyro sensor, and an acceleration sensor. The sensor group 23 outputs the detection results of the respective sensors to the ECU 22.
[0027] The input / output I / F 20G is an interface for communicating with on-board devices mounted in the vehicle 12.
[0028] The in-vehicle devices are various devices mounted on the vehicle 12. The vehicle 12 includes a monitor 24, a speaker 25, a shift lever 26, and a camera 27 as examples of the in-vehicle devices.
[0029] The monitor 24 is a liquid crystal monitor provided on an instrument panel or a meter panel, etc., for displaying operation suggestions related to the functions of the vehicle 12 and images related to explanations of the functions.
[0030] The speaker 25 is provided on the instrument panel, the center console, the front pillar, the dashboard, or the like, and is a device for outputting suggestions for operation related to functions of the vehicle 12 and audio related to explanations of the functions.
[0031] The shift lever 26 is provided at a position where it can be operated by a driver of the vehicle 12 (hereinafter simply referred to as the "driver"). In this embodiment, as an example, it is configured to be able to switch between P range (parking range), R range (reverse range), N range (neutral range), and D range (drive range).
[0032] The camera 27 is an imaging device that captures an image of the driver's face.
[0033] Next, a description will be given of the functional configuration of the vehicle 12. Fig. 2 is a block diagram showing an example of the functional configuration of the vehicle 12.
[0034] 2, the CPU 20A of the vehicle-mounted device 20 has, as functional components, an acquisition unit 200, an extraction unit 210, a diagnosis unit 220, and an output unit 230. Each functional component is realized by the CPU 20A reading and executing a driving diagnosis program stored in the storage 20D.
[0035] The acquisition unit 200 has a function of acquiring vehicle information related to the operation of the vehicle 12, the driver's head swivel angle, and the driver's gaze position for a predetermined acquisition time. In this embodiment, the acquisition unit 200 acquires all of the vehicle information, the driver's head swivel angle, and the driver's gaze position from the current time to the acquisition time before (for example, 10 seconds before). However, this is not limited to this example. For example, the acquisition unit 200 may acquire only a part of the vehicle information, the driver's head swivel angle, and the driver's gaze position from the current time to the acquisition time before. In addition, the current time indicates the time when the acquisition unit 200 acquires the vehicle information, the driver's head swivel angle, and the driver's gaze position for the acquisition time.
[0036] In this embodiment, the vehicle information includes the speed of the vehicle 12 (hereinafter simply referred to as "vehicle speed") and a reverse signal. The reverse signal is information indicating whether the shift range has been switched to a reverse range in which the vehicle 12 can move backward.
[0037] Specifically, the acquisition unit 200 acquires all of the vehicle speed detected by the vehicle speed sensor included in the sensor group 23 from the current time to the acquisition time ago, and the reverse signal input from the shift lever 26. Note that the acquisition unit 200 may acquire, as vehicle information, a travel distance of the vehicle 12, etc., in addition to the vehicle speed and the reverse signal.
[0038] The acquisition unit 200 also acquires all of the driver's head-swivel angle (hereinafter simply referred to as "head-swivel angle") and the driver's line-of-sight position (hereinafter simply referred to as "line-of-sight position") from the current time to the acquisition time. The head-swivel angle is the angle at which the driver's face faces from the front of the vehicle 12 to the rear. For example, when the direction of the central axis extending in the straight forward direction of the vehicle 12 coincides with the direction of the driver's face, the head-swivel angle is set to 0 degrees, and when the direction of the central axis extending in the straight backward direction of the vehicle 12 coincides with the direction of the driver's face, the head-swivel angle is set to 180 degrees. Therefore, the head-swivel angle according to this embodiment is from 0 degrees to 180 degrees. However, this is not limited to this example. The head-swivel angle may be in any range.
[0039] Specifically, the acquisition unit 200 acquires a facial image of the driver by detecting edges of the driver's face from image data acquired by the camera 27. Then, the acquisition unit 200 acquires the head swing angle and the line of sight position from the facial image of the driver.
[0040] The extraction unit 210 has a function of extracting a confirmation period before the vehicle 12 moves backward from the acquisition result of the acquisition unit 200. In this embodiment, the extraction unit 210 extracts, as the confirmation period, a period from a predetermined time (e.g., 3 seconds) before the shift range is switched to the reverse range until the vehicle 12 starts moving backward. Specifically, the extraction unit 210 extracts, as the confirmation period, a period from a predetermined time before a reverse signal is input by the shift lever 26 until the vehicle speed becomes greater than 0 km / h. Note that the confirmation period may be any period before the vehicle 12 moves backward (specifically, before the reverse signal is input).
[0041] The diagnosis unit 220 has a function of diagnosing (hereinafter, simply referred to as "diagnosis") whether or not the driver is performing at least one of a head turning motion in which the head turning angle is equal to or greater than a predetermined angle and a rearward motion in which the line of sight is directed to the rear of the vehicle 12 during the confirmation period at predetermined intervals (for example, 1 second). Note that the intervals are shorter than the acquisition time.
[0042] In this embodiment, the predetermined angle is 90 degrees or more. Specifically, in this embodiment, the head-swivel motion is a motion in which the driver turns his / her head until the deviation from the central axis extending in the straight forward direction of the vehicle 12 becomes 90 degrees or more. However, this is not limited to this example. The predetermined angle may be less than 90 degrees. In addition, the head-swivel motion may be a motion in which the head-swivel angle becomes a predetermined angle or more at a speed less than a predetermined speed.
[0043] In addition, in this embodiment, the rearward motion is a motion of moving the gaze position to the rear of the vehicle 12, a motion of moving to a room mirror installed at the top center of the front of the vehicle 12, or a motion of moving to a side mirror installed on the outside of the door closest to the driver's seat. However, this is not limited to the above example. For example, when an image of the rear of the vehicle 12 is displayed on the monitor 24, the rearward motion may be a motion of moving the gaze position to the monitor 24.
[0044] In this embodiment, when the interval between a confirmation period and the previous confirmation period extracted by the extraction unit 210 immediately before the confirmation period is equal to or less than a predetermined extraction interval (e.g., 0.5 seconds), the diagnosis unit 220 diagnoses whether or not the driver has performed a confirmation action during a combined confirmation period that combines the confirmation period and the previous confirmation period. However, this is not limited to this example. The diagnosis unit 220 may diagnose these separately even if the interval between the confirmation period and the previous confirmation period is equal to or less than the extraction interval. Also, in this embodiment, a time shorter than the interval time is applied as the extraction interval. However, this is not limited to this example. A time longer than the interval time may be applied as the extraction interval.
[0045] In the present embodiment, the diagnosis unit 220 does not perform diagnosis when the vehicle speed during the confirmation period is equal to or lower than a predetermined speed (for example, 1.5 km / h). However, this is not limited to this example. The diagnosis unit 220 may perform diagnosis regardless of the vehicle speed during the confirmation period.
[0046] The output unit 230 outputs the diagnosis result by the diagnosis unit 220 to at least one of the monitor 24 and the speaker 25. In this embodiment, the output unit 230 outputs the diagnosis result at each interval time that is the interval at which the diagnosis unit 220 performs the diagnosis. However, this is not limited to this example. The output unit 230 may output the diagnosis result at an interval different from the interval at which the diagnosis unit 220 performs the diagnosis. In addition, the output unit 230 may output the diagnosis result by the diagnosis unit 220 to a device other than the monitor 24 and the speaker 25 (for example, a mobile terminal held by the driver). In addition, when the diagnosis unit 220 diagnoses that the driver has not performed the confirmation action during the confirmation period, the output unit 230 may further output the swing angle and the line of sight position acquired by the acquisition unit 200.
[0047] In the conventional driving diagnosis, a driving diagnosis is performed at a fixed interval, such as 10 min, and a report is created, as shown in the upper part of Fig. 3. Therefore, it is not possible to provide feedback to the driver when a safety check is actually insufficient, and even if feedback is given to the driver, there are cases where the driver does not remember it or does not know what went wrong.
[0048] Therefore, the vehicle-mounted device 20 according to this embodiment is configured to perform driving diagnosis in real time and provide feedback to the driver.
[0049] In order to perform a diagnosis in real time, as shown in the lower part of FIG. 3, the driving diagnosis is performed while overlapping data for the past acquisition time of 10 [sec], etc., with an interval time of 1 [sec], etc. being shifted. That is, the acquisition unit 200 acquires vehicle information, a head swing angle, and a gaze position for the acquisition time. Then, the extraction unit 210 extracts a confirmation period before the vehicle 12 moves backward from the acquisition result of the acquisition unit 200. Then, the diagnosis unit 220 performs a diagnosis of whether or not the driver performs a confirmation operation during the confirmation period at intervals shorter than the acquisition time. Note that overlapping diagnoses may result in the same diagnosis being performed, so the earlier diagnosis result is given priority for output of the overlapping diagnosis result.
[0050] 4 is a flowchart showing the flow of the driving diagnosis process. The driving diagnosis process is performed by the CPU 20A reading out a driving diagnosis program from the storage 20D, loading it into the RAM 20C, and executing it.
[0051] 4, the CPU 20A acquires vehicle information, a swing angle, and a gaze position for a predetermined acquisition time. Specifically, the CPU 20A acquires all vehicle information, swing angle, and gaze position from the current time to the acquisition time before.
[0052] In step S102, the CPU 20A extracts a confirmation period from the result obtained in step S100. Specifically, the CPU 20A extracts, as the confirmation period, a period from a predetermined time before the shift range is switched to the reverse range until the vehicle 12 starts to move backward.
[0053] In step S104, the CPU 20A determines whether the vehicle speed during the confirmation period is equal to or lower than a predetermined speed. If the vehicle speed during the confirmation period is equal to or lower than the predetermined speed (step S104: YES), the CPU 20A proceeds to step S116. On the other hand, if the vehicle speed during the confirmation period is higher than the predetermined speed (step S104: NO), the CPU 20A proceeds to step S106.
[0054] In step S106, the CPU 20A determines whether or not a confirmation period has been extracted prior to the confirmation period extracted in step S102. If the CPU 20A has extracted a confirmation period prior to the confirmation period extracted in step S102 (step S106: YES), the process proceeds to step S108. On the other hand, if the CPU 20A has not extracted a confirmation period prior to the confirmation period extracted in step S102 (step S106: NO), the process proceeds to step S112.
[0055] In step S108, the CPU 20A judges whether the interval between the confirmation period and the previous confirmation period extracted immediately before the confirmation period is equal to or less than the predetermined extraction interval. If the interval between the confirmation period and the previous confirmation period extracted immediately before the confirmation period is equal to or less than the predetermined extraction interval (step S108: YES), the CPU 20A proceeds to step S110. On the other hand, if the interval between the confirmation period and the previous confirmation period extracted immediately before the confirmation period is greater than the predetermined extraction interval (step S108: NO), the CPU 20A proceeds to step S112.
[0056] In step S110, the CPU 20A diagnoses whether or not the driver has performed a checking action during a combined checking period that combines the checking period and the previous checking period.
[0057] In step S112, the CPU 20A diagnoses whether or not the driver is performing a checking action during the checking period.
[0058] In step S114, CPU 20A outputs the diagnosis result in step S110 or step S112 to at least one of monitor 24 and speaker 25.
[0059] In step S116, the CPU 20A determines whether or not the interval time has elapsed since the CPU 20A executed the process of step S116 last time. If the interval time has elapsed since the CPU 20A executed the process of step S116 last time (step S116: YES), the CPU 20A returns to step S100. On the other hand, if the interval time has not elapsed since the CPU 20A executed the process of step S116 last time (step S116: NO), the CPU 20A proceeds to step S118.
[0060] In step S118, the CPU 20A determines whether or not the driving of the vehicle 12 has ended. For example, the CPU 20A determines whether or not the ignition switch of the vehicle 12 has been turned off. If the CPU 20A determines that the driving of the vehicle 12 has ended, the driving diagnosis process ends. On the other hand, if the CPU 20A determines that the driving of the vehicle 12 has not ended, the process returns to step S116.
[0061] (others) In the above embodiment, the vehicle-mounted device 20 is an example of the driving diagnosis device, but the present invention is not limited to this, and an external terminal such as a server disposed outside the vehicle 12 or a mobile terminal such as a smartphone held by the driver may be an example of the driving diagnosis device. Also, a combination of the vehicle-mounted device 20, the external terminal, and the mobile terminal may be an example of the driving diagnosis device.
[0062] In the above embodiment, the driving diagnosis process executed by the CPU 20A by reading the software (program) may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. The driving diagnosis process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). The hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0063] In the above embodiment, the driving diagnosis program is pre-stored (installed) in the storage 20D, but the present invention is not limited to this. The driving diagnosis program may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The driving diagnosis program may be downloaded from an external device via a network. [Explanation of symbols]
[0064] 10 Vehicles 20 In-vehicle device (driving diagnostic device) 200 Acquisition Department 210 Extraction part 220 Diagnostic Department
Claims
1. An acquisition unit that acquires vehicle information related to a vehicle operation, a head turning angle of a driver of the vehicle, and a line of sight position of the driver for a predetermined acquisition time; an extraction unit that extracts a confirmation period before the vehicle moves backward from the acquisition result obtained by the acquisition unit; a diagnosis unit that diagnoses whether the driver is performing at least one of a head turning motion in which the head turning angle is equal to or greater than a predetermined angle and a rearward motion in which the line of sight is directed toward the rear of the vehicle during the confirmation period at every predetermined interval time that is shorter than the acquisition time; A driving diagnosis device equipped with the above.
2. The driving diagnosis device according to claim 1 , wherein the extraction unit extracts, as the confirmation period, a period from a predetermined time before a shift range is switched to a reverse range in which the vehicle can move backwards until the vehicle starts to move backwards.
3. 2. The driving diagnosis device according to claim 1, wherein, when an interval between the confirmation period and a previous confirmation period extracted by the extraction unit immediately before the confirmation period is equal to or less than a predetermined extraction interval, the diagnosis unit diagnoses whether the driver has performed the confirmation action during a consolidated confirmation period that combines the confirmation period and the previous confirmation period.
4. The driving diagnosis device according to claim 1 , wherein the diagnosing unit does not perform the diagnosis when the vehicle speed during the confirmation period is equal to or lower than a predetermined speed.
5. The driving diagnosis device according to claim 1 , wherein the acquisition unit acquires all of the vehicle information, the head turning angle, and the line of sight position from a current time to a time immediately prior to the acquisition time.
Citation Information
Patent Citations
Image extraction device
JP2011186867A
Section setting method, fuel consumption information generation device and driving support device
JP2012068829A
Vehicle operation evaluation system
JP2015141432A
In-vehicle device and control method thereof
JP2021005286A
Method and System for Detection of Motor Vehicle Movement to Avoid Collisions
US20130184979A1
