Emergency driver status display device

Abstract

To provide an emergency-time driver state display device which enables a third party to recognize, after occurrence of a vehicle abnormal situation, driver information including driver state video information prior to the occurrence of the vehicle abnormal situation.SOLUTION: An emergency-time driver state display device 1 comprises: an imaging unit 11 which continuously images a state of a driver; a storage unit 52 for storing driver state video information imaged by the imaging unit 11; a driver state estimation unit 12 which estimates the driver state on the basis of the driver state video information; a vehicle state information acquisition unit 51 for acquiring vehicle state information; a display device 40 inside of a cabin; and a display control unit 53 which changes display information to be displayed on the display device 40. When it is determined that a vehicle abnormal situation has occurred based on at least one of the vehicle state information and the driver state estimated information, the display control unit 53 displays on the display device 40 at least driver state video information prior to determining the vehicle abnormal situation in the driver state video information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an emergency driver status display device. [Background technology]

[0002] One of the causes of a vehicle abnormality is when an abnormality occurs in the driver, for example, when the driver loses consciousness. In this case, the driver experiences an abnormality and is unable to operate the vehicle, which then causes the vehicle abnormality. After the vehicle abnormality occurs, a third party such as a medical professional enters the vehicle cabin to rescue the driver. In this case, it is possible to confirm the driver's condition after the vehicle abnormality occurs by visually inspecting the driver. However, if the driver has lost consciousness, the third party cannot obtain necessary driver information from the driver. Conventionally, there is a technology that displays the driver's emergency contact information and driver's license information on a display unit when an ambulance or other rescue vehicle arrives at the accident scene (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-212753 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even if the driver's emergency contact information and driver's license information are displayed on the display unit, a third party cannot recognize how the driver reached his or her current driving state, or that an abnormality occurring to the driver is the cause of the vehicle abnormality.

[0005] The object of the present invention has been made in consideration of the above, and is to provide an emergency driver status display device that allows a third party to recognize driver information, including video information of the driver status before the occurrence of an abnormal vehicle situation, after the occurrence of an abnormal vehicle situation. [Means for solving the problem]

[0006] In order to achieve the above object, the emergency driver state display device of the present invention comprises an imaging unit that continuously images the driver's state, a memory unit that stores driver state video information imaged by the imaging unit, a driver state estimation unit that estimates the driver's state based on the driver state image information, a vehicle state information acquisition unit that acquires vehicle state information, a display device that is provided in the vehicle cabin and displays at least normal information, and a display control unit that changes the display information displayed on the display device, wherein when the display control unit determines that an abnormal vehicle situation has occurred based on at least one of the vehicle state information and driver state estimation information, which is the driver state estimated by the driver state estimation unit, it causes the display device to display driver information, and the driver information includes driver state video information stored in the memory unit that was present before it was determined that an abnormal vehicle situation had occurred. [Effects of the Invention]

[0007] The emergency driver state display device according to the present invention has the effect of allowing a third party to recognize driver information including video information of the driver state before the occurrence of an abnormal vehicle situation after the occurrence of an abnormal vehicle situation. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of an emergency driver state display device according to an embodiment. [Figure 2] FIG. 2 is a diagram showing normal information displayed on the display device. [Figure 3] FIG. 3 is a diagram showing driver information displayed on the display device. [Figure 4] FIG. 4 is a flowchart illustrating an example of control of the driver state display device in an emergency according to the embodiment. [Figure 5] FIG. 5 is a diagram showing a modified example of the driver information displayed on the display device. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a driver status display device in an emergency according to the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to this embodiment. [Embodiment]

[0010] Fig. 1 is a block diagram showing a schematic configuration of an emergency driver status display device according to an embodiment. Fig. 2 is a diagram showing normal information displayed on the display device. Fig. 3 is a diagram showing driver information displayed on the display device. A vehicle V in which the emergency driver status display device 1 is installed is a privately owned car, a vehicle used for business purposes, etc.

[0011] When an abnormality occurs in the vehicle, the emergency driver state display device 1 allows a third party who comes to rescue the driver to reliably recognize the driver's state before the abnormality occurred, which helps them to know on the spot how the driver's state after the abnormality occurred and the cause of the accident. The emergency driver state display device 1 is realized by mounting the components shown in Figure 1 on the vehicle V. Below, each component of the emergency driver state display device 1 will be described in detail with reference to Figure 1.

[0012] In addition, in the emergency driver status display device 1 shown in Figure 1, the connection method between each component for sending and receiving power supply, control signals, various information, etc. may be any of wired connections via wiring materials such as electric wires or optical fibers (including, for example, optical communication via optical fibers), wireless connections such as wireless communication and contactless power supply, unless otherwise specified.

[0013] The emergency driver state display device 1 includes a driver monitor 10, a vehicle state detection unit 20, a third party detection unit 30, a display device 40, and a display control device 50.

[0014] The driver monitor 10 captures an image of the driver of the vehicle V, and includes an imaging unit 11 and a driver state estimation unit 12.

[0015] The imaging unit 11 continuously captures driver images including the face of the driver of the vehicle V and generates a driver state moving image in which the driver images are arranged continuously in chronological order. The imaging unit 11 is installed, for example, in a meter unit provided in front of the driver's seat or on top of a column cover in order to capture an image of the face of the driver seated in the driver's seat inside the cabin of the vehicle V. The driver monitor 10 is electrically connected to the display control device 30, and the driver state moving image generated by the imaging unit 11 is output to the display control device 30 as driver state moving image information.

[0016] The driver state estimation unit 12 estimates the driver state based on the driver state video image acquired by the imaging unit 11. For example, the driver state estimation unit 12 estimates loss of consciousness, presence or absence of breathing, number of breaths, injury, bleeding, vomiting blood, burns, weakness, convulsions, vomiting, paralysis of limbs, body temperature, pulse rate, etc. based mainly on temporal changes of the driver based on the driver state video. The driver state estimation unit 12 outputs the estimated driver state to the display control device 30 as driver state estimation information.

[0017] The vehicle state detection unit 20 detects the state of the vehicle V, and in this embodiment, at least detects that an abnormality has occurred in the vehicle V. The vehicle state detection unit 20 includes, for example, an airbag sensor 21 and a driving assistance device 22. The airbag sensor 21 is an airbag activation state detection unit that detects the activation state of an airbag corresponding to a driver (not shown). The airbag sensor 21 is electrically connected to an interface unit 31 (described later) of the display control device 30, and outputs airbag activation state information, which is information regarding whether the airbag has been activated, to the display control device 30 as vehicle state information. The driving assistance device 22 is a device that assists the driver of the vehicle V in driving the vehicle V, and is a device that automatically or assists the driver in operating the accelerator, steering, brake, etc. of the vehicle V. The driving assistance device 22 can acquire driver state estimation information, and if it determines that it is difficult for the driver to drive the vehicle V based on the acquired driver estimation information, it performs a vehicle emergency stop process to stop the vehicle V in a safe state. The driving assistance device 22 is electrically connected to the interface unit 31, and outputs vehicle emergency stop information, which is information relating to whether or not a vehicle emergency stop process has been performed, to the display control device 30 as vehicle state information.

[0018] The third party 30 detects a third party who intends to rescue the driver from the vehicle V, and in this embodiment, detects at least a third party who intends to enter the interior of the vehicle V. The third party detection unit 30 is, for example, a door opening / closing sensor 31 and a proximity sensor 32. The door opening / closing sensor 31 detects the open / closed state of a door for entering the interior of the vehicle V (not shown). The door opening / closing sensor 31 is electrically connected to the interface unit 31 and outputs door opening / closing state information, which is information regarding whether the door has changed from a closed state to an open state, to the display control device 30 as third party presence information. The proximity sensor 32 detects the presence of a third party in the surrounding area of ​​the vehicle V. The proximity sensor 32 is electrically connected to the interface unit 31 and outputs vehicle surrounding area third party information, which is information regarding whether a third party is present in the surrounding area of ​​the vehicle V, to the display control device 30 as third party presence information.

[0019] The display device 40 is provided in the cabin of the vehicle V, and in this embodiment is a center display that can be viewed by occupants including the driver. The display device 40 is composed of a display unit 41 and an operation input unit 42. The display device 40 is electrically connected to a display control device 50, and receives a display signal as an input and outputs an operation signal to the display control device 50.

[0020] The display unit 41 displays information visible to the occupant based on the display signal, providing various information to the occupant. As shown in FIG. 2, the display unit 41 normally displays normal information I1, such as navigation information and audio information. As shown in FIG. 3, when an abnormal vehicle event occurs, the display of the normal information I1 is stopped and driver information I2 is displayed. In this embodiment, the driver information I2 includes driver status video information I21 and driver personal information I22. The driver status video information I21 is driver status video information from before the abnormal vehicle event occurred to the present, and thumbnail information of the driver status video information from before the abnormal vehicle event occurred to the present is displayed on the display unit 40. The driver status video information I21 is displayed in association with a time series icon 41a displayed on the display unit 41, and is displayed in association with a playback icon 41b, for example, overlapping with the icon. In this embodiment, the playback icon 41b is an icon for chronologically playing back the driver status video information I21 in reverse order from the present to before the abnormal vehicle event occurred. The driver personal information I2 is information about the individual driver, such as age, sex, whether or not the driver has a chronic illness, underlying illness, contact information, blood type, and the like, which is stored in advance as driver personal information in a storage unit 52 described later.

[0021] The operation input unit 42 is used by the occupant to perform operations on at least the display device 40, and is composed of switches, touch panels, etc. (not shown) that are installed around the display unit 41. When the operation input unit 42 is operated, an operation signal corresponding to the operation is output to the display control device 50. When a playback operation corresponding to the playback icon 41b is performed, the operation input unit 42 outputs an operation signal related to a reverse playback instruction for playing back the driver state video information I21 in reverse to the display control device 50. The operation input unit 42 can also perform an operation to switch the display state of the display unit 41 between the normal display I1 and the driver state display I2.

[0022] As shown in FIG. 1 , the display control device 50 generates and displays display information I to be displayed on the display unit 41 of the display device 40 based on input information such as driver state video information, driver state estimation information, vehicle state information, and third-party presence information. The display control device 50 may also be shared by an electronic control unit that comprehensively controls the entire vehicle V. The display control device 30 is configured to include electronic circuits mainly composed of a well-known microcomputer, including a central processing unit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), a ROM (Read Only Memory), a RAM (Random Access Memory), and an interface. The display control device 50 may be electrically connected to the driver monitor 10, the vehicle state detection unit 20, the third-party detection unit 30, and the display device 40 via an ECU (e.g., a body ECU) that controls each part of the vehicle V. The display control device 50 can exchange various electrical signals, such as various detection signals and drive signals for driving each part, with each part.

[0023] In terms of functional concept, the display control device 50 is configured to include an interface unit 51, a storage unit 52, a display control unit 53, and a display output unit 54. The interface unit 51, the storage unit 52, the display control unit 53, and the display output unit 54 can exchange various information with various devices that are electrically connected to them.

[0024] The interface unit 51 is a driver state moving image information acquisition unit that acquires driver state moving image information, a driver state estimation information acquisition unit that acquires driver state estimation information, a vehicle state information acquisition unit that acquires vehicle state information, a third party presence information acquisition unit that acquires third party presence information, and an interface for transmitting and receiving various information to and from each unit such as a detection unit. The interface unit 51 is configured to be electrically connectable to the storage unit 52, the display control unit 53, and the display output unit 54. The interface unit 51 has a function of wired communication of information with each unit via electric wires or the like, a function of wireless communication of information with each unit via a wireless communication unit or the like, and the like.

[0025] The storage unit 52 stores the driver status moving image information captured by the imaging unit 11. The storage unit 52 may be, for example, a relatively large-capacity storage device such as a hard disk, a solid-state drive (SSD), or an optical disk, or a rewritable semiconductor memory such as a RAM, a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 52 stores conditions and information necessary for various processes in the display control unit 53, various programs and applications executed by the display control unit 53, control data, and the like. The storage unit 52 stores the driver status moving image information from the start to the end of recording. The storage unit 52 has a preset storage area capacity for storing the driver status moving image information, and when the capacity reaches an upper limit while storing the driver status moving image information, the oldest driver status moving image information is overwritten. The display control unit 53 reads this information from the storage unit 52 as needed.

[0026] The display control unit 53 displays the normal display I1 on the display device 40. The display control unit 53 changes the display information I displayed on the display device 40. The display control unit 53 in this embodiment determines whether or not an abnormal vehicle situation has occurred based on vehicle state information. The display control unit 53 determines whether or not an abnormal vehicle situation has occurred when it determines that an airbag has been deployed based on airbag deployment state information, which is the vehicle state information, or when it determines that an emergency vehicle stop process has been performed by the driving assistance device 22 based on vehicle emergency stop information, which is the vehicle state information. Note that the display control unit 53 in this embodiment determines whether or not an abnormal vehicle situation has occurred based on the vehicle state information, but this is not limited to this. The display control unit 53 in this embodiment may also determine whether or not an abnormal vehicle situation has occurred based on driver state estimation information. The display control unit 53 may also determine whether or not an abnormal vehicle situation has occurred when it determines that the driver has lost consciousness, lost strength, had convulsions, or the like. The display control unit 53 may also determine whether or not an abnormal vehicle situation has occurred based on the vehicle state information and the driver state estimation information. In this case, the display control unit 53 may determine that an abnormal vehicle situation has occurred, for example, when it determines that an abnormal vehicle situation has occurred based on the vehicle state information and also based on the driver state estimation information. When the display control unit 53 determines that an abnormal vehicle situation has occurred, it generates driver information I2 and causes the display device 40 to display the driver information I2. When the display control unit 53 determines that an abnormal vehicle situation has occurred, it determines that a door has changed from a closed state to an open state based on door open / close state information, which is third party presence information, and / or determines that a third party is present in the peripheral area of ​​the vehicle V based on vehicle peripheral area third party information, which is third party presence information, and generates driver information I2 and causes the display device 40 to display the driver information I2. Here, when the normal display I1 is displayed on the display device 40, the display control unit 53 changes the display information I displayed on the display device 40 by switching the display to driver information I2. Furthermore, when the display information I is not displayed on the display device 40, the display control unit 53 changes the display information I displayed on the display device 40 by displaying driver information I2.The display control unit 53 extracts driver status video information from the driver status video information stored in the storage unit 52, from before it is determined that a vehicle abnormality has occurred until the occurrence of the vehicle abnormality, and generates driver status video information I21. Here, "before it is determined that a vehicle abnormality has occurred" refers to a predetermined time before the occurrence of the vehicle abnormality, for example, several minutes to several tens of minutes before. The display control unit 53 also generates the driver status video information I21 by adding information about how long ago the vehicle abnormality occurred relative to the current time. The display control unit 53 also extracts one or more pieces of driver personal information useful for determining the action to be taken for a specific driver from multiple pieces of driver personal information pre-stored in the storage unit 52, and generates driver personal information I22. The display control unit 53 generates driver information I2 including the generated driver status video information I21 and driver personal information I22.

[0027] The display output unit 34 outputs a display signal corresponding to the display information I generated by the display control unit 33 to the display device 40.

[0028] Next, there will be explained an outline of the operation of the emergency driver state display device 1. Fig. 4 is a flowchart showing an example of control of the emergency driver state display device according to the embodiment.

[0029] First, as shown in Fig. 5, the driver monitor 10 determines whether the ignition is ON (step S1). Here, the driver monitor 10 determines whether the vehicle V is in a starting state.

[0030] Next, when the driver monitor 10 determines that the ignition is ON (step S1: Yes), it starts capturing images using the imaging unit 11 (step S2). When the driver monitor 10 determines that the ignition is OFF (step S1: No), it repeats step S1 until the ignition is turned ON.

[0031] Next, the display control unit 53 starts recording the driver state moving image information in the storage unit 32 (step S3).

[0032] Next, the display control unit 33 determines whether an abnormal vehicle situation has occurred based on the vehicle state information (step S4).

[0033] Next, when the display control unit 33 determines that no abnormal vehicle situation has occurred (step S4: No), the driver monitor 10 determines whether the ignition is OFF (step S5). Here, the driver monitor 10 determines whether the vehicle V is in a non-start state.

[0034] Next, when the driver monitor 10 determines that the ignition is OFF (step S5: Yes), it ends the imaging by the imaging unit 11 (step S6). Note that when the driver monitor 10 determines that the ignition is ON (step S5: No), it repeats steps S4 and S5 until the ignition is turned OFF.

[0035] Next, the display control unit 53 ends the recording of the driver state moving image information by the storage unit 32 (step S7).

[0036] Furthermore, when the display control unit 53 determines that an abnormal vehicle situation has occurred (step S4: Yes), it generates driver information I2 including driver condition image information I21 and driver personal information I22 (step S8).

[0037] Next, the display control unit 33 determines whether or not a third party is present in the surrounding area of ​​the vehicle V based on the third party presence information (step S9).

[0038] Next, when the display control unit 33 determines that no third party is present in the area surrounding the vehicle V (step S9: No), it causes the display device 40 to perform normal display (step S10). Here, if there is no need to display the driver information I2 on the display device 40, the display control unit 53 maintains the current state; for example, if normal information I1 is being displayed, it maintains the display of normal information I1, and if the display device 40 is not displayed to reduce power consumption, it maintains the display not to be displayed.

[0039] Furthermore, when the display control unit 33 determines that a third party is present in the area surrounding the vehicle V (step S9: Yes), the display control unit 33 displays driver information on the display device 40 (step S11).

[0040] Next, the display control unit 53 determines whether or not a playback instruction has been issued (step S12). Here, the display control unit 53 determines whether or not a playback operation corresponding to the playback icon 41b has been issued to the operation input unit 42 by a third party.

[0041] Next, when the display control unit 53 determines that a playback instruction has been issued (step S12: Yes), it plays back the driver state video information (step S13). Here, the display control unit 53, in response to a request from a third party, performs reverse playback of the driver state video from the time when the vehicle abnormality occurred to the time before the vehicle abnormality occurred, that is, to confirm the driver state before the vehicle abnormality occurred. Note that the display control unit 53 may perform reverse playback (double-speed reverse playback) while the playback operation corresponding to the play icon 41b is being continued, and may perform normal playback of the driver state video from that point when the playback operation corresponding to the play icon 41b is completed. This allows a third party to easily recognize the driver state that caused the vehicle abnormality to occur. Also, when the display control unit 53 determines that a playback instruction has not been issued (step S12: No), it repeats step S12 until a playback instruction is issued.

[0042] The emergency driver status display device 1 in this embodiment described above allows a third party coming to rescue the driver after an abnormal vehicle event occurs to immediately recognize driver information, including video information about the driver's status before the abnormal vehicle event occurred. If an abnormal vehicle event occurs and the driver loses consciousness or becomes unable to communicate with a third party outside the vehicle, the third party coming to rescue the driver can learn the driver's current status but cannot understand how the driver reached his or her current state. Therefore, after detecting an abnormal vehicle event, the display control unit 33 controls the display device 40 to switch from normal information I1, which is normally displayed, to driver information I2, which includes a driver status image I21 containing video information about the driver's status before the abnormal vehicle event occurred. This allows the third party coming to rescue the driver to immediately recognize the video information I21 about the driver's status before the abnormal vehicle event occurred. Furthermore, the third party can review the driver's status from before the abnormal vehicle event to view the video or other information that shows how the driver's status reached its current state, and take appropriate measures for the driver's status. Furthermore, the display control unit 33 controls the display device 40 to display the driver's personal information I22 in addition to the driver's state video information I21. When comparing the actions to be taken when rescuing a driver based only on the driver's state after the occurrence of a vehicle abnormality with the actions to be taken when the driver's personal information I22 is available, the actions that can be taken can be more appropriately performed when the driver's personal information I22 is available. Also, the effort required to determine the actions that should be taken for the driver after the occurrence of a vehicle abnormality can be reduced. Furthermore, the information I22 about the individual driver can also be used to determine the actions that should not be taken for the driver.

[0043] In this embodiment, the occurrence of a vehicle abnormality is determined based on the airbag deployment status information. When the airbag is deployed, there is a high possibility that the driver is unconscious and therefore unable to communicate the information to a third party. Therefore, by displaying the driver information I2 on the display device 40 on the condition that the airbag is deployed, third parties can accurately recognize the driver information I2.

[0044] In addition, in this embodiment, driver information I2 is displayed on display device 40 under the conditions that an abnormal vehicle situation has occurred and a third party is present in the surrounding area of ​​vehicle V. Therefore, after an abnormal vehicle situation has occurred, a third party who has come to rescue the driver approaches vehicle V and is able to see display device 40. By displaying driver information I2 on display device 40, the third party can be sure to recognize the driver information.

[0045] The emergency driver state display device 1 according to the embodiment of the present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims.

[0046] In this embodiment, the driver information I2 includes the driver status moving image information I21 and the driver personal information I22, but it may include only the driver status moving image information I21.

[0047] In this embodiment, the driver state video information I21 is driver state video information from a predetermined time before the occurrence of the abnormal vehicle event to the occurrence of the abnormal vehicle event, but this is not limited to this. The driver state video information I21 may also be from the time it is determined that the driver lost consciousness based on the driver state estimation information to the present. FIG. 5 shows a modified example of the driver information displayed on the display device. In this case, as shown in FIG. 5, the display control unit 53 may generate driver state video information I21 from the time the driver lost consciousness occurred to the time the abnormal vehicle event occurred, driver state video information I21 from the time the driver lost consciousness occurred to the present, and driver state video information I21 from the time the driver lost consciousness occurred to the present. For example, the playback icon 41b associated with each driver state video information I21 is preferably an icon for chronological playback.

[0048] In this embodiment, the driver personal information I22 is generated by extracting it from the driver personal information stored in advance in the storage unit 52, but this is not limited to this. The driver personal information I22 may include driver personal information based on driver state estimation information. As shown in FIG. 5, the driver personal information based on the driver state estimation information may include, for example, body temperature, pulse rate, and driver state (whether or not the driver is unconscious). [Explanation of symbols]

[0049] 1 Emergency driver status display device 10 Driver Monitor 11 Imaging unit 12 Driver state estimation unit 20 Vehicle condition detection unit 21 Airbag sensor 22 Driving assistance devices 30 Third Party Detection Unit 31 Door open / close sensor 32 Proximity Sensor 40 Display device 41 Display section 42 Operation input section 50 Display control device 51 Interface section 52 Storage section 53 Display control unit 54 Display output section I1 Normal Information I2 driver information I21 Driver status video information I22 Driver Personal Information

Claims

1. an imaging unit that continuously captures images of the driver's state; a storage unit that stores driver state video information captured by the imaging unit; a driver state estimation unit that estimates a driver state based on the driver state video information; a vehicle state information acquisition unit that acquires vehicle state information; a display device provided in the vehicle interior and displaying at least normal information; a display control unit that changes display information to be displayed on the display device; Equipped with the display control unit, when determining that a vehicle abnormality has occurred based on at least one of the vehicle state information and driver state estimation information, which is the driver state estimated by the driver state estimation unit, causes the display device to display driver information; the driver information includes, among the driver state moving image information stored in the storage unit, at least driver state moving image information before it is determined that the vehicle abnormality has occurred; the display control unit plays back the driver state moving image information based on a playback instruction for the driver state moving image information; An emergency driver status display device characterized by:

2. The driver information includes driver personal information that is information about an individual driver. The emergency driver status display device according to claim 1.

3. An airbag operation state detection unit that detects the operation state of the airbag is further provided, The vehicle state information is airbag operation state information output from the airbag operation state detection unit. When determining that the airbag has been deployed based on the airbag deployment state information, the display control unit determines that the vehicle abnormality has occurred.

3. The driver status display device in an emergency according to claim 1 or 2.

4. a third party presence information acquisition unit for acquiring third party presence information; the display control unit, when determining that the vehicle abnormality has occurred and determining that a third party is present based on the third party presence information, causes the display device to display the driver information. The emergency driver status display device according to any one of claims 1 to 3.

Images