Abnormality detection device

Abstract

To provide an abnormality detection device capable of respectively distinguishing and detecting a sign state indicating a sign of incapability of driving by a driver and an abnormal state indicating incapability of driving by the driver.SOLUTION: An abnormality detection device includes a control section (5) for respectively detecting a sign state indicating a sign of incapability of driving by a driver and an abnormal state indicating incapability of driving by the driver based on information related to the driver of a vehicle (1) acquired by an input section (2).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an abnormality detection device. [Background technology]

[0002] Various abnormality detection devices have been developed to detect abnormal states of drivers who drive vehicles. For example, Patent Document 1 discloses an inability to drive prediction device that includes an acquisition unit that acquires images captured by a camera that captures the interior of the vehicle, a detection unit that detects the current posture of the driver based on the captured images, and a state prediction unit that predicts the driver's future inability to drive based on the current posture of the driver detected by the detection unit and a database related to the driver's past postures. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-074964 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology such as that in Patent Document 1, although it is possible to predict an inoperable state, it is not possible to detect an abnormal state in which the vehicle is inoperable.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an abnormality detection device that can distinguish and detect a premonitory state that indicates that the driver will be unable to drive and an abnormal state that indicates that the driver is unable to drive. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the abnormality detection device of the present invention is characterized in that it includes a control unit that detects a premonitory state indicating that the driver will be unable to drive, and an abnormal state indicating that the driver is unable to drive, based on information regarding the driver of the vehicle acquired by an input unit. Effect of the Invention

[0007] The abnormality detection device according to the present invention has the advantage of being able to distinguish and detect a premonitory state which indicates that the driver may be unable to drive and an abnormal state which indicates that the driver is unable to drive. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle including an abnormality detection device according to an embodiment. [Diagram 2] FIG. 2 is a flowchart showing an example of processing in the anomaly detection method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, the components in the following embodiments include those that are replaceable and easy for a person skilled in the art, or those that are substantially the same.

[0010] [Embodiment] Fig. 1 is a block diagram showing an example of a schematic configuration of a vehicle including an abnormality detection device according to an embodiment. The vehicle 1 including the abnormality detection device according to the present embodiment shown in Fig. 1 includes an input unit 2, an output unit 3, a communication unit 4, a control unit 5 as the abnormality detection device, and a storage unit 7.

[0011] The input unit 2 includes a camera 21, a heart rate measuring device 22, a microphone 23, and a switch 24.

[0012] The camera 21 generates image data capturing an image of the driver of the vehicle 1. The camera 21 includes, for example, a complementary metal oxide semiconductor (CMOS) image sensor. The camera 21 may be another type of image sensor, such as a charge-coupled device (CCD) image sensor.

[0013] The heart rate measuring device 22 measures the heart rate of the driver of the vehicle 1. The heart rate measuring device 22 may be mounted on the steering wheel, or may be a wearable device such as a smart watch that can be worn by the driver. The pulse wave measurement may be a non-contact measurement using the Doppler effect using radio waves, a measurement that detects the pulse from the color of the face, or a measurement that measures the pressure generated by the pulse using a piezoelectric element. The heart rate measuring device 22 may also have a function of measuring the breathing of the driver of the vehicle 1 using the Doppler effect using radio waves.

[0014] The microphone 23 captures the voice of the driver of the vehicle 1 and generates voice data.

[0015] The switch 24 has, for example, a push button, and outputs a predetermined signal when the driver presses the push button.

[0016] The output section 3 includes a speaker 31 and an actuator 32 .

[0017] The speaker 31 outputs various sounds under the control of the control unit 5 .

[0018] The actuator 32 outputs various operations under the control of the control unit 5. The actuator 32 generates vibrations in, for example, a seat or a steering wheel.

[0019] The communication unit 4 is wirelessly connected to the network, and transmits and receives information, signals, and the like to and from various terminals connected to the network in a wired or wireless manner.

[0020] The control unit 5 includes an electronic circuit mainly made up of a known microcomputer including a central processing unit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a ROM (Read Only Memory), a RAM (Random Access Memory), and an interface. The control unit 5 includes a face detection unit 511, a facial expression detection unit 512, a body part detection unit 521, a motion analysis unit 522, a heart rate detection unit 53, a breathing detection unit 54, a voice detection unit 55, a response detection unit 56, a first sign detection unit 57, a second sign detection unit 58, an abnormal state detection unit 59, an abnormality determination unit 60, and a vehicle control unit 61.

[0021] The face detection unit 511 performs face detection in the image data captured by the camera 21 .

[0022] The facial expression detection unit 512 extracts the facial feature points detected by the face detection unit 511, and detects various facial expressions of the driver of the vehicle 1. Specifically, the facial expression detection unit 512 detects facial expressions such as repeatedly opening and closing the mouth widely, keeping the mouth half-open or wide open, repeatedly looking down intermittently, coughing violently, coughing frequently, keeping the eyes rolled back or closed, paleness, a distorted face (a facial expression of enduring pain), a painful expression, and the like.

[0023] The body part detection unit 521 detects the joints and bones of the human driver, and detects various body parts.

[0024] The motion analysis unit 522 analyzes various motions of the driver of the vehicle 1 based on the motions of the body parts detected by the body part detection unit 521. Specifically, the motion analysis unit 522 detects motions such as touching the head due to discomfort, driving with one hand for a long time, placing a hand firmly on the chest, touching the chest with both hands in a groping manner, rubbing the stomach, loosening the necktie or shirt collar, breathing with the shoulders, pushing out the chest, repeatedly shifting in a sitting position, staggering, poor posture (leaning face down, leaning back, falling to the side, etc.), convulsions, etc.

[0025] The heartbeat detection unit 53 detects the heartbeat of the driver of the vehicle 1 based on the measurement result of the heartbeat measuring device 22. Specifically, the heartbeat detection unit 53 detects a sudden increase or decrease in the heartbeat, arrhythmia, tachycardia, a decrease in blood pressure, and the like based on the measurement result of the heartbeat measuring device 22.

[0026] When the heart rate measuring device 22 has a function of measuring respiration, the respiration detection unit 54 detects the respiration of the driver of the vehicle 1 based on the measurement result. Specifically, the respiration detection unit 54 detects irregular breathing, etc. based on the measurement result of the heart rate measuring device 22.

[0027] The voice detection unit 55 detects the voice of the driver of the vehicle 1 from the voice data generated by the heartbeat detection unit 53. Specifically, the voice detection unit 55 detects the driver's screaming, groaning, slurred speech, loud breathing, and the like.

[0028] The response detection unit 56 detects the response of the driver to the call to the driver. The driver's response may be a voice response or a response by pressing the switch 24.

[0029] The first sign detection unit 57 and the second sign detection unit 58 detect, based on information about the driver of the vehicle 1 acquired by the input unit 2, a sign state that indicates a sign that the driver will become unable to drive.

[0030] The first sign detection unit 57 detects a first sign state in which the driver may be unable to drive. For example, when the first sign detection unit 57 detects an unconscious behavior of the driver in response to a physical discomfort of the driver, the first sign detection unit 57 determines that the first sign state exists.

[0031] The second sign detection unit 58 detects a second sign state in which the possibility that the driver will become unable to drive is higher than the first sign state. When the second sign detection unit 58 detects a driver behavior of dealing with or alleviating the driver's physical abnormality, it determines that the second sign state exists.

[0032] The abnormal state detection unit 59 detects an abnormal state indicating that the driver is unable to drive, based on information about the driver of the vehicle 1. When the abnormal state detection unit 59 detects that the driver is unable to drive, it determines that an abnormal state exists. When the abnormal state detection unit 59 detects an abnormal state, it calls out to the driver, and when the driver does not respond to the call within a predetermined period of time, it performs control to avoid a vehicle accident. The call may be a voice call from the speaker 31, or a notification by vibration from the actuator 32. Furthermore, the driver may respond by voice and the response may be detected by the microphone 23, or the driver may respond by pressing the switch 24.

[0033] The first sign detection unit 57, the second sign detection unit 58, and the abnormal condition detection unit 59 detect a first sign condition, a second sign condition, and an abnormal condition, respectively, by different sets of condition detection operations.

[0034] When a predetermined input is received from the communication unit 4, the abnormality determination unit 60 determines that an abnormality has occurred and performs control to avoid a vehicle accident. When an external operator determines that the driver is unable to drive based on information from the camera 21 and microphone 23, he or she transmits a predetermined signal to the vehicle 1.

[0035] The vehicle control unit 61 controls the driving of the vehicle 1. The vehicle control unit 61 may be provided as an ECU separate from the control unit 5. The vehicle control unit 61 performs different controls on the vehicle 1 in the first predictive state, the second predictive state, and the abnormal state. Specifically, the vehicle control unit 61 shortens a time threshold for starting a driving assist control to decelerate or stop the vehicle 1 in the first predictive state. In addition, the vehicle control unit 61 further shortens a time threshold for starting a driving assist control to decelerate or stop the vehicle 1 in the second predictive state than in the first predictive state. In addition, the vehicle control unit 61 calls out to the driver in the second predictive state and requests the driver to respond. In addition, the vehicle control unit 61 may perform preliminary control for a safe stop of the vehicle 1, such as limiting or reducing the speed of the vehicle 1 and maintaining the driving lane, in the second predictive state. In addition, the vehicle control unit 61 further shortens a time threshold for starting a driving assist control to decelerate or stop the vehicle 1 in the abnormal state than in the second predictive state. Furthermore, the vehicle control unit 61 shortens the time threshold for waiting for a response in an abnormal state. Furthermore, when the abnormality determination unit 60 determines that there is an abnormality, the vehicle control unit 61 performs control to avoid an accident involving the vehicle 1, such as slowing down and stopping the vehicle 1.

[0036] The storage unit 7 may be, for example, a relatively large-capacity storage device such as a hard disk, a solid state drive (SSD), or an optical disk, or a semiconductor memory in which data can be rewritten such as a RAM, a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 7 stores conditions and information necessary for various processes in the control unit 5, various programs and applications executed by the control unit 5, control data, and the like. The storage unit 7 also stores personal data 71, detection result data 72, and driving data 73.

[0037] The personal data 71 is information about the individual driver of the vehicle 1, such as name, sex, age, weight, blood pressure, driving history, medical history, and the like.

[0038] The detection result data 72 is information regarding the detection results by the input unit 2 and the control unit 5, and is information such as image data generated by the camera 21, the heart rate measured by the heart rate measuring device 22, audio data generated by the microphone 23, and the detection results in the first precursor detection unit 57, the second precursor detection unit 58, and the abnormal state detection unit 59.

[0039] The driving data 73 is information related to the driving of the vehicle 1, such as the history of the driver's operation of the vehicle 1, the history of the control of the vehicle 1 by the vehicle control unit 61, and the like.

[0040] Next, an abnormality detection method by the control unit 5 will be described. FIG. 2 is a flowchart showing an example of processing in the abnormality detection method. First, as shown in FIG. 2, the first sign detection unit 57 judges whether or not the vehicle is in the first predictive state (step S1). For example, when the first sign detection unit 57 detects a group of state detection actions such as the driver of the vehicle 1 touching his / her face, chest or shoulders, opening his / her mouth, opening and closing his / her mouth, or coughing frequently in image data of the driver captured by the camera 21 as an unconscious behavior of the driver in response to a physical discomfort of the driver, the first sign detection unit 57 judges that the vehicle is in the first predictive state. In addition, the first sign detection unit 57 may judge that the vehicle is in the first predictive state when the heart rate of the driver of the vehicle 1 is rapidly increasing or decreasing as measured by the heart rate measuring device 22.

[0041] When the first sign detection unit 57 determines that the vehicle is not in the first sign state (step S1: No), the second sign detection unit 58 determines whether the vehicle is in the second sign state (step S2). For example, when the second sign detection unit 58 detects a group of state detection actions, such as the driver of the vehicle 1 firmly placing his / her hands on his / her chest, having a pained expression, or a change (worsening) in facial color, in the image data of the driver captured by the camera 21 as a driver's behavior for dealing with or alleviating the driver's physical abnormality, the second sign detection unit 58 determines that the vehicle is in the second sign state. In addition, the second sign detection unit 58 may determine that the vehicle is in the second sign state when the driver's voice data acquired by the microphone 23 shows that the driver is groaning. In addition, the second sign detection unit 58 may determine that the vehicle is in the second sign state when the heart rate measurement device 22 shows that the driver's heart rate has an irregular heartbeat, a tachycardia, or irregular breathing.

[0042] When the second sign detection unit 58 determines that the vehicle is not in the second sign state (step S2: No), the abnormal state detection unit 59 determines whether the vehicle is in an abnormal state (step S3). For example, when the abnormal state detection unit 59 detects a group of state detection actions, such as the driver of the vehicle 1 having closed eyes, white eyes, convulsing, or poor posture, in the image data of the driver captured by the camera 21 as a state indicating that the driver is unable to drive, the abnormal state detection unit 59 determines that the vehicle is in an abnormal state. In addition, the abnormal state detection unit 59 may determine that the vehicle is in an abnormal state when the driver of the vehicle 1 is not operating the steering wheel based on the driving data 73. In addition, the abnormal state detection unit 59 may determine that the vehicle is in an abnormal state when the blood pressure of the driver of the vehicle 1 is reduced by the heart rate measuring device 22.

[0043] If the abnormal state detection unit 59 determines that there is no abnormal state (step S3: No), the control unit 5 determines whether or not a predetermined control signal indicating the end of operation has been detected (step S4). If the control unit 5 determines that a predetermined control signal indicating the end of operation has been detected (step S4: Yes), the series of processes ends.

[0044] On the other hand, when the control unit 5 determines that the predetermined control signal indicating the end of operation has not been detected (step S4: No), the process returns to step S1 and continues the process.

[0045] In step S1, when the first sign detection unit 57 determines that the vehicle is in the first sign state (step S1: Yes), the control unit 5 adjusts various thresholds used by the vehicle control unit 61 to control the vehicle 1 (step S5). For example, the control unit 5 shortens the time threshold at which the vehicle control unit 61 starts driving assist control to decelerate or stop the vehicle 1.

[0046] In step S2, when the second sign detection unit 58 determines that the vehicle is in the second sign state (step S2: Yes), the control unit 5 adjusts various thresholds used by the vehicle control unit 61 to control the vehicle 1 and starts calling the driver of the vehicle 1 (step S6). For example, the control unit 5 further shortens the time threshold at which the vehicle control unit 61 starts driving assistance control to decelerate or stop the vehicle 1, compared to the first sign state. In addition, the control unit 5 calls the driver by voice, such as a buzzer sound from the speaker 31, and requests a voice response. In addition, the control unit 5 may notify an external operator that the second sign state has been detected through the communication unit 4. This makes it possible to notify the external operator earlier than notifying the operator after detecting the abnormal state. In addition, the control unit 5 may perform preliminary control for safely stopping the vehicle 1, such as limiting or reducing the speed of the vehicle 1 and maintaining the driving lane, at this stage.

[0047] Next, the control unit 5 determines whether or not there is a response from the driver (step S7). For example, when the control unit 5 detects a predetermined response from the driver by the microphone 23, it determines that there is a response from the driver.

[0048] When the control unit 5 determines that there is a response from the driver (step S7: Yes), the control unit 5 proceeds to step S3 and continues the process. At this time, the control unit 5 resets various thresholds used by the vehicle control unit 61 to control the vehicle 1 to their initial values.

[0049] On the other hand, if the control unit 5 determines that there is no response from the driver (step S7: No), the control unit 5 causes the vehicle control unit 61 to execute a process of safely decelerating and stopping the vehicle 1 (step S8). The control unit 5 outputs a sound from the speaker 31 to notify the outside of the vehicle of an abnormality, and causes the hazard warning indicator lights to flash to decelerate the vehicle, stops the vehicle 1 on a safe roadside or the like, and ends the process.

[0050] In step S3, when the abnormal state detection unit 59 determines that an abnormal state exists (step S3: Yes), the control unit 5 adjusts various thresholds used by the vehicle control unit 61 to control the vehicle 1, and increases the volume of a buzzer sound to call the driver of the vehicle 1 (step S9). For example, the control unit 5 further shortens the time threshold at which the vehicle control unit 61 starts driving assistance control to decelerate or stop the vehicle 1, compared to the second predictive state. Also, the control unit 5 increases the volume of the buzzer sound from the speaker 31, and shortens the time threshold for waiting for a response. Also, the control unit 5 may notify an external operator via the communication unit 4 that an abnormal state has been detected.

[0051] Next, the control unit 5 determines whether or not there is a response from the driver (step S10). If the control unit 5 determines that there is a response from the driver (step S10: Yes), the control unit 5 proceeds to step S4 and continues the process. At this time, the control unit 5 resets various thresholds used by the vehicle control unit 61 to control the vehicle 1 to their initial values.

[0052] On the other hand, if the control unit 5 determines that there is no response from the driver (step S10: No), the process proceeds to step S8 and continues.

[0053] According to the abnormality detection device described above, the control unit 5 distinguishes between and detects a predictive state and an abnormal state, thereby improving safety compared to a case in which only one of a predictive state and an abnormal state is detected.

[0054] In addition, the abnormality detection device detects abnormalities or signs of abnormalities in the driver using image data generated by the camera 21, the heart rate measured by the heart rate measuring device 22, or audio data generated by the microphone 23, thereby improving the accuracy of abnormality detection compared to when detecting abnormalities in the driver using images alone.

[0055] In addition, according to the abnormality detection device, different controls are performed in a predictive state and an abnormal state, so that in a predictive state, preliminary control is performed to ensure the safety of the vehicle 1, and in an abnormal state, control is performed to decelerate and stop the vehicle 1 while ensuring the safety of the vehicle 1, thereby improving the safety of the vehicle 1.

[0056] Furthermore, the abnormality detection device detects the first premonitory state and the second premonitory state, making it possible to detect an abnormality premonition from a slight change in state, such as an unconscious behavior of the driver.

[0057] Furthermore, the abnormality detection device detects the first premonitory state, the second premonitory state, and the abnormal state through different sets of state detection operations, thereby making it possible to accurately determine the state of the driver in a step-by-step manner.

[0058] Furthermore, according to the abnormality detection device, when a call is made to the driver and there is no response to the call, the vehicle 1 can be safely controlled.

[0059] Furthermore, the abnormality detection device can detect anomalies at an early stage by detecting unconscious behavior of the driver in response to a physical discomfort felt by the driver.

[0060] Furthermore, the abnormality detection device can accurately detect signs of an abnormality by detecting the driver's behavior in dealing with a physical abnormality in the driver.

[0061] Furthermore, the abnormality detection device can detect when the driver is unable to drive, thereby making it possible to detect the inability to drive at an early stage and ensure the safety of the vehicle 1.

[0062] The abnormality detection device according to the embodiment of the present invention described above is not limited to the above embodiment, and various modifications are possible within the scope of the claims.

[0063] In the above description, an example has been described in which the vehicle 1 is equipped with the control unit 5 which is an abnormality detection device, but a terminal outside the vehicle 1 may have the function of the abnormality detection device.

[0064] The abnormality detection device according to this embodiment may be configured by appropriately combining the components of the embodiment and modified examples described above. [Explanation of symbols]

[0065] 1 vehicle 2 Input section 3. Output section 4. Communications Department 5. Control section (abnormality detection device) 7 Memory section 21 Camera 22 Heart rate monitor 23. Mike 24 Switch 31 Speakers 32 Actuator 53 Heart rate detector 54 Breath detection unit 55 Voice detection section 56 Response detection unit 57 First Predictor Detection Unit 58 Second Predictor Detection Unit 59 Abnormal condition detection section 60 Abnormality determination section 61 Vehicle control unit 71 Personal Data 72 Detection result data 73 Operational Data 511 Face detection unit 512 Facial Expression Detection Unit 521 Body Part Detection Unit 522 Motion analysis section

Claims

1. Based on the information about the driver of the vehicle acquired by the input unit, A predictive state indicating a predictive state that the driver will be unable to drive; an abnormal state indicating that the driver is unable to drive; A control unit for detecting each of the above. Anomaly detection device.

2. the input unit includes a camera that generates image data; The information is the image data of the driver captured by the camera. The abnormality detection device according to claim 1 .

3. the input unit includes a microphone that generates voice data; The information is voice data obtained by acquiring the voice of the driver by the microphone. The abnormality detection device according to claim 1 .

4. the input unit includes a heart rate measuring device that measures a heart rate; The information is the heart rate of the driver measured by the heart rate measuring device. The abnormality detection device according to claim 1 .

5. The control unit performs different controls on the vehicle depending on whether the predictive state is in the abnormal state or not. The abnormality detection device according to claim 1 .

6. The control unit is As the predictive state, a first predictive state in which the driver may be unable to drive; a second predictive state in which the possibility that the driver will become unable to drive is higher than the first predictive state; and Detect each of The abnormality detection device according to claim 1 .

7. The control unit detects the first predictive state, the second predictive state, and the abnormal state by different state detection operation groups, respectively. The abnormality detection device according to claim 6.

8. The control unit is When the abnormal state is detected, a call is made to the driver, If the driver does not respond to the call within a predetermined period of time, control is performed to avoid an accident involving the vehicle. The abnormality detection device according to claim 1 .

9. The control unit determines that the driver is in the predictive state when detecting an unconscious behavior of the driver in response to a physical discomfort of the driver. The abnormality detection device according to claim 1 .

10. The control unit determines that the driver is in the predictive state when detecting an action of the driver that deals with or alleviates the driver's physical abnormality. The abnormality detection device according to claim 1 .

11. The control unit determines that the abnormal state occurs when detecting that the driver is unable to drive. The abnormality detection device according to claim 1 .

Images