Anomaly detection device

Abstract

To provide an abnormality detection device capable of respectively distinguishing and detecting a sign state indicating a sign of incapability of driving by a driver and an abnormal state indicating incapability of driving by the driver.SOLUTION: An abnormality detection device includes a control section (5) for respectively detecting a sign state indicating a sign of incapability of driving by a driver and an abnormal state indicating incapability of driving by the driver based on information related to the driver of a vehicle (1) acquired by an input section (2).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an abnormality detection device. [Background technology]

[0002] Various abnormality detection devices have been developed to detect abnormal states of drivers who drive vehicles. For example, Patent Document 1 discloses an inability to drive state prediction device that includes an acquisition unit that acquires images captured by a camera that captures images of the interior of the vehicle, a detection unit that detects the current posture of the driver based on the captured images, and a state prediction unit that predicts the driver's future inability to drive based on the current posture of the driver detected by the detection unit and a database related to the driver's past postures. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-074964 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology such as that disclosed in Patent Document 1, although it is possible to predict an inoperable state, it is not possible to detect an abnormal state that makes the vehicle inoperable.

[0005] The present invention has been made in consideration of the above circumstances, and aims to provide an abnormality detection device that can distinguish between and detect a premonitory state that indicates that the driver may be unable to drive and an abnormal state that indicates that the driver is unable to drive. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the abnormality detection device of the present invention is characterized by having a control unit that detects a premonitory state that indicates that the driver will be unable to drive, and an abnormal state that indicates that the driver is unable to drive, based on information about the driver of the vehicle acquired by an input unit. [Effects of the Invention]

[0007] The abnormality detection device according to the present invention has the advantage of being able to distinguish between a premonitory condition that indicates that the driver may become unable to drive and an abnormal condition that indicates that the driver is unable to drive. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle including an abnormality detection device according to an embodiment. [Figure 2] FIG. 2 is a flowchart showing an example of processing in the anomaly detection method. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, the components in the following embodiments include those that are easily replaceable by those skilled in the art, or those that are substantially the same.

[0010] [Embodiment] 1 is a block diagram showing an example of the schematic configuration of a vehicle including an abnormality detection device according to an embodiment. The vehicle 1 including the abnormality detection device of this embodiment shown in FIG. 1 includes an input unit 2, an output unit 3, a communication unit 4, a control unit 5 as the abnormality detection device, and a storage unit 7.

[0011] The input unit 2 includes a camera 21, a heart rate measuring device 22, a microphone 23, and a switch 24.

[0012] The camera 21 generates image data of an image of the driver of the vehicle 1. The camera 21 includes, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 21 may also be another type of image sensor, such as a CCD (Charge-Coupled Device) image sensor.

[0013] The heart rate measuring device 22 measures the heart rate of the driver of the vehicle 1. The heart rate measuring device 22 may be mounted on the steering wheel, for example, or may be a wearable device such as a smart watch that can be worn by the driver. Pulse wave measurement may be performed by non-contact measurement using the Doppler effect with radio waves, by detecting the pulse from the color of the face, or by measuring the pressure generated by the pulse with a piezoelectric element. The heart rate measuring device 22 may also have a function of measuring the breathing of the driver of the vehicle 1 with the Doppler effect with radio waves.

[0014] The microphone 23 captures the voice of the driver of the vehicle 1 and generates voice data.

[0015] The switch 24 has, for example, a push button, and outputs a predetermined signal when the driver presses the push button.

[0016] The output unit 3 includes a speaker 31 and an actuator 32 .

[0017] The speaker 31 outputs various sounds under the control of the control unit 5.

[0018] The actuator 32 outputs various operations under the control of the control unit 5. The actuator 32 generates vibrations in the seat or steering, for example.

[0019] The communication unit 4 is wirelessly connected to the network, and transmits and receives information, signals, etc. to and from various terminals connected to the network by wire or wirelessly.

[0020] The control unit 5 is configured by an electronic circuit mainly consisting of a well-known microcomputer including a central processing unit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a ROM (Read Only Memory), a RAM (Random Access Memory), and an interface. The control unit 5 includes a face detection unit 511, a facial expression detection unit 512, a body part detection unit 521, a motion analysis unit 522, a heartbeat detection unit 53, a breathing detection unit 54, a voice detection unit 55, a response detection unit 56, and a first sign detection unit 57. situation Detector 57 and second predictor situation The vehicle control system includes a detection unit 58, an abnormal state detection unit 59, an abnormality determination unit 60, and a vehicle control unit 61.

[0021] The face detection unit 511 detects faces in the image data captured by the camera 21.

[0022] The facial expression detection unit 512 extracts the facial feature points detected by the face detection unit 511 and detects various facial expressions of the driver of the vehicle 1. Specifically, the facial expression detection unit 512 detects facial expressions such as repeatedly opening and closing the mouth widely, keeping the mouth half-open or wide open, repeatedly looking down intermittently, coughing violently, coughing frequently, keeping the eyes rolled back or closed, paleness of the face, facial distortion (an expression of enduring pain), and a pained expression.

[0023] The body part detection unit 521 detects the joints and bones of the driver, and detects various body parts.

[0024] The motion analysis unit 522 analyzes various motions of the driver of the vehicle 1 based on the movements of the body parts detected by the body part detection unit 521. Specifically, the motion analysis unit 522 detects motions such as touching the head due to discomfort, driving with one hand for a long time, placing a hand firmly on the chest, touching the chest with both hands in a groping manner, rubbing the stomach, loosening the necktie or shirt collar, breathing with the shoulders, pushing out the chest, repeatedly shifting in position, staggering, poor posture (leaning face down, leaning backward, falling to the side, etc.), and convulsions.

[0025] The heartbeat detection unit 53 detects the heartbeat of the driver of the vehicle 1 based on the measurement results of the heartbeat measuring device 22. Specifically, the heartbeat detection unit 53 detects a sudden increase or decrease in the heartbeat, arrhythmia, tachycardia, a decrease in blood pressure, etc. based on the measurement results of the heartbeat measuring device 22.

[0026] If the heart rate measuring device 22 has a function of measuring respiration, the respiration detection unit 54 detects the respiration of the driver of the vehicle 1 based on the measurement result of the heart rate measuring device 22. Specifically, the respiration detection unit 54 detects irregular breathing, etc. based on the measurement result of the heart rate measuring device 22.

[0027] The voice detection unit 55 detects the voice of the driver of the vehicle 1 from the voice data generated by the heartbeat detection unit 53. Specifically, the voice detection unit 55 detects the driver's shouting, groaning, slurred speech, loud breathing, and the like.

[0028] The response detection unit 56 detects the driver's response to the call to the driver. The driver's response may be a voice response or may be a response by pressing the switch 24.

[0029] First Sign situation Detection unit 57 and second sign situation The detection unit 58 detects a predictive state that indicates that the driver will become unable to drive, based on information about the driver of the vehicle 1 acquired by the input unit 2.

[0030] First Sign situation The detection unit 57 detects a first predictive state that may cause the driver to become unable to drive. situation For example, when detecting an unconscious action of the driver in response to a physical discomfort of the driver, the detection unit 57 determines that the first predictive state exists.

[0031] Second Sign situation The detection unit 58 detects a second predictive state in which the possibility that the driver will become unable to drive is higher than that of the first predictive state. situation When the detection unit 58 detects an action of the driver that deals with or alleviates the physical abnormality of the driver, it determines that the second predictive state exists.

[0032] The abnormal condition detection unit 59 detects an abnormal condition indicating that the driver is unable to drive, based on information about the driver of the vehicle 1. When the abnormal condition detection unit 59 detects that the driver is unable to drive, it determines that an abnormal condition has occurred. When the abnormal condition detection unit 59 detects an abnormal condition, it calls out to the driver, and if the driver does not respond to the call within a predetermined period of time, it performs control to avoid a vehicle accident. The call may be an audio call from the speaker 31, or a notification by vibration from the actuator 32. The driver may respond by voice and the response may be detected by the microphone 23, or the driver may respond by pressing the switch 24.

[0033] First Sign situation Detection unit 57, second sign situation The detection unit 58 and the abnormal condition detection unit 59 detect the first predictive condition, the second predictive condition, and the abnormal condition, respectively, by different sets of condition detection operations.

[0034] When a predetermined input is received from the communication unit 4, the abnormality determination unit 60 determines that an abnormality has occurred and performs control to avoid a vehicle accident. When an external operator determines that the driver is unable to drive based on information from the camera 21 and microphone 23, the external operator transmits a predetermined signal to the vehicle 1.

[0035] The vehicle control unit 61 controls the driving of the vehicle 1. The vehicle control unit 61 may be provided as an ECU separate from the control unit 5. The vehicle control unit 61 performs different controls on the vehicle 1 in the first predictive state, the second predictive state, and the abnormal state. Specifically, the vehicle control unit 61 shortens the time threshold for starting driving assist control to decelerate or stop the vehicle 1 in the first predictive state. Furthermore, the vehicle control unit 61 further shortens the time threshold for starting driving assist control to decelerate or stop the vehicle 1 in the second predictive state compared to the first predictive state. Furthermore, the vehicle control unit 61 calls out to the driver in the second predictive state and requests a response from the driver. Furthermore, the vehicle control unit 61 may perform preliminary control to safely stop the vehicle 1, such as limiting or reducing the speed of the vehicle 1 or maintaining the vehicle in its lane, in the second predictive state. Furthermore, the vehicle control unit 61 further shortens the time threshold for starting driving assist control to decelerate or stop the vehicle 1 in the abnormal state compared to the second predictive state. Furthermore, the vehicle control unit 61 shortens the time threshold for waiting for a response in an abnormal state. Furthermore, when the abnormality determination unit 60 determines that an abnormality has occurred, the vehicle control unit 61 performs control to avoid an accident involving the vehicle 1, such as slowing down or stopping the vehicle 1.

[0036] The storage unit 7 may be, for example, a relatively large-capacity storage device such as a hard disk, SSD (Solid State Drive), or optical disk, or a data-rewritable semiconductor memory such as RAM, flash memory, or NVSRAM (Non Volatile Static Random Access Memory). The storage unit 7 stores conditions and information required for various processes in the control unit 5, various programs and applications executed by the control unit 5, control data, etc. The storage unit 7 also stores personal data 71, detection result data 72, and driving data 73.

[0037] The personal data 71 is information about the individual driver of the vehicle 1, such as name, sex, age, weight, blood pressure, driving history, medical history, and the like.

[0038] The detection result data 72 is information about the detection results by the input unit 2 and the control unit 5, and includes image data generated by the camera 21, the heart rate measured by the heart rate measuring device 22, audio data generated by the microphone 23, and the first predictor. situation Detection unit 57, second sign situation This information includes the detection results of the detection unit 58 and the abnormal state detection unit 59.

[0039] The driving data 73 is information relating to the driving of the vehicle 1, such as the history of the driver's operation of the vehicle 1, the history of the control of the vehicle 1 by the vehicle control unit 61, and the like.

[0040] Next, an abnormality detection method performed by the control unit 5 will be described. Fig. 2 is a flowchart showing an example of processing in the abnormality detection method. First, as shown in Fig. 2, situation The detection unit 57 determines whether or not the first predictive state exists (step S1). situation For example, when the detection unit 57 detects a group of state detection actions such as the driver of the vehicle 1 touching his / her face, chest or shoulders, opening his / her mouth, opening and closing his / her mouth, or coughing frequently in the image data of the driver captured by the camera 21 as an unconscious behavior of the driver in response to a physical discomfort of the driver, the detection unit 57 determines that the first predictive state exists. situation The detection unit 57 may determine that the first predictive state exists when the heart rate measuring device 22 detects a sudden increase or decrease in the heart rate of the driver of the vehicle 1.

[0041] First Sign situation If the detection unit 57 determines that the first predictive state is not present (step S1: No), the second predictive state is present. situation The detection unit 58 determines whether or not the second predictive state exists (step S2). situation For example, when the detection unit 58 detects a group of state detection actions, such as the driver of the vehicle 1 placing his / her hands firmly on his / her chest, looking distressed, or showing a change (worsening) in facial color, in the image data of the driver captured by the camera 21 as a driver's behavior to deal with or alleviate the driver's physical abnormality, the detection unit 58 determines that the second predictive state exists. situation The detection unit 58 may determine that the driver is in the second predictive state when the driver's voice data acquired by the microphone 23 includes a groan. situation The detection unit 58 may determine that the second predictive state exists when the heart rate measuring device 22 detects that the driver has irregular heart rate, tachycardia, or irregular breathing.

[0042] Second Sign situation If the detection unit 58 determines that the second predictive state is not present (step S2: No), the abnormal state detection unit 59 determines whether or not an abnormal state exists (step S3). For example, the abnormal state detection unit 59 determines that an abnormal state exists when, in image data of the driver captured by the camera 21, it detects a group of state detection actions indicating that the driver is unable to drive, such as the driver of the vehicle 1 having closed eyes, white eyes, convulsing, or poor posture. The abnormal state detection unit 59 may also determine that an abnormal state exists when, based on the driving data 73, the driver of the vehicle 1 is not operating the steering wheel. The abnormal state detection unit 59 may also determine that an abnormal state exists when the heart rate measuring device 22 detects a drop in the blood pressure of the driver of the vehicle 1.

[0043] If the abnormal state detection unit 59 determines that there is no abnormal state (step S3: No), the control unit 5 determines whether or not a predetermined control signal indicating the end of operation has been detected (step S4). If the control unit 5 determines that a predetermined control signal indicating the end of operation has been detected (step S4: Yes), the series of processes ends.

[0044] On the other hand, if the control unit 5 determines that the predetermined control signal indicating the end of operation has not been detected (step S4: No), the process returns to step S1 and continues the process.

[0045] In step S1, the first sign situationIf the detection unit 57 determines that the vehicle is in the first predictive state (step S1: Yes), the control unit 5 adjusts (step S5) various thresholds that the vehicle control unit 61 uses to control the vehicle 1. For example, the control unit 5 shortens the time threshold at which the vehicle control unit 61 starts driving assist control to decelerate or stop the vehicle 1.

[0046] In step S2, the second sign situation If the detection unit 58 determines that the vehicle 1 is in the second predictive state (step S2: Yes), the control unit 5 adjusts various thresholds used by the vehicle control unit 61 to control the vehicle 1 and starts calling out to the driver of the vehicle 1 (step S6). For example, the control unit 5 further shortens the time threshold at which the vehicle control unit 61 starts driving assist control to slow down or stop the vehicle 1 compared to the first predictive state. The control unit 5 also requests the driver to respond by voice, for example, by making a buzzer sound from the speaker 31. The control unit 5 may also notify an external operator via the communication unit 4 that the second predictive state has been detected. This makes it possible to notify an external operator earlier than if the abnormal state is detected and then notified. At this stage, the control unit 5 may also perform preliminary control to safely stop the vehicle 1, such as limiting or reducing the speed of the vehicle 1 or maintaining the vehicle in its lane.

[0047] Next, the control unit 5 determines whether or not there has been a response from the driver (step S7). For example, when the control unit 5 detects a predetermined response from the driver via the microphone 23, it determines that there has been a response from the driver.

[0048] If the control unit 5 determines that there is a response from the driver (step S7: Yes), the control unit 5 proceeds to step S3 and continues the process. At this time, the control unit 5 resets various thresholds used by the vehicle control unit 61 to control the vehicle 1 to their initial values.

[0049] On the other hand, if the control unit 5 determines that there is no response from the driver (step S7: No), the control unit 5 causes the vehicle control unit 61 to execute processing to safely decelerate and stop the vehicle 1 (step S8). The control unit 5 outputs a sound from the speaker 31 to notify the outside of the vehicle of an abnormality, flashes the hazard warning flasher indicator light, decelerates the vehicle 1, and stops the vehicle 1 on a safe side of the road or the like, and then ends the processing.

[0050] If the abnormal state detection unit 59 determines in step S3 that an abnormal state exists (step S3: Yes), the control unit 5 adjusts various thresholds used by the vehicle control unit 61 to control the vehicle 1 and increases the volume of the buzzer sound that calls the driver of the vehicle 1 (step S9). For example, the control unit 5 further shortens the time threshold at which the vehicle control unit 61 starts driving assist control to slow down or stop the vehicle 1 compared to the second predictive state. Furthermore, the control unit 5 increases the volume of the call made by the buzzer sound from the speaker 31 and shortens the time threshold for waiting for a response. Furthermore, the control unit 5 may notify an external operator via the communication unit 4 that an abnormal state has been detected.

[0051] Next, the control unit 5 determines whether or not there has been a response from the driver (step S10). If the control unit 5 determines that there has been a response from the driver (step S10: Yes), the control unit 5 proceeds to step S4 and continues processing. At this time, the control unit 5 resets various thresholds used by the vehicle control unit 61 to control the vehicle 1 to their initial values.

[0052] On the other hand, if the control unit 5 determines that there is no response from the driver (step S10: No), the process proceeds to step S8 and continues the process.

[0053] According to the abnormality detection device described above, the control unit 5 distinguishes between and detects a predictive state and an abnormal state, thereby improving safety compared to when detecting only one of the predictive state and the abnormal state.

[0054] Furthermore, the abnormality detection device detects abnormalities or signs of abnormalities in the driver using image data generated by the camera 21, the heart rate measured by the heart rate measuring device 22, or audio data generated by the microphone 23, thereby improving the accuracy of detecting abnormalities compared to when detecting abnormalities in the driver using only images.

[0055] In addition, the abnormality detection device performs different controls in predictive states and abnormal states, so that in predictive states, preliminary control is performed to ensure the safety of vehicle 1, and in abnormal states, control is performed to slow down and stop vehicle 1 while ensuring its safety, thereby improving the safety of vehicle 1.

[0056] Furthermore, the abnormality detection device detects the first premonition state and the second premonition state, so that it is possible to detect an abnormality premonition from a slight change in the state such as an unconscious behavior of the driver.

[0057] Furthermore, the abnormality detection device can accurately determine the driver's condition in stages by detecting the first predictive condition, the second predictive condition, and the abnormal condition through different sets of condition detection operations.

[0058] Furthermore, the abnormality detection device can call out to the driver, and if there is no response to the call, the vehicle 1 can be safely controlled.

[0059] Furthermore, the abnormality detection device can detect signs of an abnormality at an early stage by detecting unconscious behavior of the driver in response to a physical discomfort felt by the driver.

[0060] Furthermore, the abnormality detection device can accurately detect signs of an abnormality by detecting the driver's behavior in dealing with the driver's physical abnormality.

[0061] Furthermore, the abnormality detection device can detect the driver's inability to drive, thereby enabling early detection of the inability to drive and ensuring the safety of the vehicle 1.

[0062] The abnormality detection device according to the embodiment of the present invention described above is not limited to the above embodiment, and various modifications are possible within the scope of the claims.

[0063] In the above description, an example has been described in which the vehicle 1 is provided with the control unit 5 which is an abnormality detection device, but a terminal outside the vehicle 1 may also have the function of the abnormality detection device.

[0064] The abnormality detection device according to this embodiment may be configured by appropriately combining the components of the above-described embodiments and modifications. [Explanation of symbols]

[0065] 1 vehicle 2 Input section 3 Output section 4. Communications Department 5. Control unit (abnormality detection device) 7 Memory section 21 Camera 22 Heart rate monitor 23. Mike 24 Switch 31 speakers 32 Actuator 53 Heart rate detector 54 Breath detection unit 55 Voice detection unit 56 Response detection unit 57 First Sign situation Detection unit 58 The Second Omen situation Detection unit 59 Abnormal condition detection unit 60 Abnormality determination section 61 Vehicle control unit 71 Personal Data 72 Detection result data 73 Operational Data 511 Face detection unit 512 Facial Expression Detection Unit 521 Body Part Detection Unit 522 Motion analysis section

Claims

1. Based on the information about the driver of the vehicle acquired by the input unit, a predictive state indicating that the driver will become unable to drive; an abnormal condition indicating that the driver is unable to drive; a control unit for detecting each of the The control unit determines that the driver is in the premonition state when detecting an unconscious behavior of the driver in response to a physical discomfort of the driver. Anomaly detection device.

2. the input unit includes a camera that generates image data; The information is the image data of the driver captured by the camera. The abnormality detection device according to claim 1 .

3. the input unit includes a microphone that generates voice data; The information is the voice data obtained by capturing the voice of the driver using the microphone. The abnormality detection device according to claim 1 .

4. the input unit includes a heart rate measuring device that measures a heart rate; The information is the heart rate of the driver measured by the heart rate measuring device. The abnormality detection device according to claim 1 .

5. The control unit performs different controls on the vehicle depending on whether the predictive state or the abnormal state is detected. The abnormality detection device according to claim 1 .

6. The control unit As the predictive state, a first predictive state in which the driver may be unable to drive; a second predictive state in which the possibility that the driver will become unable to drive is higher than the first predictive state; and Detect each of The abnormality detection device according to claim 1 .

7. the control unit detects the first predictive state, the second predictive state, and the abnormal state by different state detection operation groups, respectively. The abnormality detection device according to claim 6.

8. The control unit When the abnormal state is detected, a call is made to the driver, If the driver does not respond to the call within a predetermined period of time, control is performed to avoid an accident involving the vehicle. The abnormality detection device according to claim 1 .

9. The control unit determines that the abnormal state exists when detecting that the driver is unable to drive. The abnormality detection device according to claim 1 .

10. Based on the information about the driver of the vehicle acquired by the input unit, a predictive state indicating that the driver will become unable to drive; an abnormal condition indicating that the driver is unable to drive; a control unit for detecting each of the The control unit determines that the driver is in the predictive state when detecting an action of the driver that addresses or alleviates the driver's physical abnormality. Anomaly detection device.

Images