Driving aptitude confirmation device and driving aptitude confirmation system

Abstract

To provide a driving aptitude confirmation device and a driving aptitude confirmation system capable of easily detecting abnormal conditions, including driver illness and attention.SOLUTION: A driving aptitude confirmation device 100 includes: a heart rate measuring device 20 that measures a heart rate of a driver US; an electroencephalogram measuring device 30 that measures an electroencephalogram of the driver US; a stimulus-generating device 13 that applies a stimulus to the driver US when measuring the electroencephalogram as an attention test; and a determination device 11b that determines an activation status from the heart rate variation based on a determination model JM that learned about the heart rates and electroencephalograms, and determines the driving capacity from the electroencephalogram.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a driving suitability confirmation device and a driving suitability confirmation system that determine the state of a driver based on the driver's heartbeat and brain waves.

Background Art

[0002] In a safe driving support system, driving risk based on the driver's fatigue, stress, diseases, etc. is evaluated by driving ability check (specifically, evaluating the diagnosis result by the driving ability evaluation means and the determination result by the drive simulator) and health information check (specifically, evaluating the determination result of fatigue degree by the acceleration pulse wave and the determination result by the fluctuating blood pressure) (Patent Document 1). In the system of Patent Document 1, it is not possible to detect the driver's distracted state or drowsiness, and there is a possibility of an accident occurring. Further, in the system of Patent Document 1, since it uses driving ability evaluation based on cognitive reaction, predictive reaction, and discriminative reaction in the driving ability check, as well as the drive simulation result, evaluation or diagnosis takes time and effort.

[0003] In a driving attention amount determination device, it is disclosed to determine the attention amount in the peripheral visual field area of the driver from the brain wave signal measured starting from the occurrence time of the visual stimulus generated in the peripheral visual field area of the driver (Patent Document 2). Although the device of Patent Document 2 can detect the driver's temporary distracted state, it cannot detect the continuous distracted state caused by dementia, schizophrenia, etc., and there is a possibility of an accident occurring.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

[0005] The present invention has been made in view of the above points, and an object thereof is to provide a driving suitability confirmation device and a driving suitability confirmation system capable of easily detecting an abnormal state including a driver's illness and attention.

[0006] To achieve the above object, a driving suitability confirmation device according to the present invention includes a heartbeat measurement device that measures a driver's heartbeat, an electroencephalogram measurement device that measures a driver's electroencephalogram, a stimulation generation device that gives a stimulus to the driver during measurement of the electroencephalogram as an attention test, and a determination device that determines an activity state from fluctuations in the heartbeat and determines a driving ability from the electroencephalogram based on a learned determination model of the heartbeat and the electroencephalogram.

[0007] According to the above driving suitability confirmation device, the activity state and driving ability of the driver can be easily determined based on a learned determination model.

[0008] To achieve the above object, a driving suitability confirmation system according to the present invention includes a driving suitability confirmation device including a heartbeat measurement device that measures a driver's heartbeat, an electroencephalogram measurement device that measures a driver's electroencephalogram, a stimulation generation device that gives a stimulus to the driver during measurement of the electroencephalogram as an attention test, and a determination device that determines an activity state from fluctuations in the heartbeat and determines a driving ability from the electroencephalogram based on a learned determination model of the heartbeat and the electroencephalogram, and a management device that receives the determination result of the determination device.

[0009] According to the above driving suitability confirmation system, the activity state and driving ability of the driver are easily determined based on a learned determination model by the driving suitability confirmation device, and the management device can reflect the determination result in driving support.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0011] 〔First Embodiment〕 Hereinafter, with reference to the drawings, the driving suitability confirmation device of the present embodiment will be described.

[0012] FIG. 1 is a conceptual diagram for explaining a usage example of the driving suitability confirmation device 100. FIGS. 2(A) to 2(C) are diagrams for explaining an example of the electroencephalogram measurement device 30 in the driving suitability confirmation device 100. FIG. 2(A) is a view of the hat-type electroencephalogram measurement device 30 seen from the back side. FIGS. 2(B) and 2(C) are a plan view and a side view for explaining the electrode arrangement of the electroencephalogram measurement device 30. FIG. 3 is a conceptual block diagram for explaining the driving suitability confirmation device 100. The driving suitability confirmation device 100 determines the activity state and driving ability of the driver US from the heartbeat data and electroencephalogram data of the driver US based on an AI (artificial intelligence) determination model. Thereby, according to the determination result, it is possible to arouse the driver US's attention and the like and support safe driving. Here, the activity state corresponds to the health state and includes, for example, any one of the states of fatigue, attention and tension, drowsiness, stress, and relaxation. The driving ability includes, for example, the presence or absence of signs of illness and the presence or absence of attention. The signs of illness are, for example, dementia, schizophrenia, bipolar disorder, and epilepsy, etc. Although details will be described later, the determination result is represented by a normal state, an abnormal state, and a caution state according to the combination of the above activity state and driving ability.

[0013] The driving suitability confirmation device 100 is installed in a station or in a train. Also, the driving suitability confirmation device 100 may be installed at an arbitrary place other than the vehicle installation place such as at home, for example. The suitability confirmation using the driving suitability confirmation device 100 is performed before getting on the vehicle or during the ride. Also, the suitability confirmation may be performed at home or the like outside the workplace. The driving suitability confirmation device 100 can continuously confirm the driving suitability of the driver US.

[0014] The driving suitability confirmation device 100 includes a device main body 10, a heartbeat measurement device 20, and an electroencephalogram measurement device 30.

[0015] The heartbeat measurement device 20 measures the heartbeat of the driver US. Specifically, the heartbeat measurement device 20 measures the heart rate and heartbeat variation of the driver US. The heartbeat measurement device 20 analyzes the heartbeat variation regarding the heartbeat of the driver US and obtains the low-frequency component (LF), high-frequency component (HF), ratio of the low-frequency component to the high-frequency component (LF / HF), etc. of the pulse wave signal.

[0016] The measurement result of the heartbeat measurement device 20 is recorded as heartbeat data in the storage device 12 of the device main body 10 described later. The heartbeat data is used to determine the activity state (fatigue, attention and tension, drowsiness, stress, and relaxation) of the driver US.

[0017] Heart rate variability is a periodic variation in the heart rate interval (RRI: RR Interval). The heart rate interval is the interval in the time of the heartbeat, and is, for example, the interval between peaks of signals generated at regular intervals in a pulse wave signal. The heart rate interval includes two components, a low frequency component (LF; 0.04 Hz to 0.15 Hz) and a high frequency component (HF; 0.15 Hz to 0.40 Hz). LF is derived from blood pressure fluctuations and reflects the activities of both the sympathetic nervous system and the parasympathetic nervous system. HF is derived from the respiration of the living body and becomes smaller when the activity of the parasympathetic nervous system decreases. The ratio of LF to HF is an index for evaluating the degree of fatigue or stress of the living body. Thus, by evaluating the autonomic nerve function by LF and HF, etc., the activity state of the driver US can be determined.

[0018] The heartbeat measurement device 20 is, for example, a non-contact heartbeat sensor that does not contact the body of the driver US (for example, a non-contact heartbeat sensor using radio waves (specifically, millimeter waves) or ultrasonic waves) (see FIG. 1). Note that the heartbeat measurement device 20 may be a heartbeat sensor worn by the driver US (for example, a non-invasive sensor worn around the arm).

[0019] The measured heartbeat is sampled so that it can be processed by the control device 11 of the device main body 10 described later. The heartbeat data is collected at a sampling rate of, for example, 200 Hz to 500 Hz. The control device 11 calculates LF, HF, etc. by performing frequency analysis or the like on the heartbeat data.

[0020] The electroencephalogram measurement device 30 measures the electroencephalogram of the driver US. The measurement result of the electroencephalogram measurement device 30 is recorded as electroencephalogram data in the storage device 12 of the device main body 10 described later. The electroencephalogram data is used to determine the driving ability (signs of illness and attention) of the driver US.

[0021] The electroencephalogram measuring device 30 is a simple electroencephalograph 31. As shown in FIGS. 2(A) to 2(C), etc., the simple electroencephalograph 31 has a reference electrode A disposed on the earlobe ER of the driver US and an electrode Pz disposed on the midline vertex Q1 of the driver US. In addition, the simple electroencephalograph 31 may further have an electrode Cz disposed on the midpoint Q2 of the driver US in order to improve the accuracy of electroencephalogram data. As shown in FIG. 2(A), these electrodes A, Pz, and Cz are attached to, for example, a hat HT. As shown in FIGS. 2(B) and 2(C), when the driver US wears the hat HT shown in FIG. 2(A), the reference electrode A is attached to one earlobe ER, the electrode Pz is disposed on the midline vertex Q1, and the electrode Cz is further disposed on the midpoint Q2. The simple electroencephalograph 31 can communicate with the device main body 10 by means of short-range wireless communication, wired communication, or the like. Note that the electroencephalogram measuring device 30 may be any device that measures electroencephalograms, such as a head-mounted type or earphone-type simple electroencephalograph.

[0022] The measured electroencephalogram is sampled so that it can be processed by the control device 11 of the device main body 10 described later. The electroencephalogram data is collected at a sampling rate of, for example, 200 Hz to 500 Hz. Various data processes such as noise processing by a filter, baseline correction, averaging, and frequency analysis are performed on the electroencephalogram data.

[0023] Note that the simple electroencephalograph 31 illustrated in FIG. 1 can also measure the heartbeat and can be used as the heartbeat measuring device 20.

[0024] As shown in FIG. 3, the device main body 10 has a control device 11, a storage device 12, a stimulation generation device 13, and an output device 14.

[0025] The control device 11 controls the operations of a storage device 12, a stimulation generator 13, an output device 14, a heart rate measurement device 20, an electroencephalogram measurement device 30, and the like. The control device 11 includes a learning device 11a and a determination device 11b. The learning device 11a and the determination device 11b are functions of the control device 11 that operate with corresponding application software. The learning device 11a creates a learned determination model (hereinafter, determination model JM) by supervised machine learning using a machine learning model LM. The determination device 11b determines the activity state (fatigue, attention and tension, drowsiness, stress, and relaxation) from the heart rate and heart rate fluctuations of the driver US based on the determination model JM regarding the heart rate created by the learning device 11a. Further, the determination device 11b determines the driving ability (signs of illness and attention) from the electroencephalogram of the driver US based on the determination model JM regarding the electroencephalogram created by the learning device 11a.

[0026] (Determination of Activity State) The determination device 11b determines the heart rate data based on the determination model JM corresponding to the following characteristics of the heart rate data. Thereby, the determination device 11b can confirm the balance of the autonomic nerves from the heart rate and heart rate fluctuations and estimate the activity state (health state). At the time of determination, the state during wakefulness with eyes open is compared with the state during quiet with eyes closed. Regarding the characteristics of the heart rate data, in the case of fatigue, LF / HF increases. In the case of attention and tension, HF increases, but the heart rate does not decrease. In the case of drowsiness, continuous HF increases, but the heart rate decreases. In the case of stress, LF / HF increases. In the case of relaxation, LF / HF decreases. When the determination device 11b obtains a high confidence level for either attention and tension or relaxation, it determines that the state is normal. When the determination device 11b obtains a high confidence level for any one of fatigue, drowsiness, and stress, it determines that the state is abnormal. Note that when the determination device 11b obtains a somewhat high confidence level for any one of fatigue, drowsiness, and stress, it determines that the state is an attention state.

[0027] (Determination of Driving Ability) The determination device 11b determines the electroencephalogram data based on a determination model JM corresponding to the following characteristics of the electroencephalogram data. Thereby, the determination device 11b can confirm the driving ability (signs of illness and attention) from the electroencephalogram and the result of the attention test. The electroencephalogram measurement for confirming the signs of illness is about 1 minute, and then an attention test (oddball task) for confirming attention is performed.

[0028] Regarding the characteristics of the electroencephalogram data, among the signs of illness, in the case of dementia, the alpha band increases predominantly in the dementia group compared to the MCI group and the healthy group. In the case of schizophrenia, the MMN amplitude decreases. In the case of bipolar disorder, the MMN amplitude decreases. In the case of epilepsy, spike waves appear. When the determination device 11b obtains a high confidence level regarding the signs of illness, it determines that it is an abnormal state.

[0029] As described above, in the confirmation of attention, an attention test is carried out to estimate attention. The attention test is a simple test that measures the amplitude of P300, which is a waveform component of the event-related potential (electroencephalogram generated in relation to a specific event), for a task for measuring attention. As the attention test, for example, the oddball task is used. The oddball task is a task in which two types of stimulus events (for example, a high tone and a low tone) are presented with different appearance frequencies, and the subject is asked to respond only when a low-frequency stimulus (deviant stimulus) appears. The stimulus may be an auditory stimulus or a visual stimulus.

[0030] FIG. 4 is a diagram showing the waveform and amplitude of the extracted electroencephalogram data (event-related potential). After the stimulus, a positive event-related potential P300 appears around 300 milliseconds. In the oddball task, when the same sound (standard stimulus) is repeatedly heard and sometimes a different sound (deviant stimulus) is heard, a positive potential change appears in the electroencephalogram around 300 milliseconds after hearing the different sound. The determination device 11b determines that the attention is high when the value of the event-related potential P300 has a large amplitude with respect to the reference line PL, like the waveform EA. Also, the determination device 11b determines that the attention is low when the value of the event-related potential P300 has a small amplitude with respect to the reference line PL, like the waveform NA. The reference line PL corresponds to the determination threshold at the time of determination. When the determination device 11b obtains a high confidence level regarding the attention, it determines that it is an abnormal state.

[0031] Note that the attention test may be based on the measurement of CNV (negative slow potential variation, an event-related potential that occurs in relation to concentration and expectation).

[0032] In the storage device 12 shown in FIG. 3 and the like, a program for operating the driving suitability confirmation device 100 is stored. In the storage device 12, a database 12a related to the processing in the driving suitability confirmation device 100 is stored.

[0033] FIG. 5 is a diagram for explaining the outline of the database 12a. The database 12a is provided with a learning database 12b, a determination database 12c, and the like. Learning data is recorded in the learning database 12b. The learning database 12b has a normal data area 2X and an abnormal data area 2Y. The normal data area 2X has a heartbeat database 2a of a healthy person, an electroencephalogram database 2b of a healthy person, and an electroencephalogram database 2c of a person with attention. The abnormal data area 2Y has a heartbeat database 2d of a patient or an unhealthy person, an electroencephalogram database 2e of a patient or an unhealthy person, and an electroencephalogram database 2f of a person without attention.

[0034] Figs. 6 and 7 are diagrams for explaining an example of the content of the learning database 12b. Fig. 6 is a diagram for explaining an example of the content of the normal data area 2X in the learning database 12b. Fig. 7 is a diagram for explaining an example of the content of the abnormal data area 2Y in the learning database 12b. As shown in Figs. 6 and 7, each data recorded in the learning database 12b is given a tag representing the feature of the data. Each data in the normal data area 2X shown in Fig. 6 is given "0" which means normal as a tag. Also, in Fig. 6, the alphabet in the tag column indicates the state that is the basis for normality. Specifically, regarding the activity state, "A" indicates attention and tension, "B" indicates relaxation, and "Z" indicates other normal states not applicable to the above. Each data in the abnormal data area 2Y shown in Fig. 7 is given "1" which means abnormal as a tag. Also, in Fig. 7, the alphabet in the tag column indicates the state that is the basis for abnormality. Specifically, regarding the activity state, "C" indicates fatigue, "D" indicates drowsiness, and "E" indicates stress. Regarding the driving ability, "F" indicates dementia, "G" indicates schizophrenia, "H" indicates bipolar disorder, "I" indicates epilepsy, and "J" indicates lack of attention.

[0035] The determination database 12c shown in Fig. 5 records determination data. The determination database 12c has an individual heartbeat database 2g, an individual electroencephalogram database 2h, and a determination result database 2i.

[0036] FIG. 8 is a diagram for explaining an example of the content of the determination database 12c. As shown in FIG. 8, the determination database 12c is associated with a driver ID. In the determination database 12c, in the personal heartbeat database 2g and the personal electroencephalogram database 2h, the heartbeat data and electroencephalogram data measured in advance by the driver US during eyes-closed rest until the day before boarding are recorded. The prior personal heartbeat data and personal electroencephalogram data are pre-tagged with the determination results by the determination device 11b. For the prior personal heartbeat data and personal electroencephalogram data, usually, the data determined to be normal (specifically, the data with the tag "0") is recorded, but the data determined to be abnormal (specifically, the data with the tag "*-1"; "*" represents the above "C" to "J") may be recorded for reference. Corresponding to the personal heartbeat database 2g and the personal electroencephalogram database 2h, a determination threshold for determination by the determination device 11b is set for each driver US. Note that the determination threshold is corrected with personal data based on the learning data and is different for each driver US and each determination model JM. The determination result database 2i lists the date and time, determination result, determination reason, notification presence or absence, etc. together with the driver ID. The notification presence or absence indicates whether or not the driver US etc. has been notified when the determination result is in a caution state or an abnormal state.

[0037] Returning to FIG. 3, the stimulation generation device 13 gives a stimulation to the driver US during electroencephalogram measurement in the attention test. When the stimulation is a sound, the stimulation generation device 13 outputs two types of sound signals (specifically, a high sound and a low sound) to a speaker, earphone, etc. at different timings. When the stimulation is an image, the stimulation generation device 13 outputs two types of image signals to the output device 14 at different timings. The driver US can view the stimulation image via the output device 14. The stimulation generation device 13 outputs information regarding the time or generation timing when the stimulation occurs as a trigger to the control device 11, specifically, the determination device 11b. The trigger corresponds to the reference point of the appearance of the event-related potential P300 in the attention test.

[0038] The output device 14 alerts the driver US using images, sounds, etc., based on the determination result of the determination device 11b. Thereby, when an abnormal state or a caution state is detected regarding the activity state or driving ability, it is possible to notify the driver US that it is an abnormal state or a caution state and prompt the driver to return to the normal state. The output device 14 is a device that outputs images, sounds, etc. In the case of an image, the output device 14 is a display device such as a display. In the case of sound, the output device 14 is an acoustic device such as a speaker or earphone.

[0039] Hereinafter, details of the learning and determination in the control device 11 (learning device 11a and determination device 11b) will be described. As described above, the learning device 11a creates a determination model JM for determining driving suitability, and the determination device 11b determines the driving suitability from the personal heartbeat data and personal electroencephalogram data of the driver US based on the determination model JM.

[0040] (Learning) Hereinafter, an example of creating the determination model JM for the above determination, that is, an example of learning for the determination, will be described.

[0041] FIG. 9 is a conceptual diagram for explaining the learning in the control device 11. As shown in FIG. 9, as learning data, learning heartbeat data and learning electroencephalogram data (each data shown in FIGS. 6 and 7) that are pre-tagged and sorted to correspond to a predetermined state are used. A large number of learning heartbeat data and learning electroencephalogram data in a predetermined state are prepared and recorded in the database 12a of the storage device 12. The learning device 11a creates a determination model JM as an estimator 50 for each common tag, that is, for each predetermined state.

[0042] The learning device 11a calculates a feature quantity in a predetermined state from each data by using annotation information of tags added to each data. For example, in the case of heartbeat data, the feature quantity is based on, for example, the heart rate, the low-frequency component (LF), the high-frequency component (HF) of the pulse wave signal, the ratio of the low-frequency component to the high-frequency component (LF / HF), and the like. In the case of electroencephalogram data, the feature quantity is based on the waveform, amplitude, etc. of the electroencephalogram. In calculating the feature quantity, for the waveform of the electroencephalogram, the one obtained by cutting out a predetermined range of the feature part and coordinate-transforming it is used.

[0043] The learning device 11a prepares a classification-type machine learning model LM for learning with a teacher, inputs a plurality of feature quantities into the machine learning model LM in data units, and causes the machine learning model LM to output a confidence level regarding a predetermined state. At this time, the information regarding the predetermined state given to the tag of the learning data is used for correcting the machine learning model LM. By repeating the learning of inputting and processing a feature quantity group in which a plurality of feature quantities are grouped into the machine learning model LM a predetermined number of times or more for the data indicating the predetermined state, the derivation accuracy of the confidence level regarding the driving suitability can be improved. By undergoing the learning as described above, the machine learning model LM can be made to function as a determination model JM. Thereby, the learned parameters in the determination model JM can be provided to the determination device 11b. Since the machine learning model LM is prepared and individually learned for each of a large number of predetermined states, the determination model JM is also obtained for each predetermined state. Specifically, the learning device 11a creates a determination model JM for each predetermined state based on any pattern of the low-frequency component, the high-frequency component, the ratio of the low-frequency component to the high-frequency component, and the heart rate among the heartbeat fluctuations. The learning device 11a creates a determination model JM for each predetermined state based on the pattern of the electroencephalogram. Examples of the determination model JM include estimators 50 for fatigue, drowsiness, stress, attention / tension, relaxation, attention, and signs of illness. Regarding the signs of illness, estimators 50 for dementia, schizophrenia, bipolar disorder, and epilepsy may be created respectively, or one estimator 50 for the signs of illness may be created.

[0044] FIG. 10 is a flowchart for explaining the creation of the determination model JM in the driving suitability confirmation device 100.

[0045] First, the control device 11 acquires data to be learning data (step S11). Specifically, the control device 11 operates the heart rate measurement device 20 and the electroencephalogram measurement device 30 to measure and store the heart rate data and the electroencephalogram data. A large number of learning data are prepared and recorded in the database 12a of the storage device 12. Note that data may be acquired from an external institution for the learning data. During learning, the heart rate data and electroencephalogram data of an unspecified number of people are used. For improving accuracy, the personal data of the driver US may be included in the learning data.

[0046] Next, the control device 11 creates learning data (step S12). The learning data is created by annotating the data saved in step S11. Here, annotation means attaching an information tag to the data. Specifically, "data (heart rate or electroencephalogram)" and "predetermined state (corresponding; 0, not corresponding; 1)" are added to the data as annotation information (tags shown in FIGS. 6 and 7). Specifically, the control device 11 associates and assigns a value of 0 if normal and 1 if abnormal to each data. Note that the tagged learning data may be acquired from an external institution. In this case, step S12 can be omitted.

[0047] Next, the control device 11 performs supervised machine learning on the learning data in step S12 as the learning device 11a (step S13). The machine learning is performed for each predetermined state using the annotation information (tags) added to the data constituting the learning data with respect to the machine learning model LM. The learning device 11a learns one by one with one set of the data and the attached tag (numerical value). In the machine learning, a plurality of feature amounts are input to the machine learning model LM, and the confidence level regarding driving suitability is output from the machine learning model LM.

[0048] In the above learning, the data in step S11 is input one by one, and steps S11 to S14 are repeated until learning is completed (Yes in step S14) (No in step S14).

[0049] After learning is completed (Yes in step S14), the control device 11, as a learning device 11a, creates learned parameters corresponding to the machine learning in step S13 (step S15). The learning device 11a creates function data for calculating the confidence level of a predetermined state from feature amounts for data with an unknown predetermined state using the learned parameters, and by incorporating the function data into the determination device 11b, it functions as a determination model JM. Through the above learning, the determination model JM is created as the estimator 50.

[0050] (Determination) An example of determination based on the above determination model JM will be described below.

[0051] FIG. 11 is a conceptual diagram for explaining the determination in the control device 11. During operation, the determination device 11b of the control device 11 makes a determination about the real-time data PD of the driver US. Specifically, the determination device 11b determines the activity state (any state among fatigue, attention and tension, drowsiness, stress, and relaxation) about the personal heartbeat data of the driver US. Further, the determination device 11b determines the driving ability (signs of illness and attention) about the personal electroencephalogram data of the driver US.

[0052] The determination device 11b creates personal heartbeat data and personal electroencephalogram data in advance. The personal heartbeat data is the average value of the personal heartbeat data in the normal state. The personal electroencephalogram data is the average value of the personal electroencephalogram data in the normal state.

[0053] The determination device 11b inputs data related to heartbeats or data related to electroencephalograms into the determination model JM, and sets a determination threshold from the output value of the determination model JM. The data used for setting the determination threshold is based on, for example, learning data, and the prior personal data is used for correction corresponding to the individual. The determination threshold is set for each determination model JM (estimator 50).

[0054] Input the heartbeat data and electroencephalogram data collected in real time into the determination model JM. At this time, do not input numerical values of 1 or 0. The output from the determination model JM is a numerical value between 0 and 1. This can be considered as a numerical value representing whether the input heartbeat and electroencephalogram patterns resemble an abnormal state. The determination device 11b makes a determination of "abnormal" or "normal" based on the output numerical value with respect to the determination threshold. Also, when the determination device 11b provides a determination threshold corresponding to a state close to "abnormal" between "abnormal" and "normal", it can make a determination of "caution".

[0055] The determination threshold is a value for classifying the numerical values output between 0 and 1. Note that the threshold used in the learning process classifies numerical values representing multiple dimensions, which is different from the threshold used at the time of determination. When the output is 0.75, if the determination threshold for abnormality is 0.8 or more, it is determined as "normal", and if it is 0.7 or more, it is determined as "abnormal". Although the above numerical range is set as 0 to 1, the numerical range is a setting item and can be freely changed, such as 0 to 100, -1 to 1, etc. Depending on how accurate a state, for example, "normal", "abnormal", "caution" is to be represented in the final determination, the fineness of the threshold, etc. can be changed.

[0056] Hereinafter, an example of determining the determination threshold for heartbeat data will be described. In the case of confirming fatigue, the determination device 11b inputs learning data (normal data and abnormal data related to fatigue) into the fatigue estimator. The determination device 11b outputs a numerical value between 0 and 1 for each data. The determination device 11b calculates the average value from the values of the normal data and the abnormal data respectively. The determination device 11b determines the determination threshold from the above average value. At this time, the results of the individual heartbeat data obtained in advance are also taken into account.

[0057] The following is an example of determining the determination threshold of electroencephalogram data. In the case of confirming attention, the determination device 11b inputs learning data (normal data and abnormal data related to attention) to the attention estimator. The determination device 11b outputs a numerical value from 0 to 1 for each data. The determination device 11b calculates the average value from the values of the normal data and the abnormal data respectively. The determination device 11b determines the determination threshold from the above average value. At this time, the results of the individual electroencephalogram data obtained in advance are also taken into account. In the case of confirming attention, if the determination threshold for determining attention decline (the reference line PL shown in FIG. 4) is fixed, it can be determined without pre-training of the AI.

[0058] The determination device 11b determines the normal state, abnormal state, or attention state for each individual's data based on the determination threshold. That is, the determination device 11b determines whether it is normal, abnormal, or attention from the output value of the determination model JM. After determining each individual's data, the determination device 11b combines a plurality of determination models JM for comprehensive determination.

[0059] FIG. 12 is a diagram for explaining the comprehensive determination result. As shown in FIG. 12, if both the heartbeat data and the electroencephalogram data are normal, the determination device 11b determines it as normal. If there is an abnormality in either the heartbeat data or the electroencephalogram data, the determination device 11b determines it as abnormal. If there is normality in one of the heartbeat data and the electroencephalogram data and attention in the other, the determination device 11b determines it as attention. Note that the determination device 11b may notify the causes of "abnormality" and "attention" (specifically, fatigue, sleepiness, stress, attention, signs of illness) together with the determination result.

[0060] FIG. 13 is a flowchart for explaining an example of the operation of the driving suitability confirmation device 100.

[0061] The driving suitability confirmation using the driving suitability confirmation device 100 is performed before and during boarding. As a result, even when the driver US's concentration decreases during boarding, etc., the driver US can be notified accordingly at any time. In FIG. 13, steps S21 and S22 are processes that are performed in advance, for example, by the day before boarding, as preparations before operation.

[0062] First, as described above, the control device 11 acquires the driver US's prior personal data (step S21). Specifically, the control device 11 operates the heart rate measurement device 20 and the electroencephalogram measurement device 30 to measure and store the driver US's heart rate data and electroencephalogram data.

[0063] Next, the control device 11 sets the determination threshold value for determination as the determination device 11b (step S22).

[0064] On the day of operation, the control device 11 acquires the driver US's personal data (step S23). Specifically, the control device 11 operates the heart rate measurement device 20 and the electroencephalogram measurement device 30 to measure and store the driver US's heart rate data and electroencephalogram data. The personal data in step S23 is acquired in real time. Predetermined data processing is performed on the personal data for the processing by the determination device 11b.

[0065] The control device 11 makes a determination as the determination device 11b from the personal data acquired in step S23 (step S24). Specifically, the determination device 11b makes a determination for each determination model JM and finally makes a comprehensive determination based on each determination (see FIG. 12).

[0066] If the determination result in step S25 is "abnormal" (Yes in step S25) and the driver US has not been notified (No in step S26), the control device 11 operates the output device 14 to notify the driver US that they are not suitable for driving by means of sound, image, etc. (step S27). That is, in the case of step S27, the output device 14 notifies the driver US to request improvement of the state.

[0067] If the determination result in step S25 is "abnormal" (Yes in step S25) and the driver US has already been notified (Yes in step S26), the control device 11 operates the output device 14 to warn the driver US that the driving suitability is lacking by means of voice, image, etc. (step S28). That is, in the case of step S28, the control device 11, as the determination device 11b, determines that improvement of the state is not expected, operates the output device 14, and issues a warning requesting the driver US to stop driving.

[0068] If the determination result in step S25 is "normal" (No in step S25), the process returns to step S23.

[0069] In addition, if the determination result in step S25 is "caution", the control device 11 operates the output device 14 to notify or warn the driver US that the driving suitability is low. If "caution" has already been notified in step S27, the output device 14 may issue a notification again instead of a warning.

[0070] According to the above driving suitability confirmation device 100, it is possible to easily determine based on the learned determination model JM of the activity state and driving ability of the driver US.

[0071] Generally, a train driver US makes a self-declaration about their health condition and diseases before boarding the train. However, even with a self-declaration, accidents may actually occur due to the driver US's poor physical condition or illness, and a self-declaration is insufficient to ensure driving safety and peace of mind. The driving suitability confirmation device 100 of the present invention can objectively and easily detect abnormalities of the driver US, specifically, fatigue, drowsiness, stress, attention, signs of illness, etc.

[0072] 〔Second Embodiment〕 Hereinafter, with reference to FIG. 14, a second embodiment obtained by modifying the first embodiment will be described.

[0073] FIG. 14 is a conceptual block diagram for explaining the driving suitability confirmation system 200 of the present embodiment. As shown in FIG. 14, the driving suitability confirmation device 100 constitutes a driving suitability confirmation system 200 combined with the management device 90. That is, the driving suitability confirmation system 200 includes the driving suitability confirmation device 100 and the management device 90 installed at a location remote from the driving suitability confirmation device 100. For convenience of explanation, FIG. 14 shows a single driving suitability confirmation device 100, but the driving suitability confirmation system 200 of the present embodiment can perform collective operation management for a plurality of driving suitability confirmation devices 100.

[0074] In the driving suitability confirmation system 200, the management device 90 is communicably connected to the driving suitability confirmation device 100 and the like via the communication network NT. Specifically, this communication network NT is the Internet and accepts connections from the driving suitability confirmation device 100 and the like.

[0075] The management device 90 is a computer that operates under the management of the system administrator and manages the use or operation of the driving suitability confirmation device 100. The management device 90 receives the determination result of the determination device 11b of the driving suitability confirmation device 100.

[0076] As shown in the figure, the management device 90 has a control device 91, a storage device 92, and a communication device 93. The management device 90 operates based on a program stored in the storage device 92 by the control device 91. That is, the control device 91 controls the operations of the storage device 92, the communication device 93, and the like. The management device 90 communicates with the driving suitability confirmation device 100 and the like via the communication network NT shown in FIG. 14 by the communication device 93.

[0077] The storage device 92 stores a program for operating the management device 90, enabling the operation of the driving suitability confirmation system 200. The storage device 92 stores a database 92a necessary for the operation of the driving suitability confirmation system 200. In the database 92a, for example, a determination result database similar to the determination result database 2i shown in FIG. 8 is provided. Note that a learning database or the like shown in FIGS. 6 and 7 may be provided in the database 92a of the management device 90, and the determination model JM may be created on the management device 90 and the created determination model JM may be provided to the driving suitability confirmation device 100.

[0078] The driving suitability confirmation device 100 shown in FIG. 14 performs driving suitability confirmation of the driver US at the site and communication with the management device 90. The driving suitability confirmation device 100 is installed at an arbitrary location such as a station, inside a train, or at home.

[0079] When the driving suitability confirmation device 100 determines that the driver US is in an abnormal state in the determination result, it alerts the driver US. After the alert, if the driver US does not return to the normal state, it notifies the management device 90 that the driver US is in an abnormal state.

[0080] The driving suitability confirmation device 100 includes a control device 11, a storage device 12, a stimulus generation device 13, an output device 14, a communication device 15, a heart rate measurement device 20, and an electroencephalogram measurement device 30. The driving suitability confirmation device 100 operates based on a program stored in the storage device 12 by the control device 11. That is, the control device 11 controls the operations of the storage device 12, the stimulus generation device 13, the output device 14, the communication device 15, the heart rate measurement device 20, the electroencephalogram measurement device 30, and the like. The driving suitability confirmation device 100 communicates with the management device 90 via the communication network NT by the communication device 15.

[0081] FIG. 15 is a flowchart for explaining an example of the operation of the driving suitability confirmation system 200. With reference to FIG. 15, a series of operations for driving suitability confirmation performed among the driving suitability confirmation device 100, the management device 90, etc. will be described. Steps S21 to S28 are the same as those in the first embodiment, so the description will be omitted.

[0082] After warning the driver US in step S28, the control device 11 notifies the management device 90 via the communication device 15 that the driver US is in an abnormal state together with the driver ID (step S29).

[0083] Note that the determination result by the driving suitability confirmation device 100 may be periodically transmitted from the driving suitability confirmation device 100 to the management device 90. The management device 90 records the received determination result in the database 92a of the storage device 92.

[0084] In the driving suitability confirmation system 200 described above, the driving suitability confirmation device 100 can easily determine the activity state and driving ability of the driver US based on the determination model JM, and the management device 90 can reflect the determination result in driving support.

[0085] 〔Other matters〕 The present invention is not limited to the above-described embodiments, and can be implemented in various modes without departing from the gist thereof.

[0086] In the above embodiment, the driver US is the subject of driving suitability confirmation, but the subject may be other crew members such as conductors related to the business. Further, the driving suitability confirmation using the driving suitability confirmation device 100 is not limited to the crew members related to trains, and may be applied to crew members related to other vehicles such as cars and airplanes.

[0087] In the above embodiment, the determination device 11b may determine the driving ability based on the response time to the stimulus in the attention test. That is, the determination device 11b may add the response time to the stimulus as a determination material for attention. For example, in the oddball task, the subject performs an operation of pressing a button or the like for a low-frequency stimulus (deviant stimulus) among two types of stimulus events, and the determination device 11b measures the response time based on the difference between the generation timing of the deviant stimulus and the timing when the button is pressed, and determines that the attention is low when the response time is longer than a predetermined time. The determination device 11b uses the average value of the electroencephalogram data and the median value of the response time for estimating attention in creating the electroencephalogram data and determining the determination threshold value.

[0088] In the above embodiment, the machine learning may be supervised learning or unsupervised learning. In the case of unsupervised learning, for example, deep learning is used.

[0089] In the above embodiment, the management device 90 may be provided with the function of the determination device. For example, the management device 90 receives the personal data acquired by the driving suitability confirmation device 100 during operation, and the control device 91 of the management device 90 makes a determination as the determination device.

Description of Reference Numerals

[0090] 10... Device main body, 11... Control device, 11a... Learning device, 11b... Determination device, 12... Storage device, 12a... Database, 12b... Learning database, 12c... Determination database, 13... Stimulus generation device, 14... Output device, 15... Communication device, 20... Heart rate measurement device, 30... Electroencephalogram measurement device, 31... Simple electroencephalograph, 50... Estimator, 90... Management device, 91... Control device, 92... Storage device, 92a... Database, 93... Communication device, 100... Driving suitability confirmation device, 200... Driving suitability confirmation system, JM... Determination model, LM... Machine learning model, NT... Communication network, US... Driver

Claims

1. A heart rate measuring device for measuring the driver's heart rate, An electroencephalogram measuring device for measuring the electroencephalogram of the driver, A stimulus generator for giving a stimulus to the driver during the measurement of the electroencephalogram as an attention test, A determination device for determining an activity state from fluctuations in the heart rate and a driving ability from the electroencephalogram based on a learned determination model of the heart rate and the electroencephalogram, Comprising A driving suitability confirmation device.

2. Comprising an output device for alerting the driver based on the determination result of the determination device, The driving suitability confirmation device according to claim 1.

3. The determination model is created by supervised learning based on normal data and abnormal data regarding the heart rate and the electroencephalogram in advance, The driving suitability confirmation device according to claim 1.

4. The activity state includes any one of fatigue, attention and tension, drowsiness, stress, and relaxation, The determination device creates the determination model based on any one of a low-frequency component, a high-frequency component, a ratio of the low-frequency component to the high-frequency component, and a heart rate among the fluctuations of the heart rate, and determines the activity state, The driving suitability confirmation device according to claim 1.

5. The driving ability includes the presence or absence of signs of any one of dementia, schizophrenia, bipolar disorder, and epilepsy, The determination device creates the determination model based on the pattern of the electroencephalogram and determines the signs of the disease, The driving suitability confirmation device according to claim 1.

6. The driving ability includes attention, The attention test is an oddball task, In the oddball task, the determination device determines that the attention is low when the amplitude of the positive event-related potential that appears after the stimulus is smaller than the determination threshold, The driving suitability confirmation device according to claim 1.

7. The determination device determines the driving ability based on the response time to the stimulus, The driving suitability confirmation device according to claim 1.

8. The determination device inputs data regarding the heart rate or data regarding the electroencephalogram into the determination model and sets a determination threshold from the output value of the determination model, The driving suitability confirmation device according to claim 1.

9. The determination device determines a normal state or an abnormal state for the driver based on the determination threshold, The driving suitability confirmation device according to claim 8.

10. A driving suitability confirmation device comprising: a heart rate measurement device for measuring the driver's heart rate; an electroencephalogram measurement device for measuring the driver's electroencephalogram; a stimulation generator for giving a stimulus to the driver during the measurement of the electroencephalogram as an attention test; and a determination device for determining an activity state from the variation of the heart rate and determining a driving ability from the electroencephalogram based on a learned determination model of the heart rate and the electroencephalogram. A management device for receiving the determination result of the determination device. Comprising. A driving suitability confirmation system.

11. When the driving suitability confirmation device determines that the driver is in an abnormal state in the determination result, it alerts the driver, and if the driver does not return to the normal state after the alert, it notifies the management device that the driver is in an abnormal state. The driving suitability confirmation system according to claim 10.

Images