Arousal state determination method and arousal state determination device

Abstract

To provide an arousal state determination method and an arousal state determination device capable of determining an arousal state of a subject more accurately in a short period of time.SOLUTION: An arousal state determination method includes: a step (ST1) of measuring a brain activity signal of a subject; a step (ST3) of extracting the brain activity signal during a certain period of time set in advance when an attention target that attracts the subject's attention appears; a step (ST4) of calculating features of the extracted brain activity signal; and a step (ST5 to ST7) of determining the arousal state of the subject from the calculated features of the brain activity signal.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD OF THE INVENTION The present invention relates to an arousal state determination method and an arousal state determination device. [Background technology]

[0002] Patent Document 1 discloses a drowsiness warning system that measures the driver's brain activity state, judges the driver's drowsiness based on the measured data, and issues an alert to the driver if drowsiness is judged. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-018779 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the data used to determine the driver's drowsiness in the drowsiness warning system of Patent Document 1 is obtained as a result of measuring the brain activity state over a long period of time, such as 60 minutes.

[0005] While brain activity is measured over such a long period of time, data indicating drowsiness is obtained. However, it is possible that the data indicating drowsiness may be affected by human physiological activity. If the data is affected by such physiological activity, the accuracy of the data indicating drowsiness cannot be guaranteed. Furthermore, the need for long-term measurement makes it difficult to analyze driver drowsiness in real time.

[0006] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide an awake state discrimination method and an awake state discrimination device that can more accurately determine the awake state of a subject in a short period of time. [Means for solving the problem]

[0007] The method for determining the state of arousal in an embodiment includes the steps of measuring the brain activity signal of a subject, extracting the brain activity signal for a predetermined period of time when an object of attention that attracts the subject's attention appears, calculating the characteristics of the extracted brain activity signal, and determining the state of arousal of the subject from the characteristics of the calculated brain activity signal. [Effects of the Invention]

[0008] By adopting such a configuration, the present invention can provide an arousal state discrimination method and an arousal state discrimination device that can more accurately determine the arousal state of a subject in a short period of time. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing the overall configuration of an awakening state determination device according to an embodiment of the present invention; [Figure 2] 1 is a flowchart showing the overall flow of an arousal state determination method according to an embodiment of the present invention. [Figure 3] 10 is a flowchart showing a flow of calculating features of a brain activity signal used to determine whether or not an effort to wake up is being made in an embodiment of the present invention. [Figure 4] 10 is a flowchart showing a flow of determining whether or not an effort to wake up is made in an embodiment of the present invention. [Figure 5] 10 is a flowchart showing a process for classifying the level of an effort to wake up when the effort to wake up is recognized in an embodiment of the present invention. [Figure 6] 10 is a flowchart showing a flow of calculating features of a brain activity signal used when determining objective vigilance in an embodiment of the present invention. [Figure 7] 10 is a flowchart showing a flow of categorizing the level of objective alertness when an effort to alert is recognized in an embodiment of the present invention. [Figure 8]10 is a flowchart showing the flow of a process for classifying the level of objective alertness when no effort to alert is recognized in an embodiment of the present invention. [Figure 9] 10 is a flowchart showing a flow of calculating features of a brain activity signal used when determining subjective alertness in an embodiment of the present invention. [Figure 10] 10 is a flowchart showing a flow of classifying the level of subjective alertness when an effort to alert is recognized in an embodiment of the present invention. [Figure 11] 10 is a flowchart showing a process flow for classifying the level of subjective alertness when no effort to alert is recognized in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the drawings are schematic and may differ from the actual product. Furthermore, the embodiments of the present invention shown below are merely examples of devices and methods for embodying the technical concept of the present invention, and the technical concept of the present invention does not limit the structure, arrangement, etc. of the components to those described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims.

[0011] 1 is a block diagram showing the overall configuration of an arousal state discrimination device A according to an embodiment of the present invention. The arousal state discrimination device A is a device that discriminates the arousal state of a subject based on data obtained from the subject. The subject here is assumed to be, for example, a person who drives or operates a moving object such as a car, train, or airplane.

[0012] The arousal state discrimination device A includes an external situation detection device 1, a brain activity detection device 2, and an arousal level discrimination device 3. The external situation detection device 1 is a device for detecting the environment (external situation) surrounding the subject. The external situation detection device 1 in the embodiment of the present invention includes an external monitoring device 11 and an attention object discrimination device 12.

[0013] The external monitoring device 11 is a device that monitors the outside world of the subject. Here, the outside world refers to, for example, the environment around the subject and the environment in the direction the subject is facing. In addition, the environmental information acquired by the external monitoring device 11 may be information that the subject can perceive with their five senses, such as sight and hearing. From the perspective of vision, the external monitoring device 11 may be, for example, an imaging device such as various types of cameras. It may also be the subject's eyes.

[0014] The attention object discrimination device 12 is a device that discriminates whether or not an event (attention object) that attracts the subject's attention has occurred in the situation of the external world of the subject that is being monitored by the external monitoring device 11. As described above, the external monitoring device 11 monitors the environment around the subject, and therefore, when an object, movement, etc. that attracts the subject's attention (attention object) appears in the monitored environment, it is recognized that the attention object has appeared.

[0015] The brain activity detection device 2 is a device that detects the brain activity of a subject. The brain activity of the subject detected by the brain activity detection device 2 is understood from the brain activity signal of the subject. Here, the brain activity signal is understood from, for example, cerebral blood flow or cranial nerve signals. The brain activity detection device 2 includes a brain activity signal measurement device 21 and a brain activity signal extraction device 22.

[0016] The brain activity signal measuring device 21 is attached to the head of the subject and measures the brain activity signal of the subject. The brain activity signal is constantly measured while the brain activity signal measuring device 21 is attached to the subject.

[0017] Brain activity signals are used to determine the subject's state of alertness (hereinafter, this "state of alertness" will also be referred to as "level of alertness" where appropriate). The part of the brain that provides information on the body's level of alertness (state of alertness) is called the insular cortex. For example, if the body's level of alertness decreases, this is transmitted to the insular cortex, which is subjectively perceived as sleepiness. However, the insular cortex is located deep in the brain and cannot be measured directly.

[0018] On the other hand, the middle frontal gyrus is strongly connected to the insular cortex (and is affected by changes in the insular cortex). The middle frontal gyrus responds to external stimuli. If the state of arousal changes, it will affect the insular cortex, which in turn will affect the middle frontal gyrus, which is connected to it. Therefore, instead of directly examining the insular cortex, which cannot be directly examined, we can estimate the influence of the state of arousal on the insular cortex by examining the middle frontal gyrus, which is strongly connected to it. To do this, we measure brain activity signals in the middle frontal gyrus to understand the state of arousal.

[0019] In addition, when the attention object discrimination device 12 detects the appearance of an attention object while the brain activity signal measuring device 21 is measuring the brain activity signal of the subject, the brain activity signal extraction device 22 extracts a brain activity signal for a predetermined period of time.

[0020] The predetermined period of time here refers to, for example, the number of seconds from the time when the attention target appears, i.e., between 5 and 8 seconds from the time when the attention target appears, for example, when the time is set to zero.

[0021] When determining the level of arousal of a subject, it is possible to grasp some correlation between the change in the amount of oxyhemoglobin in the subject and the level of arousal. Since this correlation can be grasped even within the aforementioned period of, for example, 5 to 8 seconds, this is set as a predetermined fixed time in the embodiment of the present invention.

[0022] The start and end times of this fixed time can be set arbitrarily. In addition, the start time of the fixed time can also be set arbitrarily, with the time interval after the appearance of the attention object as the reference point.

[0023] On the other hand, if the fixed time is set too long, it will take a long time to determine the subject's wakefulness, which will cause disturbances and reduce the accuracy rate. Therefore, by setting the time as described above, it is possible to reduce disturbances and increase the accuracy rate in determining the subject's wakefulness. It also enables real-time determination.

[0024] In this way, the brain activity signal extraction device 22 extracts brain activity signals for a certain period of time using the appearance of an attention target as a trigger. That is, while the brain activity signal measurement device 21 is measuring the subject's brain activity signals, the brain activity signal extraction device 22 constantly receives the measurement results. In this state, when the attention target discrimination device 12 determines that an attention target has appeared, the attention target discrimination device 12 notifies the brain activity signal extraction device 22 that an attention target has appeared.

[0025] When the brain activity signal extraction device 22 receives a signal indicating that an object of attention has appeared from the brain activity signal measurement device 21, it extracts a brain activity signal for a predetermined period of time, setting the time when the object of attention appeared to be zero.

[0026] Here, the brain activity signal extraction device 22 directly receives a signal indicating that an object of attention has appeared from the attention object discrimination device 12. However, for example, while the brain activity signal measurement device 21 is measuring the brain activity signal of the subject, a process of receiving a signal indicating that an object of attention has appeared from the attention object discrimination device 12 may be performed. In this case, the brain activity signal measurement device 21 transmits a signal indicating that an object of attention has appeared to the brain activity signal extraction device 22.

[0027] The wakefulness discrimination device 3 is a device that discriminates the wakefulness of a subject using the measured brain activity signal of the subject. The "wakefulness (wakeful state)" discriminated here can be classified into three types: "wakefulness effort," "objective wakefulness," and "subjective wakefulness."

[0028] "Wakefulness efforts" refer to the efforts made by a subject to wake themselves up when they feel sleepy. For example, if the subject shakes their head, consciously yawns and moves their mouth widely, or consciously takes a deep breath, the subject is determined to be making an effort to wake up. On the other hand, if the subject makes few body movements or mouth movements, or shows little change in facial expression, the subject is determined to be not making an effort to wake up.

[0029] "Objective alertness" refers to the level of alertness of a subject as assessed by a third party observing the subject. On the other hand, "subjective alertness" refers to the level of alertness assessed by the subject themselves. Alertness levels are divided into several stages, ranging from not appearing sleepy at all to being asleep. "Objective alertness" and "subjective alertness" are determined using several indicators, such as the subject's body movements and facial expressions, that are determined for each level.

[0030] The arousal level discrimination device 3 is composed of a calculation unit 31, a storage unit 32, a comparison unit 33, and a discrimination unit 34. The calculation unit 31 receives the brain activity signal extracted by the brain activity signal extraction device 22, and calculates the brain activity signal to be used when discriminating the arousal level of the subject.

[0031] The brain can be divided into regions known as Brodmann areas, which are groups of areas believed to have the same tissue structure. In these areas, the brain activity signals measured when a subject feels drowsy vary. For example, the amount of oxyhemoglobin, which is used to determine a subject's drowsiness, may increase or decrease depending on the area.

[0032] Therefore, for all regions or for each region, and for each of the three levels of alertness, the tendency of increase and decrease in the amount of oxyhemoglobin when the subject is in a drowsy state or when not drowsy (awake) is grasped in advance. By doing so, if it is possible to grasp which region the calculated brain activity signal is from, the level of alertness of the subject can also be grasped.

[0033] In other words, it is possible to select which brain activity signal in which region is to be used as the brain activity signal measured when determining the subject's level of alertness. That is, for example, it is possible to use brain activity signals from all regions and grasp the subject's level of alertness based on the characteristics of the brain activity signals.

[0034] Furthermore, by utilizing the area where the brain activity signal measured shows a significant difference between when the subject's level of arousal is high and when it is low, it is possible to perform even more accurate determination. In this case, the calculation unit 31 calculates the area where a significant difference is observed between the brain activity signals of the subject when the subject is aroused and when not (hereinafter, such an area will be referred to as a "specific area"), particularly during the above-mentioned certain time period, and calculates the characteristics of the brain activity signal in the specific area.

[0035] There are various possible methods for the calculation unit 31 to calculate the specific region, and for example, the following two methods can be adopted: The calculation unit 31 first analyzes the brain activity signals extracted by the brain activity signal extraction device 22. This determines whether to use features obtained from all the extracted brain activity signals to determine the subject's level of arousal, or to use features obtained from the brain activity signals at the specific region.

[0036] If all extracted brain activity signals are used to determine the level of alertness of the subject, the calculation unit 31 calculates, for example, the average value of all these brain activity signals as the feature of the brain activity signal.

[0037] When cerebral blood flow signals are used as brain activity signals, the average value is calculated as described above. Alternatively, cranial nerve signals, such as electroencephalograms or magnetoencephalograms, can also be used as brain activity signals. In this case, the brain activity signal extraction device 22 first extracts cranial nerve signals, which are electroencephalogram signals over a certain period of time. The calculation unit 31 then performs a process of convolving the extracted cranial nerve signals with a standard hemodynamic response function (e.g., a multiple gamma function) to calculate the cerebral blood flow signals. The average value of the calculated cerebral blood flow signals is then calculated.

[0038] On the other hand, when brain activity signals from specific regions are used, the calculation unit 31 checks, based on the brain activity signals, whether or not a significant difference that allows discrimination of the presence or absence of each level of wakefulness of the subject ("wakefulness effort," "objective wakefulness," and "subjective wakefulness") is found. Then, the region in which a significant difference that allows discrimination of the presence or absence of wakefulness is found is calculated as the specific region to be used for discriminating the wakefulness.

[0039] Then, using the brain activity signal of the calculated specific area, the calculation unit 31 calculates its characteristics. Note that, since this specific area may differ for each of "awakening effort," "objective arousal level," and "subjective arousal level," it is calculated each time each arousal level is calculated.

[0040] Furthermore, as a method by which the calculation unit 31 analyzes the brain activity signals extracted by the brain activity signal extraction device 22 and calculates the specific region, it is possible to use, for example, an averaging method or a GLM (General Linear Model) method.

[0041] To take the arithmetic averaging method as an example, assuming that the time when the attention target appears is 0 seconds, brain activity signals are extracted for a total of 15 seconds, from the 5 seconds before that to the 10 seconds after, and the average value for the 5 seconds before the attention target appears is set to zero. Meanwhile, arithmetic averaging is performed on brain activity signals for the 10 seconds after the attention target appears, and differences between experimental conditions in "alertness effort," "objective alertness," and "subjective alertness" are confirmed. Then, for each level of alertness, areas that reflect the differences in the experimental conditions are extracted, and the specific areas for each are calculated.

[0042] Furthermore, when the GLM method is used, the results of analyzing all extracted brain activity signals using the GLM method are used to calculate whether the standard partial regression coefficient values for the regressors of "awakening effort," "objective arousal," and "subjective arousal" for each brain region are significantly greater than 0. If a value significantly greater than 0 is calculated, the region in question is calculated as the respective specific region.

[0043] As will be described later, the storage unit 32 stores, for example, an effort threshold determined in advance by experiment or the like, which is used to classify the level of the awakening effort when, for example, an awakening effort is recognized. The storage unit 32 also stores various thresholds used to classify the levels of objective awakening and subjective awakening.

[0044] The comparison unit 33 compares the value of the brain activity signal used to determine the subject's level of alertness with the various thresholds described above. The reason for using various thresholds here is that different thresholds are used for the subject's levels of alertness, namely, "alertness effort," "objective alertness," and "subjective alertness." For example, the threshold used to determine whether or not an effort to alert is being made is the aforementioned effort threshold.

[0045] The effort threshold is determined, for example, by associating the measured brain activity signal with a change in the external situation acquired by the external situation detection device 1. That is, a brain activity signal when the attention target does not appear and a brain activity signal for a predetermined period of time when the attention target appears are acquired.

[0046] The calculation unit 31 then averages the brain activity signals when a wake-up effort is made and when no wake-up effort is made, and sets the average of the average values when a wake-up effort is made and the average values when no wake-up effort is made as the effort threshold. The comparison unit 33 then uses the set effort threshold to perform a comparison process required to determine whether or not a wake-up effort is made, as will be described later.

[0047] Furthermore, there are various stages of the subject's level of alertness. For example, when taking alertness effort as an example, even when alertness effort is observed, there can be cases where the level of alertness effort is high and cases where it is low. Furthermore, it is possible to categorize the level of alertness effort into three or more levels, rather than just categorizing it into two levels, high and low.

[0048] When classifying the levels of alertness effort in this way, as described above, the level of alertness of the subject is classified by comparison using thresholds by the comparison unit 33 and, in some cases, the discrimination unit 34. Similarly, various thresholds used when classifying the levels of objective alertness and subjective alertness are set, for example, by the calculation unit 31 using brain activity signals.

[0049] The effort threshold and the various thresholds used for the objective and subjective arousal levels may be set based on experimental results. Alternatively, for example, the objective and subjective arousal levels in each specific body part may be divided into a plurality of levels in advance based on the tendency of the change over time in the amount of oxyhemoglobin, and the respective thresholds may be set.

[0050] In this case, the comparison unit 33 compares the value of the brain activity signal calculated by the calculation unit 31 to determine which position it falls within, and the level of the alertness effort is finally determined by the determination unit 34. These thresholds are stored in the above-mentioned storage unit 32 and are used by the comparison unit 33 and determination unit 34 when determining the level of alertness of the subject.

[0051] Regarding the level of alertness, alertness effort is classified into levels only when alertness effort is recognized. This is because there is no need to classify the level of alertness effort when alertness effort is not recognized.

[0052] On the other hand, in the case of objective and subjective arousal, the levels can be distinguished regardless of whether or not the subject makes an effort to wake up. In other words, it is possible to distinguish the levels of objective and subjective arousal not only when an effort to wake up is recognized, but also when no effort to wake up is recognized, simply because the subject is not making an effort to wake up.

[0053] When the comparison unit 33 compares the brain activity signal with the threshold, the discrimination unit 34 discriminates the level of alertness of the subject or the level set for each level of alertness based on the comparison result. Although not particularly shown in the embodiment of the present invention, for example, the level of alertness of the subject discriminated by the discrimination unit 34 may be notified to the subject, etc., as needed.

[0054] [Operation] Next, the flow of the process for determining the wakefulness level of the subject will be described. First, the general flow of the wakefulness level determination process will be described. Then, the flow of the determination process for each of the wakefulness effort, objective wakefulness level, and subjective wakefulness level will be described.

[0055] 2 is a flowchart showing the overall flow of the wakefulness state discrimination method according to the embodiment of the present invention. First, the brain activity signal of the subject is measured (ST1). As described above, this is measured by the brain activity signal measuring device 21 of the brain activity detection device 2 worn on the head of the subject.

[0056] Furthermore, while the brain activity signal measuring device 21 is measuring the brain activity signal, the environment around the subject is monitored by the external monitoring device 11 of the external situation detection device 1. Furthermore, the attention object discrimination device 12 determines whether or not an attention object has appeared (ST2).

[0057] That is, the brain activity signal measuring device 21 constantly measures the brain activity signal while it is worn by the subject, but the attention object discrimination device 12 also constantly determines whether or not an attention object has appeared. Therefore, when an attention object has not appeared (NO in ST2), the attention object discrimination device 12 continues to determine whether or not an attention object has appeared.

[0058] On the other hand, when the attention object discrimination device 12 determines that an attention object has appeared (YES in ST2), it notifies the brain activity signal extraction device 22 that an attention object has appeared.

[0059] The brain activity signal extraction device 22 extracts a brain activity signal upon receiving the notification from the attention object discrimination device 12. Specifically, it extracts a brain activity signal for a certain period such as that described above from the brain activity signals measured by the brain activity signal measurement device 21, using the appearance of the attention object as a criterion (ST3).

[0060] The brain activity signals extracted by the brain activity signal extraction device 22 are transmitted to the alertness determination device 3. The transmitted brain activity signals are received by the calculation unit 31, and the characteristics of the extracted brain activity signals are calculated (ST4). Thereafter, as will be described later, the presence or absence of an effort to wake up is determined (ST5), and the process proceeds to determination of the objective alertness (ST6) and the subjective alertness (ST7), thereby performing the process of determining the alertness of the subject.

[0061] In this description, it is assumed that the determination process of the wakefulness level is performed in the order of wakefulness effort, objective wakefulness, and subjective wakefulness. However, the order of determining the objective wakefulness and subjective wakefulness is not limited to this, and the subjective wakefulness may be determined before the objective wakefulness, or they may be determined in parallel.

[0062] Next, the flow of the process of determining whether or not a wake-up effort is being made will be described. Fig. 3 is a flowchart showing the flow of calculating the characteristics of the brain activity signal used to determine whether or not a wake-up effort is being made in an embodiment of the present invention. Fig. 4 is a flowchart showing the flow of determining whether or not a wake-up effort is being made in an embodiment of the present invention.

[0063] The calculation unit 31 receives the brain activity signals extracted by the brain activity signal extraction device 22 and analyzes the extracted brain activity signals (ST11). This is a process performed to determine which of the transmitted brain activity signals should be used to determine the subject's level of alertness, as described above.

[0064] If all the brain activity signals extracted by the brain activity signal extraction device 22 are used to determine whether or not an effort to wake up is being made (YES in ST12), the calculation unit 31 calculates the features of the extracted brain activity signals (ST13). As for the method of calculating the features, as described above, for example, the average value of all the brain activity signals is calculated as the feature.

[0065] On the other hand, if brain activity signals at a specific site are used instead of all extracted brain activity signals (NO in ST12), the calculation unit 31 calculates the specific site using a calculation method such as the arithmetic averaging method or GLM method described above (ST14).

[0066] Specifically, all brain activity signals are checked to see if a significant difference is found that can distinguish whether or not an effort to wake up is being made (ST15). Then, a region where a significant difference is found that can distinguish whether or not an effort to wake up is being made is calculated as a specific region to be used when determining whether or not an effort to wake up is being made (ST16). Then, the calculation unit 31 calculates a value indicating the characteristic of the calculated specific region using the brain activity signals from the specific region (ST17).

[0067] The calculated value indicating the characteristic of the brain activity signal is transmitted from the calculation unit 31 to the comparison unit 33. The comparison unit 33 compares the received value indicating the characteristic of the brain activity signal with a threshold value (ST21 in FIG. 4). At this time, the comparison unit 33 acquires a threshold value (effort threshold value) from the storage unit 32.

[0068] The effort threshold obtained here is, for example, a threshold set using all brain activity signals when a characteristic value is calculated using all brain activity signals, whereas, when a characteristic value is calculated using brain activity signals from a specific region, the effort threshold set for that specific region is used.

[0069] The comparison unit 33 compares the value indicating the characteristics of the brain activity signal with the effort threshold (ST22), and transmits the comparison result to the discrimination unit 34. If the value indicating the characteristics of the brain activity signal is greater than the effort threshold (YES in ST22), the discrimination unit 34 determines that the subject is making an effort to stay awake (wakefulness effort present) (ST23).

[0070] On the other hand, if the value indicating the characteristic of the brain activity signal is equal to or less than the effort threshold (NO in ST22), the discrimination unit 34 determines that the subject is not making an effort to stay awake (no effort to stay awake) (ST24). This completes the process of discriminating whether or not an effort to stay awake is being made.

[0071] As described above, when an effort to stay awake is recognized, the level of the effort to stay awake made by the subject may be further classified. Fig. 5 is a flowchart showing the flow of a process for classifying the level of the effort to stay awake when an effort to stay awake is recognized in an embodiment of the present invention.

[0072] The level of the wake-up effort is classified in this way only when the discrimination unit 34 determines that a wake-up effort is being made, so in Figure 5, the discrimination process of step ST23, which is the premise for this, is shown by a dashed line.

[0073] The process of classifying the level of the awakening effort is subsequently performed by the discrimination unit 34. Alternatively, after the discrimination unit 34 has once determined whether or not an awakening effort is being made, in order to classify the level of the awakening effort, the information may be transmitted again to the comparison unit 33, which may compare the information with a threshold value, and then the comparison result may be transmitted again to the discrimination unit 34. In the following description, an example will be given in which the discrimination unit 34 classifies the level of the awakening effort.

[0074] The determination unit 34 first accesses the memory unit 32 to obtain thresholds used to determine the level of the awakening effort. For convenience, these are referred to as a "first level threshold" and a "second level threshold."

[0075] The first level threshold is a threshold used to determine whether the level of the arousal effort made by the subject is low, and the second level threshold is used to determine whether the level of the arousal effort made by the subject is high.

[0076] The discrimination unit 34 compares the value indicating the characteristic of the brain activity signal with a first level threshold (ST31). As a result, if the value indicating the characteristic of the brain activity signal is greater than the first level threshold (YES in ST31), the discrimination unit 34 determines that the level of the wakefulness effort is low (ST32).

[0077] On the other hand, if it is determined that the value indicating the characteristic of the brain activity signal is equal to or less than the first level threshold (NO in ST31), the value indicating the characteristic of the brain activity signal is further compared with the second level threshold (ST33).

[0078] As a result, if the value indicating the characteristic of the brain activity signal is greater than the second level threshold (YES in ST33), the discrimination unit 34 determines that the level of the wakefulness effort is medium (ST34).On the other hand, if the value indicating the characteristic of the brain activity signal is determined to be equal to or less than the second level threshold (NO in ST33), the discrimination unit 34 determines that the level of the wakefulness effort is high (ST35).

[0079] Next, the flow of the process for determining the objective alertness will be described below. Fig. 6 is a flowchart showing the flow of calculating the features of the brain activity signal used when determining the objective alertness in the embodiment of the present invention.

[0080] The calculation unit 31 receives the brain activity signals extracted by the brain activity signal extraction device 22 and analyzes the extracted brain activity signals (ST41). This is a process performed to determine which of the transmitted brain activity signals should be used to determine the subject's level of alertness, as described above.

[0081] If all of the brain activity signals extracted by the brain activity signal extraction device 22 are used to determine the presence or absence of objective arousal (YES in ST42), the calculation unit 31 calculates the features of the extracted brain activity signals (ST43). As for the method of calculating the features, as described above, for example, the average value of all of the brain activity signals is calculated as the feature.

[0082] On the other hand, if brain activity signals at a specific region are used instead of all extracted brain activity signals (NO in ST42), the calculation unit 31 calculates the specific region using a calculation method such as the arithmetic averaging method or GLM method described above (ST44).

[0083] Specifically, all brain activity signals are checked to see if a significant difference is found that allows the objective arousal level to be determined (ST45). Then, a region where a significant difference is found to the extent that the objective arousal level can be determined is calculated as a specific region to be used when determining the objective arousal level (ST46). Then, the calculation unit 31 calculates a value indicating the characteristic of the calculated specific region using the brain activity signals from the specific region (ST47).

[0084] In this state, the presence or absence of an effort to wake up has already been determined. Regarding objective arousal, the presence or absence of objective arousal is not determined, but the level of objective arousal is classified. Therefore, the level of objective arousal is classified on the premise that an effort to wake up is recognized. Similarly, the level of objective arousal is classified on the premise that an effort to wake up is not recognized.

[0085] Furthermore, as described above, the brain activity signals used to classify the levels of objective alertness may be calculated using values that indicate the characteristics of all brain activity signals, or may be calculated using brain activity signals from specific areas.

[0086] Fig. 7 is a flowchart showing the process of classifying the level of objective alertness when an effort to wake up is recognized in an embodiment of the present invention, and Fig. 8 is a flowchart showing the process of classifying the level of objective alertness when an effort to wake up is not recognized in an embodiment of the present invention.

[0087] As described above, the level of objective alertness may be classified into two cases: "with alertness effort" and "without alertness effort." Therefore, the flow of the process of classifying the level of objective alertness when the discrimination unit 34 determines that an alertness effort is made will be described with reference to Fig. 7. Therefore, the discrimination process shown in step ST23 in the discrimination unit 34, which is the premise for this process, is indicated by a dashed line.

[0088] As mentioned above, the process of classifying the levels of objective arousal may be performed either by the discrimination unit 34 or by the comparison unit 33. In the following explanation, as in the previous examples, the case where the levels of objective arousal are classified by the discrimination unit 34 will be described as an example.

[0089] The discrimination unit 34 first accesses the storage unit 32 to obtain thresholds used to discriminate the level of objective alertness. Here, for convenience, these are referred to as a "third level threshold" and a "fourth level threshold."

[0090] The third level threshold is a threshold used to determine whether the subject's level of objective arousal is high or not, and the fourth level threshold is used to determine whether the subject's level of objective arousal is low or not.

[0091] The discrimination unit 34 compares the value indicating the characteristic of the brain activity signal with a third level threshold (ST51). As a result, if the value indicating the characteristic of the brain activity signal is greater than the third level threshold (YES in ST51), the discrimination unit 34 determines that the level of the objective alertness is high (ST52).

[0092] On the other hand, if it is determined that the value indicating the characteristic of the brain activity signal is equal to or less than the third level threshold (NO in ST51), the value indicating the characteristic of the brain activity signal is further compared with a fourth level threshold (ST53).

[0093] As a result, if the value indicating the characteristic of the brain activity signal is greater than the fourth level threshold (YES in ST53), the discrimination unit 34 determines that the level of the objective alertness is medium (ST54).On the other hand, if the value indicating the characteristic of the brain activity signal is determined to be equal to or less than the fourth level threshold (NO in ST53), the discrimination unit 34 determines that the level of the objective alertness is low (ST55).

[0094] Next, the flow of the process of classifying the level of the objective alertness when the discrimination unit 34 determines that no effort to wake up is being made will be described with reference to Fig. 8. Therefore, the discrimination process shown in step ST24 in the discrimination unit 34, which is the premise for this, is indicated by a dashed line.

[0095] The discrimination unit 34 first accesses the storage unit 32 to obtain thresholds used to discriminate the level of objective alertness. Here, for convenience, these are referred to as a "fifth level threshold" and a "sixth level threshold."

[0096] The fifth level threshold is a threshold used to determine whether the subject's level of objective arousal is high or not, and the sixth level threshold is used to determine whether the subject's level of objective arousal is low or not.

[0097] The discrimination unit 34 compares the value indicating the characteristic of the brain activity signal with the fifth level threshold (ST61). As a result, if the value indicating the characteristic of the brain activity signal is greater than the fifth level threshold (YES in ST61), the discrimination unit 34 determines that the level of the objective alertness is high (ST62).

[0098] On the other hand, if it is determined that the value indicating the characteristic of the brain activity signal is equal to or less than the fifth level threshold (NO in ST61), the value indicating the characteristic of the brain activity signal is further compared with the sixth level threshold (ST63).

[0099] As a result, if the value indicating the characteristic of the brain activity signal is greater than the sixth level threshold (YES in ST63), the discrimination unit 34 determines that the level of the objective alertness is medium (ST64).On the other hand, if the value indicating the characteristic of the brain activity signal is determined to be equal to or less than the sixth level threshold (NO in ST63), the discrimination unit 34 determines that the level of the objective alertness is low (ST65).

[0100] Next, the flow of the process for determining the subjective alertness will be described below. Fig. 9 is a flowchart showing the flow of calculating the features of the brain activity signal used when determining the subjective alertness in the embodiment of the present invention.

[0101] The calculation unit 31 receives the brain activity signals extracted by the brain activity signal extraction device 22 and analyzes the extracted brain activity signals (ST71). This is a process performed to determine which of the transmitted brain activity signals should be used to determine the subject's level of alertness, as described above.

[0102] If all of the brain activity signals extracted by the brain activity signal extraction device 22 are used to determine whether or not the subjective alertness is low (YES in ST72), the calculation unit 31 calculates the features of the extracted brain activity signals (ST73). As for the method of calculating the features, as described above, for example, the average value of all of the brain activity signals is calculated as the feature.

[0103] On the other hand, if brain activity signals at a specific region are used instead of all extracted brain activity signals (NO in ST72), the calculation unit 31 calculates the specific region using a calculation method such as the arithmetic averaging method or GLM method described above (ST74).

[0104] Specifically, all brain activity signals are checked to see if a significant difference that can distinguish the subjective arousal level is found (ST75). Then, the region where a significant difference that can distinguish the subjective arousal level is found is calculated as a specific region to be used when distinguishing the subjective arousal level (ST76). Then, the calculation unit 31 uses the brain activity signals at the calculated specific region to calculate a value indicating its characteristics (ST77).

[0105] In this state, the presence or absence of an effort to wake up has already been determined. Regarding subjective arousal, the level of subjective arousal is not determined, but rather categorized. Therefore, the level of subjective arousal is categorized on the premise that an effort to wake up is recognized. Similarly, the level of subjective arousal is categorized on the premise that an effort to wake up is not recognized.

[0106] Furthermore, as described above, the brain activity signals used to classify the levels of subjective alertness may be calculated using values that indicate the characteristics of all brain activity signals, or may be calculated using brain activity signals from specific areas.

[0107] Fig. 10 is a flowchart showing the flow of categorizing the level of subjective alertness when an effort to wake up is recognized in an embodiment of the present invention, and Fig. 11 is a flowchart showing the flow of processing of categorizing the level of subjective alertness when an effort to wake up is not recognized in an embodiment of the present invention.

[0108] As described above, the level of subjective alertness may be classified either as "with an effort to wake up" or "without an effort to wake up." First, the flow of the process of classifying the level of subjective alertness when the discrimination unit 34 determines that an effort to wake up is made will be described with reference to Fig. 10. Therefore, the discrimination process shown in step ST23 in the discrimination unit 34, which is the premise for this, is indicated by a dashed line.

[0109] As described above, the process of classifying the levels of subjective arousal may be performed either by the discrimination unit 34 or by the comparison unit 33. In the following explanation, as in the previous examples, a case where the levels of subjective arousal are classified by the discrimination unit 34 will be described as an example.

[0110] The determination unit 34 first accesses the storage unit 32 to obtain thresholds used to determine the level of subjective alertness. Here, for convenience, these are referred to as a "seventh level threshold" and an "eighth level threshold."

[0111] The seventh level threshold is a threshold used to determine whether the subject's level of subjective arousal is high or not, and the eighth level threshold is used to determine whether the subject's level of subjective arousal is low or not.

[0112] The discrimination unit 34 compares the value indicating the characteristic of the brain activity signal with the seventh level threshold (ST81). As a result, if the value indicating the characteristic of the brain activity signal is greater than the seventh level threshold (YES in ST81), the discrimination unit 34 determines that the level of subjective alertness is high (ST82).

[0113] On the other hand, if it is determined that the value indicating the characteristic of the brain activity signal is equal to or lower than the seventh level threshold (NO in ST81), the value indicating the characteristic of the brain activity signal is further compared with the eighth level threshold (ST83).

[0114] As a result, if the value indicating the characteristics of the brain activity signal is greater than the eighth level threshold (YES in ST83), the discrimination unit 34 determines that the level of the subjective alertness is medium (ST84).On the other hand, if the value indicating the characteristics of the brain activity signal is determined to be equal to or less than the eighth level threshold (NO in ST83), the discrimination unit 34 determines that the level of the subjective alertness is low (ST85).

[0115] Next, the flow of the process of classifying the level of the objective alertness when the discrimination unit 34 determines that no effort to wake up is being made will be described with reference to Fig. 11. For this reason, the discrimination process shown in step ST24 in the discrimination unit 34, which is the premise for this, is indicated by a dashed line.

[0116] The discrimination unit 34 first accesses the storage unit 32 to obtain thresholds used to discriminate the level of subjective alertness. Here, for convenience, these are referred to as a "ninth level threshold" and a "tenth level threshold."

[0117] The ninth level threshold is a threshold used to determine whether the subject's level of subjective arousal is high or not, and the tenth level threshold is used to determine whether the subject's level of subjective arousal is low or not.

[0118] The discrimination unit 34 compares the value indicating the characteristic of the brain activity signal with the ninth level threshold (ST91). As a result, if the value indicating the characteristic of the brain activity signal is greater than the ninth level threshold (YES in ST91), the discrimination unit 34 determines that the level of subjective alertness is high (ST92).

[0119] On the other hand, if it is determined that the value indicating the characteristic of the brain activity signal is equal to or less than the ninth level threshold (NO in ST91), the value indicating the characteristic of the brain activity signal is further compared with the tenth level threshold (ST93).

[0120] As a result, if the value indicating the characteristics of the brain activity signal is greater than the tenth level threshold (YES in ST93), the discrimination unit 34 determines that the level of subjective alertness is medium (ST94). On the other hand, if the value indicating the characteristics of the brain activity signal is determined to be equal to or less than the tenth level threshold (NO in ST93), the discrimination unit 34 determines that the level of subjective alertness is low (ST95).

[0121] [Effects of the Example] (1) A method for determining the state of arousal according to an embodiment of the present invention includes the steps of measuring a brain activity signal of a subject, extracting a brain activity signal for a predetermined period of time when an object of attention that attracts the subject's attention appears, calculating characteristics of the extracted brain activity signal, and determining the state of arousal of the subject from the calculated characteristics of the brain activity signal.

[0122] By adopting this type of discrimination method, it is possible to more accurately discriminate the subject's state of arousal in a short time. In other words, since the subject's state of arousal is discriminated based on brain activity signals for a short time after the appearance of an attention target, it is possible to increase the accuracy rate with less disturbance. Furthermore, discrimination can be performed in real time.

[0123] (2) In the above (1), the step of determining the wakefulness state of the subject is a step of determining whether or not the subject is making an effort to wake up. Indicators for measuring the wakefulness state (wakefulness level) of the subject include "wakefulness effort," "objective wakefulness," and "subjective wakefulness." The wakefulness state determination method according to the embodiment of the present invention can determine not only objective wakefulness and subjective wakefulness, but also whether or not the subject is making an effort to wake up. Therefore, the wakefulness effort is determined as a prerequisite for determining objective wakefulness and subjective wakefulness.

[0124] (3) In the method for discriminating the wakefulness state in (2) above, the step of calculating the characteristics of the brain activity signal when discriminating the wakefulness effort of the subject includes the steps of calculating a specific region where a brain activity signal is measured that can significantly distinguish between cases where a wakefulness effort is observed and cases where a wakefulness effort is not observed in the extracted brain activity signal, and calculating the characteristics of the brain activity signal using the calculated brain activity signal in the specific region, characterized in that the wakefulness effort of the subject is discriminated based on the characteristics of the brain activity signal in the specific region.

[0125] In determining whether a subject is making an effort to stay awake, the system not only uses the characteristics of all measured brain activity signals but also calculates specific regions and then uses the characteristics of brain activity signals from those specific regions. By performing this processing, it is possible to more accurately determine whether a subject is making an effort to stay awake, thereby improving the accuracy rate.

[0126] (4) In the step of determining the alertness effort in (2) or (3) above, if it is determined that the alertness effort is recognized, a step of classifying the alertness effort into a plurality of levels is further provided. By classifying the level of alertness effort not only based on whether or not the alertness effort is recognized, but also based on whether or not the alertness effort is recognized, the state of alertness effort can be determined in more detail, and the rate of correct answers can be improved.

[0127] (5) The step of determining the subject's wakefulness state in any of (2) to (4) above is a step of determining the subject's objective wakefulness level, which is performed after the step of determining the wakefulness effort. After determining whether or not the subject is making an effort to wake up, determining the subject's objective wakefulness level also makes it possible to more accurately determine the subject's wakefulness state.

[0128] (6) In the method for discriminating the state of alertness in (5) above, the step of calculating the characteristics of the brain activity signal when discriminating the subject's objective level of alertness includes the steps of calculating a specific region at which a brain activity signal is measured that can significantly distinguish between cases in which objective alertness is observed and cases in which objective alertness is not observed in the extracted brain activity signal, and calculating the characteristics of the brain activity signal using the calculated brain activity signal at the specific region, and the subject's objective level of alertness is discriminated based on the characteristics of the brain activity signal at the specific region.

[0129] In determining the subject's objective level of arousal, not only are the characteristics of all measured brain activity signals used, but specific regions are calculated and then the characteristics of the brain activity signals at those specific regions are used. By performing this processing, the objective level of arousal can be determined more accurately, leading to an improved rate of correct answers.

[0130] (7) In the step of determining the objective level of alertness in (5) or (6) above, a step of classifying the objective level of alertness into a plurality of levels is provided in each of the cases where an effort to alert is recognized and where an effort to alert is not recognized.

[0131] By dividing the objective arousal level into multiple levels regardless of whether or not an effort to awaken is being made, it becomes possible to grasp the state of the objective arousal level in more detail, and the rate of correct answers can be improved.

[0132] (8) The step of determining the subject's state of alertness in any of the above (2) to (4) is a step of determining the subject's subjective level of alertness, which is performed after the step of determining the alertness effort. By determining the presence or absence of alertness effort and then determining the subject's subjective level of alertness, the subject's state of alertness can be more accurately determined, and the rate of correct answers can be improved.

[0133] (9) In the method for discriminating the state of alertness in (8) above, the step of calculating the characteristics of the brain activity signal when discriminating the subject's subjective alertness includes the steps of calculating a specific region at which a brain activity signal is measured that can significantly distinguish between cases in which subjective alertness is observed and cases in which subjective alertness is not observed in the extracted brain activity signal, and calculating the characteristics of the brain activity signal using the calculated brain activity signal at the specific region, and the subject's subjective alertness is discriminated based on the characteristics of the brain activity signal at the specific region.

[0134] In determining the subject's subjective arousal level, not only are the characteristics of all measured brain activity signals used, but specific regions are calculated and then the characteristics of the brain activity signals at those specific regions are used. By performing this processing, the subjective arousal level can be determined more accurately, and the accuracy rate can be improved.

[0135] (10) In the step of determining the subjective level of alertness in (8) or (9) above, a step of classifying the subjective level of alertness into a plurality of levels is provided in each of the cases where an effort to alert is recognized and where an effort to alert is not recognized.

[0136] By dividing the subjective arousal level into multiple levels regardless of whether or not an effort to wake up is being made, it becomes possible to grasp the state of the subjective arousal level in more detail, and the rate of correct answers can be improved.

[0137] (11) The wakefulness state discrimination device includes an external situation detection device that detects whether an object of attention that attracts the subject's attention has appeared, a brain activity detection device that measures the subject's brain activity signals and extracts the brain activity signals for a predetermined period of time when the external situation detection device detects the appearance of an object of attention that attracts the subject's attention, and an wakefulness level discrimination device that calculates the characteristics of the brain activity signals extracted by the brain activity detection device and discriminates the wakefulness state of the subject from the calculated characteristics of the brain activity signals.

[0138] By using an arousal state discrimination device with such a configuration to discriminate the arousal state of a subject, it becomes possible to discriminate more accurately in a short time. That is, since the arousal state of a subject is discriminated based on brain activity signals for a short time after the appearance of an attention target, it becomes possible to increase the accuracy rate with less disturbance. Furthermore, it is possible to discriminate in real time.

[0139] Furthermore, as indicators of the wakefulness state, it is possible to distinguish between the wakefulness effort, objective wakefulness, and subjective wakefulness. In particular, by calculating specific regions and using brain activity signals at the specific regions for discrimination, it is possible to distinguish between the wakefulness effort, objective wakefulness, and subjective wakefulness for each specific region. [Explanation of symbols]

[0140] 1. External situation detection device, 11. External monitoring device, 12. Attention target discrimination device, 2. Brain activity detection device, 21. Brain activity signal measurement device, 22. Brain activity extraction device, 3. Arousal level determination device, 31. Calculation unit, 32. Memory unit, 33. Comparison unit, 34. Discrimination unit, A. Arousal state discrimination device

Claims

1. measuring brain activity signals of a subject; extracting the brain activity signal for a predetermined period of time when an attention target that attracts the subject's attention appears; calculating features of the extracted brain activity signals; determining a wakefulness state of the subject based on the calculated characteristics of the brain activity signal; An arousal state discrimination method comprising:

2. The step of determining the wakefulness state of the subject includes:

2. The method for determining a state of alertness according to claim 1, further comprising a step of determining whether or not the subject is making an effort to stay alert.

3. In determining the wakefulness effort of the subject, the step of calculating characteristics of the brain activity signal includes: Calculating a specific region where the extracted brain activity signal is measured, which can significantly distinguish between a case where the wake-up effort is recognized and a case where the wake-up effort is not recognized in the extracted brain activity signal; and calculating a feature of the brain activity signal using the calculated brain activity signal at the specific region, 3. The method for determining a state of alertness according to claim 2, wherein the alertness effort of the subject is determined based on characteristics of the brain activity signal in the specific region.

4. The method for determining a state of alertness described in claim 2 or claim 3, characterized in that if it is determined that the alertness effort is recognized in the step of determining the alertness effort, it further comprises a step of classifying the alertness effort into a plurality of levels.

5. The step of determining the wakefulness state of the subject includes:

3. The method for determining a state of arousal according to claim 2, further comprising a step of determining an objective level of arousal in the subject, the step being performed after the step of determining the effort to arouse.

6. In determining the objective alertness level of the subject, the step of calculating characteristics of the brain activity signal includes: calculating a specific region where the brain activity signal is measured, which can significantly distinguish between a case where the objective arousal level is recognized and a case where the objective arousal level is not recognized in the extracted brain activity signal; and calculating a feature of the brain activity signal using the calculated brain activity signal at the specific region, 6. The method for determining a state of arousal according to claim 5, wherein the objective level of arousal of the subject is determined based on characteristics of the brain activity signal in the specific region.

7. The method for determining a state of alertness described in claim 5 or claim 6, characterized in that the step of determining the objective level of alertness includes a step of classifying the objective level of alertness into multiple levels in each of the cases where the alertness effort is recognized and where the alertness effort is not recognized.

8. The step of determining the wakefulness state of the subject includes:

3. The method for determining a state of alertness according to claim 2, further comprising a step of determining a subjective level of alertness of the subject, which step is performed after the step of determining the alertness effort.

9. In determining the subjective alertness of the subject, the step of calculating characteristics of the brain activity signal includes: calculating a specific region where the brain activity signal is measured, which can significantly distinguish between a case where the subjective arousal level is recognized and a case where the subjective arousal level is not recognized in the extracted brain activity signal; and calculating a feature of the brain activity signal using the calculated brain activity signal at the specific region, 9. The method for determining an arousal state according to claim 8, wherein the subjective arousal level of the subject is determined based on characteristics of the brain activity signal in the specific region.

10. The method for determining a state of alertness described in claim 8 or claim 9, characterized in that the step of determining the subjective level of alertness includes a step of classifying the subjective level of alertness into a plurality of levels in each of the cases where the effort to wake up is recognized and where the effort to wake up is not recognized.

11. an external situation detection device that detects whether an attention target that attracts the subject's attention has appeared; a brain activity detection device that measures a brain activity signal of the subject and extracts the brain activity signal for a predetermined period of time when the external situation detection device detects that an attention target that attracts the subject's attention has appeared; an alertness discrimination device that calculates features of the brain activity signal extracted by the brain activity detection device and discriminates the alertness state of the subject from the calculated features of the brain activity signal; An awakening state determination device comprising:

Images