Brain activity estimation device

Abstract

To provide a brain activity estimation device capable of estimating a degree of brain activity accurately.SOLUTION: A brain activity estimation device includes a Doppler sensor for detecting a pulse wave of a human body in a non-contact manner, and an analysis unit for analyzing a signal of the pulse wave output from the Doppler sensor. The analysis unit adjusts an input signal amplification rate to increase the height of the signal of the pulse wave when the height of the signal of the pulse wave is small, calculates a Liapunov index from the signal of the pulse wave, and estimates a degree of the brain activity of the human body in the work or action on the basis of the Liapunov index.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a brain activity estimation device that estimates brain activity of a human body. [Background technology]

[0002] Conventionally, there is a concentration estimation device that determines the number of blinks, the amount of hand or body movement, the degree of pupil opening and closing from moving images acquired by an imaging means or continuously acquired still images, and estimates the level of concentration of a user based on the determination results (see, for example, Patent Document 1). This concentration estimation device estimates the level of concentration on the premise that when the level of concentration is increasing, the user's actions and states will be such that hand movement increases, the amount of body movement decreases, the number of blinks decreases, and the pupils become slightly dilated. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-082311 Summary of the Invention [Problem to be solved by the invention]

[0004] The concentration level estimation device in Patent Document 1 estimates the concentration level based on pre-determined assumptions, such as assuming that a decrease in the number of blinks indicates an increase in the concentration level. However, a decrease in the number of blinks does not necessarily mean that the concentration level is increasing, and the concentration level estimation device in Patent Document 1 has room for improvement in estimation accuracy.

[0005] Furthermore, the degree of concentration is thought to be related to the degree of brain activity, but evaluation of the degree of brain activity is not considered in Patent Document 1. Since the degree of brain activity is an index related to human emotions, if the degree of brain activity can be estimated, it can also be used to estimate human emotions, and estimating the degree of brain activity is important in light of future developments.

[0006] The present invention is intended to solve the above-mentioned problems, and aims to provide a brain activity estimation device that can estimate the degree of brain activity with high accuracy. [Means for solving the problem]

[0007] A brain activity estimation device according to the present disclosure includes a Doppler sensor that detects a pulse wave of a human body in a non-contact manner, and an analysis unit that analyzes a pulse wave signal output from the Doppler sensor, wherein the analysis unit: A chaos analysis is performed in which a first step is to calculate a vector specified from time series data of the height of the pulse wave signal and a preset delay time, a second step is to generate an attractor in which the vectors are arranged in time series order in a state space of three or more dimensions, and a third step is to calculate a Lyapunov exponent based on the trajectory of the attractor. It estimates the degree of human brain activity during tasks and actions based on Lyapunov exponents. So, before doing chaos analysis, The deviation of the pulse wave signal height is calculated, and when the deviation is smaller than a preset threshold, the input signal amplification rate is increased to increase the pulse wave signal height. Then, chaos analysis is performed based on the enlarged pulse wave signal. It is something. [Effects of the Invention]

[0008] According to the present disclosure, the brain activity estimation device adjusts the input signal amplification rate to increase the height of the pulse wave signal when the height of the pulse wave signal detected non-contactly by the Doppler sensor is small, thereby making the shape of the pulse wave clearer and allowing the analysis unit to perform highly accurate analysis. Here, the pulse wave is vital data related to the pulsation of the heart and, by extension, the activity of the nervous system of the brain, and the brain activity estimation device estimates brain activity based on the Lyapunov exponent quantified based on such pulse wave, thereby enabling highly accurate estimation of the degree of brain activity. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of a brain activity estimation device according to Embodiment 1 and a configuration for using the brain activity estimation device. [Figure 2] 2 is a schematic diagram of an antenna surface of the Doppler sensor according to the first embodiment. FIG. [Figure 3] 2 is a schematic diagram of a component mounting surface of a board of the Doppler sensor according to the first embodiment. FIG. [Figure 4] 3 is a diagram showing an example of a pulse wave detected by the Doppler sensor according to the first embodiment. FIG. [Figure 5] FIG. 3 is a diagram showing time-series data of pulse waveform changes in the brain activity estimation device according to the first embodiment. [Figure 6] FIG. 2 is a conceptual diagram of an attractor in chaos analysis of the brain activity estimation device according to the first embodiment. [Figure 7] FIG. 2 is a conceptual diagram of Lyapunov exponent conversion in the brain activity estimation device according to the first embodiment. [Figure 8] This is a bar graph showing the relationship between Lyapunov exponents and various behaviors that differ in central nervous system brain activity. [Figure 9] FIG. 10 is a diagram showing an example of changes in Lyapunov exponents when a human body sequentially performs actions that result in different brain activities. [Figure 10] 4 is a flowchart of a brain activity estimation process in the brain activity estimation device according to the first embodiment. [Figure 11] FIG. 10 is a block diagram showing a configuration of a brain activity estimation device according to a second embodiment and a configuration for using the brain activity estimation device. [Figure 12] This is a diagram showing Russell's circumplex model of emotions. [Figure 13] FIG. 10 is a diagram showing an example of an emotion model of the brain activity estimation device according to the second embodiment. [Figure 14] FIG. 10 is a diagram showing the configuration of an air conditioner according to a third embodiment. [Figure 15] FIG. 10 is a block diagram of an air conditioner according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals, and their description will be omitted or simplified as appropriate.

[0011] FIG. 1 is a block diagram showing the configuration of a brain activity estimation device 1 according to embodiment 1 and a configuration for using the brain activity estimation device 1. FIG. 2 is a schematic diagram of the antenna surface of a Doppler sensor 10 according to embodiment 1. FIG. 3 is a schematic diagram of the board component mounting surface of the Doppler sensor 10 according to embodiment 1. The brain activity estimation device 1 is a device that estimates the degree of brain activity of a human body. The brain activity estimation device 1 can objectify the degree of brain activity by quantifying the degree of brain activity of a human body.

[0012] The brain activity estimation device 1 includes a Doppler sensor 10 and an analysis unit 103. The Doppler sensor 10 emits a constant sinusoidal wave of approximately 24 GHz, known as the microwave or quasi-millimeter wave band, toward the human body, whose brain activity is to be estimated. Blood flow in the human body changes with the pulsation of the heart, and when the distance between the body surface and the Doppler sensor 10 changes, the wave reflected from the body surface changes due to the Doppler effect.

[0013] The Doppler sensor 10 receives reflected waves from the human body in response to blood vessel movement, and detects the pulse wave of the human central nervous system based on the frequency difference between the reflected waves and the transmitted waves emitted from the Doppler sensor 10. A pulse wave is a waveform that indicates changes in the movement of a person's body surface due to the pulsation of the heart, and includes waveforms of changes in blood vessel movement and waveforms of changes in the body surface around the heart. Blood vessels run throughout the human body, and the Doppler sensor 10 can detect blood vessel movement not only in the heart but also in parts of the human body, such as parts of the head or arms.

[0014] For distance measurement, a measurement frequency of 60 GHz to 79 GHz is often used due to its high resolution. However, unlike large body movements, the purpose here is to detect pulse waves, and it is necessary to analyze minute fluctuations with ultra-low frequency characteristics of approximately 1 Hz. For this reason, analog detection using the 24 GHz Doppler method is suitable for detecting pulse waves.

[0015] As described above, the Doppler sensor 10 has the advantage of being able to detect the human body's pulse wave without contact by using radio waves. Therefore, the Doppler sensor 10 can measure a wide range of the human body. Note that the Doppler sensor 10 can measure vital data such as pulse rate, respiratory rate, body movement, sleep state, and autonomic nervous balance by analyzing the peak interval of the pulse wave, but in the first embodiment, it is used to detect the pulse wave.

[0016] Although the sensor for detecting the human body's pulse waves has been described as the Doppler sensor 10, it is not limited to the Doppler sensor 10. The sensor for detecting the human body's pulse waves may also be an FMCW sensor of 24 GHz to 79 GHz. The FMCW sensor can measure the pulse wave by detecting changes in distance to the target and converting the velocity.

[0017] Furthermore, sensors that detect human pulse waves are not limited to non-contact sensors; they can also be contact sensors that detect by contacting the human body. Non-contact sensors have the potential to measure pulse waves with higher accuracy than contact sensors. While there are many contact-type devices for measuring pulse, photoelectric pulse wave sensors are commonly used in wearable devices. Pulse wave sensors capture the changes in blood vessel volume that occur as the heart pumps blood as a waveform, and are equipped with a detector that monitors this volume change. A pulse wave sensor can obtain the pulse interval by counting the interval between peaks of the pulse wave obtained. The number of pulse beats per minute can be calculated by reciprocalizing the pulse interval. For example, if the average pulse interval is 800 ms (0.8 seconds), the pulse rate is 60 / 0.8, or 75 beats per minute.

[0018] Pulse wave sensors are divided into transmission and reflection types, which differ in their measurement method. Transmission pulse wave sensors measure pulse waves by irradiating the body surface with infrared or red light and measuring the change in blood flow that changes with the heartbeat as the amount of light that passes through the body. However, transmission pulse wave sensors are limited to measuring areas where infrared or red light can easily pass through, such as the fingertips or earlobes.

[0019] On the other hand, reflective pulse wave sensors shine infrared light, red light, or light with a green wavelength of around 550 nanometers toward the living body and measure the light reflected from the body using a photodiode or phototransistor. Oxygenated hemoglobin is present in arterial blood, which has the property of absorbing incident light. For this reason, reflective pulse wave sensors can measure pulse wave signals by sensing the blood flow rate (changes in vascular pressure) that changes in response to the heart's pulsation in a time series. Because reflective pulse wave sensors measure reflected light, they have the advantage of not needing to limit the measurement location, as is the case with transmissive sensors. For pulse wave measurements in outdoor applications such as sports watches, a green LED is often used for the illumination light, as a green light source is suitable because it has a high absorption rate for hemoglobin in the blood and is less affected by ambient light.

[0020] However, contact-type measuring devices require the device to be worn to measure, which can be cumbersome and makes it difficult to measure from a distance, so non-contact sensors are more suitable for incorporation into equipment. In the following, the explanation will be given assuming that the sensor that detects the pulse waves of the human body is a Doppler sensor 10.

[0021] Specifically, the Doppler sensor 10 has an antenna unit 100, a radio unit 101, an analog circuit unit 102, and a substrate unit 10a. The antenna unit 100 is a unit that acquires the human body's pulse wave, which is a central nervous activity. The antenna unit 100 has a TX, which is an oscillator, and an RX, which is a receiver. As shown in FIG. 2, the antenna unit 100 has multiple (12 in this example) antennas 100a. Each of the TX and RX has six antennas 100a in this example.

[0022] The radio unit 101 generates 24 GHz radio waves called RF (Radio Frequency), transmits the radio waves from TX, and receives the reflected waves from RX. The analog circuit unit 102 has a circuit that converts the frequency components, which are the Doppler shift of the reflected waves. The analog circuit unit 102 also has an analog amplification filter unit (OPAMP) that extracts and amplifies the necessary frequency band, as shown in Figure 3, and an analog-to-digital converter (LDO) that enables numerical analysis.

[0023] The substrate unit 10a includes a connector unit and a memory for outputting information to the analysis unit 103 or a device equipped with the brain activity estimation device 1. The wireless unit 101, the analog circuit unit 102, and the analysis unit 103 are covered with a metal shielding case as shown in Fig. 3. In Fig. 3, the shielding case portion is indicated by dots.

[0024] The analysis unit 103 is a part that analyzes the pulse wave detected by the Doppler sensor 10. The analysis unit 103 generates an index value by quantifying the pulse wave based on chaos analysis using pulse wave shape displacement, which is a time-series displacement of the waveform shape of the pulse wave (hereinafter referred to as pulse wave shape), as the analysis source, and estimates the degree of brain activity of the human body based on the index value. The estimation of brain activity in the analysis unit 103 will be described later. The analysis unit 103 outputs brain activity information that indicates the estimated result of brain activity. The brain activity information is information that indicates the degree of brain activity. The brain activity information output from the analysis unit 103 is input to the control content determination unit 104 or the cloud unit 106, which will be described later.

[0025] The analysis unit 103 is configured with a microprocessor unit. The analysis unit 103 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and a control program and the like are stored in the ROM. The analysis unit 103 is not limited to a microprocessor unit. For example, the analysis unit 103 may be configured with updatable firmware or the like. The analysis unit 103 may also be a program module that is executed by commands from a CPU or the like (not shown). The analysis unit 103 may be provided outside the Doppler sensor 10 as a separate entity from the Doppler sensor 10, or may be provided on a board within the Doppler sensor 10 and edge processed within the single sensor.

[0026] The degree of brain activity estimated by analysis unit 103 can be used to control equipment equipped with brain activity estimation device 1. A specific example of equipment equipped with brain activity estimation device 1 is an air conditioning device, the details of which will be described again in embodiment 3 below. Equipment equipped with brain activity estimation device 1 has control content determination unit 104 and equipment control unit 105. Control content determination unit 104 determines control content for the equipment based on the brain activity information obtained by analysis unit 103, generates control data, and outputs it to equipment control unit 105. Equipment control unit 105 controls various actuators of the equipment based on the control data from control content determination unit 104.

[0027] Furthermore, as a form of use of the brain activity estimation device 1, the brain activity information output from the analysis unit 103 may be input to the cloud unit 106. The cloud unit 106 accumulates the brain activity information input from the analysis unit 103. The analysis unit 103 may also output the vital data itself, which is the pulse wave data obtained by the Doppler sensor 10, and accumulate it in the cloud unit 106. The brain activity information and vital data accumulated in the cloud unit 106 can be visualized by displaying it on the display unit 107. Furthermore, the brain activity information and vital data accumulated in the cloud unit 106 are not limited to being visualized, and can also be provided to the data collection unit 108 from another cloud via the cloud unit 106 and used for various purposes.

[0028] The display unit 107 is a display such as a liquid crystal panel for displaying information. The display unit 107 may be a display unit of a smartphone, or may be one that displays and visualizes brain activity information on an application installed on the smartphone.

[0029] Conventional methods for estimating a person's level of concentration from images acquired by imaging devices estimate the level of concentration based on pre-defined assumptions, such as assuming that a decrease in the number of blinks indicates an increase in the level of concentration. However, a decrease in the number of blinks does not necessarily mean an increase in the level of concentration, and this method leaves room for improvement in estimation accuracy. Furthermore, while there are conventional devices that estimate people's emotions using electroencephalographs, quantifying alpha and beta brain waves is not easy. Furthermore, electroencephalograph-based estimation devices have issues with practicality and analysis time, such as the need for measurement data over a certain period of time for estimation processing, which makes it impossible to perform estimation in real time, the need to wear the electroencephalograph on the head, and the complexity of the system.

[0030] In contrast, the brain activity estimation device 1 of the first embodiment can estimate the degree of brain activity in the central nervous system in a short time from pulse waves detected in a non-contact manner using Doppler sensor 10 by chaos analysis, which will be described below.

[0031] Here, we will explain the details of chaos analysis. Physiological psychology is the study of physiological and psychological states based on physiological changes observed in biosignals. Traditional physiological psychology has analyzed various biosignals, such as electroencephalograms (EEGs), electrocardiograms (ECGs), heartbeat intervals, blood pressure, and respiratory pulse waves, using various techniques, resulting in significant insights. However, most analyses have relied on linear theory. However, biosignals contain nonlinear properties, which are known to fluctuate due to a nonlinear property called chaos. Chaos refers to the phenomenon in which, despite the system's state transition rules being deterministic, the nonlinearity of the system itself creates complexity equivalent to that of a stochastic system. While the state of an object can be deterministically described using equations, the behavior of the object's state lacks any discernible pattern, exhibiting extremely complex behavior resembling randomness.

[0032] However, chaotic phenomena are phenomena that appear chaotic at first glance, but in fact have definite rules behind them. In other words, the next phenomenon that occurs is not determined by probability, but is determined deterministically according to certain rules. The reason why an object appears chaotic despite following rules is that even if the movements of each of the elements that make up the object are simple, they become complex when they behave as a group. It has been determined that there is a high possibility that chaotic information exists in the biological signals generated by such complex systems.

[0033] In recent years, the effectiveness of chaos analysis for estimating human physiological and psychological states has been proven through various experiments. In conventional chaos analysis, information that cannot be determined univocally, such as a person's sense of warmth or cold or their psychological state, is treated as a chaotic object, and by performing chaos analysis, correlations are found among information that at first glance appear unrelated. In chaos analysis, the choice of what to treat as a chaotic object is important. If the object changes, it becomes a completely different concept.

[0034] Brain activity estimation device 1 targets brain activity of the central nervous system as a chaos target. Conventionally, there has been no device that targets brain activity of the central nervous system as a chaos target. Brain activity estimation device 1 performs chaos analysis using pulse wave shape fluctuations, which focus on the movement of the waveform itself, as the analysis source. Pulse wave shape fluctuations are expressed as time-series data of the displacement of the waveform shape of the pulse wave, in other words, time-series displacement data of the waveform shape of the pulse wave. Brain activity estimation device 1 generates an index value that quantifies the pulse wave based on chaos analysis using pulse wave shape displacement, which is the time-series displacement of the waveform shape of the pulse wave, as the analysis source, and estimates the degree of brain activity of the human body based on the index value.

[0035] Chaos analysis involves the sequential implementation of the following three steps (1) to (3). Details of each step will be explained later. (1) The first step is to calculate a vector specified from the time series data of pulse waveform changes and a preset delay time. (2) The second step is to generate an attractor by arranging vectors in a time-series order in a three-dimensional state space. (3) The third step is to calculate the Lyapunov exponent, which is an index value based on the attractor trajectory.

[0036] The heart's pulsation interval varies from beat to beat, but this originates in the brain and is transmitted to the heart via the autonomic nervous system. Therefore, the inventors of the present invention have conceived the idea of ​​estimating the degree of brain activity by finding a correlation between heart pulsation and brain activity, considering that the central nervous system, as well as the autonomic nervous system, is also related to heart pulsation.

[0037] Furthermore, there is a conventional technology that estimates the level of brain arousal using changes in pulse rate. For example, this technology determines that a large change in pulse rate indicates a high level of arousal, and a small change in pulse rate indicates a low level of arousal and drowsiness. Pulse rate changes are the time fluctuations in the intervals between peaks of the pulse wave, and are detected using pinpoint information on the pulse wave peaks alone. Hereinafter, the time fluctuations in the intervals between peaks of the pulse wave may be referred to as one-dimensional pattern pulse rate displacement or pulse rate fluctuation. The term one-dimensional is used because the time fluctuations in the intervals between peaks of the pulse wave are not related to the height of the pulse wave, and vital data is obtained by frequency conversion of only the pulse rate displacement.

[0038] While the conventional techniques for measuring arousal levels described above use pulse rate changes, the inventors realized that pulse rate changes alone are insufficient to fully interpret complex neural activity information in living organisms. Therefore, the inventors sought a method different from pinpoint detection based solely on pulse wave peak information, and focused on pulse waveform displacement. However, when comparing the complexity of peak-to-peak fluctuations with the two-dimensional waveform pattern fluctuations of the pulse wave itself, the two-dimensional waveform pattern fluctuations (i.e., pulse waveform fluctuations) are incomparably more complex than the one-dimensional pulse interval fluctuations. This is thought to be because pulse waveform fluctuations are related to neuronal activity in the six layers of the cerebral cortex. For this reason, the inventors decided to use chaos as a method for analyzing brain activity, rather than relying on conventional analytical methods.

[0039] Next, we will explain the chaos analysis performed by the analysis unit 103 of the brain activity estimation device 1. First, the analysis unit 103 obtains a pulse wave from the Doppler sensor 10. The following FIG. 4 shows an example of a pulse wave detected by the Doppler sensor 10.

[0040] FIG. 4 is a diagram illustrating an example of a pulse wave detected by the Doppler sensor 10 according to the first embodiment. The horizontal axis of FIG. 4 represents time, and the vertical axis represents analog pulse wave height. Here, pulse wave height refers to power. The pulse wave height does not refer to the power of only the peak portion of the pulse wave, where the power is highest, but rather refers to the power at each time point in a time series including the peak portion of the pulse wave. The Doppler sensor 10 obtains the analog waveform shown in FIG. 4 and outputs it to the analysis unit 103. The pulse wave height decreases as the distance between the Doppler sensor 10 and the human body being measured increases. However, in the chaos analysis of the first embodiment, the variation in pulse wave height is analyzed, so the absolute value of the pulse wave height is not necessary information for quantifying the pulse wave. However, a larger pulse wave height results in a clearer pulse wave shape and higher accuracy in attractor generation, as described below. For this reason, a shorter distance between the Doppler sensor 10 and the human body is preferable.

[0041] When analyzing time-series data of the pulse waveform, the analysis unit 103 can obtain the pulse wave height necessary to ensure accuracy by doing the following: The analysis unit 103 calculates the deviation of the pulse wave height, and if the deviation is smaller than a preset threshold, automatically adjusts the input signal amplification factor to artificially increase the analog pulse waveform. This clarifies the shape of the pulse wave, allowing the analysis unit 103 to perform a highly accurate analysis even if the distance from the Doppler sensor 10 to the human body is great.

[0042] For example, when the distance from the Doppler sensor 10 to the human body is short, the input signal magnification may be 1x. On the other hand, as the distance from the Doppler sensor 10 to the human body increases, the pulse wave itself becomes smaller, making the pulse waveform variation less visible and reducing the deviation. Therefore, the analysis unit 103 may determine the magnification depending on the magnitude of the deviation. Specifically, for example, when the deviation is smaller than a first threshold, the analysis unit 103 may set the input signal magnification to 2x, and when the deviation is smaller than a second threshold that is smaller than the first threshold, the analysis unit 103 may set the input signal magnification to 3x. In other words, the farther the distance from the Doppler sensor 10 to the human body, the greater the input signal amplification factor, thereby increasing the magnitude of the analog waveform. This allows the analysis unit 103 to analyze pulse waveform fluctuations (time-series deviation data of the pulse waveform) even when the distance from the Doppler sensor 10 to the human body is long.

[0043] Furthermore, even if the distance from the Doppler sensor 10 to the human body is the same, if the area of ​​the human body and the power of the blood flow sent to the blood vessels are small, such as in the case of an infant, the pulse wave level is small and analysis becomes difficult. For this reason, the analysis unit 103 automatically increases the input signal amplification factor and increases the analog waveform as the area of ​​the human body becomes smaller and the deviation in pulse wave height becomes smaller. This allows the analysis unit 103 to solve the problem of difficulty in analysis when the pulse wave level is small.

[0044] Next, the analysis unit 103 generates an attractor, described below, from the pulse waveform. A pulse waveform is a two-dimensional waveform pattern of a pulse wave. Although it is difficult to find patterns in time-series data of pulse waveforms from one waveform to another, when the time-series data of pulse waveforms is converted into an attractor, a certain pattern exists. An attractor is a set toward which a dynamical system evolves over time. When a dynamical system moves from a point sufficiently close to an attractor, the system continues to remain sufficiently close to the attractor. There are no restrictions on the trajectories contained in an attractor other than that they must remain within the attractor.

[0045] Next, the first to third steps of the chaos analysis will be explained in order. The first embodiment is characterized in that the brain activity of the central nervous system is the subject of chaos analysis, and pulse wave shape displacement, which is the time series displacement of the waveform shape of the pulse wave, is used as the analysis source of the chaos analysis, and the chaos analysis method itself uses a conventionally known method. Therefore, the following explanation of the chaos analysis will be an outline.

[0046] (1st step) The first step is a step of calculating a vector specified from the time series data of the pulse waveform variation and a preset delay time, as described above.

[0047] FIG. 5 is a diagram showing time-series data of pulse waveform displacement in the brain activity estimation device 1 according to the first embodiment. The horizontal axis of FIG. 5 represents time, and the vertical axis represents analog pulse wave height. x(i) (i=1, 2, ..., n) represents time-series data of the pulse waveform based on sensor data obtained by the Doppler sensor 10. The difference between the pulse wave height at x(i) and the pulse wave height at x(i+1) represents the pulse waveform displacement. Note that x(0) is the initial value of the sensor data obtained over a period of the calculation window length. The calculation window length is an arbitrarily set time length, such as 20 seconds. A shorter calculation window length enables faster calculation, but a longer calculation window length provides more pulse waveform data and higher accuracy.

[0048] Using this time series data, in order to embed two-dimensional time series changes into a d-dimensional state space, in other words, to draw a trajectory in the d-dimensional state space, the analysis unit 103 sets a time delay τ that is an appropriate delay time and creates a vector. Specifically, the analysis unit 103 creates a vector X(i) = {x(i), x(i + τ), x(i + 2τ), ..., x(i + (d - 1)τ)}. For example, if d is set to 3 and two-dimensional time series changes are embedded into a three-dimensional state space, the analysis unit 103 creates a vector with three state variables. In the case of a three-dimensional state space, the vector X(i) is X(i) = {x(i), x(i + τ), x(i + 2τ)}. Since i = 1, 2, ..., n, n vectors X are created. Here, τ is a parameter called an embedding delay time.

[0049] (Second step) The second step is to generate an attractor by arranging the vectors in a time-series order in the three-dimensional state space.

[0050] FIG. 6 is a conceptual diagram of an attractor in the chaos analysis of the brain activity estimation device 1 according to the first embodiment. By sequentially plotting this vector X(i) on the coordinate axes x(i), x(i+τ), x(i+2τ), ..., x(i+(d-1)τ), a trajectory is obtained. FIG. 6 shows a three-dimensional state space with three coordinate axes: x(i), x(i+τ), and x(i+2τ). For example, by setting d=3 and a delay time of 0.05 seconds, the vectors obtained in the first step are plotted (arranged) in chronological order in the three-dimensional state space, resulting in a trajectory as shown in FIG. 6. This trajectory is the trajectory of the attractor. The shape of the attractor trajectory reveals a spiral trajectory, demonstrating the presence of chaos information in the fluctuations of the two-dimensional pulse waveform pattern.

[0051] (Third Step) The third step is to calculate the Lyapunov exponent, which is an index value based on the attractor trajectory. The Lyapunov exponent is calculated by assessing the instability or divergence of the attractor trajectory.

[0052] FIG. 7 is a conceptual diagram of Lyapunov exponent calculation in the brain activity estimation device 1 according to the first embodiment. To ultimately estimate the degree of brain activity, it is necessary to quantify the time series displacement of the pulse waveform. The Lyapunov exponent is used as this quantified index value. An attractor in a three-dimensional state space has trajectory instability. Instability can also be described as divergence. The Lyapunov exponent quantifies this trajectory instability. The Lyapunov exponent is a measure of how far two trajectories starting from two nearby points move apart. In other words, the Lyapunov exponent is a value that represents the degree to which very close trajectories move apart in a dynamical system. Here, the larger the Lyapunov exponent, the greater the fluctuation range of the attractor, and the greater the width of the fluctuation.

[0053] As shown in Figure 7, suppose a tiny sphere (hypersphere) of radius ε is given as an initial value to a three-dimensional chaotic dynamical system. Then, after an arbitrarily set sliding time S, what was initially a sphere is mapped once, resulting in it being stretched in the e1 direction, remaining almost unchanged in the e2 direction, and being squashed in the e3 direction, resulting in the sphere becoming an ellipsoid. Here, if the logarithms of the expansion rates per unit time in the e1, e2, and e3 directions are λ1, λ2, and λ3, respectively, these λ1, λ2, and λ3 are the Lyapunov exponents.

[0054] This set of Lyapunov exponents is called the Lyapunov spectrum. The sphere is repeatedly stretched for each slide time S, and the expansion rate is calculated. The sum of these values ​​is averaged to calculate the overall Lyapunov spectrum. The largest of the calculated Lyapunov exponents is called the maximum Lyapunov exponent. The maximum Lyapunov exponent indicates the degree to which the system's trajectory deviates from its initial value. In other words, the Lyapunov exponent indicates how far the nth state is from the initial state. In terms of brain activity, the farther away the nth state is from the initial state, the more active the brain is, and the less active the brain is. Of course, accuracy will decrease, but instead of calculating multiple times for each slide time S, it is also possible to calculate the Lyapunov exponent by comparing two states over a single slide time interval.

[0055] Specifically, the Lyapunov exponents are calculated as follows. For example, assume that the radius of the sphere is 0.08, the calculation window length is 20 seconds, the slide time is 1 second, and the measurement frequency is 200 Hz. Because the measurement frequency is 200 Hz, 200 pieces of data are obtained per second, and 4,000 pieces of data are obtained from the start of calculation through the calculation window length. If the number of dimensions of the state space is three, a Lyapunov spectrum consisting of λ1, λ2, and λ3 for the three dimensions is obtained using 4,000 pieces of data 20 seconds after the start of measurement. Then, a similar Lyapunov spectrum consisting of λ1, λ2, and λ3 is obtained over the next second.

[0056] If the output window length is set to 60 seconds, the above process is repeated for 60 seconds. In other words, the first Lyapunov spectrum is obtained in the first 20 seconds, and then a Lyapunov spectrum is obtained every second of the slide time for the next 40 seconds, resulting in a total of 40 Lyapunov spectra. The 40 Lyapunov spectra are then summed and averaged for λ1, λ2, and λ3 to determine the averaged λ1, λ2, and λ3. The largest Lyapunov exponent among these is the maximum Lyapunov exponent, and this maximum Lyapunov exponent is one piece of data obtained over the total output length. In other words, the maximum Lyapunov exponent is used as the degree to which the system is deviating from its initial value.

[0057] Here, the number of pulse wave data measurements, which are vital data, will be considered. For example, if the measurement frequency is set to 200 Hz and the output window length is set to 60 seconds, 12,000 pieces of data will be obtained in 60 seconds. The analysis unit 103 creates an attractor trajectory using the 12,000 pieces of data and calculates the Lyapunov exponents. Note that the output window length is the time required to output one piece of data, as described above, and is the time required to calculate one Lyapunov exponent. Furthermore, if the measurement frequency is set to 500 Hz and the output window length is set to 60 seconds, for example, 30,000 pieces of data will be obtained in 60 seconds. In this case, the analysis unit 103 creates an attractor trajectory using the 30,000 pieces of data and calculates the Lyapunov exponents.

[0058] The more data there is, the greater the accuracy is, which is an advantage. However, when there is a large amount of data, there are disadvantages such as longer measurement time, an inability to track changes in the level of brain activity, and longer CPU processing time or the inability to perform calculations. Conversely, the fewer data there is, the lower the accuracy is. However, when there is a small amount of data, there are advantages such as shorter measurement time, shorter CPU processing time, and better tracking.

[0059] Therefore, the measurement frequency is preferably approximately 100 Hz to 1000 Hz. The output window length is preferably 30 seconds or more and 5 minutes or less. Because the central nervous system has a fast response time, the output window length should be shorter than when measuring the autonomic nervous system, preferably 30 seconds or more and 60 seconds or less. Furthermore, the analysis unit 103 may be configured as follows to improve accuracy: The analysis unit 103 may collect and average one piece of output data every set time. For example, if the output window length is 30 seconds and the set time is 5 seconds, the analysis unit 103 collects one piece of output data every 5 seconds based on vital data from the 30 seconds prior to that time. Once 30 pieces of data have been collected, the analysis unit 103 may average the 30 pieces of data to obtain one piece of data. In this case, the analysis unit 103 can obtain one piece of highly accurate data (Lyapunov exponent) after spending 3 minutes on digitization.

[0060] Next, the correlation between the Lyapunov exponents and various behaviors with different brain activities of the central nervous system, which the present inventors have found, will be explained using Figures 8 and 9. Figures 8 and 9 are created based on the results of an experiment in which a human body is made to perform various behaviors with different brain activities, and the Lyapunov exponents are calculated based on pulse waves acquired from the human body performing the behaviors.

[0061] Figure 8 is a bar graph showing the relationship between the Lyapunov exponent and various behaviors that involve different levels of central nervous system brain activity. The values ​​on the vertical axis are normalized with resting as the value of 1. In other words, the values ​​on the vertical axis for behaviors other than resting are the Lyapunov exponent for that behavior divided by the Lyapunov exponent for resting. The horizontal axis shows human behaviors with different levels of brain activity. The following four behaviors were used for the horizontal axis: resting, reading an article, typing an email, and typing out documents. Reading an article refers to the behavior of reading documents on a computer or smartphone, for example. Typing out documents refers to the behavior of typing out the characters written in the documents into a computer.

[0062] Figure 8 confirms that heavy tasks, such as typing emails and copying documents, which are subjectively reported to involve a higher degree of brain activity, have larger values ​​on the vertical axis than light tasks such as resting and reading articles. In other words, it was confirmed that a higher degree of brain activity results in a higher Lyapunov exponent. Therefore, it was confirmed that there is a relationship between the degree of brain activity in the central nervous system and the Lyapunov exponent.

[0063] Furthermore, when resting, the level of concentration is low, while when typing calculations, the level of concentration is high. This has been confirmed to correlate with the results of subjective evaluation. Therefore, there is a correlation between behavior, Lyapunov exponent, and concentration level, and the Lyapunov exponent can be treated as a mental index of concentration. Specifically, a large Lyapunov exponent can be evaluated as a high level of concentration, and a small Lyapunov exponent can be evaluated as a low level of concentration. Therefore, the vertical axis in Figure 8 can also be seen as an index of concentration level. In other words, a larger value on the vertical axis indicates a higher level of concentration, and a smaller value on the vertical axis indicates a lower level of concentration. Therefore, the value on the vertical axis can be considered, for example, as a concentration index that indicates the level of concentration.

[0064] Figure 9 shows an example of changes in Lyapunov exponents when a human body sequentially performs actions that involve different brain activities. The vertical axis represents the elapsed time (minutes) of the action, and the horizontal axis represents the Lyapunov exponents during that action. Figure 9 also shows changes in the Lyapunov exponents when the action is changed from a resting state before the task, to a typing task, and then to a cessation of the typing task. The Lyapunov exponents, which were small during the resting state, increased during the typing task and then decreased when the task was halted. This indicates a relationship between the Lyapunov exponents and the level of brain activity. In other words, a large Lyapunov exponent indicates a high level of brain activity, while a small Lyapunov exponent indicates a low level of brain activity.

[0065] Based on the above relationship, the brain activity estimation device 1 estimates brain activity of the central nervous system based on the Lyapunov exponents. Specifically, the analysis unit 103 estimates that if the Lyapunov exponents are large, the degree of brain activity is high, and if the Lyapunov exponents are small, the degree of brain activity is low.

[0066] The Lyapunov exponents on the vertical axis of FIG. 9 are values ​​obtained by calculation using brain activity estimation device 1 temporarily as an experimental device. For this reason, it can be seen from FIG. 9 that the Lyapunov exponents are obtained at one-minute intervals by brain activity estimation device 1. Furthermore, FIG. 9 proves that the process of calculating Lyapunov exponents from pulse waves can be performed in one minute, which is generally considered a short time. Note that although FIG. 9 is described here as the experimental results obtained using brain activity estimation device 1, FIG. 9 merely shows the experimental results of measuring the correlation between Lyapunov exponents and behavior, and may be created using a dedicated experimental device rather than brain activity estimation device 1.

[0067] FIG. 10 is a flowchart of the brain activity estimation process in the brain activity estimation device 1 according to the first embodiment. The brain activity estimation device 1 includes a step of acquiring pulse waveform data (step S1) and a step of performing chaos analysis based on the electroencephalogram waveform data (step S2). As described above, the step of performing chaos analysis includes three steps (steps S21 to S23). The first step is a step of calculating a vector identified from time-series data of pulse waveform displacement and a preset delay time (step S21). The second step is a step of generating an attractor in which the vectors are arranged in chronological order in a three-dimensional state space (step S22). The third step is a step of calculating a Lyapunov exponent, which is an index value, based on the trajectory of the attractor (step S23).

[0068] The analysis unit 103 estimates the degree of brain activity based on the Lyapunov exponents (step S3) and outputs brain activity information indicating the estimation result. The analysis unit 103 converts the Lyapunov exponents, for example, into numerical scales (e.g., 1 to 10) indicating the degree of brain activity and outputs the resulting scale. The numerical scales indicate, for example, a low to a high degree of brain activity, from a low to a high level. The analysis unit 103 may also store, for example, Lyapunov exponents at rest and output a percentage (%) of the Lyapunov exponents analyzed by the analysis unit 103 relative to the Lyapunov exponents at rest as a numerical value indicating the degree of brain activity. The analysis unit 103 is not limited to outputting the degree of brain activity as numerical values ​​as described above, and may also output, for example, a stage image. A stage image refers to, for example, an image in which facial expressions change in stages according to the degree of brain activity.

[0069] The brain activity information output from the analysis unit 103 is input to the display unit 107 via the cloud unit 106 and displayed. In this way, the brain activity estimation device 1 outputs and visualizes brain activity information from the analysis unit 103, allowing the user to grasp the degree of brain activity. The brain activity information output from the analysis unit 103 may be accumulated in the data collection unit 108 via the cloud unit 106. The brain activity information output from the analysis unit 103 may also be input to the display unit 107 directly and displayed, without going through the cloud unit 106. The brain activity information output from the analysis unit 103 may also be input to the control content determination unit 104 and used to determine the control content of a device equipped with the brain activity estimation device 1.

[0070] Since the degree of brain activity is related to the degree of concentration, the analysis unit 103 may output, as brain activity information, concentration level information indicating the degree of concentration estimated based on the Lyapunov exponent. The concentration level information may be, for example, the concentration index described above, or, similar to the degree of brain activity, may be a percentage (%) numerical value or a stage image. Note that the concentration level information indicates that a larger Lyapunov exponent indicates a higher degree of concentration, and a smaller Lyapunov exponent indicates a lower degree of concentration. The concentration level information may also be information indicating the current concentration level, the time-series concentration level, or both.

[0071] As described above, behavior with a high degree of concentration as reported subjectively is associated with high brain activity and a high Lyapunov exponent. Therefore, brain activity estimation device 1 suitably outputs from analysis unit 103 that the larger the Lyapunov exponent, the higher the degree of concentration.

[0072] Note that since the concentration level can be regarded as an index that is the opposite of the drowsiness level, the analysis unit 103 may output the drowsiness level information instead of the concentration level information, thereby allowing the user to visually confirm the drowsiness level.

[0073] As described above, the brain activity estimation device 1 of the first embodiment includes the Doppler sensor 10, which is a sensor that detects the pulse wave of the human body, and the analysis unit 103 that analyzes the pulse wave detected by the Doppler sensor 10. The analysis unit 103 generates an index value that digitizes the pulse wave based on chaos analysis using pulse wave shape displacement, which is a time-series displacement of the waveform shape of the pulse wave, as the analysis source, and estimates the degree of brain activity of the human body based on the index value.

[0074] In this way, brain activity estimation device 1 can estimate the degree of brain activity based on index values ​​generated based on time-series changes in the waveform shape of a pulse wave. Furthermore, pulse waves are vital data related to the pulsation of the heart and, ultimately, to the activity of the nervous system of the brain. Because brain activity estimation device 1 estimates brain activity based on index values ​​calculated from such pulse waves, it can estimate the degree of brain activity with higher accuracy than conventional estimation methods that use eye blinking, etc.

[0075] Brain activity estimation device 1 also uses Doppler sensor 10 as a sensor for non-contact detection of the human body's pulse waves, allowing for non-contact detection of the human body's pulse waves and non-contact estimation of the degree of brain activity. Brain activity estimation device 1 also does not use an electroencephalograph. Therefore, brain activity estimation device 1 does not require the effort of wearing an electroencephalograph, and because it does not use an electroencephalograph, analysis does not take long and can be performed in a short time.

[0076] Furthermore, while conventional devices for estimating human activity use imaging means to measure the degree of pupil dilation, these devices require a short distance between the device and the eye in order to capture an image of the pupil. In contrast, brain activity estimation device 1 uses Doppler sensor 10, making it easy to use and capable of measuring from a long distance. Furthermore, because it does not measure information specific to an individual, such as pupil size, privacy can be respected.

[0077] Chaos analysis is a process that sequentially performs the following steps: calculating a vector identified from time-series data of pulse waveform changes and a preset delay time; generating an attractor in which the vectors are arranged in chronological order in a three-dimensional state space; and calculating the Lyapunov exponent, which is an index value, based on the trajectory of the attractor.

[0078] In this way, the brain activity estimation device 1 can calculate the Lyapunov exponent, which is an index value, by chaos analysis.

[0079] The analysis unit 103 outputs concentration level information indicating the concentration level estimated based on the index value as brain activity information indicating the estimation result of the degree of brain activity. Furthermore, the analysis unit 103 outputs concentration level information indicating that the larger the index value, the higher the concentration level, and that the smaller the index value, the lower the concentration level.

[0080] This allows the user to understand the level of concentration based on the concentration level information.

[0081] The analysis unit 103 calculates the deviation of the pulse wave height based on the pulse wave shape, and if the deviation is smaller than a preset threshold, automatically adjusts the input signal amplification rate to increase the waveform of the pulse wave acquired by the Doppler sensor 10.

[0082] This allows the brain activity estimation device 1 to clarify the shape of the pulse wave, and improve the accuracy of analysis even if the distance from the Doppler sensor 10 to the human body is long.

[0083] Embodiment 2 The brain activity estimation device 1 of embodiment 1 estimates the degree of brain activity of the central nervous system by analyzing fluctuations in the pulse waveform. The brain activity estimation device 1 of embodiment 2 estimates the degree of activity of the autonomic nervous system based on pulse interval fluctuations in addition to the degree of brain activity. In other words, the brain activity estimation device 1 of embodiment 2 estimates both central nervous activity and autonomic nervous activity. The following description will focus on differences between embodiment 2 and embodiment 1, and configurations not described in embodiment 2 are the same as embodiment 1.

[0084] 11 is a block diagram showing the configuration of a brain activity estimation device 1 according to Embodiment 2 and a usage configuration of the brain activity estimation device 1. The Doppler sensor 10 detects the pulse of the human body in addition to the pulse wave of the human body.

[0085] The pulse interval is generally called the RR Interval (RRI). RRI is converted into various information, such as autonomic nervous balance, which will be discussed later, through frequency conversion. Vital sign analysis, which analyzes pulse, blood pressure, breathing, and other vital signs, analyzes minute fluctuations with ultra-low frequency characteristics of approximately 1 Hz, unlike body movement analysis. For this reason, analog detection using the 24 GHz Doppler method is preferable for vital sign analysis, compared to the 60 GHz to 79 GHz range that has high resolution and is often used for distance measurement.

[0086] The Doppler sensor 10 detects autonomic nervous balance from the variation in the pulse intervals of the human body (one-dimensional pattern pulse variation). Autonomic nervous balance is the balance between the sympathetic and parasympathetic nervous systems. Autonomic nervous balance is the ratio between LF (Low Frequency) and HF (High Frequency), calculated as LF / HF. LF indicates sympathetic nervous activity, and HF indicates parasympathetic nervous activity. It is said that the sympathetic nervous system is dominant during the day or in an active state, while the parasympathetic nervous system is dominant at night or in a sedated state.

[0087] LF is calculated by integrating the power in a low frequency band, for example, 0.05 Hz to 0.15 Hz, on the characteristic curve. HF is calculated by integrating the power in a high frequency band, for example, 0.15 Hz to 0.40 Hz, on the characteristic curve. A characteristic curve is a curve obtained by frequency-expanding the time series interval of the pulse interval, and is drawn on a coordinate axis with frequency on the horizontal axis and power on the vertical axis.

[0088] Since autonomic nervous balance is LF / HF, when LF is relatively large, it can be inferred that the sympathetic nervous system is dominant and the person is in an excited or active state, and when LF is relatively small, it can be inferred that the parasympathetic nervous system is dominant and the person is in a relaxed state. Also, when the autonomic nervous balance value is large, it can be inferred that the person is in an excited state, and when the autonomic nervous balance value is small, it can be inferred that the person is in a relaxed state. Therefore, autonomic nervous balance is used as an index showing the degree of activity of the autonomic nervous system. When the human body is in an excited or active state, the degree of activity of the autonomic nervous system is high and the autonomic nervous balance value is large. On the other hand, when the human body is in a relaxed state, the degree of activity of the autonomic nervous system is low and the autonomic nervous balance value is small.

[0089] In this way, the Doppler sensor 10 can detect the degree of activity of the autonomic nervous system, and can also detect the degree of brain activity of the central nervous system based on the pulse wave, as described above. In other words, the Doppler sensor 10 can detect both autonomic nervous activity and central nervous activity by itself.

[0090] Note that, like in the first embodiment, the sensor used in the brain activity estimation device 1 of the second embodiment is not limited to the Doppler sensor 10, but may be any sensor that can measure pulse waves and pulses. Here, the pulse that can be measured by the sensor refers to the pulse rate or pulse movement, and includes the magnitude of the pulse rate or a time-series increase or decrease in the pulse rate, the magnitude of the pulse interval or a time-series increase or decrease in the pulse interval, the magnitude of LF or a time-series increase or decrease in LF, and the magnitude of LF / HF or a time-series increase or decrease in LF / HF.

[0091] The analysis unit 103 estimates a person's emotions by applying both the degree of brain activity and the degree of activity of the autonomic nervous system to an emotion model created in advance.

[0092] A commonly used emotional model is the psychological model known as Russell's Circumplex of Emotions, shown in Figure 12 below.

[0093] Figure 12 shows Russell's emotional circumplex model. This emotional circumplex model is a two-axis plane with the vertical axis representing activation (arousal level) and the horizontal axis representing pleasantness (pleasantness), and arranges emotions such as happy, peaceful, bored, and tense on a circle. The vertical axis represents sleepiness-alertness, and the horizontal axis represents discomfort-pleasure.

[0094] This emotional cycle model allows a person's emotions to be estimated by combining the comfort level on the horizontal axis and the arousal level on the vertical axis.

[0095] Here, the "vertical axis of alertness" can be estimated from images acquired by imaging means or electroencephalograms obtained by an electroencephalograph. Also, TP (Total Power, unit [ms 2Previous techniques used TP (pulse width) as a measure of arousal. TP is the sum of the LF and HF powers obtained from one-dimensional pattern pulse displacement. One literature document describes the "vertical axis arousal" as entropy, which is calculated based on the LF and HF. These conventional methods were unable to accurately estimate arousal without contact. Accurate detection of the high-frequency component, HF, was particularly difficult due to the inclusion of respiratory and body movement components as noise. Furthermore, the high-frequency component, HF, faced challenges in terms of conversion accuracy and detection accuracy in identifying the frequency band between 0.15 Hz and 0.40 Hz. In technologies that use TP or entropy as arousal, the source data for estimating arousal is the pulse interval for both TP and entropy.

[0096] As described above, the arousal level on the vertical axis of the emotion model used to estimate emotions has traditionally been estimated from data obtained using imaging means or an electroencephalograph, as described above, or TP or entropy calculated from the autonomic nervous system, etc. Methods using imaging means or an electroencephalograph cannot estimate the index indicating the vertical axis of the emotion model with high accuracy in a short period of time.

[0097] In brain activity estimation device 1 of embodiment 2, the vertical axis of the emotion model used to estimate emotions uses the degree of brain activity of the central nervous system estimated based on chaos analysis using pulse wave shape displacement as the analysis source. In other words, brain activity estimation device 1 uses the degree of brain activity of the central nervous system estimated based on pulse waves as the index set on the vertical axis of the emotion model, and in this respect it differs significantly from conventional devices.

[0098] As described above, brain activity estimation device 1 can estimate the degree of autonomic nervous activity. For this reason, in the second embodiment, a unique emotion model is created in advance, with the vertical axis representing the "degree of central nervous brain activity" and the horizontal axis representing the "degree of autonomic nervous activity," and brain activity estimation device 1 estimates emotions based on this emotion model.

[0099] FIG. 13 is a diagram showing an example of an emotion model of the brain activity estimation device 1 according to the second embodiment. The vertical axis represents the degree of brain activity obtained from pulse wave shape fluctuations, which is central nervous activity. The horizontal axis represents pulse fluctuations, which is autonomic nervous system activity. In other words, the vertical axis represents the degree of brain activity of the central nervous system estimated based on two-dimensional pulse wave pattern shape variations (fluctuation patterns). The horizontal axis represents the degree of activity of the autonomic nervous system estimated based on one-dimensional pattern pulse rate variations (fluctuation patterns). In this emotion model, multiple emotions are displayed on a two-dimensional plane, with the vertical axis representing "concentration-sleepiness," which indicates the level of concentration or sleepiness, and the horizontal axis representing "relaxation-excitement-activity." In this way, the emotion model shows the relationship between emotions and the levels of brain activity and activity of the autonomic nervous system. Here, emotions refer to emotions resulting from, for example, concentration, sleepiness, fatigue, activity, and relaxation.

[0100] As shown in FIG. 9 above, the Lyapunov exponent can take on values ​​ranging from 0 to 15, for example. Therefore, in the emotion model of FIG. 13, for example, "drowsiness-concentration" on the vertical axis is assigned to Lyapunov exponents of 0-15. In other words, a Lyapunov exponent of 0 corresponds to "drowsiness," a Lyapunov exponent of 15 corresponds to "concentration," and an intermediate Lyapunov exponent of 7 corresponds to the position where the vertical axis intersects with the horizontal axis. Note that the assignment of Lyapunov exponents to the vertical axis described here is merely an example, and it is also possible to assign values ​​as a percentage of the Lyapunov exponent.

[0101] In addition, in the emotion model of Figure 13, the horizontal axis of "Relaxed - Excited / Active" is assigned according to the value of autonomic nervous balance. The larger the value of autonomic nervous balance, the closer the assigned position is to the "Excited / Active" side, and the smaller the value of autonomic nervous balance, the closer the assigned position is to the "Relaxed" side.

[0102] When the LF sympathetic nervous activity is high, people feel excitement, activity, good daytime stress, and discomfort. Also, when the pulse interval RRI is narrow, the pulse rate is high (beats per minute (bpm)), or the pulse interval deviation (RRI SD) is small, people feel excitement, activity, good daytime stress, and discomfort.

[0103] Conversely, when the sympathetic nervous system (LF) is low, people feel relaxed or comfortable. Furthermore, when the parasympathetic nervous system (HF) is high, the pulse interval is wide, the pulse rate is low, or the pulse interval deviation is large, people also feel relaxed or comfortable. The relationship between these autonomic nervous activity levels and emotions has long been known. It has also been confirmed that the Lyapunov exponent increases when performing tasks that require high levels of concentration. Furthermore, the Lyapunov exponent decreases when sleepiness is high. Concentration is enhanced by highly reactive central nervous activity, with little influence from pleasant or unpleasant emotions or the autonomic nervous system. The emotional model in Figure 13 was created based on these existing findings.

[0104] In the emotional model of Figure 13, when "high concentration" and "high relaxation" are present, emotions such as happiness or joy are assigned. When "high concentration" and "high excitement or active emotion" are present, emotions such as tension or progress are assigned. When "low concentration" and "high relaxation" are present, emotions such as relaxation, calmness, or tranquility are assigned. When "low concentration" and "high excitement or active emotion" are present, emotions such as depression or boredom are assigned. Note that the horizontal axis may represent comfort level, as in Russell's emotional circumplex model, and may be the axis of "pleasant-unpleasant."

[0105] Brain activity estimation device 1 estimates emotions using the above emotion model. The operation of brain activity estimation device 1 according to the second embodiment will be described below.

[0106] Brain activity estimation device 1 detects the pulse wave and pulse rate of the human body using Doppler sensor 10. Analysis unit 103 analyzes the pulse wave detected by Doppler sensor 10 to estimate the degree of brain activity of the central nervous system, and estimates the degree of activity of the autonomic nervous system from the pulse rate. Analysis unit 103 estimates emotions based on the degree of brain activity, the degree of activity of the autonomic nervous system, and an emotion model. Specifically, analysis unit 103 identifies the position of the vertical axis in the emotion model from the Lyapunov exponent, which indicates the degree of brain activity, and identifies the position of the horizontal axis in the emotion model from the autonomic nervous balance, which indicates the degree of activity of the autonomic nervous system.

[0107] The analysis unit 103 then applies the identified positions on the vertical axis and the horizontal axis to an emotion model to estimate the emotion, and outputs emotion information indicating the estimation result as the estimation result. The emotion information may be text such as joy, calm, melancholy, or tension, or a facial image from which the emotion can be identified.

[0108] Here, the "comfort level on the horizontal axis" can be determined by the activity level of the autonomic nervous system based on pulse fluctuations. Specifically, the activity level of the autonomic nervous system can be determined by the autonomic nervous balance obtained by the Doppler sensor 10 as described above. In other words, the "comfort level on the horizontal axis" can be determined based on vital data detected from the variation in the pulse intervals of the human body.

[0109] The "comfort level on the horizontal axis" does not necessarily have to be determined by the autonomic nervous balance, but may be determined by other data. The "comfort level on the horizontal axis" can also be determined by, for example, "changes or deviations in pulse interval," "changes or deviations in pulse rate," or "increases or decreases in or changes in balance of the LF of the autonomic nervous system described above." In any of these methods, the original data for determining the position of the "comfort level on the horizontal axis" is the pulse. In other words, the "comfort level on the horizontal axis" is determined by the pulse. When detecting the pulse interval, the analysis unit 103 eliminates the influence of noise, breathing, body movement, or other factors to accurately detect the pulse interval.

[0110] The emotional information output from the analysis unit 103 is input to the display unit 107 via the cloud unit 106 and displayed. In this way, the brain activity estimation device 1 outputs the emotional information from the analysis unit 103 and visualizes it, allowing the user to understand the emotion. The emotional information output from the analysis unit 103 may be accumulated in the data collection unit 108 via the cloud unit 106. Alternatively, the emotional information output from the analysis unit 103 may be input to the display unit 107 directly and displayed, without going through the cloud unit 106. Alternatively, the emotional information output from the analysis unit 103 may be input to the control content determination unit 104 and used to determine the control content of the equipment of the brain activity estimation device 1.

[0111] Humans operate their autonomic nervous system throughout the day, but it is primarily during the daytime when they are working or studying that the sympathetic nervous system, known as good stress, is in a high state. Furthermore, when the sympathetic nervous system is in a high state, it excites the brain, dilates the trachea, speeds up the heartbeat, constricts blood vessels, raises blood pressure, suppresses gastrointestinal activity, and promotes sweating. Therefore, a high sympathetic state of the autonomic nervous system is considered to be a good state for work and study. In other words, when the sympathetic nervous system in the LF is high, or when the autonomic nervous system balance ratio (LF / HF) is high, the human body is in a good state for work and study. Furthermore, when "concentration is high" and "excitement or active feelings are high," the human body is in a state of thriving.

[0112] In other words, when the level of concentration is high and the ratio of autonomic nervous system balance is high, it can be evaluated as the state most suitable for work or study, in other words, a state in which the human body's work efficiency is high. Therefore, when the level of brain activity is higher than a predetermined first threshold and the level of activity of the autonomic nervous system is higher than a predetermined second threshold, the analysis unit 103 may output work efficiency information indicating that the human body's work efficiency is high. In this way, in addition to outputting emotion information based on the emotion model, the analysis unit 103 may also output other information determined from the level of brain activity and the level of activity of the autonomic nervous system.

[0113] As described above, brain activity estimation device 1 of embodiment 2 can obtain the same effects as embodiment 1, and can estimate the activity level of the autonomic nervous system by detecting pulse with Doppler sensor 10. Therefore, brain activity estimation device 1 can estimate the activity level of the autonomic nervous system in addition to the brain activity level of the central nervous system, and can estimate emotions from the brain activity level of the central nervous system, the activity level of the autonomic nervous system, and an emotion model. Furthermore, because central nervous activity and autonomic nervous activity can be measured with a single Doppler sensor 10, brain activity estimation device 1 does not require wearing and can estimate emotions in a short time and over a long distance, compared to devices that use an electroencephalograph or imaging means.

[0114] Embodiment 3 The third embodiment relates to equipment including the brain activity estimation device 1 described in the first or second embodiment, and in particular, an air conditioner will be described here.

[0115] <Configuration of Air Conditioner 201> 14 is a diagram showing the configuration of an air conditioner 201 according to embodiment 3. The air conditioner 201 is equipment that conditions an indoor space 271, which is an air-conditioned space. Air conditioning refers to adjusting the temperature, humidity, cleanliness, airflow, etc. of the air in the air-conditioned space, and specifically includes heating, cooling, dehumidification, humidification, and air purification.

[0116] As shown in Fig. 14, air conditioner 201 is installed in house 203. Air conditioner 201 is a heat pump type air conditioning facility that uses, for example, HFC (hydrofluorocarbon) as a refrigerant. Air conditioner 201 is equipped with a vapor compression type refrigerant circuit and operates by obtaining power from a commercial power source, power generation equipment, or power storage equipment (not shown).

[0117] 14, air conditioner 201 includes outdoor unit 211 provided outside house 203, indoor unit 213 provided inside house 203, and remote controller 255 operated by a user. Outdoor unit 211 and indoor unit 213 are connected via refrigerant piping 261 through which refrigerant flows and communication line 263 through which various signals are transferred. Air conditioner 201 cools indoor space 271 by blowing conditioned air, for example, cool air, from indoor unit 213, and heats indoor space 271 by blowing warm air.

[0118] The outdoor unit 211 includes a compressor 221, a four-way valve 222, an outdoor heat exchanger 223, an expansion valve 224, an outdoor fan 231, and an outdoor unit control unit 251. The indoor unit 213 includes an indoor heat exchanger 225, an indoor fan 233, and an indoor unit control unit 252. A refrigerant pipe 261 connects the compressor 221, the four-way valve 222, the outdoor heat exchanger 223, the expansion valve 224, and the indoor heat exchanger 225 in a ring shape. The air conditioner 201 has a refrigerant circuit configured by connecting the compressor 221, the four-way valve 222, the outdoor heat exchanger 223, the expansion valve 224, and the indoor heat exchanger 225 by the refrigerant pipe 261. The refrigerant circuit circulates refrigerant to perform a refrigeration cycle operation.

[0119] The compressor 221 compresses the refrigerant and circulates it through the refrigerant pipe 261. Specifically, the compressor 221 compresses a low-temperature, low-pressure refrigerant and discharges the high-pressure, high-temperature refrigerant to the four-way valve 222. The compressor 221 is equipped with an inverter circuit that can change the operating capacity according to the drive frequency. The operating capacity is the amount of refrigerant that the compressor 221 delivers per unit time. The compressor 221 changes the operating capacity according to instructions from the outdoor unit control unit 251.

[0120] The four-way valve 222 is installed on the discharge side of the compressor 221. The four-way valve 222 switches the flow direction of the refrigerant in the refrigerant pipe 261 depending on whether the air conditioner 201 is operating in cooling operation, dehumidification operation, or heating operation. The expansion valve 224 is installed between the outdoor heat exchanger 223 and the indoor heat exchanger 225, and reduces the pressure of the refrigerant flowing through the refrigerant pipe 261 to expand it. The expansion valve 224 is an electronic expansion valve whose opening degree can be controlled to be variable. The expansion valve 224 changes its opening degree in accordance with instructions from the outdoor unit control unit 251 to adjust the pressure of the refrigerant.

[0121] The outdoor heat exchanger 223 exchanges heat between the refrigerant flowing through the refrigerant piping 261 and the air in an outdoor space (external space) 272 that is outside the indoor space 271. The outdoor fan 231 is provided near the outdoor heat exchanger 223, draws in air from the outdoor space 272, and sends the drawn-in air to the outdoor heat exchanger 223. The air sent to the outdoor heat exchanger 223 exchanges heat with the refrigerant flowing through the refrigerant piping 261, and is then blown out into the outdoor space 272.

[0122] The indoor heat exchanger 225 exchanges heat between the refrigerant flowing through the refrigerant piping 261 and the air in the indoor space 271. The indoor blower 233 is provided near the indoor heat exchanger 225, draws in air from the indoor space 271, and sends the drawn-in air to the indoor heat exchanger 225. The air sent to the indoor heat exchanger 225 exchanges heat with the refrigerant flowing through the refrigerant piping 261, and is then blown out into the indoor space 271. The air that has undergone heat exchange in the indoor heat exchanger 225 is supplied to the indoor space 271 as conditioned air. In this way, the indoor space 271 is conditioned.

[0123] The outdoor unit control unit 251 controls the operation of the outdoor unit 211. The indoor unit control unit 252 controls the operation of the indoor unit 213.

[0124] A remote controller 255 is placed in the indoor space 271. The remote controller 255 transmits and receives various signals to and from an indoor unit control unit 252 provided in the indoor unit 213. The remote controller 255 is provided with a display unit 255a as shown in FIG. 15 described later. The display unit 255a is provided with a touch screen, a liquid crystal display, an LED (Light Emitting Diode), and the like. 255 is equipped with a push button (not shown). Remote controller 255 functions as a command receiving unit that receives various commands from the user and as a display unit that displays various information to the user. The user inputs commands to air conditioner 201 by operating remote controller 255. The commands are, for example, a command to switch between operation and stop, or a command to switch the operation mode, set temperature, set humidity, air volume, air direction, or timer. Air conditioner 201 operates in accordance with the input command.

[0125] 15 is a block diagram of an air conditioner 201 according to Embodiment 3. The air conditioner 201 includes a control device 250, an air conditioning unit 280, and the brain activity estimation device 1 according to Embodiment 1. An information device 290 operated by a user is connected to the air conditioner 201 via a network N.

[0126] The control device 250 controls the entire air conditioner 201. The control device 250 also controls the operation of the device itself, in other words, the air conditioner 280, based on the degree of brain activity estimated by the brain activity estimation device 1. The control device 250 includes the outdoor unit control unit 251 and the indoor unit control unit 252 described above. Although not shown in FIG. 15 , the control device 250 also includes the control content determination unit 104 and the device control unit 105 described in the first embodiment. The control content determination unit 104 and the device control unit 105 may be provided in either the outdoor unit control unit 251 or the indoor unit control unit 252.

[0127] The outdoor unit control unit 251 includes a control unit 251a, a storage unit 251b, a clock unit 251c, and a communication unit 251d, which are connected to each other via a bus (not shown).

[0128] The control unit 251a controls the entire outdoor unit. The storage unit 251b is composed of memory such as RAM or ROM, and stores data necessary for control. The timing unit 251c is a unit that measures time. The communication unit 251d is an interface for communicating with the indoor unit control unit 252 via a communication line 263 (see FIG. 14).

[0129] 14, the outdoor unit control unit 251 is connected to the indoor unit control unit 252 via a communication line 263. The outdoor unit control unit 251 cooperates with the outdoor unit control unit 251 by receiving various signals from the indoor unit control unit 252 via the communication line 263.

[0130] The indoor unit control unit 252 includes a communication unit 252a that communicates with the outdoor unit control unit 251 and the remote controller 255. The communication unit 252a is an interface for communicating with the outdoor unit control unit 251 and the remote controller 255. The communication unit 252a is further connected to the information device 290 via the network N. The communication unit 252a performs a process of receiving various commands from the user from the remote controller 255 and a process of transmitting the various commands received from the remote controller 255 to the indoor unit control unit 252. The communication unit 252a also performs a process of transmitting notification information to the remote controller 255 to notify the user.

[0131] The outdoor unit control unit 251 and the indoor unit control unit 252 are configured with a microprocessor unit. The outdoor unit control unit 251 and the indoor unit control unit 252 are equipped with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and control programs and the like are stored in the ROM. Note that the outdoor unit control unit 251 and the indoor unit control unit 252 are not limited to microprocessor units. For example, the outdoor unit control unit 251 and the indoor unit control unit 252 may be configured with updatable firmware, etc. Furthermore, the outdoor unit control unit 251 and the indoor unit control unit 252 may be program modules that are executed by commands from a CPU or the like (not shown). Furthermore, although an example has been shown in which the control device 250 has the outdoor unit control unit 251 and the indoor unit control unit 252 and is configured separately for the outdoor unit 211 and the indoor unit 213, they may also be configured with a single control unit that has both functions.

[0132] The air conditioning unit 280 is a part that conditions the air in the indoor space 271, and corresponds to the refrigerant circuit, the outdoor fan 231, and the indoor fan 233 in FIG.

[0133] The air conditioner 201 configured as described above controls the operation of the air conditioner 280 based on the level of brain activity estimated by the brain activity estimation device 1. That is, the control device 250 controls the operation of the air conditioner 280 according to the level of concentration based on the level of brain activity. Specifically, for example, when the level of concentration is low, the control device 250 performs the following control to wake up the user. It has been previously confirmed that exposure to wind wakes up users. Therefore, when the level of concentration is low, the control device 250 increases the rotation speed of the indoor blower 233 to increase the amount of air blown.

[0134] Furthermore, if the air conditioner 201 is equipped with a human sensor (not shown) that detects the position of the user, the control device 250 may perform the following control. The control device 250 swings an up-and-down air deflector (not shown) provided on the indoor unit 213 up and down so that air intermittently hits the user, or swings a left-and-right air deflector (not shown) left and right so that air intermittently hits the user. Through the above control, the air conditioner 201 can encourage the user to wake up when the level of concentration is low.

[0135] It is also said that a temperature about 1°C lower than what the user feels is comfortable has the effect of cooling the brain and improving work efficiency. In particular, when heating, it is said that a set temperature that is too high reduces concentration. For this reason, when the concentration level is low, the control device 250 controls the room set temperature to be slightly lower than the current temperature.

[0136] Conversely, when the level of concentration is high, the control device 250 reduces the rotation speed of the indoor blower 233 to reduce the amount of air blown, or controls the vertical air deflectors to point as upward as possible to prevent the wind from blowing on the user. Furthermore, when the air conditioner 201 is equipped with a human sensor (not shown), the control device 250 controls one or both of the vertical air deflectors (not shown) and the horizontal air deflectors (not shown) to prevent the wind from blowing on the user. Through the above control, the air conditioner 201 can prevent the user's attention from being directed toward the wind, which would otherwise cause a decrease in the level of concentration. In other words, the air conditioner 201 has the effect of allowing a user who is highly concentrated to work without being conscious of the wind.

[0137] Meanwhile, the remote controller 255 is positioned as one of the components of the air conditioner 201, while the information device 290 is positioned as a device owned by the user. The information device 290 has a display unit 281 such as a liquid crystal panel, and various information is displayed on the display unit 281. The information device 290 is configured as, for example, a smartphone or a tablet. An application for displaying the concentration level of the human body and the like is installed in the information device 290. Furthermore, the information device 290 can also be used in place of the remote controller 255 by installing an application for air conditioning control.

[0138] When operated by the user, the information device 290 launches an application, acquires information related to the degree of brain activity estimated by the brain activity estimation device 1 via the network N, and displays the information on the display unit 281. The information related to the degree of brain activity may be information indicating the degree of brain activity, concentration information related to the concentration level, or emotion information related to emotions. The concentration information is information indicating the current concentration level, the time-series concentration level, or both. As a specific control, the control device 250 of the air conditioner 201 transmits information related to the degree of brain activity obtained by the brain activity estimation device 1 to the information device 290 via the communication unit 252a in response to a request from the information device 290, and performs processing to display the information on the information device 290. Note that although the information related to the degree of brain activity is displayed on the information device 290 here, it may also be displayed on the display unit 255a of the remote controller 255.

[0139] In this way, the air conditioner 201 displays and visualizes the concentration level information on the information device 290 or the display unit 255a of the remote controller 255, allowing the user to visually confirm the concentration level.

[0140] Here, the device equipped with the brain activity estimation device 1 has been described as being an air conditioner 201, but the device is not limited to the air conditioner 201 and can be incorporated into various devices such as electrical equipment, cars, or entertainment equipment. The brain activity estimation device 1 can also be incorporated into, for example, a labor management device or a learning management device.

[0141] Furthermore, the brain activity estimation device 1 can be incorporated into devices in a wide range of fields, such as healthcare, labor, education, sleep, mindfulness, meditation, customer service, marketing, or sports mental training. When the brain activity estimation device 1 is incorporated into these devices, the devices can control the devices using the brain activity determination results and present the brain activity determination results to the user. In this way, by applying the brain activity estimation device 1 to various devices, the degree of brain activity, concentration level, and emotions can be visualized and presented to the user of the device. [Explanation of symbols]

[0142] 1 Brain activity estimation device, 10 Doppler sensor, 10a board unit, 100 antenna unit, 100a antenna, 101 wireless unit, 102 analog circuit unit, 103 analysis unit, 104 control content determination unit, 105 equipment control unit, 106 cloud unit, 107 display unit, 108 data collection unit, 201 air conditioning device, 203 house, 211 outdoor unit, 213 indoor unit, 221 compressor, 222 four-way valve, 223 outdoor heat exchanger, 224 expansion valve, 225 indoor heat exchanger, 231 outdoor blower, 233 indoor blower, 250 control device, 251 outdoor unit control unit, 251a control unit, 251b memory unit, 251c timing unit, 251d communication unit, 252 indoor unit control unit, 252a Communication unit, 255 remote controller, 255a display unit, 261 refrigerant piping, 263 communication line, 271 indoor space, 272 outdoor space, 280 air conditioning unit, 281 display unit, 290 information equipment, 300 air conditioning system.

Claims

[Claim 1] A Doppler sensor that detects the human pulse wave without contact, an analysis unit that analyzes the pulse wave signal output from the Doppler sensor, The analysis unit a first step of calculating a vector specified from time series data of the pulse wave signal height and a preset delay time; a second step of generating an attractor by arranging the vectors in a time-series order in a three-dimensional or more state space; a third step of calculating Lyapunov exponents based on the trajectory of the attractor; and estimating the degree of brain activity of the human body in a task or behavior based on the Lyapunov exponents, Before performing the chaos analysis, A brain activity estimation device that calculates a deviation in height of the pulse wave signal, and when the deviation is smaller than a preset threshold, adjusts an input signal amplification rate to be large to increase the height of the pulse wave signal, and then performs the chaos analysis based on the increased pulse wave signal.

Images