Biological information processing device and biological information processing system

Abstract

To provide a biometric information processing apparatus capable of reducing mismatch, and a biometric information processing system.SOLUTION: A biometric information processing apparatus includes a derivation unit and a classification unit. The derivation unit derives emotion information on an object living body based on at least one of biometric information and behavior information which are obtained from the object living body executing a specific task. The classification unit classifies the emotion information obtained by the derivation unit on the basis of a predetermined classification index.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a biological information processing device and a biological information processing system. [Background technology]

[0002] Acquiring quality human resources is essential for any organization to prosper. However, identifying good human resources is difficult. Conventionally, when selecting people for recruitment activities or team building within an organization, people's intuition, experience, and subjectivity have often been used, or attribute information such as age, gender, and educational background has been used. Examples of prior art that judges people based on their attributes include Patent Document 1 below. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-101720 Summary of the Invention [Problem to be solved by the invention]

[0004] When relying on human intuition, experience, and subjectivity, decisions are often made based on the interviewer's subjective opinion during a short interview. Such short subjective confirmations can lead to oversights or the interviewer's field of expertise being different from that of the desired candidate, resulting in a hiring mismatch. Also, when an organization forms a team to run a project, members may be selected based solely on superficial expertise or the superior's subjective opinion, which can lead to the same mismatch described above.

[0005] Even when attribute information such as a person's age, gender, and educational background is used, the judgment does not reflect the individual's objective abilities, and there is a possibility of a uniform judgment based on age, etc., resulting in lost opportunities. Furthermore, such mismatches can occur not only in the hiring of people and the selection of project members, but also in the selection of various non-human living organisms, for example. Therefore, it is desirable to provide a biometric information processing device and a biometric information processing system that can reduce mismatches. [Means for solving the problem]

[0006] A biological information processing device according to a first aspect of the present disclosure includes a derivation unit and a classification unit. The derivation unit derives emotional information of a target living organism based on at least one of biological information and behavioral information obtained from the target living organism while the target living organism is performing a specific task. The classification unit classifies the emotional information obtained by the derivation unit based on a predetermined classification index.

[0007] A biological information processing system according to a second aspect of the present disclosure includes an acquisition unit, a derivation unit, and a classification unit. The acquisition unit acquires at least one of biological information and behavioral information from a target living organism while the target living organism is performing a specific task. The derivation unit derives emotional information of the target living organism based on the information acquired by the acquisition unit. The classification unit classifies the emotional information acquired by the derivation unit based on a predetermined classification index.

[0008] In a biometric information processing device according to a first aspect of the present disclosure and a biometric information processing system according to a second aspect of the present disclosure, emotional information is classified based on a predetermined classification index. This allows target living organisms to be classified using emotional information, which is objective data. As a result, for example, in recruiting personnel, it becomes possible to determine whether an applicant is a desirable candidate based on the applicant's emotional information. Furthermore, for example, in deciding project members, it becomes possible to determine whether a member is suitable to form a specific group based on the emotional information of multiple members.

[0009] A biological information processing device according to a third aspect of the present disclosure includes a storage unit and a derivation unit. The derivation unit derives emotional information of a target living organism based on at least one of biological information and behavioral information obtained from the target living organism while the target living organism is performing a specific task. The derivation unit further associates the derived emotional information with an identifier of the target living organism and stores the emotional information in the storage unit.

[0010] A biological information processing system according to a fourth aspect of the present disclosure includes a storage unit, an acquisition unit, and a derivation unit. The acquisition unit acquires at least one of biological information and behavioral information from a target living organism while the target living organism is performing a specific task. The derivation unit derives emotional information of the target living organism based on the information acquired by the acquisition unit. The derivation unit further associates the derived emotional information with an identifier of the target living organism and stores it in the storage unit.

[0011] In a biometric information processing device according to a third aspect of the present disclosure and a biometric information processing system according to a fourth aspect of the present disclosure, the derived emotional information is stored in a storage unit in association with an identifier of the target living organism. This allows the target living organism to be classified using the emotional information, which is objective data. As a result, for example, in a recruitment situation, it becomes possible to determine whether an applicant is a desirable candidate based on the applicant's emotional information. Furthermore, for example, in a situation where project members are being selected, it becomes possible to determine whether a member is suitable to form a specific group based on the emotional information of multiple members. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a biological information processing system according to a first embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of the electronic device of FIG. [Figure 3] FIG. 10 is a diagram illustrating an example of the relationship between task processing time and alertness. [Figure 4] This is a diagram showing the relationship between duration and rise time classified into four categories. [Figure 5] FIG. 10 is a diagram illustrating an example of a display screen. [Figure 6] FIG. 10 is a diagram illustrating an example of a display screen. [Figure 7] FIG. 10 is a diagram illustrating an example of a display screen. [Figure 8] FIG. 2 is a diagram illustrating an example of a processing procedure in the biological information processing system of FIG. [Figure 9] FIG. 10 is a diagram illustrating an example of a schematic configuration of a biological information processing system according to a second embodiment of the present disclosure. [Figure 10] 10 is a diagram illustrating an example of functional blocks of the electronic device of FIG. 9. FIG. [Figure 11] FIG. 10 is a diagram illustrating an example of a processing procedure in the biological information processing system of FIG. [Figure 12] FIG. 10 is a diagram illustrating an example of a schematic configuration of an information processing system according to a third embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram illustrating an example of functional blocks of the electronic device of FIG. [Figure 14] FIG. 13 is a diagram illustrating an example of functional blocks of the electronic device of FIG. [Figure 15] FIG. 13 is a diagram illustrating an example of a processing procedure in the information processing system of FIG. [Figure 16] FIG. 10 is a diagram illustrating an example of a schematic configuration of an information processing device according to a fourth embodiment of the present disclosure. [Figure 17] 17A and 17B are diagrams illustrating examples in which an estimation model is used in the biological information processing systems of FIGS. 1 and 9, the information processing system of FIG. 12, and the information processing device of FIG. [Figure 18] 17A and 17B are diagrams illustrating examples in which attribute information is used in the biometric information processing systems of FIGS. 1 and 9, the information processing system of FIG. 12, and the information processing device of FIG. [Figure 19] FIG. 10 is a diagram illustrating an example of time-series data of reaction times to questions of low difficulty. [Figure 20] FIG. 10 is a diagram illustrating an example of time-series data of reaction times to questions of high difficulty. [Figure 21]FIG. 10 is a diagram showing an example of power spectrum density obtained by performing an FFT (Fast Fourier Transform) on observed data of a user's electroencephalogram (α waves) when solving a low-difficulty problem. [Figure 22] FIG. 10 is a diagram showing an example of power spectrum density obtained by performing an FFT (Fast Fourier Transform) on observed data of a user's electroencephalogram (α waves) when solving a highly difficult problem. [Figure 23] FIG. 10 is a diagram showing an example of the relationship between task differences in reaction time variability and task differences in peak values ​​of electroencephalogram power in the low frequency band. [Figure 24] FIG. 10 is a diagram illustrating an example of the relationship between task differences in reaction time variability and task differences in accuracy rates. [Figure 25] FIG. 10 is a diagram illustrating an example of the relationship between the task difference in alertness and the task difference in the peak value of the power of electroencephalograms in the low frequency band. [Figure 26] FIG. 10 is a diagram illustrating an example of the relationship between a task difference in arousal level and a task difference in accuracy rate. [Figure 27] FIG. 10 is a diagram illustrating an example of the relationship between the variation in reaction time and the accuracy rate. [Figure 28] FIG. 10 is a diagram illustrating an example of the relationship between the level of awakening and the accuracy rate. [Figure 29] FIG. 1 is a diagram illustrating an example of a head-mounted display equipped with a sensor. [Figure 30] FIG. 10 is a diagram illustrating an example of a headband equipped with a sensor. [Figure 31] FIG. 1 is a diagram illustrating an example of headphones equipped with a sensor. [Figure 32] FIG. 1 is a diagram illustrating an example of an earphone equipped with a sensor. [Figure 33] FIG. 1 is a diagram illustrating an example of a watch equipped with a sensor. [Figure 34] FIG. 1 is a diagram illustrating an example of glasses equipped with a sensor. [Figure 35] FIG. 10 is a diagram showing an example of the relationship between the task difference of pnn50 of the pulse wave and the accuracy rate. [Figure 36]FIG. 10 is a diagram illustrating an example of the relationship between the task difference in the variation of pnn50 of the pulse wave and the accuracy rate. [Figure 37] FIG. 10 is a diagram showing an example of the relationship between the task difference in pnn50 power of the pulse wave in the low frequency band and the accuracy rate. [Figure 38] FIG. 10 is a diagram showing an example of the relationship between the task difference in rmssd of the pulse wave and the accuracy rate. [Figure 39] FIG. 10 is a diagram showing an example of the relationship between the task difference in rmssd variability of the pulse wave and the accuracy rate. [Figure 40] This figure shows an example of the relationship between the task difference in rmssd power of the low-frequency band pulse wave and the accuracy rate. [Figure 41] FIG. 10 is a graph showing an example of the relationship between the task difference in the variability of the number of SCRs for mental sweating and the accuracy rate. [Figure 42] FIG. 10 is a graph showing an example of the relationship between the task difference in the number of SCRs for psychological sweating and the accuracy rate. [Figure 43] FIG. 10 is a diagram illustrating an example of the relationship between task difference in median reaction time and accuracy rate. [Figure 44] FIG. 10 is a diagram illustrating an example of the relationship between the level of awakening and the accuracy rate. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0014] <1. About alertness> A person's level of alertness is closely related to their ability to concentrate. People perform at a high level when they are concentrating. Therefore, by knowing a person's level of alertness, it is possible to estimate a person's objective ability. A person's level of alertness can be derived based on biological information or behavioral information obtained from a person (hereinafter referred to as the "target organism") while performing a specific task.

[0015] Examples of biological information from which the level of arousal of a subject living body can be derived include electroencephalograms, sweating, pulse waves, electrocardiograms, blood flow, skin temperature, facial myoelectric potentials, electrooculography, or information on specific components contained in saliva.

[0016] (EEG) It is known that alpha waves contained in electroencephalograms increase when relaxed, such as when at rest, and beta waves contained in electroencephalograms increase when one is actively thinking or concentrating. Therefore, for example, when the power spectrum area of ​​the frequency band of alpha waves contained in electroencephalograms is smaller than a predetermined threshold th1 and the power spectrum area of ​​the frequency band of beta waves contained in electroencephalograms is larger than a predetermined threshold th2, it is possible to estimate that the level of arousal of the target living organism is high.

[0017] Furthermore, when estimating the arousal level of a target living organism using electroencephalograms, an estimation model such as machine learning can be used instead of the thresholds th1 and th2. This estimation model is, for example, a model trained using the power spectrum of electroencephalograms when the arousal level is clearly high as training data. When the power spectrum of electroencephalograms is input, for example, this estimation model estimates the arousal level of the target living organism based on the input power spectrum of electroencephalograms. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0018] Alternatively, the electroencephalogram may be divided into a plurality of segments on the time axis, a power spectrum may be derived for each divided segment, and the power spectrum area of ​​the α wave frequency band may be derived for each derived power spectrum. In this case, for example, if the derived power spectrum area is smaller than a predetermined threshold tha, it can be estimated that the level of arousal of the target living organism is high.

[0019] It is also possible to estimate the level of arousal of a target living organism using, for example, an estimation model that estimates the level of arousal of a target living organism based on the derived power spectrum area. This estimation model is, for example, a model that is trained using a power spectrum area when the level of arousal is clearly high as training data. When, for example, a power spectrum area is input, this estimation model estimates the level of arousal of the target living organism based on the input power spectrum area. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0020] (sweating) Psychological sweating is sweating released from the eccrine glands when the sympathetic nervous system is tense due to mental or psychological issues such as stress, tension, or anxiety. For example, by attaching a sweat meter probe to the palms or soles of the hands or feet and measuring the sweating (psychological sweating) of the palms or soles induced by various stress stimuli, the sympathetic sweating response (SSwR) can be obtained as a signal voltage. When the values ​​of certain high-frequency and low-frequency components in this signal voltage are higher than a predetermined threshold, it is possible to estimate that the target organism's level of arousal is high.

[0021] Furthermore, it is also possible to estimate the level of arousal of a target living organism using an estimation model that estimates the level of arousal of a target living organism based on a predetermined high-frequency component or a predetermined low-frequency component contained in the signal voltage. This estimation model is, for example, a model trained using a predetermined high-frequency component or a predetermined low-frequency component contained in the signal voltage when the level of arousal is clearly high as training data. When a predetermined high-frequency component or a predetermined low-frequency component is input, this estimation model estimates the level of arousal of the target living organism based on the input predetermined high-frequency component or predetermined low-frequency component. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0022] (pulse wave, electrocardiogram, blood flow) A high heart rate is generally considered to indicate a high level of arousal. The heart rate can be derived from a pulse wave, an electrocardiogram, or a blood flow velocity. Therefore, for example, if the heart rate is derived from a pulse wave, an electrocardiogram, or a blood flow velocity, and the derived heart rate is greater than a predetermined threshold, it can be estimated that the level of arousal of the target living body is high.

[0023] It is also possible to estimate the level of arousal of a target living organism using an estimation model that estimates the level of arousal of a target living organism based on, for example, a heart rate derived from a pulse wave, an electrocardiogram, or a blood flow velocity. This estimation model is, for example, a model trained using heart rates at which the level of arousal is clearly high as training data. When, for example, a heart rate derived from a pulse wave, an electrocardiogram, or a blood flow velocity is input, this estimation model estimates the level of arousal of the target living organism based on the input heart rate. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0024] Furthermore, when the heart rate variability (HRV) is small, it is generally said that the parasympathetic nervous system is subordinated and the level of arousal is high. Therefore, for example, by deriving the heart rate variability (HRV) from the pulse wave, electrocardiogram, or blood flow velocity, it is possible to estimate that the level of arousal of the target living body is high when the derived heart rate variability (HRV) is smaller than a predetermined threshold.

[0025] It is also possible to estimate the level of arousal of a target living organism using an estimation model that estimates the level of arousal of a target living organism based on, for example, heart rate variability (HRV) derived from a pulse wave, an electrocardiogram, or blood flow velocity. This estimation model is, for example, a model trained using heart rate variability (HRV) when the level of arousal is clearly high as training data. When, for example, heart rate variability (HRV) derived from a pulse wave, an electrocardiogram, or blood flow velocity is input, this estimation model estimates the level of arousal of the target living organism based on the input heart rate variability (HRV). This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0026] (skin temperature) It is generally said that a high skin temperature indicates a high level of arousal. Skin temperature can be measured, for example, by thermography. Therefore, for example, when the skin temperature measured by thermography is higher than a predetermined threshold, it is possible to estimate that the level of arousal of the target living body is high.

[0027] It is also possible to estimate the arousal level of a target living organism using, for example, an estimation model that estimates the arousal level of a target living organism based on skin temperature. This estimation model is, for example, a model that is trained using skin temperatures at times when the arousal level is clearly high as training data. When, for example, a skin temperature is input, this estimation model estimates the arousal level of the target living organism based on the input skin temperature. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0028] (Facial myoelectric potential) It is known that the corrugator supercilii, which frowns when thinking, shows high activity. It is also known that the zygomaticus major muscle does not change much when imagining happy thoughts. In this way, it is possible to estimate emotion and arousal level based on facial regions. Therefore, for example, by measuring the facial myoelectric potential of a specific region and finding that the measured value is higher than a specific threshold, it is possible to estimate the level of arousal level of the target living organism.

[0029] It is also possible to estimate the level of arousal of a target organism using, for example, an estimation model that estimates the level of arousal of a target organism based on facial myoelectric potentials. This estimation model is, for example, a model trained using, as training data, facial myoelectric potentials when the level of arousal is clearly high. When, for example, facial myoelectric potentials are input, this estimation model estimates the level of arousal of the target organism based on the input facial myoelectric potentials. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0030] (eye electroencephalogram) A method for measuring eye movement is known, taking advantage of the fact that the corneal side of the eye is positively charged and the retina side is negatively charged. The measurement value obtained using this measurement method is an electrooculogram. For example, it is possible to estimate eye movement from the acquired electrooculogram, and if the estimated eye movement follows a predetermined trend, it is possible to estimate the level of arousal of the target living organism.

[0031] It is also possible to estimate the level of arousal of a target living organism using, for example, an estimation model that estimates the level of arousal of a target living organism based on an electrooculogram. This estimation model is, for example, a model that is trained using electrooculograms obtained when the level of arousal is clearly high as training data. When, for example, an electrooculogram is input, this estimation model estimates the level of arousal of the target living organism based on the input electrooculogram. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0032] (saliva) Saliva contains cortisol, a type of stress hormone. It is known that the amount of cortisol in saliva increases when a subject is under stress. Therefore, for example, when the amount of cortisol in saliva is higher than a predetermined threshold, it can be estimated that the subject's level of arousal is high.

[0033] It is also possible to estimate the level of arousal of a target organism using an estimation model that estimates the level of arousal of a target organism based on the amount of cortisol contained in saliva. This estimation model is a model that is trained using, for example, the amount of cortisol contained in saliva when the level of arousal is clearly high as training data. When the amount of cortisol contained in saliva is input, for example, this estimation model estimates the level of arousal of the target organism based on the input amount of cortisol. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0034] On the other hand, examples of behavioral information from which the arousal level of the target living organism can be derived include information on facial expressions, voice, blinking, breathing, or reaction times of behavior.

[0035] (Facial expression) It is known that the zygomaticus major muscle does not change much when one is frowning while thinking, or when one is imagining happy things. In this way, it is possible to estimate emotions and arousal levels based on facial expressions. For example, by capturing a face image with a camera and estimating facial expressions based on the resulting video data, it is possible to estimate the level of arousal of a target living organism based on the estimated facial expressions.

[0036] It is also possible to estimate the level of arousal of a target living organism using an estimation model that estimates the level of arousal of a target living organism based on video data of facial expressions. This estimation model is a model that is trained using video data of facial expressions clearly showing high levels of arousal as training data. When video data of facial expressions is input, this estimation model estimates the level of arousal of the target living organism based on the input video data. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0037] (audio) It is known that voice, like facial expressions, changes depending on emotions and arousal levels. Therefore, for example, it is possible to acquire voice data using a microphone and estimate the level of arousal level of a target living organism based on the acquired voice data.

[0038] It is also possible to estimate the level of arousal of a target living organism using, for example, an estimation model that estimates the level of arousal of a target living organism based on audio data. This estimation model is, for example, a model trained using audio data from when the level of arousal is clearly high as training data. When audio data is input, for example, this estimation model estimates the level of arousal of the target living organism based on the input audio data. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0039] (blinks) It is known that blinking, like facial expressions, changes depending on emotions and arousal levels. Therefore, for example, blinking can be captured with a camera, the blinking frequency can be measured based on the resulting video data, and the level of arousal level of the target organism can be estimated based on the measured blinking frequency. It is also possible to measure blinking frequency from an electrooculogram, and the level of arousal level of the target organism can be estimated based on the measured blinking frequency.

[0040] It is also possible to estimate the level of arousal of a target living organism using an estimation model that estimates the level of arousal of a target living organism based on, for example, video data of blinking or an electrooculogram. This estimation model is a model that is trained using, for example, video data of blinking captured when the level of arousal is clearly high, or an electrooculogram, as training data. When, for example, video data of blinking or an electrooculogram is input, this estimation model estimates the level of arousal of the target living organism based on the input video data or electrooculogram. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0041] (breathing) It is known that breathing, like facial expressions, changes depending on emotions and arousal levels. Therefore, for example, by measuring the breathing volume or breathing rate, it is possible to estimate the level of arousal of a target living body based on the measurement data obtained.

[0042] It is also possible to estimate the level of arousal of a target living organism using, for example, an estimation model that estimates the level of arousal of a target living organism based on the respiration volume or respiration rate. This estimation model is, for example, a model trained using the respiration volume or respiration rate when the level of arousal is clearly high as training data. When, for example, the respiration volume or respiration rate is input, this estimation model estimates the level of arousal of the target living organism based on the input respiration volume or respiration rate. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0043] (behavioral reaction time) It is known that the processing time (reaction time) and variability of the processing time (reaction time) when a person sequentially processes multiple tasks depend on the person's level of arousal. Therefore, for example, by measuring the processing time (reaction time) and variability of the processing time (reaction time), it is possible to estimate the level of arousal of the target organism based on the measurement data obtained.

[0044] 19 and 20 are graphs showing the time (reaction time) required for a user to answer a number of questions in succession. FIG. 19 shows a graph showing the time when a relatively low-difficulty question was answered, and FIG. 20 shows a graph showing the time when a relatively high-difficulty question was answered. FIG. 21 shows the power spectrum density obtained by performing an FFT (Fast Fourier Transform) on the observed data of a user's electroencephalogram (α waves) when the user answered a number of low-difficulty questions in succession. FIG. 22 shows the power spectrum density obtained by performing an FFT on the observed data of a user's electroencephalogram (α waves) when the user answered a number of high-difficulty questions in succession. FIG. 21 and 22 show graphs obtained by measuring electroencephalograms (α waves) in segments of about 20 seconds and performing an FFT with an analysis window of about 200 seconds.

[0045] 19 and 20 show that when solving high-difficulty problems, not only is reaction time longer, but the variance in reaction time also increases compared to when solving low-difficulty problems. From FIGS. 21 and 22, it can be seen that when solving high-difficulty problems, the power of brain waves (α waves) around 0.01 Hz is greater and the power of brain waves (α waves) around 0.02 to 0.04 Hz is smaller compared to when solving low-difficulty problems. In this specification, the power of brain waves (α waves) around 0.01 Hz is appropriately referred to as "slow (low-frequency band) brain wave (α waves) fluctuations."

[0046] Figure 23 shows the task difference Δtv [s] in the variance of the user's reaction time (75% percentile-25% percentile) when solving a high-difficulty problem and a low-difficulty problem, and the task difference ΔP [(mV 2 / Hz) 2 / Hz]. The task difference Δtv[s] is a vector quantity obtained by subtracting the variability in the user's reaction time when solving a low-difficulty problem from the variability in the user's reaction time when solving a high-difficulty problem. The task difference ΔP is a vector quantity obtained by subtracting the peak value of the power of the user's slow electroencephalogram (α waves) when solving a low-difficulty problem from the peak value of the power of the user's slow electroencephalogram (α waves) when solving a high-difficulty problem. Note that the type of variability in reaction time is not limited to 75% percentile-25% percentile, and may be, for example, standard deviation.

[0047] FIG. 24 shows an example of the relationship between the task difference Δtv [s] in the variance (75% percentile-25% percentile) of a user's reaction time when solving a high-difficulty problem and when solving a low-difficulty problem, and the task difference ΔR [%] in the correct answer rate when solving a high-difficulty problem and when solving a low-difficulty problem. The task difference ΔR is a vector quantity obtained by subtracting the correct answer rate when solving a low-difficulty problem from the correct answer rate when solving a high-difficulty problem. Note that the type of variance in reaction time is not limited to 75% percentile-25% percentile, and may be, for example, standard deviation.

[0048] 23 and 24, data for each user is plotted, and the characteristics of all users are expressed by a regression equation (regression line). In Fig. 23, the regression equation is expressed as ΔP=a1×Δtv+b1, and in Fig. 24, the regression equation is expressed as ΔR=a2×Δtv+b2.

[0049] A small task difference Δtv in the variability of reaction time means that there is a small difference in the variability of reaction time when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that as the difficulty of the problem increases, the task difference in the variability of the time it takes to solve the problem tends to be smaller than that of other users. On the other hand, a large task difference Δtv in the variability of reaction time means that there is a large difference in the variability of reaction time when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that as the difficulty of the problem increases, the task difference in the variability of the time it takes to solve the problem tends to be larger than that of other users.

[0050] From Figure 23, it can be seen that when the task difference Δtv in the variability of reaction time is small, the task difference ΔP in the peak value of the power of slow electroencephalograms (α waves) is large, and when the task difference Δtv in the variability of reaction time is large, the task difference ΔP in the peak value of the power of slow electroencephalograms (α waves) is small. From this, it can be seen that people who can answer difficult questions in about the same reaction time as easy questions tend to have a large task difference ΔP in the peak value of the power of slow electroencephalograms (α waves). Conversely, it can be seen that people who show large variability in reaction time to difficult questions tend to have a task difference ΔP in the peak value of the power of slow electroencephalograms (α waves) that does not change much, regardless of the difficulty of the questions.

[0051] From Figure 24, we can see that when the task difference Δtv in the variability of reaction time is large, the task difference ΔR in the correct answer rate of the problem is small, and when the task difference Δtv in the variability of reaction time is small, the task difference ΔR in the correct answer rate of the problem is large. From this, we can see that people who have large variability in reaction time for difficult problems tend to have a small task difference ΔR in the correct answer rate (i.e., a lower correct answer rate for difficult problems). Conversely, we can see that people who have small variability in reaction time even for difficult problems tend to have a large task difference ΔR in the correct answer rate (i.e., they can answer difficult problems as well as easy problems correctly).

[0052] From the above, when the task difference Δtv in the variation of reaction time is large, it can be inferred that the user's cognitive resource is lower than a predetermined standard. Also, when the task difference Δtv in the variation of reaction time is small, it can be inferred that the user's cognitive resource is higher than a predetermined standard. When the user's cognitive capacity is lower than a predetermined standard, the difficulty of the problem may be too high for the user. On the other hand, when the user's cognitive capacity is higher than a predetermined standard, the difficulty of the problem may be too low for the user.

[0053] Figure 25 shows the task difference Δk [%] in the user's level of alertness when solving a high-difficulty problem and a low-difficulty problem, and the task difference ΔP [(mV 2 / Hz) 2 / Hz]. FIG. 26 shows an example of the relationship between the task difference Δk [%] of the user's alertness when solving a high-difficulty problem and when solving a low-difficulty problem, and the task difference ΔR [%] of the correct answer rate when solving a high-difficulty problem and when solving a low-difficulty problem. The task difference Δk [%] is a vector quantity obtained by subtracting the user's alertness when solving a low-difficulty problem from the user's alertness when solving a high-difficulty problem. The alertness can be obtained, for example, by using the estimation model for estimating alertness using electroencephalograms, as described above.

[0054] In Figures 25 and 26, data for each user is plotted, and the characteristics of all users are expressed by a regression equation (regression line). In Figure 25, the regression equation is expressed as ΔP = a3 × Δk + b3, and in Figure 26, the regression equation is expressed as ΔR = a4 × Δk + b4.

[0055] 23 to 26 show that there is a correspondence between the task difference Δtv in the reaction time variation and the task difference Δk in the alertness level. Therefore, it is possible to estimate the task difference Δk in the alertness level by measuring the task difference Δtv in the reaction time variation.

[0056] FIG. 27 shows an example of the relationship between the variance in user reaction time (75% percentile-25% percentile) tv [s] when solving a high-difficulty problem and the correct answer rate R [%] when solving a high-difficulty problem. In FIG. 27, data for each user is plotted, and the characteristics of all users are expressed by a regression equation (regression line). In FIG. 27, the regression equation is expressed as R = a5 × tv + b5.

[0057] FIG. 28 shows an example of the relationship between the user's level of alertness k [%] when solving a high-difficulty problem and the correct answer rate R [%] when solving the high-difficulty problem. In FIG. 28, data for each user is plotted, and the characteristics of all users are expressed by a regression equation (regression line). In FIG. 28, the regression equation is expressed as R = a6 × k + b6.

[0058] 27 and 28, it can be seen that there is a correspondence between the reaction time variation tv and the awakening level k. Therefore, it can be seen that it is possible to estimate the awakening level k by measuring the reaction time variation tv.

[0059] <2. Comfort and Discomfort> A person's comfort and discomfort, like their level of alertness, are closely related to their ability to concentrate. When a person is concentrating, they have a high level of interest in the object of their concentration. Therefore, by knowing a person's comfort and discomfort, it is possible to estimate their objective level of interest (emotions). A person's comfort and discomfort can be derived from biological information or behavioral information obtained from the person or the communication partner (hereinafter referred to as the "target organism") during a conversation with the communication partner.

[0060] Examples of biological information from which the comfort or discomfort of a target living body can be derived include information on brain waves and sweating, and examples of motion information from which the comfort or discomfort of a target living body can be derived include facial expressions.

[0061] (EEG) It is known that a person's comfort or discomfort can be estimated from the difference in alpha waves contained in electroencephalograms between the left and right sides of the frontal lobe. For example, suppose the alpha waves contained in electroencephalograms obtained from the left side of the frontal lobe (hereinafter referred to as "left alpha waves") are compared with the alpha waves contained in electroencephalograms obtained from the right side of the frontal lobe (hereinafter referred to as "right alpha waves"). When the left alpha waves are lower than the right alpha waves, it can be estimated that the subject is feeling comfort, and when the left alpha waves are higher than the right alpha waves, it can be estimated that the subject is feeling discomfort.

[0062] Furthermore, when estimating the pleasure / discomfort of a target organism using electroencephalograms, it is possible to use an estimation model such as machine learning instead of deriving the difference between the left and right sides of the frontal region in alpha waves contained in electroencephalograms. This estimation model is, for example, a model trained using alpha waves or beta waves contained in electroencephalograms obtained when the target organism is clearly experiencing pleasure as training data. When alpha waves or beta waves contained in electroencephalograms are input, this estimation model estimates the pleasure / discomfort of the target organism based on the input alpha waves or beta waves. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0063] (sweating) Psychological sweating is sweating released from eccrine glands when the sympathetic nervous system is tense due to mental or psychological issues such as stress, tension, or anxiety. For example, by attaching a sweat meter probe to the palms or soles of the feet and measuring the sweating (psychological sweating) of the palms or soles induced by various stress stimuli, the sympathetic sweating response (SSwR) can be obtained as a signal voltage. When the values ​​of the predetermined high-frequency and low-frequency components obtained from the left hand in this signal voltage are higher than those obtained from the right hand, it can be inferred that the subject is experiencing pleasure. When the values ​​of the predetermined high-frequency and low-frequency components obtained from the left hand are lower than those obtained from the right hand, it can be inferred that the subject is experiencing discomfort. When the amplitude value of this signal voltage obtained from the left hand is higher than that obtained from the right hand, it can be inferred that the subject is experiencing pleasure. Furthermore, when the amplitude value obtained from the left hand is lower than the amplitude value obtained from the right hand in the signal voltage, it can be estimated that the subject living body is feeling discomfort.

[0064] Furthermore, it is also possible to estimate the level of arousal of a target living organism using an estimation model that estimates the level of arousal of a target living organism based on a predetermined high-frequency component or a predetermined low-frequency component contained in the signal voltage. This estimation model is, for example, a model trained using a predetermined high-frequency component or a predetermined low-frequency component contained in the signal voltage when the level of arousal is clearly high as training data. When a predetermined high-frequency component or a predetermined low-frequency component is input, this estimation model estimates the level of arousal of the target living organism based on the input predetermined high-frequency component or predetermined low-frequency component. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0065] (Facial expression) It is known that when we feel uncomfortable, we frown, while when we feel comfortable, the zygomatic major muscle does not change much. In this way, it is possible to estimate comfort or discomfort based on facial expressions. For example, by photographing the face with a camera and estimating facial expressions based on the resulting video data, it is possible to estimate the comfort or discomfort of the target living organism based on the estimated facial expressions.

[0066] It is also possible to estimate the comfort / discomfort of a target organism using an estimation model that estimates the comfort / discomfort of a target organism based on video data of facial expressions. This estimation model is a model that is trained using video data of facial expressions captured when the subject is clearly at a high level of arousal as training data. When video data of facial expressions is input, this estimation model estimates the comfort / discomfort of the target organism based on the input video data. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0067] The frequency components of brain waves are described in the following documents, for example: Wang, Xiao-Wei, Dan Nie, and Bao-Liang Lu. "EEG-based emotion recognition using frequency domain features and support vector machines." International conference on neural information processing. Springer, Berlin, Heidelberg, 2011. Estimation models using EEG are described in the following literature, for example: Patent application 2020-203058 Sweating is described, for example, in the following literature: Jing Zhai, AB Barreto, C. Chin and Chao Li, "Realization of stress detection using psychophysiological signals for improvement of human-computer interactions," Proceedings. IEEE SoutheastCon, 2005., Ft. Lauderdale, FL, USA, 2005, pp. 415-420, doi: 10.1109 / SECON.2005.1423280. Boucsein, Wolfram. Electrodermal activity. Springer Science & Business Media, 2012. Heart rate is described in the following literature, for example: Veltman, JA, and AWK Gaillard. "Physiological indices of workload in a simulated flight task." Biological psychology 42.3 (1996): 323-342. Heart rate variability intervals are described in, for example, the following literature: Appelhans, Bradley M., and Linda J. Luecken. "Heart rate variability as an index of regulated emotional responding." Review of general psychology 10.3 (2006): 229-240. Salivary cortisol levels are described in the following literature, for example: Lam, Suman, et al. "Emotion regulation and cortisol reactivity to a social-evaluative speech task." Psychoneuroendocrinology 34.9 (2009): 1355-1362. Facial expressions are described in the following literature, for example: Lyons, Michael J., Julien Budynek, and Shigeru Akamatsu. "Automatic classification of single facial images." IEEE transactions on pattern analysis and machine intelligence 21.12 (1999): 1357-1362. The facial muscles are described in the following literature, for example: Ekman, Paul. "Facial action coding system." (1977). The frequency of blinks is described in the following literature, for example: Chen, Siyuan, and Julien Epps. "Automatic classification of eye activity for cognitive load measurement with emotion interference." Computer methods and programs in biomedicine 110.2 (2013): 111-124. The respiratory volume / respiratory rate is described in, for example, the following literature: Zhang Q., Chen X., Zhan Q., Yang T., Xia S. Respiration-based emotion recognition with deep learning. Comput. Ind. 2017;92-93:84-90. doi: 10.1016 / j.compind.2017.04.005. Skin surface temperature is described in the following literature, for example: Nakanishi R., Imai-Matsumura K. Facial skin temperature decreases in infants with joyful expression. Infant Behav. Dev. 2008;31:137-144. doi: 10.1016 / j.infbeh.2007.09.001. Multimodal approaches are described in the following literature, for example: Choi J.-S., Bang J., Heo H., Park K. Evaluation of Fear Using Nonintrusive Measurement of Multimodal Sensors. Sensors. 2015;15:17507-17533. doi: 10.3390 / s150717507.

[0068] An embodiment of an information processing system that uses the above-described algorithm for deriving the arousal level and the comfort / discomfort level will be described below.

[0069] 2. First Embodiment [composition] A biological information processing system 100 according to a first embodiment of the present disclosure will be described. FIG. 1 illustrates a schematic configuration example of the biological information processing system 100. The biological information processing system 100 is an objective evaluation system that evaluates a target organism based on at least one of biological information and behavioral information obtained from the target organism. In this embodiment, the target organism is a human. However, in the biological information processing system 100, the target organism is not limited to a human.

[0070] The biometric information processing system 100 includes a biometric sensor 10 that detects biometric information of a subject to be evaluated, and an electronic device 20 that processes a detection signal output from the biometric sensor 10. The biometric sensor 10 and the electronic device 20 are connected to each other via a network 30 so as to be able to send and receive data to and from each other. The network 30 is a wireless or wired communication means, such as the Internet, a wide area network (WAN), a local area network (LAN), a public communication network, or a dedicated line.

[0071] The biosensor 10 may be, for example, a sensor that comes into contact with the subject, or a sensor that does not come into contact with the subject. The biosensor 10 is a sensor that acquires information (biometric information) about at least one of, for example, electroencephalogram, sweating, pulse wave, electrocardiogram, blood flow, skin temperature, facial myoelectric potential, electrooculography, and specific components contained in saliva. The biosensor 10 may be, for example, a sensor that acquires information (behavioral information) about at least one of facial expression, voice, and reaction time. The biosensor 10 may be, for example, a sensor that acquires at least one of biometric information and behavioral information. The biosensor 10 outputs the acquired information (at least one of biometric information and behavioral information) to the electronic device 20.

[0072] The electronic device 20 includes a sensor input receiving unit 21, a user input receiving unit 22, a signal processing unit 23, a memory unit 24, a video data generating unit 25, and a video display unit 26. The signal processing unit 23 corresponds to a specific example of a "derivation unit," "classification unit," "receiving unit," and "selection unit" in the present disclosure. The memory unit 24 corresponds to a specific example of a "memory unit" in the present disclosure. The video signal generating unit 25 corresponds to a specific example of a "video data generating unit" in the present disclosure.

[0073] The sensor input receiving unit 21 receives input from the biosensor 10 and outputs it to the signal processing unit 23. The input from the biosensor 10 is at least one of biometric information and behavioral information. The sensor input receiving unit 21 is configured, for example, by an interface capable of communicating with the biosensor 10. The user input receiving unit 22 receives input from the user and outputs it to the signal processing unit 23. Examples of input from the user include attribute information (such as name) of the person being evaluated and an instruction to start evaluation. The user input receiving unit 22 is configured, for example, by an input interface such as a keyboard, a mouse, or a touch panel.

[0074] The storage unit 24 is, for example, a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The storage unit 24 stores a biometric information processing program 24a for evaluating the subject, task data 24b, and classification index 24c used in the biometric information processing program 24a. The classification index 24c corresponds to a specific example of a "predetermined classification index" in the present disclosure. Furthermore, the storage unit 24 stores an identifier 24d, a wakefulness level 24e, a feature amount 24f, a classification result 24g, and an evaluation result 24h obtained by processing using the biometric information processing program 24a. The processing details of the biometric information processing program 24a will be described in detail later.

[0075] The task data 24b includes, for example, multiple pieces of question data. The multiple pieces of question data are tasks assigned to the person being evaluated while the biometric information of the person being evaluated is being acquired, and correspond to a specific example of a "specific task" in the present disclosure. The task data 24b can be omitted as necessary. In that case, the task to be assigned to the person being evaluated while the biometric information of the person being evaluated is being acquired may be, for example, a device (e.g., a test electronic device or a game console) or paper media (e.g., test paper) provided separately from the electronic device 20. In the following description, it is assumed that the task using the task data 24b is provided by the electronic device 20.

[0076] The classification index 24c includes one or more indices used to evaluate the subject, such as the duration of alertness and the rise time of alertness. The duration of alertness refers to the period (duration Δt1) during which a high level of alertness is maintained, as shown in FIG. 3, for example. The rise time of alertness refers to the time (rise time Δt2) required for a transition from a low level of alertness to a high level, as shown in FIG. 3, for example. The duration Δt1 is an index relating to the sustainability of concentration, and a longer duration Δt1 indicates a greater ability to maintain high levels of concentration. The rise time Δt2 is an index relating to the speed of on / off switching, and a shorter rise time Δt2 indicates a faster ability to concentrate on a task.

[0077] The identifier 24d is numerical data for identifying the person to be evaluated, and is, for example, an identification number assigned to each person to be evaluated. The identifier 24d is generated, for example, at the timing when the attribute information of the person to be evaluated is input by the person to be evaluated. The alertness level 24e is numerical data on the alertness level derived based on the input (detection signal) from the biosensor 10. The alertness level 24e is, for example, numerical data on the alertness level that changes over time, as shown in FIG. 3. The feature amount 24f is numerical data on one or more indices included in the classification index 24c.

[0078] The feature 24f includes, for example, a duration Δt1 and a rise time Δt2 derived from the wakefulness level 24e. The classification result 24g indicates one of a plurality of classifications divided according to the magnitude of the feature 24f (for example, the magnitude of the duration Δt1 and the rise time Δt2). Examples of the plurality of classifications include the classifications shown in FIG. 4. Category (1): Both the duration Δt1 and rise time Δt2 are large Classification (2): A classification in which the duration Δt1 is small and the rise time Δt2 is large Classification (3): Long duration Δt1 and short rise time Δt2 Category (4): Both the duration Δt1 and rise time Δt2 are small

[0079] The evaluation result 24h is, for example, an evaluation result of suitability or unsuitability in the selection of people for recruitment activities, team building within an organization, etc. The evaluation result 24h is, for example, a result of evaluation based on the classification result 24g. For example, if the feature amount 24f corresponds to classification (1), the evaluation result 24h is "suitable." Also, for example, if the feature amount 24f corresponds to classification (4), the evaluation result 24h is "unsuitable."

[0080] The signal processing unit 23 is configured by, for example, a processor. The signal processing unit 23 executes a biometric information processing program 24a stored in the storage unit 24. The functions of the signal processing unit 23 are realized, for example, by the signal processing unit 23 executing the biometric information processing program 24a. The signal processing unit 23 executes a series of processes required for evaluating the person to be evaluated. The signal processing unit 23 reads, for example, a predetermined number of pieces of question data from the task data 24b, and sequentially outputs the read number of pieces of question data to the video data generation unit 25. The video data generation unit 25 generates video data including the question data input from the signal processing unit 23, and outputs the video data to the video display unit 26. The video display unit 26 displays a video based on the video data input from the video data generation unit 25. The person to be evaluated solves the problems while watching the video displayed on the video display unit 26.

[0081] The person to be evaluated inputs an answer corresponding to question data to the user input receiving unit 22, for example. When the signal processing unit 23 acquires an answer corresponding to a question displayed on the video display unit 26 from the user input receiving unit 22, for example, the signal processing unit 23 outputs the next question data to the video data generating unit 25. The person to be evaluated may, for example, write an answer corresponding to the question data on paper and input an answer completion notification to the user input receiving unit 22. In this case, when the signal processing unit 23 acquires an answer completion notification from the user input receiving unit 22, for example, the signal processing unit 23 outputs the next question data to the video data generating unit 25. The person to be evaluated may, for example, write an answer corresponding to the question data on paper and not input anything to the user input receiving unit 22. In this case, the signal processing unit 23 periodically outputs the next question data to the video data generating unit 25, for example.

[0082] The signal processing unit 23 derives the alertness level 24e of the subject based on at least one of biometric information and behavioral information obtained from the subject while the subject is performing a task (specific task) of solving multiple problems. The signal processing unit 23 derives the alertness level 24e of the subject using one of the various methods described above. The signal processing unit 23 derives, for example, time-series data as the alertness level 24e. The signal processing unit 23 further stores, for example, the derived time-series data in the storage unit 24 in association with the identifier 24d of the subject.

[0083] The signal processing unit 23 derives a feature 24f corresponding to the classification index 24c based on the awakening level 24e. The signal processing unit 23 derives, for example, a duration Δt1 and a rise time Δt2 from the awakening level 24e. The signal processing unit 23 selects one of a plurality of categories (1) to (4) based on the magnitude of the derived feature 24f (for example, the magnitude of the duration Δt1 and the rise time Δt2). The signal processing unit 23 evaluates the person to be evaluated based on, for example, the selected category (classification result 24g). The signal processing unit 23 stores, for example, an evaluation result 24h of the person to be evaluated in the storage unit 24.

[0084] The evaluation criteria for the person to be evaluated are stored in the memory unit 24. The evaluation criteria for the person to be evaluated are, for example, the hiring criteria for the person to be evaluated or the criteria for team building within the organization. Typically, the hiring criteria for a person are often attribute information such as the person's age, gender, and educational background. Furthermore, the criteria for team building within the organization are often based on a person's intuition, experience, and subjectivity. However, in this embodiment, the hiring criteria for a person and the criteria for team building within the organization are based on the classification result 24g. The hiring criteria for a person are, for example, that the classification result 24g corresponds to category (1). The criteria for team building within the organization may differ from the hiring criteria for a person because they also take into account relationships with other members. The criteria for team building within the organization are, for example, three people whose classification result 24g corresponds to category (1), one person whose classification result 24g corresponds to category (2), one person whose classification result 24g corresponds to category (3), and one person whose classification result 24g corresponds to category (4).

[0085] The biometric information processing system 100 may evaluate an applicant (evaluation target) in order to hire a person. In this case, when evaluating the evaluation target, the signal processing unit 23 assigns an identifier 24d to the evaluation target and stores the assigned identifier 24d in the storage unit 24. When the signal processing unit 23 obtains the alertness level 24e, the signal processing unit 23 associates the classification result 24g derived from the obtained alertness level 24e with the assigned identifier 24d and stores it in the storage unit 24. If the classification result 24g stored in the storage unit 24 matches the hiring criteria for a person, the signal processing unit 23 stores the identifier 24d of the matching person in the storage unit 24 as a hiring result 24h.

[0086] The biological information processing system 100 may sequentially evaluate multiple evaluation subjects to select people suitable for forming a specific group (e.g., a team within an organization). In this case, the signal processing unit 23 assigns an identifier 24d to the evaluation subject each time the evaluation subject is evaluated and stores the assigned identifier 24d in the storage unit 24. Each time an alertness level 24e is obtained, the signal processing unit 23 associates a classification result 24g derived from the obtained alertness level 24e with the assigned identifier 24d and stores the classification result 24g in the storage unit 24. The signal processing unit 23 selects multiple identifiers 24d suitable for forming a specific group (e.g., a team within an organization) based on the classification results 24g of each of the multiple evaluation subjects stored in the storage unit 24. The signal processing unit 23 extracts, from the multiple classification results 24g corresponding to the multiple evaluation subjects stored in the storage unit 24, those that meet the criteria for forming a specific group (e.g., a team within an organization). The signal processing unit 23 stores a plurality of identifiers 24d corresponding to the extracted plurality of classification results 24g in the storage unit 24 as adoption results 24h.

[0087] The video data generator 25 generates video data in which the classification index 24c and the feature amount 24f derived for evaluation are associated with each other. The video data generator 25 generates video data in which the classification index 24c and the time-series data of the awakening level 24e are associated with each other. The video data generator 25 outputs the generated video data to the video display unit 26.

[0088] Video display unit 26 displays a video based on the video data input from video data generator 25. At this time, video display unit 26 displays, for example, a video as shown in FIG. 4 on display screen 26A. On display screen 26A, for example, as shown in FIG. 4, classification index 24c is displayed in a two-dimensional graph format, and feature value 24f is displayed as a plot in one of the quadrants of the two-dimensional graph. On display screen 26A, time-series data of wakefulness level 24e is further displayed as a waveform, for example, as shown in FIG. 4.

[0089] When multiple evaluation subjects are evaluated sequentially to select people suitable for forming a specific group (e.g., a team within an organization), the video data generation unit 25 generates video data in which the classification index 24c is associated with multiple feature amounts 24f derived for evaluating the multiple evaluation subjects. The video data generation unit 25 generates video data in which the classification index 24c is associated with time-series data of the awakening levels 24e of the multiple evaluation subjects. The video data generation unit 25 outputs the generated video data to the video display unit 26.

[0090] Video display unit 26 displays a video based on the video data input from video data generator 25. At this time, video display unit 26 displays, for example, videos such as those shown in FIGS. 5 and 6 on display screen 26A. On display screen 26A, for example, as shown in FIGS. 5 and 6, classification index 24c is displayed in a two-dimensional graph format, and multiple feature amounts 24f are displayed as plots in one or multiple quadrants of the two-dimensional graph. On display screen 26A, time-series data for multiple levels of arousal 24e are further displayed overlapping each other with the same time, for example, as shown in FIGS. 5 and 6.

[0091] FIG. 5 illustrates an example of a waveform when the time-series data of the multiple arousal levels 24e are substantially synchronized. FIG. 6 illustrates an example of a waveform when the time-series data of the multiple arousal levels 24e are completely asynchronous. As shown in FIG. 6, when the time-series data of the multiple arousal levels 24e are substantially synchronized, the people to be evaluated for whom the multiple arousal levels 24e have been calculated can be classified into a common classification index 24c. On the other hand, as shown in FIG. 7, when the time-series data of the multiple arousal levels 24e are completely asynchronous, the people to be evaluated for whom the multiple arousal levels 24e have been calculated can be classified into different classification indexes 24c. From these facts, the user can evaluate the people to be evaluated for whom the multiple arousal levels 24e have been calculated, based on the synchronicity of the time-series data of the multiple arousal levels 24e displayed on the video display unit 26. For example, when an image (time-series data of a plurality of arousal levels 24e) such as those shown in FIGS. 6 and 7 is displayed on the image display unit 26, the signal processing unit 23 receives, via the user input receiving unit 22, a selection of a plurality of arousal levels 24e among the plurality of arousal levels 24e or a plurality of identifiers 24d among the plurality of identifiers 24d. Based on the received content (selection results), the signal processing unit 23 selects a plurality of identifiers 24d suitable for forming a specific group (e.g., a team within an organization). The signal processing unit 23 stores the selected plurality of identifiers 24d in the storage unit 24 as hiring results 24h. In this way, the user can evaluate the people to be evaluated, for whom the plurality of arousal levels 24e have been calculated, based on the synchronicity of the time-series data of the plurality of arousal levels 24e displayed on the image display unit 26.

[0092] [Operation] Next, a description will be given of the operation of the biological information processing system 100. Fig. 8 shows an example of an evaluation procedure in the biological information processing system 100.

[0093] First, the electronic device 20 (signal processing unit 23) loads the biometric information processing program 24a from the storage unit 24 and starts executing a series of procedures for evaluation described in the biometric information processing program 24a. The signal processing unit 23 reads out a predetermined number of pieces of question data from the task data 24b and sequentially outputs the read out number of pieces of question data to the video data generation unit 25. The video data generation unit 25 generates video data including the question data input from the signal processing unit 23 and outputs the video data to the video display unit 26. The video display unit 26 displays a video based on the video data input from the video data generation unit 25. At this time, the person to be evaluated solves the problems while watching the video displayed on the video display unit 26.

[0094] The signal processing unit 23 outputs an information acquisition request to the biosensor 10. The information acquisition request is a series of control signals for causing the biosensor 10 to acquire at least one of bioinformation and behavioral information of the person being evaluated while the person being evaluated is performing a task (specific task) of solving multiple problems. In response to the input of the information acquisition request, the biosensor 10 acquires at least one of bioinformation and behavioral information and outputs it to the electronic device 20.

[0095] When the electronic device 20 (signal processing unit 23) acquires information (at least one of biological information and behavioral information) from the biosensor 10, it derives a wakefulness level 24e based on the acquired information. The signal processing unit 23 derives a feature value 24f corresponding to the classification index 24c based on the derived wakefulness level 24e. The signal processing unit 23 selects one classification from a plurality of classifications (1) to (4) depending on the magnitude of the derived feature value 24f (for example, the magnitude of the duration Δt1 and the rise time Δt2). The signal processing unit 23 evaluates the evaluation subject based on the selected classification (classification result 24g). The signal processing unit 23 stores, for example, an evaluation result 24h of the evaluation subject in the storage unit 24.

[0096] Video data generation unit 25 generates video data in which classification index 24c and feature amount 24f derived for evaluation are associated with each other. Video data generation unit 25 generates video data in which classification index 24c and time-series data of awakening level 24e are associated with each other. Video data generation unit 25 outputs the generated video data to video display unit 26. Video display unit 26 displays video based on the video data input from video data generation unit 25. Video display unit 26 displays, for example, videos such as those shown in FIGS. 5 to 7 on display screen 26A.

[0097] [effect] Next, the effects of the biological information processing system 100 will be described.

[0098] In this embodiment, the arousal level 24e is classified based on a predetermined classification index 24c. This allows the evaluation subjects to be classified using the arousal level 24e, which is objective data. As a result, for example, when recruiting personnel, it becomes possible to determine whether the evaluation subject is a desirable candidate based on the arousal level 24e of the evaluation subject. Furthermore, for example, when deciding on project members, it becomes possible to determine whether the evaluation subject is suitable to form a specific group based on the arousal levels 24e of multiple evaluation subjects. This makes it possible to reduce mismatches.

[0099] In this embodiment, the person to be evaluated is evaluated based on the classification result 24g. Here, the classification result 24g is derived from the awakening level 24e, which is objective data. Therefore, for example, in the recruitment of personnel, it is possible to determine whether or not the person is a desirable person based on the objective data. Therefore, it is possible to reduce mismatches.

[0100] In this embodiment, each time an alertness level 24e is obtained, a classification result 24g derived from the alertness level 24e is stored in the storage unit 24 in association with an identifier 24d of the person being evaluated. Furthermore, based on the multiple classification results 24g stored in the storage unit 24, multiple identifiers 24d suitable for forming a specific group are selected. Therefore, for example, when deciding on project members, it is possible to determine from objective data whether or not a member is suitable for forming a specific group. Therefore, it is possible to reduce mismatches.

[0101] In this embodiment, a feature 24f corresponding to the classification index 24c is derived based on the arousal level 24e, and the derived feature 24f is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. This makes it possible to classify the person being evaluated using the feature 24f, which is objective data. As a result, for example, in a recruitment situation, it becomes possible to determine whether or not the person being evaluated is a desirable candidate based on the feature 24f of the person being evaluated. Furthermore, for example, in a situation where project members are being selected, it becomes possible to determine whether or not the person is suitable to be a member of a specific group based on the feature 24f of multiple people being evaluated. This makes it possible to reduce mismatches.

[0102] In this embodiment, video data is generated in which the classification index 24c and the feature amount 24f are associated with each other. This allows a user to view a video displayed based on the video data and evaluate the person being evaluated. As a result, for example, when recruiting personnel, it becomes possible to determine whether or not the person being evaluated is a desirable candidate based on the feature amount 24f of the person being evaluated. Also, for example, when deciding on project members, it becomes possible to determine whether or not a person is suitable to be part of a specific group based on the feature amounts 24f of multiple people being evaluated. This makes it possible to reduce mismatches.

[0103] In this embodiment, time-series data is derived as the arousal level 24e, and the derived time-series data is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. This makes it possible to classify the person being evaluated using the arousal level 24e, which is objective data. As a result, for example, when recruiting personnel, it becomes possible to determine whether the person is a desirable candidate based on the arousal level 24e of the person being evaluated. Furthermore, for example, when deciding on project members, it becomes possible to determine whether the person is suitable to form a specific group based on the arousal levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0104] In this embodiment, video data is generated in which the classification index 24c and the time-series data of the arousal level 24e are associated with each other. This allows a user to evaluate the person being evaluated by viewing the video displayed based on the video data. As a result, for example, when recruiting personnel, it becomes possible to determine whether the person is a desirable candidate based on the time-series data of the arousal level 24e of the person being evaluated. Furthermore, for example, when deciding on project members, it becomes possible to determine whether the person is suitable to form a specific group based on the synchronous time-series data of the arousal levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0105] In this embodiment, the derived arousal level 24e is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. This allows the person being evaluated to be classified using the arousal level 24e, which is objective data. As a result, for example, when recruiting personnel, it becomes possible to determine whether the person being evaluated is a desirable candidate based on the arousal level 24e of the person being evaluated. Furthermore, for example, when deciding on project members, it becomes possible to determine whether the person is suitable to form a specific group based on the arousal levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0106] In this embodiment, each time a wakefulness level 24e is derived, the derived wakefulness level 24e is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. Furthermore, video data is generated in which the wakefulness levels 24e corresponding to multiple identifiers 24d are compiled in a manner that allows them to be compared with one another. This allows a user to view a video displayed based on the video data and evaluate the person being evaluated. As a result, for example, when deciding on project members, it is possible to determine whether a person is suitable to form a specific group based on the wakefulness levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0107] In this embodiment, a selection of multiple arousal levels 24e from the multiple arousal levels 24e or multiple identifiers 24d from the multiple identifiers 24d is received from the user. Then, multiple identifiers 24d suitable for forming a specific group are selected based on the received selection. This makes it possible to determine, for example, whether a member is suitable for forming a specific group based on objective data when deciding on project members. Therefore, it is possible to reduce mismatches.

[0108] In this embodiment, each time a wakefulness level 24e is derived, the derived wakefulness level 24e is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. Furthermore, based on the multiple wakefulness levels 24e stored in the storage unit 24 and a predetermined classification index 24c, multiple identifiers 24d suitable for forming a specific group are selected. This makes it possible, for example, when deciding on project members, to determine from objective data whether or not a member is suitable for forming a specific group. This makes it possible to reduce mismatches.

[0109] In this embodiment, time-series data is derived as the arousal level 24e, and the derived time-series data is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. Then, video data is generated by aligning and superimposing the time-series data of the arousal levels 24e corresponding to the multiple identifiers 24d. This allows the user to evaluate the person being evaluated by viewing the video displayed based on the video data. As a result, for example, when deciding on project members, it is possible to determine whether or not a person is suitable to be part of a specific group based on the arousal levels 24e of multiple people being evaluated. This reduces mismatches.

[0110] 3. Second Embodiment [composition] Next, a biological information processing system 110 according to a second embodiment of the present disclosure will be described. FIG. 9 shows a schematic configuration example of the biological information processing system 110. The biological information processing system 110 is an objective evaluation system that evaluates a target organism based on at least one of biological information and behavioral information obtained from the target organism. In this embodiment, the target organism is a human. However, in the biological information processing system 110, the target organism is not limited to a human.

[0111] The biometric information processing system 110 includes an electronic device 40 incorporating a biometric sensor 41 that detects biometric information of a subject to be evaluated. The biometric sensor 41 has a configuration similar to that of the biometric sensor 10 according to the above-described embodiment. For example, as shown in FIG. 10 , the electronic device 40 corresponds to the electronic device 20 in which the biometric sensor 41 is provided instead of the sensor input receiving unit 21. The biometric sensor 41 outputs acquired information (at least one of biometric information and behavioral information) to the signal processing unit 23.

[0112] [Operation] Next, a description will be given of the operation of the biological information processing system 110. Fig. 11 shows an example of an evaluation procedure in the biological information processing system 110.

[0113] First, the signal processing unit 23 loads the biometric information processing program 24a from the storage unit 24 and starts executing a series of procedures for evaluation described in the biometric information processing program 24a. The signal processing unit 23 reads out a predetermined number of pieces of question data from the task data 24b and sequentially outputs the read out number of pieces of question data to the video data generation unit 25. The video data generation unit 25 generates video data including the question data input from the signal processing unit 23 and outputs the video data to the video display unit 26. The video display unit 26 displays a video based on the video data input from the video data generation unit 25. At this time, the person to be evaluated solves the problems while watching the video displayed on the video display unit 26.

[0114] The signal processing unit 23 outputs an information acquisition request to the biosensor 41. In response to the input of the information acquisition request, the biosensor 41 acquires at least one of bioinformation and behavioral information and outputs the information to the signal processing unit 23.

[0115] When the signal processing unit 23 acquires information (at least one of biological information and behavioral information) from the biosensor 41, it derives a wakefulness level 24e based on the acquired information. The signal processing unit 23 derives a feature value 24f corresponding to the classification index 24c based on the derived wakefulness level 24e. The signal processing unit 23 selects one classification from a plurality of classifications (1) to (4) depending on the magnitude of the derived feature value 24f (for example, the magnitude of the duration Δt1 and the rise time Δt2). The signal processing unit 23 evaluates the evaluation subject based on the selected classification (classification result 24g). The signal processing unit 23 stores, for example, an evaluation result 24h of the evaluation subject in the storage unit 24.

[0116] Video data generation unit 25 generates video data in which classification index 24c and feature amount 24f derived for evaluation are associated with each other. Video data generation unit 25 generates video data in which classification index 24c and time-series data of awakening level 24e are associated with each other. Video data generation unit 25 outputs the generated video data to video display unit 26. Video display unit 26 displays video based on the video data input from video data generation unit 25. Video display unit 26 displays, for example, videos such as those shown in FIGS. 5 to 7 on display screen 26A.

[0117] [effect] Next, the effects of the biological information processing system 110 will be described.

[0118] In this embodiment, similar to the above embodiment, the arousal level 24e is classified based on a predetermined classification index 24c. This allows the evaluation subjects to be classified using the arousal level 24e, which is objective data. As a result, for example, when recruiting personnel, it is possible to determine whether the evaluation subject is a desirable candidate based on the arousal level 24e of the evaluation subject. Furthermore, for example, when deciding on project members, it is possible to determine whether the evaluation subject is suitable to form a specific group based on the arousal levels 24e of multiple evaluation subjects. This makes it possible to reduce mismatches.

[0119] In this embodiment, the derived arousal level 24e is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. This allows the person being evaluated to be classified using the arousal level 24e, which is objective data. As a result, for example, when recruiting personnel, it becomes possible to determine whether the person being evaluated is a desirable candidate based on the arousal level 24e of the person being evaluated. Furthermore, for example, when deciding on project members, it becomes possible to determine whether the person is suitable to form a specific group based on the arousal levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0120] 4. Third Embodiment [composition] Next, an information processing system 120 according to a third embodiment of the present disclosure will be described. FIG. 12 illustrates a schematic configuration example of the information processing system 120. The information processing system 120 is an objective evaluation system that evaluates multiple target organisms based on at least one of biometric information and behavioral information obtained from the multiple target organisms. In this embodiment, the target organisms are humans. Note that in the information processing system 120, the target organisms are not limited to humans.

[0121] The information processing system 120 includes an electronic device 50 and a plurality of electronic devices 60. The electronic device 50 and each electronic device 60 are connected via a network 70 so as to be able to transmit and receive data to and from each other. The information processing system 120 also includes a plurality of biosensors 10. Each of the plurality of biosensors 10 is assigned to each electronic device 60, and each biosensor 10 is connected to the electronic device 60. The network 70 is a wireless or wired communication means, such as the Internet, a WAN, a LAN, a public communication network, or a dedicated line.

[0122] As shown in FIG. 13 , the electronic device 50 includes a communication unit 51, a user input receiving unit 22, a signal processing unit 23, a storage unit 24, a video data generating unit 25, and a video display unit 26. The communication unit 51 is configured with an interface capable of communicating with each electronic device 60 via a network 70. The signal processing unit 23 receives, from each electronic device 60, detection information 65b, which is at least one of biometric information and behavioral information, and an identifier 24d of the subject of evaluation, via the communication unit 51. The signal processing unit 23 derives the level of alertness 24e of the subject of evaluation based on the received detection information 65b. At this time, the signal processing unit 23 derives the level of alertness 24e of the subject of evaluation using one of the various methods described above. The signal processing unit 23 derives, for example, time-series data as the level of alertness 24e. The signal processing unit 23 further stores, for example, the derived time-series data in the storage unit 24 in association with the received identifier 24d.

[0123] The electronic device 60 has, for example, as shown in FIG. 14, a communication unit 61, a sensor input receiving unit 62, a user input receiving unit 63, a signal processing unit 64, a memory unit 65, a video data generation unit 66, and a video display unit 67.

[0124] The communication unit 61 is configured with an interface capable of communicating with the electronic device 50 via the network 70. The sensor input receiving unit 62 receives input from the biosensor 10 and outputs it to the signal processing unit 64. The input from the biosensor 10 is at least one of biometric information and behavioral information (detection information 65b). The sensor input receiving unit 62 is configured with, for example, an interface capable of communicating with the biosensor 10. The user input receiving unit 63 receives input from the user and outputs it to the signal processing unit 64. Examples of input from the user include attribute information (such as name) of the person being evaluated and an instruction to start evaluation. The user input receiving unit 63 is configured with an input interface such as a keyboard, mouse, or touch panel.

[0125] The storage unit 65 is, for example, a volatile memory such as a DRAM, or a non-volatile memory such as an EEPROM or a flash memory. The storage unit 65 stores a biometric information processing program 65a and task data 24b used in the biometric information processing program 65a. The biometric information processing program 65a includes a series of procedures for acquiring the detection information 65b. Furthermore, the storage unit 65 stores an identifier 24d obtained by processing using the biometric information processing program 65a.

[0126] The signal processing unit 64 is configured by, for example, a processor. The signal processing unit 64 executes a biometric information processing program 65a stored in the storage unit 65. The function of the signal processing unit 64 is realized, for example, by the signal processing unit 64 executing the biometric information processing program 65a. The signal processing unit 64 executes a series of processing steps for acquiring the detection information 65b. The signal processing unit 64 reads, for example, a predetermined number of pieces of question data from the task data 24b and sequentially outputs the read number of pieces of question data to the video data generation unit 66. The video data generation unit 66 generates video data including the question data input from the signal processing unit 64 and outputs the video data to the video display unit 67. The video display unit 67 displays a video based on the video data input from the video data generation unit 67. The subject solves the questions while watching the video displayed on the video display unit 67.

[0127] The person to be evaluated, for example, inputs an answer corresponding to question data to the user input accepting unit 63. For example, when the signal processing unit 64 acquires an answer corresponding to a question displayed on the video display unit 67 from the user input accepting unit 63, the signal processing unit 64 outputs the next question data to the video data generating unit 66. Note that the person to be evaluated may, for example, write the answer corresponding to the question data on paper and input an answer completion notice to the user input accepting unit 63. In this case, for example, when the signal processing unit 64 acquires the answer completion notice from the user input accepting unit 63, the signal processing unit 64 outputs the next question data to the video data generating unit 66. The person to be evaluated may, for example, write the answer corresponding to the question data on paper and not input anything to the user input accepting unit 63. In this case, the signal processing unit 64, for example, periodically outputs the next question data to the video data generating unit 66. The signal processing unit 64 transmits detection information 65b obtained from the person to be evaluated while the person to be evaluated is performing a task (specific task) of solving multiple questions, together with an identifier 24d of the person to be evaluated, to the electronic device 50 via the communication unit 61.

[0128] The information processing system 120 may evaluate an applicant (evaluation target) in order to hire a person. In this case, when the alertness level 24e is obtained, the signal processing unit 23 associates the classification result 24g derived from the obtained alertness level 24e with the identifier 24d of the evaluation target and stores it in the storage unit 24. If the classification result 24g stored in the storage unit 24 matches the hiring criteria for a person, the signal processing unit 23 stores the identifier 24d of the matching person in the storage unit 24 as the hiring result 24h.

[0129] The information processing system 120 may evaluate multiple individuals to select people suitable for forming a specific group (e.g., a team within an organization). In this case, each time an arousal level 24e is obtained from an individual to be evaluated, the signal processing unit 23 associates a classification result 24g derived from the obtained arousal level 24e with an identifier 24d of the individual to be evaluated and stores the classification result 24g in the storage unit 24. The signal processing unit 23 selects multiple identifiers 24d suitable for forming a specific group (e.g., a team within an organization) based on the classification results 24g of each of the multiple individuals to be evaluated, stored in the storage unit 24. The signal processing unit 23 extracts, from the multiple classification results 24g corresponding to the multiple individuals to be evaluated, those that meet the criteria for forming a specific group (e.g., a team within an organization). The signal processing unit 23 stores, in the storage unit 24, the multiple identifiers 24d corresponding to the extracted classification results 24g.

[0130] When multiple evaluation subjects are evaluated to select people suitable for forming a specific group (e.g., a team within an organization), the video data generation unit 25 generates video data in which the classification index 24c is associated with multiple feature amounts 24f derived for evaluating the multiple evaluation subjects. The video data generation unit 25 generates video data in which the classification index 24c is associated with time-series data of the awakening levels 24e of the multiple evaluation subjects. The video data generation unit 25 outputs the generated video data to the video display unit 26.

[0131] Video display unit 26 displays a video based on the video data input from video data generator 25. At this time, video display unit 26 displays, for example, videos such as those shown in FIGS. 6 and 7 on display screen 26A. On display screen 26A, for example, as shown in FIGS. 6 and 7, classification index 24c is displayed in a two-dimensional graph format, and multiple feature amounts 24f are displayed as plots in one or multiple quadrants of the two-dimensional graph. On display screen 26A, time-series data for multiple levels of awakening 24e are further displayed overlapping each other with the same time, for example, as shown in FIGS. 6 and 7.

[0132] As shown in FIG. 6, when the time-series data of the multiple arousal levels 24e are substantially synchronized, the people to be evaluated for whom the multiple arousal levels 24e have been calculated may be classified under a common classification index 24c. On the other hand, as shown in FIG. 7, when the time-series data of the multiple arousal levels 24e are not synchronized at all, the people to be evaluated for whom the multiple arousal levels 24e have been calculated may be classified under different classification indexes 24c. For these reasons, the user can evaluate the people to be evaluated for whom the multiple arousal levels 24e have been calculated based on the synchronicity of the time-series data of the multiple arousal levels 24e displayed on the video display unit 26. For example, when an image (time-series data of the multiple arousal levels 24e) such as those shown in FIGS. 6 and 7 is displayed on the video display unit 26, the signal processing unit 23 receives, via the user input receiving unit 22, a selection of multiple arousal levels 24e among the multiple arousal levels 24e or multiple identifiers 24d among the multiple identifiers 24d. Based on the received content (selection result), the signal processing unit 23 selects multiple identifiers 24d suitable for forming a specific group (e.g., a team within an organization). The signal processing unit 23 stores the selected identifiers 24d as hiring results 24h in the storage unit 24. In this way, the user can evaluate the people to be evaluated, for whom the multiple levels of arousal 24e have been calculated, based on the synchronicity of the time-series data of the multiple levels of arousal 24e displayed on the video display unit 26.

[0133] [Operation] Next, a description will be given of the operation of the biological information processing system 120. Fig. 15 shows an example of an evaluation procedure in the biological information processing system 120.

[0134] First, the electronic device 50 (signal processing unit 23) loads the biometric information processing program 24a from the storage unit 24 and starts executing a series of procedures for evaluation described in the biometric information processing program 24a. The electronic device 60 (signal processing unit 64) loads the biometric information processing program 65a from the storage unit 65 and starts executing a series of procedures for evaluation described in the biometric information processing program 65a.

[0135] The electronic device 50 (signal processing unit 23) transmits a task execution request to each electronic device 60 via the communication unit 51. When the task execution request is input, the electronic device 60 (signal processing unit 64) reads out a predetermined number of pieces of problem data from the task data 24b and sequentially outputs the read out number of pieces of problem data to the video data generation unit 66. The video data generation unit 66 generates video data including the problem data input from the signal processing unit 64 and outputs the video data to the video display unit 67. The video display unit 67 displays a video based on the video data input from the video data generation unit 66. At this time, the person to be evaluated solves the problem while watching the video displayed on the video display unit 67.

[0136] The electronic device 60 (signal processing unit 64) acquires detection information 65b of the person to be evaluated from the biosensor 10 while the person to be evaluated is performing a task (specific task) of solving a plurality of problems. Upon acquiring the detection information 65b from the biosensor 10, the electronic device 60 (signal processing unit 64) transmits the detection information 65b and the identifier 24d of the person to be evaluated to the electronic device 50 via the communication unit 61.

[0137] The electronic device 50 (signal processing unit 23) acquires the detection information 65b and the identifier 24d of the subject from each electronic device 50 via the communication unit 61, and derives a level of awakening 24e for each subject based on the acquired information. The electronic device 50 (signal processing unit 23) derives a feature 24f corresponding to the classification index 24c for each subject based on the derived level of awakening 24e. The signal processing unit 23 selects one of a plurality of categories (1) to (4) for each subject based on the magnitude of the derived feature 24f (e.g., the magnitude of the duration Δt1 and the rise time Δt2). The signal processing unit 23 evaluates the subject based on the selected category (classification result 24g). The signal processing unit 23 stores, for example, an evaluation result 24h for each subject in the storage unit 24.

[0138] The video data generation unit 25 generates video data for each evaluation subject, in which the classification index 24c and the feature amount 24f derived for evaluation are associated with each other. The video data generation unit 25 generates video data for each evaluation subject, in which the classification index 24c and the time-series data of the awakening level 24e are associated with each other. The video data generation unit 25 outputs the generated video data to the video display unit 26. The video display unit 26 displays a video based on the video data input from the video data generation unit 25. The video display unit 26 displays, for example, videos such as those shown in FIGS. 6 and 7 on the display screen 26A.

[0139] [effect] Next, the effects of the biological information processing system 120 will be described.

[0140] In this embodiment, similarly to the first and second embodiments and their modifications, the arousal levels 24e are classified based on a predetermined classification index 24c. This allows the evaluation subjects to be classified using the arousal levels 24e, which are objective data. As a result, for example, when deciding on project members, it becomes possible to determine whether or not a given subject is suitable to be a member of a specific group based on the arousal levels 24e of a large number of evaluation subjects. This makes it possible to reduce mismatches.

[0141] In this embodiment, the derived awakening level 24e is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. This allows the person being evaluated to be classified using the awakening level 24e, which is objective data. As a result, for example, when deciding on project members, it is possible to determine whether or not a person is suitable to form a specific group based on the awakening levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0142] 5. Fourth Embodiment [composition] Next, an information processing device 130 according to a fourth embodiment of the present disclosure will be described. FIG. 16 illustrates a schematic configuration example of the information processing device 130. The information processing device 130 is an objective evaluation system that evaluates a plurality of target living organisms based on at least one of biometric information and behavioral information obtained from the plurality of target living organisms. In this embodiment, the target living organisms are humans. Note that in the information processing device 130, the target living organisms are not limited to humans.

[0143] The information processing device 130 includes a plurality of (for example, two) devices 131, a signal processing unit 23 connected to the plurality of (for example, two) devices 131, a user input receiving unit 22, and a storage unit 24. Each device 131 is, for example, a device such as eyeglasses, and performs operations similar to those of the electronic devices 20, 40, 50 and the information processing system 120 according to the first to fourth embodiments and their modifications under the control of the signal processing unit 23. That is, in this embodiment, one information processing device 130 is shared by a plurality of users.

[0144] Each device 131 has, for example, a sensor input receiving unit 21a, a video data generating unit 25a, and a video display unit 26a. For example, each device 131 has one biosensor 10 attached thereto.

[0145] In this embodiment, similar to the above-described first and second embodiments and their modified examples, emotional information 16c of the target living organism is estimated based on information of the target living organism (at least one of biometric information and motion information) obtained by biosensor 10, and is displayed on the display surface of video display unit 26a.

[0146] In this embodiment, similarly to the first and second embodiments and their modifications, the arousal levels 24e are classified based on a predetermined classification index 24c. This allows the evaluation subjects to be classified using the arousal levels 24e, which are objective data. As a result, for example, when deciding on project members, it becomes possible to determine whether or not a given subject is suitable to be a member of a specific group based on the arousal levels 24e of a large number of evaluation subjects. This makes it possible to reduce mismatches.

[0147] In this embodiment, the derived awakening level 24e is stored in the storage unit 24 in association with the identifier 24d of the person being evaluated. This allows the person being evaluated to be classified using the awakening level 24e, which is objective data. As a result, for example, when deciding on project members, it is possible to determine whether or not a person is suitable to form a specific group based on the awakening levels 24e of multiple people being evaluated. This makes it possible to reduce mismatches.

[0148] 4. Modifications of each embodiment Next, modifications of the above-described biological information processing systems 100 and 110, the information processing system 120, and the information processing device 130 will be described.

[0149] [Variation A] In the first to fourth embodiments, the storage unit 24 may have, for example, an estimation model 24k for estimating the level of awakening 24e, as shown in FIG. 17. The estimation model 24k estimates the level of awakening 24e based on information (at least one of biological information and behavioral information) obtained from the biosensor 10. The estimation model 24k is, for example, the estimation model described in <1. Regarding the Level of Arousal>. In this case, the storage unit 24 includes, instead of the bioinformation processing program 24a, a bioinformation processing program 24i that realizes the functions (series of processing procedures) of the bioinformation processing program 24a excluding the functions of the estimation model 24k. In this way, by using the estimation model 24k, the level of awakening 24e can be estimated with higher accuracy. As a result, it is possible to further reduce mismatches.

[0150] [Variation B] In the first to fourth embodiments and their modifications, the storage unit 24 may store attribute information 24m, for example, as shown in FIG. 18. The attribute information 24m is, for example, attribute information such as the age, gender, and educational background of the person being evaluated. In this modification, the signal processing unit 23 may evaluate the person being evaluated using, for example, not only the feature amount 24f but also the attribute information 24m. In this manner, by evaluating the person being evaluated using not only the feature amount 24f but also the attribute information 24m, it is possible to estimate the awakening level 24e with higher accuracy. As a result, it is possible to further reduce mismatches.

[0151] [Variation C] In the first to fourth embodiments and their modifications, the electronic devices 20, 40, and 50 and the information processing device 130 may be connected to a server device via an external network. In this case, the server device may include a program or an estimation model that executes a series of processes to estimate the wakefulness level 24e. In this case, it is not necessary to provide the electronic devices 20, 40, and 50 and the information processing device 130 with a program or an estimation model that executes a series of processes to estimate the wakefulness level 24e. As a result, the program or the estimation model that executes a series of processes to estimate the wakefulness level 24e, which is provided in the server device, can be shared among multiple electronic devices 20, multiple electronic devices 40, multiple electronic devices 50, or multiple information processing devices 130.

[0152] [Variation D] In the first to fourth embodiments and their modifications, pleasantness / unpleasantness may be used instead of or in addition to the arousal level 24e. Pleasure / unpleasantness is a type of emotional information, similar to the arousal level 24e. In this modification, a duration of pleasantness may be used instead of or in addition to the duration Δt1. Also, in this modification, an index relating to the rapidity of switching between pleasantness and unpleasantness may be used instead of or in addition to the rise time Δt2.

[0153] In this modified example, at least one of the arousal level 24e and the pleasant / unpleasant level is classified based on a predetermined classification index 24c. This allows the evaluation subjects to be classified using the objective data of the arousal level 24e and the pleasant / unpleasant level. As a result, for example, when recruiting personnel, it becomes possible to determine whether the evaluation subject is a desirable candidate based on the arousal level 24e and the pleasant / unpleasant level of the evaluation subject. Furthermore, for example, when deciding on project members, it becomes possible to determine whether a large number of evaluation subjects are suitable members to form a specific group based on the arousal levels 24e and the pleasant / unpleasant level of the evaluation subject. This makes it possible to reduce mismatches.

[0154] [Variation E] In the first embodiment and its modifications, for example, some functions of the electronic device 20 may be provided in an external device (for example, a server device) separate from the electronic device 20. In this case, the electronic device 20 and the external device (for example, the server device) may be connected via some kind of network, for example.

[0155] Furthermore, in the second embodiment and its modified examples, for example, some functions of the electronic device 40 may be provided in an external device (for example, a server device) separate from the electronic device 40. In this case, the electronic device 40 and the external device (for example, the server device) may be connected via some kind of network, for example.

[0156] Furthermore, in the third embodiment and its modified examples, for example, some functions of the electronic device 50 may be provided in an external device (for example, a server device) separate from the electronic device 50. In this case, the electronic device 50 and the external device (for example, the server device) may be connected, for example, via some kind of network.

[0157] Furthermore, in the above-described fourth embodiment and its modified examples, for example, some functions of the information processing device 130 may be provided in an external device (for example, a server device) separate from the information processing device 130. In this case, the information processing device 130 and the external device (for example, the server device) may be connected, for example, by some kind of network.

[0158] [Variation F] In the first embodiment and its modifications, the electronic device 20 and the biosensor 10 may be connected to each other by means other than the network 30.

[0159] [Variation G] In the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, a head-mounted display (HMD) 200 as shown in Fig. 29. In the head-mounted display 200, the detection electrodes 203 of the biosensor 10 can be provided on, for example, the inner surfaces of the pad part 201 and the band part 202.

[0160] In the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, a headband 300 as shown in Fig. 30. In the headband 300, the detection electrodes 303 of the biosensor 10 can be provided on, for example, the inner surfaces of the band portions 301 and 302 that come into contact with the head.

[0161] Furthermore, in the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, headphones 400 as shown in Fig. 31. In the headphones 400, the detection electrodes 403 of the biosensor 10 can be provided on, for example, the inner surface of a band part 401 that comes into contact with the head, ear pads 402, or the like.

[0162] In the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, an earphone 500 as shown in Fig. 32. In the earphone 500, for example, a detection electrode 502 of the biosensor 10 can be provided on an earpiece 501 that is inserted into the ear.

[0163] Furthermore, in the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, a watch 600 as shown in Fig. 33. In the watch 600, the detection electrodes 604 of the biosensor 10 can be provided on, for example, the inner surface of a display unit 601 that displays the time, etc., or the inner surface of a band unit 602 (for example, the inner surface of a buckle unit 603).

[0164] In the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, eyeglasses 700 as shown in Fig. 34. In the eyeglasses 700, the detection electrodes 702 of the biosensor 10 can be provided on the inner surfaces of temples 701, for example.

[0165] In the first to fourth embodiments and their modifications, the biosensor 10 can be mounted on, for example, a glove, a ring, a pencil, a pen, or a controller for a game machine.

[0166] [Variation H] In the above first to fourth embodiments and their variations, the signal processing unit 23 may derive, for example, the following features based on the electrical signals of the pulse wave, electrocardiogram, and blood flow of the person being evaluated obtained by a sensor, and derive the alertness level 24e of the person being evaluated based on the derived features.

[0167] (pulse wave, electrocardiogram, blood flow) The level of alertness 24e of the person being evaluated can be derived by using, for example, the following feature quantities obtained based on the pulse wave, electrocardiogram, and electrical signals of blood flow obtained by the sensors. Heart rate per second Average heart rate per second within a given period (window) rmssd (root mean square successive difference): root mean square of successive heartbeat intervals pnn50 (percentage of adjacent normal-to-normal intervals): The percentage of consecutive heartbeat intervals exceeding 50 ms LF: Area between 0.04~0.15Hz of PSD of heartbeat interval HF: PSD area between 0.15 and 0.4 Hz LF / (LF+HF) HF / (LF+HF) LF / HF Heartbeat entropy SD1: Standard deviation of the Poincaré plot (a scatter plot with the tth heartbeat interval on the x-axis and the t+1th heartbeat interval on the y-axis) SD2: Standard deviation of the Poincaré plot along the vertical axis of y=x SD1 / SD2 SDRR (standard deviation of RR interval): Standard deviation of heartbeat interval

[0168] Furthermore, in the above-described first to fourth embodiments and their variations, the signal processing unit 23 may derive, for example, the following feature quantities based on the electrical signal (EDA: electrodermal activity) of mental sweating of the subject obtained by a sensor, and derive the alertness level 24e of the subject based on the derived feature quantities.

[0169] (psychotic sweating) By using the following feature values, for example, obtained based on the electrical signal of mental sweating obtained by the sensor, it is possible to derive the level of arousal 24e of the person being evaluated. - Number of SCRs (skin conductance responses) occurring per minute SCR amplitude SCL (skin conductance level) value Rate of change of SCL

[0170] For example, by using the method described in the following document, it is possible to separate SCR and SCL from EDA. Benedek, M., & Kaernbach, C. (2010). A continuous measure of phasic electrodermal activity. Journal of neuroscience methods, 190(1), 80-91.

[0171] In deriving the level of arousal 24e, a single modal (one physiological index) may be used, or a combination of multiple modal (multiple physiological indexes) may be used.

[0172] The signal processing unit 23 derives the above-mentioned feature amounts by using, for example, the regression equations shown in FIGS. 35 to 42, which will be described later.

[0173] FIG. 35 shows an example of the relationship between the task difference Δha [%] of the pnn50 of the pulse wave when solving a high-difficulty problem and a low-difficulty problem, and the accuracy rate R [%] when solving the high-difficulty problem. The task difference Δha is a vector quantity obtained by subtracting the pnn50 of the pulse wave when solving a low-difficulty problem from the pnn50 of the pulse wave when solving a high-difficulty problem. In FIG. 35, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 35, the regression equation is represented by R = a10 × Δha + b10.

[0174] A small task difference Δha in the pnn50 of the pulse wave means that the difference in the pnn50 of the pulse wave is small when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that as the difficulty of the problem increases, the task difference in the pnn50 of the pulse wave tends to be smaller compared to other users. On the other hand, a large task difference Δha in the pnn50 of the pulse wave means that the difference in the pnn50 of the pulse wave is large when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that as the difficulty of the problem increases, the task difference in the pnn50 of the pulse wave tends to be larger compared to other users.

[0175] From Figure 35, it can be seen that when the task difference Δha of the pulse wave pnn50 is large, the correct answer rate R of the problem increases, and when the task difference Δha of the pulse wave pnn50 is small, the correct answer rate R of the problem decreases. From this, it can be seen that people whose pulse wave pnn50 is large for difficult problems tend to have a high correct answer rate R (i.e., they can answer difficult problems as well as easy problems correctly). Conversely, it can be seen that people whose pulse wave pnn50 is small for difficult problems tend to have a low correct answer rate R (i.e., they can answer difficult problems less correctly).

[0176] As described above, it can be seen from Figure 28 that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δha of the pulse wave pnn50 is large, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δha of the pulse wave pnn50 is small, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0177] From the above, it can be seen that it is possible to derive the user's level of alertness by using the task difference Δha of the pulse wave pnn50 and the regression equations of FIGS.

[0178] FIG. 36 shows an example of the relationship between the task difference Δhb [%] in the variability of the pnn50 of the pulse wave when solving a high-difficulty problem and a low-difficulty problem, and the accuracy rate R [%] when solving the high-difficulty problem. The task difference Δhb is a vector quantity obtained by subtracting the variability of the pnn50 of the pulse wave when solving a low-difficulty problem from the variability of the pnn50 of the pulse wave when solving a high-difficulty problem. In FIG. 36, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 36, the regression equation is represented by R = a11 × Δhb + b11.

[0179] A small task difference Δhb in the variation of the pnn50 of the pulse wave means that the difference in the variation of the pnn50 of the pulse wave is small when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that as the difficulty of the problem increases, the task difference in the variation of the pnn50 of the pulse wave tends to become smaller compared to other users. On the other hand, a large task difference Δhb in the variation of the pnn50 of the pulse wave means that the difference in the variation of the pnn50 of the pulse wave is large when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that as the difficulty of the problem increases, the task difference in the variation of the pnn50 of the pulse wave tends to become larger compared to other users.

[0180] From Figure 36, it can be seen that when the task difference Δhb in the variance of the pnn50 of the pulse wave is large, the correct answer rate R of the problem increases, and when the task difference Δhb in the variance of the pnn50 of the pulse wave is small, the correct answer rate R of the problem decreases. From this, it can be seen that people who have a large variance in the pnn50 of the pulse wave for difficult problems tend to have a high correct answer rate R (i.e., they can answer difficult problems as well as easy problems correctly). Conversely, it can be seen that people who have a small variance in the pnn50 of the pulse wave for difficult problems tend to have a low correct answer rate R (i.e., they can answer difficult problems correctly less).

[0181] As described above, it can be seen from Figure 28 that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δhb in the variation of the pulse wave pnn50 is large, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δha in the variation of the pulse wave pnn50 is small, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0182] From the above, it can be seen that it is possible to derive the user's level of wakefulness by using the task difference Δhb in the variation in pnn50 of the pulse wave and the regression equations of FIGS.

[0183] Figure 37 shows the difference in power in the low frequency band (around 0.01 Hz) of the power spectrum obtained by performing FFT on the pnn50 of the pulse wave when solving a high difficulty problem and a low difficulty problem. -2 37 shows an example of the relationship between the frequency [Hz] and the accuracy rate R [%] when solving a high-difficulty problem. Hereinafter, the "power in the low-frequency band (around 0.01 Hz) of the power spectrum obtained by performing an FFT on the pulse wave pnn50" will be referred to as the "power of the low-frequency band of the pulse wave pnn50." The task difference Δhc is a vector quantity obtained by subtracting the power of the low-frequency band of the pulse wave pnn50 when solving a low-difficulty problem from the power of the low-frequency band of the pulse wave pnn50 when solving a high-difficulty problem. In FIG. 37, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 37, the regression equation is expressed as R = a12 × Δhc + b12.

[0184] A large task difference Δhc in the power of the low frequency band of the pulse wave pnn50 means that there is a large difference in the power of the low frequency band of the pulse wave pnn50 when solving a high-difficulty problem and when solving a low-difficulty problem. It can be said that users who obtain such results tend to have a larger task difference in the power of the low frequency band of the pulse wave pnn50 when solving a high-difficulty problem compared to other users. On the other hand, a small task difference Δhc in the power of the low frequency band of the pulse wave pnn50 means that there is a small difference in the power of the low frequency band of the pulse wave pnn50 when solving a high-difficulty problem and when solving a low-difficulty problem. It can be said that users who obtain such results tend to have a smaller task difference in the power of the low frequency band of the pulse wave pnn50 compared to other users when solving a high-difficulty problem.

[0185] From Figure 37, it can be seen that when the task difference Δhc in the power of the low frequency band of the pulse wave pnn50 is large, the correct answer rate R for the problem increases, and when the task difference Δhc in the power of the low frequency band of the pulse wave pnn50 is small, the correct answer rate R for the problem decreases. From this, it can be seen that people with high power in the low frequency band of the pulse wave pnn50 tend to have a high correct answer rate R even for difficult problems (i.e., they can answer difficult problems as well as easy problems correctly). Conversely, it can be seen that people with low power in the low frequency band of the pulse wave pnn50 tend to have a low correct answer rate R (i.e., they can answer difficult problems as well correctly).

[0186] As described above, it can be seen from Figure 28 that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δhc of the power of the low-frequency band of the pulse wave pnn50 is small, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δhc of the power of the low-frequency band of the pulse wave pnn50 is large in the negative direction, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0187] From the above, it can be seen that it is possible to derive the user's level of alertness by using the subject difference Δhc in the power of the low frequency band of the pulse wave pnn50 and the regression equations of Figures 28 and 37.

[0188] FIG. 38 shows an example of the relationship between the task difference Δhd [ms] of the rmssd of the pulse wave when solving a high-difficulty problem and a low-difficulty problem, and the accuracy rate R [%] when solving the high-difficulty problem. The task difference Δhd is a vector quantity obtained by subtracting the rmssd of the pulse wave when solving a low-difficulty problem from the rmssd of the pulse wave when solving a high-difficulty problem. In FIG. 38, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 38, the regression equation is represented by R = a13 × Δhd + b13.

[0189] A large task difference Δhd in the rmssd of the pulse wave means that there is a large difference in the rmssd of the pulse wave when solving a high-difficulty problem and when solving a low-difficulty problem. It can be said that users who obtain such results tend to have a larger task difference in the rmssd of the pulse wave when solving a high-difficulty problem compared to other users. On the other hand, a small task difference Δhd in the rmssd of the pulse wave means that there is a small difference in the rmssd of the pulse wave when solving a high-difficulty problem and when solving a low-difficulty problem. It can be said that users who obtain such results tend to have a smaller task difference in the rmssd of the pulse wave compared to other users when solving a high-difficulty problem.

[0190] From Figure 38, it can be seen that when the task difference Δhd of the rmssd of the pulse wave is large, the correct answer rate R of the problem increases, and when the task difference Δhd of the rmssd of the pulse wave is small, the correct answer rate R of the problem decreases. From this, it can be seen that people with a large rmssd of the pulse wave tend to have a high correct answer rate R even for difficult problems (i.e., they can solve difficult problems as well as easy problems correctly). Conversely, it can be seen that people with a small rmssd of the pulse wave for difficult problems tend to have a low correct answer rate R (i.e., a lower correct answer rate for difficult problems).

[0191] As described above, Fig. 28 shows that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δhd of the rmssd of the pulse wave is small, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δhd of the rmssd of the pulse wave is large in the negative direction, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0192] From the above, it can be seen that it is possible to derive the user's level of alertness by using the subject difference Δhd of the rmssd of the pulse wave and the regression equations of FIGS.

[0193] FIG. 39 shows an example of the relationship between the task difference Δhe [ms] in the rmssd variability of the pulse wave when solving a high-difficulty problem and a low-difficulty problem, and the accuracy rate R [%] when solving the high-difficulty problem. The task difference Δhe is a vector quantity obtained by subtracting the rmssd variability of the pulse wave when solving a low-difficulty problem from the rmssd variability of the pulse wave when solving a high-difficulty problem. In FIG. 39, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 39, the regression equation is expressed as R = a14 × Δhe + b14.

[0194] A large task difference Δhe in the variability of the pulse wave rmssd means that there is a large difference in the variability of the pulse wave rmssd between solving a high-difficulty problem and solving a low-difficulty problem. It can be said that users who obtain such results tend to have a larger task difference in the variability of the pulse wave rmssd when solving a high-difficulty problem compared to other users. On the other hand, a small task difference Δhe in the variability of the pulse wave rmssd means that there is a small difference in the variability of the pulse wave between solving a high-difficulty problem and solving a low-difficulty problem. It can be said that users who obtain such results tend to have a smaller task difference in the variability of the pulse wave rmssd compared to other users as the difficulty of the problem increases.

[0195] From Figure 39, it can be seen that when the task difference Δhe in the variance of the rmssd of the pulse wave is large, the correct answer rate R for the problem increases, and when the task difference Δhe in the variance of the rmssd of the pulse wave is small, the correct answer rate R for the problem decreases. From this, it can be seen that people with large variance in the rmssd of the pulse wave tend to have a high correct answer rate R even for difficult problems (i.e., they can solve difficult problems as well as easy problems correctly). Conversely, it can be seen that people with small variance in the rmssd of the pulse wave tend to have a low correct answer rate R for difficult problems (i.e., a lower correct answer rate for difficult problems).

[0196] As described above, Figure 28 shows that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δh e in the rmssd variation of the pulse wave is small, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δh e in the rmssd variation of the pulse wave is large in the negative direction, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0197] From the above, it can be seen that it is possible to derive the user's level of alertness by using the subject difference Δhe in the rmssd variation of the pulse wave and the regression equations of Figures 28 and 39.

[0198] Figure 40 shows the difference in power in the low frequency band (around 0.01 Hz) of the power spectrum obtained by performing FFT on the rmssd of the pulse wave when solving a high difficulty problem and a low difficulty problem. 2 This figure shows an example of the relationship between the frequency [Hz] of the pulse wave and the accuracy rate R [%] when solving a high-difficulty problem. Hereinafter, the "power in the low-frequency band (around 0.01 Hz) of the power spectrum obtained by performing an FFT on the rmssd of the pulse wave" will be referred to as the "power of the low-frequency band of the rmssd of the pulse wave." The task difference Δhf is a vector quantity obtained by subtracting the power of the low-frequency band of the rmssd of the pulse wave when solving a low-difficulty problem from the power of the low-frequency band of the rmssd of the pulse wave when solving a high-difficulty problem. Figure 40 plots data for each user, and the characteristics of all users are represented by a regression equation (regression line). In Figure 40, the regression equation is expressed as R = a15 × Δhf + b15.

[0199] A large task difference Δhf in the power of the low-frequency band of the rmssd of the pulse wave means that there is a large difference in the power of the low-frequency band of the rmssd of the pulse wave between solving a high-difficulty problem and solving a low-difficulty problem. It can be said that users who obtain such results tend to have a larger task difference in the power of the low-frequency band of the rmssd of the pulse wave when solving a high-difficulty problem compared to other users. On the other hand, a small task difference Δhf in the power of the low-frequency band of the rmssd of the pulse wave means that there is a small difference in the power of the low-frequency band of the rmssd of the pulse wave between solving a high-difficulty problem and solving a low-difficulty problem. It can be said that users who obtain such results tend to have a smaller task difference in the power of the low-frequency band of the rmssd of the pulse wave compared to other users when solving a high-difficulty problem.

[0200] From Figure 40, it can be seen that when the task difference Δhf in the power of the low-frequency band of the rmssd of the pulse wave is large, the correct answer rate R for the problem increases, and when the task difference Δhf in the power of the low-frequency band of the rmssd of the pulse wave is small, the correct answer rate R for the problem decreases. From this, it can be seen that people with a large power in the low-frequency band of the rmssd of the pulse wave tend to have a high correct answer rate R even for difficult problems (i.e., they can solve difficult problems as well as easy problems). Conversely, it can be seen that people with a small power in the low-frequency band of the rmssd of the pulse wave tend to have a low correct answer rate R for difficult problems (i.e., a lower correct answer rate for difficult problems).

[0201] As described above, Fig. 28 shows that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δhf of the power of the low-frequency band of the rmssd of the pulse wave is small, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δhf of the power of the low-frequency band of the rmssd of the pulse wave is large in the negative direction, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0202] From the above, it can be seen that it is possible to derive the user's level of alertness by using the target difference Δhf in the power of the low frequency band of the rmssd of the pulse wave and the regression equations of Figures 28 and 40.

[0203] FIG. 41 shows an example of the relationship between the task difference Δhg [min] in the variance in the number of SCRs for mental sweating when solving a high-difficulty problem and a low-difficulty problem, and the accuracy rate R [%] when solving a high-difficulty problem. The task difference Δhg is a vector quantity obtained by subtracting the variance in the number of SCRs for mental sweating when solving a low-difficulty problem from the variance in the number of SCRs for mental sweating when solving a high-difficulty problem. In FIG. 41, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 41, the regression equation is expressed as R = a16 × Δhg + b16.

[0204] A large task difference Δhg in the variance of the number of SCRs in mental sweating means that there is a large difference in the variance of the number of SCRs in mental sweating when solving high-difficulty problems and when solving low-difficulty problems. For users who obtain such results, it can be said that the task difference in the variance of the number of SCRs in mental sweating tends to be larger when solving high-difficulty problems compared to other users. On the other hand, a small task difference Δhg in the variance of the number of SCRs in mental sweating means that there is a small difference in the variance of the number of SCRs in mental sweating when solving high-difficulty problems and when solving low-difficulty problems. For users who obtain such results, it can be said that there is a tendency for the task difference in the variance of the number of SCRs in mental sweating to be smaller compared to other users as the difficulty of the problems increases.

[0205] Figure 41 shows that when the task difference Δhg in the variance in the number of SCRs for mental sweating is large, the correct answer rate R for the problem increases, and when the task difference Δhg in the variance in the number of SCRs for mental sweating is small, the correct answer rate R for the problem decreases. This shows that people with a large variance in the number of SCRs for mental sweating, even for difficult problems, tend to have a high correct answer rate R (i.e., they can answer difficult problems as well as easy problems correctly). Conversely, people with a small variance in the number of SCRs for mental sweating for difficult problems tend to have a low correct answer rate R (i.e., a lower correct answer rate for difficult problems).

[0206] As described above, Fig. 28 shows that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δhg of the variation in the number of SCRs for mental sweating is small, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δhg of the variation in the number of SCRs for mental sweating is large in the negative direction, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0207] From the above, it can be seen that it is possible to derive the user's level of alertness by using the task difference Δhgf of the variation in the number of SCRs of mental sweating and the regression equations of FIGS.

[0208] FIG. 42 shows an example of the relationship between the task difference Δhh [ms2 / Hz] between the number of SCRs for mental sweating when solving a high-difficulty problem and a low-difficulty problem, and the accuracy rate R [%] when solving a high-difficulty problem. The task difference Δhh is a vector quantity obtained by subtracting the number of SCRs for mental sweating when solving a low-difficulty problem from the number of SCRs for mental sweating when solving a high-difficulty problem. In FIG. 42, data for each user is plotted, and the characteristics of all users are represented by a regression equation (regression line). In FIG. 42, the regression equation is expressed as R = a17 × Δhh + b17.

[0209] A large task difference Δhh in the number of SCRs for mental sweating means that there is a large difference in the number of SCRs for mental sweating when solving a high-difficulty problem and when solving a low-difficulty problem. For users who obtain such results, it can be said that the task difference in the number of SCRs for mental sweating tends to be larger compared to other users when solving high-difficulty problems. On the other hand, a small task difference Δhh in the number of SCRs for mental sweating means that there is a small difference in the number of SCRs for mental sweating when solving high-difficulty problems and when solving low-difficulty problems. For users who obtain such results, it can be said that there is a tendency that the task difference in the number of SCRs for mental sweating tends to be smaller compared to other users as the difficulty of the problems increases.

[0210] From Figure 42, we can see that when the task difference Δhh in the number of SCRs for mental sweating is large, the correct answer rate R for the problem increases, and when the task difference Δhh in the number of SCRs for mental sweating is small, the correct answer rate R for the problem decreases. From this, we can see that people with a large number of SCRs for mental sweating tend to have a high correct answer rate R even for difficult problems (i.e., they can answer difficult problems as well as easy problems correctly). Conversely, we can see that people with a small number of SCRs for mental sweating for difficult problems tend to have a low correct answer rate R (i.e., a lower correct answer rate for difficult problems).

[0211] As described above, Figure 28 shows that when the accuracy rate is high, the level of alertness is low, and when the accuracy rate is low, the level of alertness is high. From the above, when the task difference Δhh in the number of SCRs for mental sweating is small, it can be inferred that the user's level of alertness is lower than a predetermined standard. Also, when the task difference Δhh in the number of SCRs for mental sweating is large in the negative direction, it can be inferred that the user's level of alertness is higher than a predetermined standard.

[0212] From the above, it can be seen that it is possible to derive the user's level of alertness by using the task difference Δhh in the number of SCRs for mental sweating and the regression equations of FIGS.

[0213] Furthermore, in the regression equations according to the first to fourth embodiments and their modified examples, for example, as shown in FIG. 43, the task difference Δtv of the median of reaction time may be used instead of the task difference Δtv of the variability of reaction time.

[0214] In the first to fourth embodiments and their modifications, the regression equation is not limited to a straight line (regression line) and may be, for example, a curve (regression curve). The curve (regression curve) may be, for example, a quadratic function. The regression equation defining the relationship between the awakening level k [%] and the accuracy rate R [%] may be, for example, a quadratic function (R = a × k2 + bk + c) as shown in FIG.

[0215] Furthermore, for example, the present disclosure can be configured as follows. (1) a derivation unit that derives emotional information of a target living organism based on at least one of biological information and behavioral information obtained from the target living organism while the target living organism is performing a specific task; a classification unit that classifies the emotional information obtained by the derivation unit based on a predetermined classification index; Equipped with Biometric information processing device. (2) The apparatus further includes an evaluation unit that evaluates the target living body based on the classification result by the classification unit. The biometric information processing device according to (1). (3) Further comprising a storage unit, the classification unit stores the classification result by the classification unit in the storage unit in association with an identifier of the target living body each time the derivation unit obtains the emotional information; The evaluation unit selects a plurality of the identifiers suitable for forming a specific group based on the plurality of classification results stored in the storage unit. (2) A biometric information processing device. (4) Further comprising a storage unit, The derivation unit derives a feature corresponding to the classification index based on the emotional information, and stores the derived feature in the storage unit in association with an identifier of the target living organism. The biometric information processing device according to any one of (1) to (3). (5) The image data generating unit generates image data in which the classification index and the feature amount are associated with each other. (4) A biometric information processing device. (6) The derivation unit derives time-series data as the emotional information, and stores the derived time-series data in the storage unit in association with an identifier of the target living body. The biometric information processing device according to any one of (1) to (3). (7) The image data generating unit generates image data in which the classification index and the time-series data are associated with each other. (6) A biometric information processing device. (8) The biological information is information on brain waves, sweating, heart rate, blood flow velocity, or specific components contained in saliva. The biometric information processing device according to any one of (1) to (7). (9) The behavioral information is information about facial expressions, vocalizations, or reaction times. The biometric information processing device according to any one of (1) to (7). (10) The emotional information is at least one of the arousal level and the pleasantness or unpleasantness of the target living body. The biometric information processing device according to any one of (1) to (9). (11) A memory unit; a derivation unit that derives emotional information of a target living organism based on at least one of biological information and behavioral information obtained from the target living organism while the target living organism is performing a specific task, and stores the derived emotional information in the storage unit in association with an identifier of the target living organism; Equipped with Biometric information processing device. (12) each time the derivation unit derives the emotional information, the derivation unit associates the derived emotional information with an identifier of the target living organism and stores the associated emotional information in the storage unit; The biometric information processing device further includes a video data generation unit that generates video data in which the emotional information corresponding to the plurality of identifiers is compiled in a manner that allows comparison with each other. (11) The biometric information processing device. (13) a receiving unit that receives, from a user, a selection of a plurality of pieces of emotional information among the plurality of pieces of emotional information or a selection of a plurality of identifiers among the plurality of identifiers; The system further includes a selection unit that selects a plurality of the identifiers suitable for forming a specific group based on the content received by the reception unit. (12) The biometric information processing device according to (12). (14) each time the derivation unit derives the emotional information, the derivation unit associates the derived emotional information with an identifier of the target living organism and stores the associated emotional information in the storage unit; The biometric information processing device further includes a selection unit that selects a plurality of the identifiers suitable for forming a specific group based on the plurality of pieces of emotional information stored in the storage unit and a predetermined classification index. (11) The biometric information processing device. (15) the derivation unit derives time-series data as the emotional information, associates the derived time-series data with an identifier of the target living body, and stores the associated time-series data in the storage unit; The video data generating unit generates, as the video data, video data in which the time-series data corresponding to the plurality of identifiers are superimposed on each other in a time-aligned manner. (12) The biometric information processing device according to (12). (16) The biological information is information on brain waves, sweating, pulse waves, electrocardiograms, blood flow, skin temperature, facial myoelectric potential, electrooculography, or specific components contained in saliva. The biometric information processing device according to any one of (11) to (15). (17) The behavioral information is information about facial expressions, voice, blinking, breathing, or behavioral reaction times. The biometric information processing device according to any one of (11) to (15). (18) The emotional information is at least one of the arousal level and the pleasantness or unpleasantness of the target living body. The biometric information processing device according to any one of (11) to (17). (19) an acquisition unit that acquires at least one of biological information and behavioral information from a target living organism while the target living organism is performing a specific task; a derivation unit that derives emotional information of the target living organism based on the information obtained by the acquisition unit; a classification unit that classifies the emotional information obtained by the derivation unit based on a predetermined classification index; Equipped with Biometric information processing system. (20) A memory unit; an acquisition unit that acquires at least one of biological information and behavioral information from a target living organism while the target living organism is performing a specific task; a derivation unit that derives emotional information of the target living organism based on the information obtained by the acquisition unit, and stores the derived emotional information in the storage unit in association with an identifier of the target living organism; Equipped with Biometric information processing system.

[0216] In a biometric information processing device according to a first aspect of the present disclosure and a biometric information processing system according to a second aspect of the present disclosure, arousal levels are classified based on a predetermined classification index. This allows target living organisms to be classified using arousal levels, which are objective data. As a result, for example, in recruiting personnel, it becomes possible to determine whether an applicant is a desirable candidate based on their arousal levels. Furthermore, for example, in deciding on project members, it becomes possible to determine whether a member is suitable to form a specific group based on the arousal levels of multiple members. This makes it possible to reduce mismatches.

[0217] In a biometric information processing device according to a third aspect of the present disclosure and a biometric information processing system according to a fourth aspect of the present disclosure, the derived arousal level is stored in a storage unit in association with an identifier of the target living organism. This allows the target living organism to be classified using the arousal level, which is objective data. As a result, for example, in recruiting personnel, it is possible to determine whether an applicant is a desirable candidate based on the arousal level of the applicant. Furthermore, for example, in deciding on project members, it is possible to determine whether a member is suitable to form a specific group based on the arousal levels of multiple members. This makes it possible to reduce mismatches. [Explanation of symbols]

[0218] 10...biometric sensor, 20...electronic device, 21...sensor input receiving unit, 22...user input receiving unit, 23...signal processing unit, 24...storage unit, 24a...biometric information processing program, 24b...task data, 24c...classification index, 24d...identifier, 24e...alertness level, 24f...feature amount, 24g...classification result, 24h...evaluation result, 24i...biometric information processing program, 24k...estimation model, 24m...attribute information, 25...video data generation unit, 26...video display unit, 26A...display screen, 30...network, 40...electronic device, 41...biometric sensor, 100, 110...biometric information processing Processing system, 200...head mounted display, 201...pad section, 202...band section, 203...detection electrode, 300...headband, 301, 302...band section, 303...detection electrode, 400...headphones, 401...band section, 402...ear pad, 403...detection electrode, 500...earphone, 501...earpiece, 502...detection electrode, 600...watch, 601...display section, 602...band section, 603...buckle section, 604...detection electrode, 700...glasses, 701...temple, 702...detection electrode, Δt1...duration, Δt2...rise time.

Claims

1. a derivation unit that derives emotional information of a target living organism based on at least one of biological information and behavioral information obtained from the target living organism while the target living organism is performing a specific task; a classification unit that classifies the emotional information obtained by the derivation unit based on a predetermined classification index; an evaluation unit that evaluates the target living body based on the classification result by the classification unit; a memory unit in which evaluation criteria for the person being evaluated are stored; Equipped with the emotional information is at least one of arousal level and pleasantness / unpleasantness of the target organism, the classification unit stores the classification result by the classification unit in the storage unit in association with an identifier of the target living body each time the derivation unit obtains the emotional information; The evaluation unit extracts a plurality of classification results that match the evaluation criteria from the plurality of classification results stored in the storage unit, and selects a plurality of the identifiers corresponding to the extracted plurality of classification results. Biometric information processing device.

2. The derivation unit derives a feature corresponding to the classification index based on the emotional information, and stores the derived feature in the storage unit in association with an identifier of the target living organism. The biometric information processing device according to claim 1 .

3. The image data generating unit generates image data in which the classification index and the feature amount are associated with each other. The biometric information processing device according to claim 2 .

4. The derivation unit derives time-series data as the emotional information, and stores the derived time-series data in the storage unit in association with an identifier of the target living body. The biometric information processing device according to claim 1 .

5. The image data generating unit generates image data in which the classification index and the time-series data are associated with each other. The biometric information processing device according to claim 4 .

6. The biological information is information on brain waves, sweating, heart rate, blood flow velocity, or specific components contained in saliva. The biometric information processing device according to claim 1 .

7. The behavioral information is information about facial expressions, vocalizations, or reaction times. The biometric information processing device according to claim 1 .

8. an acquisition unit that acquires at least one of biological information and behavioral information from a target living organism while the target living organism is performing a specific task; a derivation unit that derives emotional information of the target living organism based on the information obtained by the acquisition unit; a classification unit that classifies the emotional information obtained by the derivation unit based on a predetermined classification index; an evaluation unit that evaluates the target living body based on the classification result by the classification unit; a memory unit in which evaluation criteria for the person being evaluated are stored; Equipped with the emotional information is at least one of arousal level and pleasantness / unpleasantness of the target organism, the classification unit stores the classification result by the classification unit in the storage unit in association with an identifier of the target living body each time the derivation unit obtains the emotional information; The evaluation unit extracts a plurality of classification results that match the evaluation criteria from the plurality of classification results stored in the storage unit, and selects a plurality of the identifiers corresponding to the extracted plurality of classification results. Biometric information processing system.

Images