Information processing method, information processing device, and learning method

Abstract

The present technology relates to an information processing method, an information processing device, and a learning method that make it possible to accurately infer emotion. This information processing method involves: using a plurality of biological signals that are included in a signal set and are obtained by measuring the biological reactions of a user on a respective plurality of channels to extract an associative feature quantity that represents the relationship between the plurality of biological signals included in the signal set; and using the associative feature quantities extracted for respective signal sets that can be combined in one sample to infer the emotion of the user. The present technology can be applied, for example, to a system that infers emotion.

Description

Information processing method, information processing device, and learning method

[0001] The present technology relates to an information processing method, an information processing device, and a learning method, and in particular to an information processing method, an information processing device, and a learning method that enable emotion estimation to be performed with high accuracy.

[0002] When a person's internal state, such as emotion, changes, biological responses such as brain waves are expressed. Generally, emotion estimation systems estimate a user's emotion by, for example, measuring brain waves as biological signals using a measurement device, extracting features from the biological signals, and inputting the features into an emotion estimation model obtained by machine learning.

[0003] However, when such technology is applied to applications used in daily life or in real environments, signals other than EEG signals, such as electromyography, electrooculography, and body movements, are superimposed on the biosignals as noise, resulting in a decrease in signal quality and a deterioration in the accuracy of emotion estimation.In measurement devices used in daily life or in real environments, the number of electrodes that come into contact with the skin is limited, so a decrease in the quality of the biosignal measured by a single electrode significantly affects the accuracy of emotion estimation.

[0004] In response to this, Patent Document 1 describes a technology for improving the robustness of emotion estimation against noise by taking into account the signal quality of a single biological signal in emotion estimation based on features extracted from the biological signal.

[0005] International Publication No. 2023 / 286313

[0006] The brain exchanges information not only within each region but also between regions in cooperation. For example, a single process is not completed within the frontal lobe; information is exchanged between the frontal and temporal lobes. With regard to human emotions, rewards and penalties related to pleasure and displeasure are processed in the left and right frontal regions of the brain. Therefore, it is necessary to distinguish between high arousal pleasant states and high arousal unpleasant states based on the relationship between the difference, correlation, and mutual information between biosignals measured in the left and right frontal lobes.

[0007] The present technology has been made in consideration of such circumstances, and is intended to enable emotion estimation to be performed with high accuracy.

[0008] An information processing method according to a first aspect of the present technology includes extracting connectivity features indicating a relationship between a plurality of biosignals included in a signal set, based on a plurality of biosignals obtained by measuring a user's bioreactions on each of a plurality of channels included in the signal set, and estimating the user's emotions based on the connectivity features extracted for each of the signal sets that can be combined within one sample.

[0009] An information processing device according to a first aspect of the present technology includes: a feature extraction unit that extracts connectivity features that indicate relationships between multiple biological signals included in a signal set, based on multiple biological signals obtained by measuring a user's biological reactions on each of multiple channels included in the signal set; and an emotion estimation unit that estimates the user's emotions based on the connectivity features extracted for each of the signal sets that can be combined within one sample.

[0010] A learning method according to a second aspect of the present technology includes: extracting connectivity features indicating a relationship between a plurality of biosignals included in a signal set, based on a plurality of biosignals obtained by measuring a person's bioreactions on each of a plurality of channels included in the signal set; determining signal quality on a sample-by-sample basis; and performing learning using the connectivity features extracted for each of the signal sets that can be combined within the sample, the determination results of the signal quality on a sample-by-sample basis, and emotion labels that indicate the emotion of the person associated with the sample, to obtain model parameters used for emotion estimation.

[0011] In a first aspect of the present technology, a connectivity feature indicating the relationship between a plurality of biosignals included in a signal set is extracted based on a plurality of biosignals obtained by measuring a user's bioreactions on each of a plurality of channels included in the signal set, and the user's emotions are estimated based on the connectivity feature extracted for each of the signal sets that can be combined within one sample.

[0012] In a second aspect of the present technology, based on a plurality of biosignals obtained by measuring a person's bioreactions on each of a plurality of channels included in a signal set, connectivity features indicating the relationship between the plurality of biosignals included in the signal set are extracted, signal quality on a sample-by-sample basis is determined, and learning is performed using the connectivity features extracted for each of the signal sets that can be combined within the sample, the determination results of the signal quality on a sample-by-sample basis, and emotion labels that indicate the emotion of the person associated with the sample, and model parameters used for emotion estimation are obtained.

[0013] FIG. 1 is a diagram illustrating an example configuration of an emotion estimation system according to an embodiment of the present technology. FIG. 1 is a first diagram illustrating an example of a device applicable as a measurement device. FIG. 2 is a second diagram illustrating an example of a device applicable as a measurement device. FIG. 2 is a block diagram illustrating an example functional configuration of an information processing device according to a first embodiment of the present technology. FIG. 3 is a block diagram illustrating an example detailed configuration of a connectivity feature extraction unit. FIG. 4 is a flowchart illustrating processing performed by an information processing device. FIG. 4 is a block diagram illustrating an example detailed configuration of a connectivity feature extraction unit according to a second embodiment. FIG. 5 is a block diagram illustrating an example configuration of a connectivity feature extraction unit according to a third embodiment. FIG. 6 is a block diagram illustrating an example configuration of a connectivity feature extraction unit according to a fourth embodiment. FIG. 7 is a block diagram illustrating an example configuration of a connectivity feature extraction unit according to a fifth embodiment. FIG. 8 is a block diagram illustrating an example configuration of an information processing device according to a sixth embodiment. FIG. 9 is a diagram illustrating an example detailed configuration of a sample signal quality determination unit. FIG. 10 is a diagram illustrating an example table. FIG. 11 is a block diagram illustrating a modified configuration of the information processing device according to the sixth embodiment. FIG. 12 is a block diagram illustrating an example configuration of an information processing device according to a seventh embodiment. FIG. 13 is a block diagram illustrating an example detailed configuration of an emotion estimation model learning unit. FIG. 14 is a block diagram illustrating a first modified configuration of the emotion estimation model learning unit. FIG. 15 is a block diagram illustrating a second modified configuration of the emotion estimation model learning unit. It is a first block diagram showing a modified example of the configuration of the information processing device. It is a second block diagram showing a modified example of the configuration of the information processing device. It is a third block diagram showing a modified example of the configuration of the information processing device. It is a block diagram showing an example of the configuration of computer hardware.

[0014] Hereinafter, embodiments for carrying out the present technology will be described. The description will be made in the following order: 1. System configuration 2. First embodiment (basic configuration) 3. Second embodiment 4. Third embodiment 5. Fourth embodiment 6. Fifth embodiment 7. Sixth embodiment 8. Seventh embodiment 9. Modified example of system configuration

[0015] 1. System Configuration FIG. 1 is a diagram illustrating an example configuration of an emotion estimation system according to an embodiment of the present technology.

[0016] The emotion estimation system in FIG. 1 is a system that estimates the emotion of a user based on a biological signal obtained by measuring the biological reaction of the user.

[0017] 1 is configured by connecting a measurement device 1 and an information processing device 2 via a network such as a local area network (LAN) or the Internet. The measurement device 1 and the information processing device 2 may also be connected by wire.

[0018] The measuring device 1 is a device that measures biological responses such as electroencephalograms, acceleration, angular acceleration, and contact impedance as biological signals. The measuring device 1 is equipped with an electroencephalogram sensor, an acceleration sensor, a gyro sensor, an impedance sensor, and the like as biological sensors that measure the user's biological responses. The measuring device 1 is worn directly on the user's body to measure the biological responses.

[0019] The measuring device 1 transmits a biological signal to the information processing device 2 .

[0020] The information processing device 2 is configured by a PC, a smartphone, a tablet terminal, a server, etc. The information processing device 2 receives the biosignal transmitted from the measurement device 1 and performs signal processing on the biosignal to estimate the user's emotion.

[0021] 2 and 3 are diagrams showing examples of devices that can be used as the measuring device 1. FIG.

[0022] 2A, the measurement device 1 is configured by, for example, a head-mounted display. The head-mounted display has at least a pad portion 11 and a band portion 12 that come into contact with the user's head.

[0023] In the head-mounted display, a plurality of biosensors are provided at predetermined locations on the pad unit 11 and the band unit 12. For example, electrodes for measuring brain waves on the left and right sides of at least one of the user's forehead, temporal region, and occipital region are provided on the pad unit 11 and the band unit 12.

[0024] 2B, the measurement device 1 is configured, for example, as a headband. The headband has a plurality of biosensors provided at predetermined locations that come into contact with the user's head. For example, the headband has electrodes for measuring electroencephalograms on the left and right sides of at least one of the user's forehead, temporal region, and occipital region.

[0025] 3C, the measurement device 1 is configured by, for example, headphones. The headphones have at least a band portion 21 that contacts the top of the user's head and ear pads 22L and 22R that contact the areas around the ears of the user's head.

[0026] In the headphones, a plurality of biosensors (Around-Ear EEG) are provided at predetermined locations on the band 21 and the ear pads 22L, 22R. For example, electrodes for measuring brain waves at least on the top of the user's head or around the ears are provided on the band 21 and the ear pads 22L, 22R.

[0027] 3D, the measurement device 1 is configured by, for example, an earphone (in-ear headphone), which has at least an earpiece 31 to be inserted into the user's ear.

[0028] In the earphone, a plurality of biosensors (In-Ear EEG) are provided at predetermined locations on the earpiece 31. For example, electrodes for measuring brain waves inside the user's ear are provided on the earpiece 31.

[0029] 3E, the measurement device 1 is configured by, for example, smart glasses (eyeglasses). The smart glasses have at least temples 41L and 41R that are worn above the user's ears.

[0030] In the smart glasses, a plurality of biosensors are provided at predetermined locations on the temples 41L and 41R. For example, electrodes for measuring brain waves around the user's ears are provided on the temples 41L and 41R.

[0031] Furthermore, the measurement device 1 may be configured as an electroencephalograph that covers the entire head, or a hat provided with multiple electrodes for measuring electroencephalograms. As described above, the measurement device 1 is provided with multiple sensors so that potentials can be measured at multiple locations on the user's head.

[0032] 2. First Embodiment (Basic Configuration) In the brain, information is exchanged not only within each region but also between regions in cooperation with each other. For example, a single process is not completed within the frontal lobe, but information is exchanged between the frontal and temporal lobes to carry out processing. With regard to human emotions, rewards and penalties related to pleasure and displeasure are processed in the left and right frontal regions of the brain. Therefore, it is expected that a high arousal pleasant state and a high arousal unpleasant state can be distinguished based on the relationship between the difference, correlation, mutual information, etc. between biosignals measured in the left frontal lobe and the right frontal lobe.

[0033] However, it has been difficult to improve robustness against noise in emotion estimation based on the relationships between multiple biological signals.

[0034] In the following, an example will be described in which the information processing device 2 estimates the high arousal pleasant state / high arousal unpleasant state as an emotion based on a plurality of biological signals measured in a plurality of regions of the user's body.

[0035] FIG. 4 is a block diagram showing an example of a functional configuration of the information processing device 2 according to the first embodiment of the present technology.

[0036] As shown in FIG. 4, the information processing device 2 includes a feature extraction unit 101 and an emotion determination unit 102 .

[0037] The feature extraction unit 101 extracts biosignals X from channels 1 to n (n is an integer of 2 or more) measured by the measurement device 1. raw Extract the time series of features from the biological signal X raw is a signal indicating time-series changes in biological responses such as electroencephalograms, and includes, for example, at least an electroencephalogram signal obtained by measuring electroencephalograms in the left region of the user's head and an electroencephalogram signal obtained by measuring electroencephalograms in the right region of the head. Channels 1 to n correspond to n biological sensors provided in the measurement device 1.

[0038] The feature extraction unit 101 includes a single-channel feature extraction unit 111 and a connectivity feature extraction unit 112 .

[0039] The single-channel feature extraction unit 111 extracts biosignals X raw From each, the time series of features X single-feat (In the following, we will use the time-series single-channel feature X single-feat ) and extract the time-series single-channel feature quantity X single-feat is supplied to the emotion determination unit 102.

[0040] The connectivity feature extracting unit 112 extracts the biosignal X included in the signal set. raw The time series X of connectivity features that show relationships such as differences, correlations, and mutual information connect-feat (In the following, the time series connectivity feature X connect-feat The signal set includes biosignals of multiple channels (two channels in the first embodiment) among channels 1 to n. The time-series connectivity feature X connect-feat is supplied to the emotion determination unit 102.

[0041] The emotion determination unit 102 receives the biological signal X raw The emotion discrimination unit 102 has a classifier (emotion estimation model) that receives as input a time series of features extracted from the time series of features X supplied from the single-channel feature extraction unit 111 and outputs the user's emotion estimation result (a value in which a high arousal pleasant state is set to 1 and a high arousal unpleasant state is set to 0). single-feat and the time-series connectivity feature X supplied from the connectivity feature extraction unit 112 connect-featBy inputting these into the classifier, it functions as an emotion estimation unit that estimates the user's emotion.

[0042] The emotion determination unit 102 may estimate a time series of the user's emotions or estimate the user's emotions in a certain time window. Instead of a time series of features, features at a certain time may be input to the classifier, and the user's emotions at that time may be estimated.

[0043] FIG. 5 is a block diagram showing a detailed configuration example of the connectivity feature extracting unit 112.

[0044] As shown in FIG. 5, the connectivity feature extracting unit 112 includes a connectivity feature calculating unit 151 , single-channel signal quality determining units 152 i and 152 j , a correction parameter determining unit 153 , and a correcting unit 154 .

[0045] The connectivity feature calculation unit 151 calculates the biosignals X raw Among these, the biological signal X of the i (i is an arbitrary integer from 1 to n) channel included in a signal set is raw (ch=i) and j (j is any integer from 1 to n other than i) channel biosignal X raw (ch=j) and the biological signal X raw (ch=i) and biological signal X raw Time series connectivity feature X that indicates the relationship between (ch=j) connect-feat_origin The binding feature calculation unit 151 calculates the time-series binding feature X connect-feat_origin is supplied to the correction unit 154.

[0046] The single-channel signal quality determination unit 152i determines the biological signal X raw Analyze the waveform of (ch=i) and obtain the biological signal X raw The single-channel signal quality determining unit 152i determines the signal quality of the biological signal X (ch=i). raw The signal quality score S(ch=i) obtained by quantifying the signal quality of (ch=i) is supplied to the correction parameter determination unit 153 .

[0047] The single-channel signal quality determination unit 152j determines the biological signal X raw Analyze the waveform of (ch=j) and obtain the biological signal X rawThe single-channel signal quality determining unit 152j determines the signal quality of the biological signal X (ch=j). raw The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the correction parameter determination unit 153.

[0048] When there is no need to particularly distinguish between the single-channel signal quality determining unit 152i and the single-channel signal quality determining unit 152j, they will be simply referred to as the single-channel signal quality determining unit 152.

[0049] The correction parameter determination unit 153 acquires a correction parameter α based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 152i and the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 152j. The correction parameter α is calculated based on the time-series connectivity feature X connect-feat_origin The correction parameter determination unit 153 supplies the correction parameter α to the correction unit 154.

[0050] The correction unit 154 calculates the time-series binding feature X supplied from the binding feature calculation unit 151 using the correction parameter α supplied from the correction parameter determination unit 153. connect-feat_origin is corrected, and the corrected time-series connectivity feature X connect-feat Output (ch=i,j).

[0051] FIG. 6 is a diagram for explaining the details of the processing performed by the connectivity feature extracting unit 112.

[0052] As shown in the upper left of FIG. 6, the single-channel signal quality determining unit 152i determines the biological signal X raw Analyze the waveform of (ch=i) and obtain the biological signal X raw A signal quality score S(ch=i) of (ch=i) is determined. For example, if the signal quality is the worst, the signal quality score value is set to 0.0, and if the biological signal is not superimposed with noise, i.e., the signal quality is the best, the signal quality score value is set to 1.0. In the following, the signal quality score will be described as being treated as time-series information, but this is not necessarily the case.

[0053] As shown in the lower left of FIG. 6, the single-channel signal quality determining unit 152j determines the biological signal Xraw Analyze the waveform of (ch=j) and obtain the biological signal X raw Determine the signal quality score S(ch=j) for (ch=j).

[0054] The correction parameter determination unit 153 has a table T1 in which parameter values ​​corresponding to the signal quality score S(ch=i) and the signal quality score S(ch=j) are recorded, as shown on the right side of Fig. 6. The correction parameter determination unit 153 refers to the table T1 and determines a parameter value α score is set as the correction parameter α.

[0055] In the table T1, for example, when S(ch=i)=1.0 and S(ch=j)=1.0, the parameter value α score = 1.0 is recorded. For example, when the value of the signal quality score S(ch=i) determined by the single-channel signal quality determining unit 152i and the value of the signal quality score S(ch=j) determined by the single-channel signal quality determining unit 152j are both 1.0, the correction parameter determining unit 153 sets the correction parameter α to 1.0.

[0056] The correction unit 154 applies an IIR (Infinite Impulse Response) filter based on the correction parameter α determined by the correction parameter determination unit 153 to the time-series binding feature X calculated by the binding feature calculation unit 151. connect-feat_origin The IIR filter based on the correction parameter α is expressed by, for example, the following equation (1).

[0057] In the following formula (1), for the sake of simplicity, the time-series connectivity feature value X before correction is connect-feat_origin X origin The corrected time-series connectivity feature X connect-feat is set to X. In the following formula (1), the time at which the feature is measured is indicated by a subscript.

[0058]

[0059] As shown in formula (1), the correction unit 154 sets a time t to be corrected, and calculates the connectivity feature X origin_t and the connectivity feature X at time t-1, which is earlier than time t. t-1 and are weighted based on the correction parameter α to obtain the connectivity feature X origin_t By correcting the time direction, the corrected connectivity feature X at time t is ｔ is calculated.

[0060] Here, the biological signal X at time t is raw (ch=i) and biological signal X raw The worse the signal quality of (ch=j) (the lower the signal quality scores S(ch=i), S(ch=j)), the greater the connectivity feature X t-1 The correction parameter α is determined so that the weight of the biological signal X at time t is large. raw (ch=i) and biological signal X raw The better the signal quality of (ch=j) (the higher the signal quality scores S(ch=i), S(ch=j)), the greater the connectivity feature X origin_t The correction parameter α is determined so that the weight of is large.

[0061] The connectivity feature at the time to be corrected and the connectivity feature at a time earlier than the time to be corrected are weighted and added based on the signal quality at the time to be corrected, thereby reducing the effect of noise on the connectivity feature and improving the robustness of emotion estimation against noise.

[0062] The connectivity feature extraction unit 112 sets as signal sets all possible combinations of pairs from among the multiple biosignals contained in one sample among the biosignals measured by the measurement device 1, and performs the above-described processing for each signal set. In this way, the connectivity feature extraction unit 112 extracts time-series features (after correction) for the number of signal sets that can be combined in one sample.

[0063] Here, a sample is a group of bio-signals used to estimate a user's emotion in a given time window, also called an instance.

[0064] When performing correction in the spatial direction, the correction unit 154 sets a signal set (e.g., ch=i, j) to be corrected, and calculates the connectivity feature X connect-feat_origin (ch=i,j) and the connectivity feature X for other signal sets (e.g., ch=g,h) that serve as correction terms. connect-feat_origin (ch=g, h) are weighted and added to obtain the connectivity feature X connect-feat_origin Correction is made in the spatial direction for (ch=i,j).

[0065] Here, for example, a biological signal X included in the signal set to be corrected is raw (ch=i) and biological signal X raw A signal set including a biological signal whose measurement position is near the measurement position of (ch=j) is set as a signal set of a correction term. raw (ch=i) and biological signal X raw The worse the signal quality of (ch=j) (the lower the signal quality scores S(ch=i), S(ch=j)), the greater the connectivity feature X connect-feat_origin The correction parameter α is determined so that the weight of (ch=g, h) is increased.

[0066] Next, the processing performed by the information processing device 2 having the above configuration will be described with reference to the flowchart of FIG.

[0067] In step S1, the single-channel feature extracting unit 111 extracts time-series single-channel features of each biological signal contained in a sample.

[0068] In step S2, the connectivity feature calculation unit 151 calculates the time-series connectivity feature for each signal set that can be combined in a certain sample.

[0069] In step S3, the single-channel signal quality determining unit 152 determines the signal quality score of each biological signal.

[0070] In step S4, the correction parameter determination unit 153 determines correction parameters for correcting the time-series connectivity features for the signal set to be corrected, based on the signal quality scores of each channel included in the signal set to be corrected.

[0071] In step S5, the correction unit 154 corrects the time-series connectivity feature for the signal set to be corrected, using the correction parameters determined by the correction parameter determination unit 153.

[0072] In step S6, the emotion determination unit 102 estimates the user's emotion based on the time-series single-channel feature extracted by the single-channel feature extraction unit 111 and the time-series connectivity feature corrected by the correction unit 154.

[0073] As described above, in the information processing device 2 of the present technology, connectivity features indicating the relationships between multiple biological signals included in a signal set are extracted based on the multiple biological signals included in the signal set, and the user's emotions are estimated based on the connectivity features extracted for each of all signal sets that can be combined within one sample.

[0074] The information processing device 2 is capable of accurately identifying (estimating) emotions (e.g., high arousal pleasant states and high arousal unpleasant states) in which information is exchanged between parts of the brain in cooperation with one another, based on the relationship between multiple biological signals.

[0075] Furthermore, in the information processing device 2 of the present technology, a time-series connectivity feature is calculated based on a plurality of biological signals, the signal quality of each biological signal is determined, and the time-series connectivity feature is corrected based on the determination result of the signal quality (signal quality score) for each biological signal, thereby extracting the time-series connectivity feature.

[0076] Since the time-series connectivity feature is corrected based on the signal quality score for each biological signal, the information processing device 2 can improve robustness against noise in emotion estimation based on the relationship between multiple biological signals.

[0077] Furthermore, when a small number of biosensors are mounted on the measurement device 1, even if noise is superimposed on the biosignals measured by each biosensor, the influence of the noise on the connectivity features can be reduced, making it possible to accurately determine the user's emotion based on all of the biosignals. Therefore, not only measurement devices 1 equipped with a large number of biosensors but also measurement devices 1 equipped with a small number of biosensors can be used to estimate the user's emotion.

[0078] 3. Second Embodiment In the first embodiment, the signal quality determination result by the single-channel signal quality determination unit 152 includes a signal quality score. In the second embodiment, the signal quality determination result by the single-channel signal quality determination unit 152 includes not only the signal quality score but also a noise label indicating the type of noise superimposed on the biological signal. The correction parameter determination unit 153 determines correction parameters based on the signal quality score and the noise label.

[0079] FIG. 8 is a diagram for explaining details of the processing performed by the connectivity feature extracting unit 112 according to the second embodiment.

[0080] As shown in the upper left of FIG. 8, the single-channel signal quality determining unit 152i determines the biological signal X raw Analyze the waveform of (ch=i) and obtain the biological signal X raw In the example of FIG. 8, the signal quality score S(ch=i) of the biological signal X raw The portion surrounded by a dashed line in the waveform of (ch=i) indicates a portion where noise is superimposed. raw The type of noise superimposed on the portion surrounded by the dashed line in (ch=i) is determined, and a noise label S_label(ch=i) indicating the type of noise and a signal quality score S(ch=i) are supplied to the correction parameter determination unit 153.

[0081] As shown in the lower left of FIG. 8, the single-channel signal quality determining unit 152j determines the biological signal X raw Analyze the waveform of (ch=j) and obtain the biological signal X raw In the example of FIG. 8, the signal quality score S(ch=j) of the biological signal X raw The portion surrounded by a dashed line in the waveform of (ch=j) indicates a portion where noise is superimposed. raw The type of noise superimposed on the portion surrounded by the dashed line in (ch=j) is determined, and a noise label S_label(ch=j) indicating the type of noise and a signal quality score S(ch=j) are supplied to the correction parameter determination unit 153.

[0082] The correction parameter determination unit 153 has tables T11 and T12, as shown on the right side of FIG. 8, in which parameter values ​​corresponding to the signal quality score S(ch=i) and the signal quality score S(ch=j) are recorded.

[0083] Table T11 is a table of biological signals X raw (ch=i) and biological signal X raw When the same type of noise is superimposed on the biological signal X (ch=j), that is, raw Noise label S_label(ch=i) and biological signal X for (ch=i) raw Table T12 is a table to be referenced when the noise label S_label(ch=j) for the biological signal X (ch=j) is the same as the noise label S_label(ch=j) for the biological signal X (ch=j). raw (ch=i) and biological signal X raw When different types of noise are superimposed on the biological signal X (ch=j), that is, raw Noise label S_label(ch=i) and biological signal X for (ch=i) raw This is a table that is referenced when the noise label S_label(ch=j) for (ch=j) is different from the noise label S_label(ch=j).

[0084] Biosignal X raw (ch=i) and biological signal X rawIf the same type of noise (e.g., electromyographic noise) is superimposed on (ch=j), the electromyographic voltage is larger than the electroencephalogram voltage, so the electromyographic noise will raw (ch=i) and biological signal X raw The relationship between (ch=j) may fluctuate. In this way, the connectivity feature is affected not only by the signal quality but also by the type of noise superimposed on the biological signal.

[0085] Therefore, biosignal X raw (ch=i) and biological signal X raw When the same type of noise is superimposed on (ch=j), even if the signal quality score at time t to be corrected is higher than a predetermined threshold, the connectivity feature X t-1 The table T11 is referenced so that parameter values ​​are set so that the weight of the parameter is increased.

[0086] As described above, the correction parameter determination unit 153 switches the table to be referred to depending on whether the noise label S_label(ch=i) and the noise label S_label(ch=j) are the same, thereby reducing the influence of noise on the connectivity feature and further improving the robustness of emotion estimation.

[0087] 4. Third Embodiment In the first embodiment, the correction unit 154 was provided subsequent to the connectivity feature calculation unit 151. In other words, by correcting the connectivity feature, the robustness of emotion estimation against noise was improved. In the third embodiment, the correction unit is provided prior to the connectivity feature calculation unit 151. In other words, by correcting the biological signal, the robustness of emotion estimation against noise is improved.

[0088] 9 is a block diagram showing an example of the configuration of a connectivity feature extraction unit according to the third embodiment. In Fig. 9, the same components as those in Fig. 5 are denoted by the same reference numerals. Duplicate descriptions will be omitted as appropriate.

[0089] The binding feature extraction unit 112 in Figure 9 differs from the binding feature extraction unit 112 in Figure 5 in that the correction unit 201 is provided in front of the binding feature calculation unit 151, instead of the correction unit 154 being provided in the rear of the binding feature calculation unit 151.

[0090] The correction parameter determination unit 153 acquires a correction parameter α based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 152i and the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 152j. Here, the correction parameter α is determined by raw (ch=i) and biological signal X raw This is a parameter used to correct (ch=j).

[0091] The correction parameter determination unit 153 and the biological signal X raw Correction parameters used to correct (ch=i), and X raw The correction parameters used for correcting (ch=j) may be acquired separately. In addition, the single-channel signal quality determination unit 152 may determine not only the signal quality score but also the noise label. When the noise label is determined, the correction parameter determination unit 153 determines the biological signal X based on the signal quality score and the noise label. raw (ch=i) and biological signal X raw Obtain the correction parameter α used for correcting (ch=j).

[0092] The correction parameter determination unit 153 supplies the correction parameter α to the correction unit 201 .

[0093] The correction unit 201 uses the correction parameter α supplied from the correction parameter determination unit 153 to correct the biological signal X raw (ch=i) and biological signal X raw (ch=j) is corrected in the time direction or the space direction, and the corrected biological signal X adjist (ch=i) and biological signal X adjust (ch=j) is supplied to the connectivity feature calculation unit 151.

[0094] For example, the correction unit 201 sets a time t to be corrected in the same manner as the correction unit 154, and calculates the biological signal X at the time t. raw (ch=i) and the biological signal X at time t-1 raw (ch=i) and the corrected biological signal X at time t are obtained by weighting and adding them based on the correction parameter α. raw Calculate (ch=i) (correction in the time direction).

[0095] Furthermore, for example, the correction unit 201, like the correction unit 154, corrects the biological signal to be corrected (for example, biological signal X raw (ch=i)) is set, and the biological signal X to be corrected is raw (ch=i) and other biological signals (e.g., biological signals X raw (ch=g)) and the biological signal X to be corrected are obtained by weighting and adding them. raw (ch=i) is corrected in the spatial direction. For example, the biological signal X raw A biosignal whose measurement position is near the measurement position of (ch=i) is set as the biosignal of the correction term.

[0096] The connectivity feature calculation unit 151 calculates the biosignal X supplied from the correction unit 201. adjust (ch=i) and biological signal X adjust Based on the relationship of (ch=j), the time series connectivity feature X connect-feat Calculate (ch=i,j) and output.

[0097] As described above, by providing the correction unit 201 in front of the binding feature quantity calculation unit 151, the binding feature quantity calculation unit 151 can calculate the binding feature quantity more accurately.

[0098] In addition, correction units may be provided both before and after the connectivity feature calculation unit 151.

[0099] 5. Fourth Embodiment In the first embodiment, information indicating the relationship between two biosignals from among multiple biosignals contained in one sample was extracted as a connectivity feature. In the fourth embodiment, all possible combinations (networks) of m (m is an integer greater than or equal to 3) biosignals from among multiple biosignals contained in one sample are each considered to be a signal set, and information indicating the relationship (centrality) of each network is extracted as a connectivity feature. Hereinafter, a connectivity feature indicating the relationship between a network will also be referred to as a network feature.

[0100] Fig. 10 is a block diagram showing an example of the configuration of a connectivity feature extraction unit according to the fourth embodiment. In Fig. 10, the same components as those in Fig. 5 are denoted by the same reference numerals. Duplicate descriptions will be omitted as appropriate.

[0101] The connectivity feature extraction unit 112 in FIG. 10 is provided with a single-channel signal quality determination unit 152k, and the connectivity feature calculation unit 151 receives the biological signal X raw This differs from the connectivity feature extraction unit 112 in FIG. 5 in that (ch=k) is input.

[0102] The connectivity feature calculation unit 151 calculates the biosignal X included in the network. raw Time series network feature X showing the relationship between network-feat_origin The network (signal set) contains i-channel biosignal X raw (ch=i), j-channel biological signal X raw (ch=j), and k (k is any integer from 1 to n other than i and j) channels of biosignals X raw The connectivity feature calculation unit 151 calculates the network feature X network-feat_origin is supplied to the correction unit 154.

[0103] The single-channel signal quality determination unit 152k determines the biological signal X raw Analyze the waveform of (ch=k) and extract the biological signal X raw The single-channel signal quality determining unit 152k determines the signal quality of the biological signal X rawThe signal quality score S(ch=k) obtained by quantifying the signal quality of (ch=k) is supplied to the correction parameter determination unit 153.

[0104] The correction parameter determination unit 153 determines a correction parameter α based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 152i, the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 152j, and the signal quality score S(ch=k) supplied from the single-channel signal quality determination unit 152k. Here, the correction parameter α is determined based on the time-series network feature X network-feat_origin The correction parameter determination unit 153 supplies the correction parameter α to the correction unit 154.

[0105] The correction unit 154 uses the correction parameter α supplied from the correction parameter determination unit 153 to calculate the time-series network feature X supplied from the connectivity feature calculation unit 151. network-feat_origin is corrected, and the corrected time-series network feature X network-feat Output (ch=i,j,k).

[0106] For example, if the measurement device 1 is configured with headphones, the connectivity feature extraction unit 112 can extract network features for a network configured from biosignals measured by three electrodes: a right ear electrode, a parietal electrode, and a left ear electrode. Compared to cases in which emotion estimation is performed based on connectivity features that indicate the relationship between paired biosignals (e.g., the first embodiment), the information processing device 2 according to the fourth embodiment can take into account, for example, the relationships between more regions in the brain, making it possible to perform emotion estimation that precisely reflects the exchange of information between regions in the brain.

[0107] 6. Fifth Embodiment In the first embodiment, the emotion determination unit 102 estimates the user's emotion based on the single-channel feature and the connectivity feature. In the fifth embodiment, the emotion determination unit 102 estimates the user's emotion based only on the connectivity feature.

[0108] Fig. 11 is a block diagram showing an example of the configuration of a connectivity feature extraction unit according to the fifth embodiment. In Fig. 11, the same components as those in Fig. 4 are denoted by the same reference numerals. Duplicate descriptions will be omitted as appropriate.

[0109] The information processing device 2 in FIG. 11 differs from the information processing device 2 in FIG. 4 in that the single-channel feature extraction unit 111 is not provided.

[0110] The emotion determination unit 102 extracts the time-series connectivity feature X supplied from the connectivity feature extraction unit 112. connect-feat The user's emotion is estimated by inputting this into a classifier.

[0111] 7. Sixth Embodiment In the first embodiment, the time-series connectivity feature is corrected in a stage subsequent to the connectivity feature calculation unit 151. In the sixth embodiment, the emotion determination result is corrected in a stage subsequent to the emotion determination unit 102.

[0112] Fig. 12 is a block diagram showing an example of the configuration of an information processing device 2 according to the sixth embodiment. In Fig. 12, the same components as those in Fig. 4 are denoted by the same reference numerals. Duplicate descriptions will be omitted where appropriate.

[0113] The information processing device 2 in FIG. 12 differs from the information processing device 2 in FIG. 4 in that a sample signal quality determining unit 251 and a correcting unit 252 are provided.

[0114] As described above, the connectivity feature extraction unit 112 may correct the connectivity feature based on the signal quality score and supply it to the emotion determination unit 102, or may supply the connectivity feature to the emotion determination unit 102 without correcting it.

[0115] The emotion determination unit 102 supplies the emotion estimation result Y hat (the character Y with a ^ attached thereto will be written as Y hat; hereinafter, the same will be used in the present specification) to the correction unit 252 .

[0116] The sample signal quality determination unit 251 receives the biological signals X raw The sample signal quality determination unit 251 quantifies the signal quality of each sample and generates a signal quality score S ｓ is supplied to the correction unit 252.

[0117] The correction unit 252 calculates the signal quality score S supplied from the sample signal quality determination unit 251. ｓ The emotion estimation result Y hat supplied from the emotion determination unit 102 is corrected based on the above, and the corrected emotion estimation result y hat is output.

[0118] FIG. 13 is a diagram showing a detailed configuration example of the sample signal quality determining section 251. As shown in FIG.

[0119] As shown in FIG. 13, the sample signal quality determining unit 251 includes single-channel signal quality determining units 261 i and 261 j , an inter-channel signal quality determining unit 262 , and a sample signal quality calculating unit 263 .

[0120] The single-channel signal quality determination unit 261i receives the biological signals X raw Among them, biosignal X raw Analyze the waveform of (ch=i) and obtain the biological signal X raw The single-channel signal quality determining unit 261i determines the signal quality of the biological signal X (ch=i). raw The signal quality score S(ch=i) obtained by quantifying the signal quality of (ch=i) is supplied to the inter-channel signal quality determining unit 262 .

[0121] The single-channel signal quality determination unit 261j determines the biological signals X raw Among them, biosignal X raw Analyze the waveform of (ch=j) and obtain the biological signal X raw The single-channel signal quality determining unit 261j determines the signal quality of the biological signal X (ch=j). raw The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the inter-channel signal quality determining unit 262 .

[0122] When there is no need to particularly distinguish between the single-channel signal quality determining unit 261i and the single-channel signal quality determining unit 261j, they will be simply referred to as the single-channel signal quality determining unit 261.

[0123] The inter-channel signal quality determination unit 262 determines the signal quality score S(ch=i,j) for each signal set based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 261i and the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 262j.

[0124] Specifically, the inter-channel signal quality determination unit 262 has a table T101 in which score values ​​corresponding to the signal quality score S(ch=i) and the signal quality score S(ch=j) are recorded, as shown in Fig. 14. The inter-channel signal quality determination unit 262 refers to the table T101 and sets the signal quality score S(ch=i,j) to a score value determined according to the value of the signal quality score S(ch=i) determined by the single-channel signal quality determination unit 261i and the value of the signal quality score S(ch=j) determined by the single-channel signal quality determination unit 261j.

[0125] In table T101, for example, a score value of 1.0 is recorded for S(ch=i)=1.0 and S(ch=j)=1.0. For example, if the value of the signal quality score S(ch=i) determined by the single-channel signal quality determination unit 251i and the value of the signal quality score S(ch=j) determined by the single-channel signal quality determination unit 251j are 1.0, the inter-channel signal quality determination unit 262 determines the signal quality score S(ch=i,j) to be 1.0.

[0126] The sample signal quality determination unit 251 sets all possible combinations of pairs from among the multiple biological signals included in one sample among the biological signals measured by the measuring device 1 as signal sets, and performs the above-described processing for each signal set. As a result, the sample signal quality determination unit 251 determines the signal quality score S(ch=i, j) for each of the signal sets that can be combined in one sample.

[0127] The sample signal quality calculation unit 263 calculates the signal quality score S for each sample based on the signal quality score S(ch=i, j) for each signal set supplied from the inter-channel signal quality determination unit 262. ｓ is calculated and output.

[0128] For example, as shown in the following formula (2), the sample signal quality calculation unit 263 calculates a weighted average of the signal quality scores S(ch=i, j) in units of signal sets as the signal quality score S ｓ It is calculated as follows.

[0129]

[0130] In formula (2), w is the connectivity feature X connect-feat Indicates the weight according to (ch=i,j).

[0131] Furthermore, the sample signal quality calculation unit 263 calculates the simple arithmetic average of the signal quality scores S(ch=i, j) in units of signal sets as the signal quality score S in units of samples, as shown in the following formula (3): ｓ It is calculated as follows.

[0132]

[0133] The signal quality score S for each sample is calculated by weighting the average of the signals, taking into account the ease of ensuring signal quality and the brain regions that are important for emotion estimation. ｓ The signal quality score S ｓ The calculation method is not limited to the above-described method.

[0134] As described above, in the sixth embodiment, only the emotion estimation result is corrected, and therefore it is possible to improve the accuracy of emotion estimation while reducing the amount of calculation compared to when the connectivity features for each signal set and each biological signal are corrected.

[0135] Fig. 15 is a block diagram showing a modified example of the configuration of the information processing device 2 according to the sixth embodiment. In Fig. 15, the same components as those in Fig. 11 are denoted by the same reference numerals. Duplicate descriptions will be omitted where appropriate.

[0136] 15 is configured with sub-discrimination units 301-1 to 301-N and an integrated discrimination unit 302. Here, the sample used for emotion estimation includes a number of biological signals (e.g., three or more) that can be used to assemble multiple signal sets from the sample.

[0137] The sub-discrimination unit 301-1 detects the biological signal X raw (ch=i) and biological signal X raw The signal set consisting of (ch=j) is processed, and the user's emotion is estimated based on the connectivity features of the signal set.

[0138] The sub-discrimination unit 301 - 1 includes a feature extraction unit 101 , an emotion discrimination unit 102 , single-channel signal quality determination units 311 i and 311 j , an inter-channel signal quality determination unit 312 , and a correction unit 313 .

[0139] The feature extraction unit 101 includes a connectivity feature extraction unit 112. The connectivity feature extraction unit 112 extracts a time-series connectivity feature X connect-feat and supplies it to the emotion determination unit 102.

[0140] The emotion determination unit 102 extracts the time-series connectivity feature X supplied from the connectivity feature extraction unit 112. connect-feat The emotion determination unit 102 estimates the emotion of the user by inputting the emotion estimation result Y ｓ The (ch=i, j) hat is supplied to the correction unit 313 .

[0141] The single-channel signal quality determination unit 311i determines the biological signal X raw Analyze the waveform of (ch=i) and obtain the biological signal X raw The single-channel signal quality determination unit 311i determines the signal quality of the biological signal X (ch=i). raw The signal quality score S(ch=i) obtained by quantifying the signal quality of (ch=i) is supplied to the inter-channel signal quality determining unit 312 .

[0142] The single-channel signal quality determination unit 311j determines the biological signal X raw Analyze the waveform of (ch=j) and obtain the biological signal X raw The single-channel signal quality determination unit 311j determines the signal quality of the biological signal X (ch=j). raw The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the inter-channel signal quality determining unit 312 .

[0143] The inter-channel signal quality determination unit 312 determines the signal quality score S(ch=i,j) for the signal set to be processed based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 311i and the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 312j. The inter-channel signal quality determination unit 312 supplies the signal quality score S(ch=i,j) to the correction unit 313.

[0144] The correction unit 313 corrects the emotion estimation result Y supplied from the emotion determination unit 102 based on the signal quality score S(ch=i, j) supplied from the inter-channel signal quality determination unit 312. ｓ (ch=i,j) Correct the hat and calculate the emotion estimation result y ｓ (ch=i, j) hat is supplied to the integrated discrimination unit 302 .

[0145] The sub-discrimination unit 301-2 detects the biological signal X raw (ch=j) and biological signal X raw The signal set consisting of (ch=k) is processed, and the user's emotion is estimated based on the connectivity features of the signal set.

[0146] The sub-discrimination unit 301 - 2 includes a feature extraction unit 101 , an emotion discrimination unit 102 , single-channel signal quality determination units 311 j and 311 k , an inter-channel signal quality determination unit 312 , and a correction unit 313 .

[0147] The feature extraction unit 101 includes a connectivity feature extraction unit 112. The connectivity feature extraction unit 112 extracts a time-series connectivity feature X connect-feat and supplies it to the emotion determination unit 102.

[0148] The emotion determination unit 102 extracts the time-series connectivity feature X supplied from the connectivity feature extraction unit 112. connect-feat The emotion determination unit 102 estimates the emotion of the user by inputting the emotion estimation result Y ｓ The (ch=j, k) hat is supplied to the correction unit 313 .

[0149] The single-channel signal quality determination unit 311j determines the biological signal X raw Analyze the waveform of (ch=j) and obtain the biological signal Xraw The single-channel signal quality determination unit 311i determines the signal quality of the biological signal X (ch=j). raw The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the inter-channel signal quality determining unit 312 .

[0150] The single-channel signal quality determination unit 311k determines the biological signal X raw Analyze the waveform of (ch=k) and extract the biological signal X raw The single-channel signal quality determination unit 311k determines the signal quality of the biological signal X raw The signal quality score S(ch=k) obtained by quantifying the signal quality of (ch=k) is supplied to the inter-channel signal quality determining unit 312 .

[0151] The inter-channel signal quality determination unit 312 determines the signal quality score S(ch=j,k) for the signal set to be processed based on the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 311j and the signal quality score S(ch=k) supplied from the single-channel signal quality determination unit 312k. The inter-channel signal quality determination unit 312 supplies the signal quality score S(ch=j,k) to the correction unit 313.

[0152] The correction unit 313 corrects the emotion estimation result Y supplied from the emotion determination unit 102 based on the signal quality score S(ch=j, k) supplied from the inter-channel signal quality determination unit 312. ｓ (ch=j,k) Correct the hat and calculate the emotion estimation result y ｓ (ch=j, k) hat is supplied to the integrated discrimination unit 302 .

[0153] Sub-discrimination units 301-3 to 301-N (not shown) process corresponding signal sets and estimate the user's emotion based on the connectivity features for the signal sets. When there is no need to particularly distinguish between sub-discrimination units 301-1 to 301-N, they will be simply referred to as sub-discrimination units 301.

[0154] The information processing device 2 has a sub-discrimination unit 301 that processes all combinations (N combinations) of multiple biological signals contained in one sample that can be paired together as signal sets.

[0155] The integration discrimination unit 302 has an integrator that integrates the emotion estimation results from the sub-discrimination units 301 to 301-N. The integration discrimination unit 302 inputs the emotion estimation results (after correction) from the sub-discrimination units 301 to 301-N to the integrator to obtain a final emotion estimation result y ｓ Get and print the hat.

[0156] As described above, in the modification of the sixth embodiment, the sub-discrimination unit 301 estimates the user's emotion, and after the emotion estimation result is corrected, the integration discrimination unit 302 integrates the emotion estimation results of the sub-discrimination units 301.

[0157] Each sub-discrimination unit 301 corrects only the emotion estimation result, which makes it possible to improve the accuracy of emotion estimation while suppressing the amount of calculation, and by taking into account the signal quality between channels during correction, it becomes possible to perform correction with high accuracy.

[0158] 8. Seventh Embodiment In the first embodiment, connectivity features are corrected based on the signal quality scores extracted for each signal set. In the seventh embodiment, the signal quality scores extracted for each signal set are used to train a classifier (emotion estimation model). For example, the classifier is trained so that samples with better signal quality contribute more to training.

[0159] Fig. 16 is a block diagram showing an example of the configuration of an information processing device 2 according to the seventh embodiment. In Fig. 16, the same components as those in Fig. 11 are denoted by the same reference numerals. Duplicate descriptions will be omitted where appropriate.

[0160] The information processing device 2 in FIG. 16 differs from the information processing device 2 in FIG. 11 in that an emotion estimation model learning unit 401 is provided.

[0161] The emotion estimation model learning unit 401 uses a dataset consisting of training data of p samples to train a classifier used to estimate a user's emotion in the emotion determination unit 102. Here, a sample refers to a group of biological signals obtained by measuring the biological reactions of a specific person within a specific time window, and is also called an instance.

[0162] The training data for the sth sample (s is an arbitrary integer from 1 to p) includes biosignals X raw-s and the biological signal X raw-s Emotion label Y indicating the user's emotion when ｓ For example, when the user is in a high arousal uncomfortable state, the emotion label Y ｓ The value of is set to 0, and when the user is in a high arousal pleasant state, the emotional label Y ｓ The value of is set to 1.

[0163] The emotion estimation model learning unit 401 supplies the model parameters obtained by learning to the emotion determination unit 102 .

[0164] The emotion estimation model learning unit 401 may be realized by a device external to the information processing device 2.

[0165] FIG. 17 is a block diagram showing a detailed configuration example of the emotion estimation model learning unit 401.

[0166] As shown in FIG. 17, the emotion estimation model learning unit 401 has a feature extraction unit 411, single-channel signal quality determination units 412i and 412j, an inter-channel signal quality determination unit 413, a sample signal quality calculation unit 414, and a learning unit 415.

[0167] The feature extraction unit 411 includes a connectivity feature extraction unit 431. The connectivity feature extraction unit 431 extracts a time-series connectivity feature X connect-feat and supplies it to the learning unit 415.

[0168] The single-channel signal quality determination unit 412i determines the biological signals X of channels 1 to n included in the s-th sample of the learning data. raw-s Among them, biosignal X raw-sAnalyze the waveform of (ch=i) and obtain the biological signal X raw-s The single-channel signal quality determination unit 412i determines the signal quality of the biological signal X raw-s The signal quality score S(ch=i) obtained by quantifying the signal quality of (ch=i) is supplied to the inter-channel signal quality determination unit 413 .

[0169] The single-channel signal quality determination unit 412j determines the biological signals X of channels 1 to n included in the s-th sample of the learning data. raw-s Among them, biosignal X raw-s Analyze the waveform of (ch=j) and obtain the biological signal X raw-s The single-channel signal quality determining unit 261j determines the signal quality of the biological signal X (ch=j). raw-s The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the inter-channel signal quality determining unit 413 .

[0170] When there is no need to particularly distinguish between the single-channel signal quality determining unit 412i and the single-channel signal quality determining unit 412j, they will simply be referred to as the single-channel signal quality determining unit 412.

[0171] The inter-channel signal quality determination unit 413 determines the signal quality score S(ch=i,j) for each signal set based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 412i and the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 412j. The method for determining the signal quality score S(ch=i,j) is the same as the method used by the inter-channel signal quality determination unit 262 in Fig. 13. The inter-channel signal quality determination unit 413 supplies the signal quality score S(ch=i,j) to the sample signal quality calculation unit 414.

[0172] The emotion estimation model training unit 401 sets all possible combinations (N combinations) of pairs of multiple biological signals included in the s samples as signal sets, and performs the above processing for each signal set. As a result, the emotion estimation model training unit 401 determines the signal quality score S(ch=i, j) for each possible signal set in the s samples.

[0173] The sample signal quality calculation unit 414 calculates the signal quality score S for each sample based on the signal quality score S(ch=i, j) for each signal set supplied from the inter-channel signal quality determination unit 262. ｓ and supplies it to the learning unit 415.

[0174] The learning unit 415 calculates the connectivity feature X for each signal set supplied from the feature extraction unit 411. connect-feat , the signal quality score S provided by the sample signal quality calculation unit 414 ｓ , and emotion label Y ｓ Specifically, the learning unit 415 uses the connectivity feature X connect-feat and the signal quality score S ｓ and the emotion label Y ｓ The emotion estimation model is trained to output the following.

[0175] In the following, as an example, a case where a binary classification model using a deep neural network (DNN) is trained will be described. The training unit 415 uses a loss function L loss By defining as in the following equation (4), the signal quality is good (signal quality score S ｓ Learning can be performed in such a way that samples with higher values ​​(higher values) have a higher contribution during learning.

[0176]

[0177] In formula (4), Y ｓ The hat indicates the estimation result of the binary classification model (probability value between 0 and 1). Note that the learning method for the emotion estimation model is not limited to this. For example, a learning method may be considered in which the sum of a term defined by the error between the estimation result and the correct answer and a penalty term defined by the signal quality score is defined as the loss function.

[0178] The learning unit 415 outputs the model parameters obtained by learning.

[0179] As described above, the information processing device 2 can prevent the estimation accuracy of the emotion estimation model from deteriorating when the signal quality of the biological signals included in the dataset is poor by training the emotion estimation model based not only on the error between the estimation result of the emotion estimation model and the emotion label, but also on the signal quality on a sample-by-sample basis. Furthermore, training the emotion estimation model based on the signal quality on a sample-by-sample basis improves the explainability of the emotion estimation model.

[0180] In the above, the signal quality score S ｓ used to train the emotion estimation model, signal quality scores for each signal set may also be used to train the emotion estimation model. In this case, the emotion estimation model training unit 401 is provided with multiple sub-training units that train the emotion estimation model using signal quality and connectivity features calculated based on the corresponding signal sets.

[0181] When signal quality scores for each signal set are used to train an emotion estimation model, two possible methods for acquiring model parameters are, for example, Method 1 and Method 2. Method 1 calculates model parameters corresponding to each signal set. Method 2 integrates the estimation results of the classifiers in each sub-learning unit and performs backpropagation based on the integrated estimation result.

[0182] First, the emotion estimation model learning unit 401 that learns the emotion estimation model using the first method will be described.

[0183] Fig. 18 is a block diagram showing a first modified example of the configuration of the emotion estimation model learning unit 401. In Fig. 18, the same components as those in Fig. 17 are assigned the same reference numerals. Duplicate explanations will be omitted where appropriate.

[0184] The emotion estimation model learning unit 401 in FIG. 18 has emotion estimation model sub-learning units 451-1 to 451-N.

[0185] The emotion estimation model sub-learning unit 451-1 receives the biological signals X of channels 1 to n included in the s-th sample of learning data. raw-s Among them, biosignal X raw-s(ch=i) and biological signal X raw-s The emotion estimation model sub-learning unit 451-1 has a feature extraction unit 411, single-channel signal quality determination units 412i and 412j, an inter-channel signal quality determination unit 413, and a learning unit 415.

[0186] The connectivity feature extraction unit 431 extracts the time-series connectivity feature X connect-feat and supplies it to the learning unit 415.

[0187] The single-channel signal quality determination unit 412i determines the biological signals X of channels 1 to n included in the s-th sample of the learning data. raw-s Among them, biosignal X raw-s Analyze the waveform of (ch=i) and obtain the biological signal X raw-s The single-channel signal quality determination unit 412i determines the signal quality of the biological signal X raw-s The signal quality score S(ch=i) obtained by quantifying the signal quality of (ch=i) is supplied to the inter-channel signal quality determination unit 413 .

[0188] The single-channel signal quality determination unit 412j determines the biological signals X of channels 1 to n included in the s-th sample of the learning data. raw-s Among them, biosignal X raw-s Analyze the waveform of (ch=j) and obtain the biological signal X raw-s The single-channel signal quality determining unit 261j determines the signal quality of the biological signal X (ch=j). raw-s The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the inter-channel signal quality determining unit 413 .

[0189] The inter-channel signal quality determination unit 413 determines the signal quality score S(ch=i,j) for the signal set to be processed based on the signal quality score S(ch=i) supplied from the single-channel signal quality determination unit 412i and the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 412j. The inter-channel signal quality determination unit 413 supplies the signal quality score S(ch=i,j) to the learning unit 415.

[0190] The learning unit 415 uses the connectivity feature X supplied from the feature extraction unit 411 connect-feat and the signal quality score S(ch=i,j) supplied from the inter-channel signal quality determination unit 413 are input, and the emotion label Y ｓ The emotion estimation model is trained to output the following.

[0191] The learning unit 415, for example, calculates the loss function L loss By defining (ch=i,j) as in the following equation (5), learning can be performed so that the better the signal quality (the higher the signal quality score S(ch=i,j)), the higher the contribution of the signal set during learning.

[0192]

[0193] In formula (5), Y ｓ (ch=i, j) indicates the estimation result of the binary classification model (probability value between 0 and 1). The learning unit 415 calculates the loss function L loss (ch=i, j) are optimized to calculate model parameters of the emotion estimation model for biosignal X(ch=i) and biosignal X(ch=j). The emotion estimation model for biosignal X(ch=i) and biosignal X(ch=j) is a classifier (a classifier included in the emotion discrimination unit 102 of the sub-discrimination unit 301-1 in FIG. 15 ) that estimates the user's emotion based on connectivity features that indicate the relationship between biosignal X(ch=i) and biosignal X(ch=j).

[0194] The learning unit 415 outputs model parameters for the biological signal X(ch=i) and the biological signal X(ch=j) obtained by learning.

[0195] The emotion estimation model sub-learning unit 451-2 uses the biological signals X of channels 1 to n included in the s-th sample of learning data. raw-s Among them, biosignal X raw-s (ch=j) and biological signal X raw-s The emotion estimation model sub-learning unit 451-2 has a feature extraction unit 411, single-channel signal quality determination units 412j and 412k, an inter-channel signal quality determination unit 413, and a learning unit 415.

[0196] The connectivity feature extraction unit 431 extracts the time-series connectivity feature X connect-feat and supplies it to the learning unit 415.

[0197] The single-channel signal quality determination unit 412j determines the biological signals X of channels 1 to n included in the s-th sample of the learning data. raw-s Among them, biosignal X raw-s Analyze the waveform of (ch=j) and obtain the biological signal X raw-s The single-channel signal quality determining unit 261j determines the signal quality of the biological signal X (ch=j). raw-s The signal quality score S(ch=j) obtained by quantifying the signal quality of (ch=j) is supplied to the inter-channel signal quality determining unit 413 .

[0198] The single-channel signal quality determination unit 412 k determines the biological signals X of channels 1 to n contained in the s-th sample of the learning data. raw-s Among them, biosignal X raw-s Analyze the waveform of (ch=k) and extract the biological signal X raw-s The single-channel signal quality determination unit 261k determines the signal quality of the biological signal X raw-s The signal quality score S(ch=k) obtained by quantifying the signal quality of (ch=k) is supplied to the inter-channel signal quality determining unit 413 .

[0199] The inter-channel signal quality determination unit 413 determines the signal quality score S(ch=j,k) for the signal set to be processed based on the signal quality score S(ch=j) supplied from the single-channel signal quality determination unit 412j and the signal quality score S(ch=k) supplied from the single-channel signal quality determination unit 412k. The inter-channel signal quality determination unit 413 supplies the signal quality score S(ch=j,k) to the learning unit 415.

[0200] The learning unit 415 uses the connectivity feature X supplied from the feature extraction unit 411 connect-feat and the signal quality score S(ch=j, k) supplied from the inter-channel signal quality determination unit 413 are input, and the emotion label Y ｓ The emotion estimation model is trained to output the following.

[0201] The learning unit 415 uses a loss function L similar to that of Equation (5). loss (ch=j, k) are optimized to calculate model parameters of the emotion estimation model for biosignal X(ch=j) and biosignal X(ch=k). The emotion estimation model for biosignal X(ch=j) and biosignal X(ch=k) is a classifier (a classifier included in the emotion discrimination unit 102 of the sub-discrimination unit 301-2 in FIG. 15 ) that infers the user's emotion based on connectivity features that indicate the relationship between biosignal X(ch=j) and biosignal X(ch=k).

[0202] The learning unit 415 outputs model parameters for the biological signal X(ch=i) and the biological signal X(ch=j) obtained by learning.

[0203] Emotion estimation model sub-learning units 451-3 to 451-N (not shown) train emotion estimation models using signal quality scores and connectivity features calculated based on the corresponding signal sets. When there is no need to particularly distinguish between emotion estimation model sub-learning units 451-1 to 451-N, they will be simply referred to as emotion estimation model sub-learning units 451.

[0204] The emotion estimation model learning unit 401 defines all possible combinations (N combinations) of the multiple biological signals included in the s samples as signal sets, and has an emotion estimation model sub-learning unit 451 corresponding to each signal set.

[0205] In the first modified example of the seventh embodiment, it is necessary to provide the sub-discrimination unit 301 and the integrated discrimination unit 302 in the information processing device 2, as described in the sixth embodiment.

[0206] Next, the emotion estimation model learning unit 401 that learns the emotion estimation model using the second method will be described.

[0207] Fig. 19 is a block diagram showing a second modified example of the configuration of the emotion estimation model learning unit 401. In Fig. 19, the same components as those in Fig. 18 are assigned the same reference numerals. Duplicate explanations will be omitted where appropriate.

[0208] The emotion estimation model learning unit 401 in FIG. 19 has sub-discrimination units (sub-learning units) 501-1 to 501-N and an integrated discrimination unit (learning unit) 502.

[0209] The sub-discrimination unit 501-1 detects the biological signals X of channels 1 to n contained in the s-th sample of learning data. raw-s Among them, biosignal X raw-s (ch=i) and biological signal X raw-s The sub-discrimination unit 501-1 has a feature extraction unit 411, single-channel signal quality determination units 412i and 412j, an inter-channel signal quality determination unit 413, and a discriminator 511.

[0210] The connectivity feature extraction unit 431 extracts the time-series connectivity feature X connect-feat is input to the discriminator 511.

[0211] The inter-channel signal quality determination unit 413 inputs the signal quality score S(ch=i, j) for the signal set to be processed to the discriminator 511 .

[0212] The discriminator 511 extracts the connectivity feature X input from the feature extraction unit 411. connect-feat and the signal quality score S(ch=i, j) input from the inter-channel signal quality determination unit 413. The discriminator 511 estimates the emotion of the user based on the emotion estimation result Y ｓ The (ch=i, j) hat is supplied to the integrated discrimination unit 502 .

[0213] The sub-discrimination unit 501-2 detects the biological signals X of channels 1 to n contained in the s-th sample of learning data. raw-s Among them, biosignal X raw-s (ch=j) and biological signal X raw-s The sub-discrimination unit 501-2 has a feature extraction unit 411, single-channel signal quality determination units 412j and 412k, an inter-channel signal quality determination unit 413, and a discriminator 511.

[0214] The connectivity feature extraction unit 431 extracts the time-series connectivity feature X connect-feat is input to the discriminator 511.

[0215] The inter-channel signal quality determination unit 413 inputs the signal quality score S(ch=j, k) for the signal set to be processed to the discriminator 511 .

[0216] The discriminator 511 extracts the connectivity feature X input from the feature extraction unit 411. connect-feat and the signal quality score S(ch=j, k) input from the inter-channel signal quality determination unit 413. The discriminator 511 estimates the emotion of the user based on the emotion estimation result Y ｓ The (ch=j, k) hat is supplied to the integrated discrimination unit 502 .

[0217] Sub-discrimination units 501-3 to 501-N (not shown) estimate the user's emotion based on the signal quality scores and connectivity features calculated based on the corresponding signal sets. When there is no need to particularly distinguish between sub-discrimination units 501-1 to 501-N, they will be simply referred to as sub-discrimination units 501.

[0218] The emotion estimation model learning unit 401 defines all possible combinations (N combinations) of biosignals included in the s samples as signal sets, and has a sub-discrimination unit 501 corresponding to each signal set.

[0219] The integration discrimination unit 502 has an integrator that integrates the emotion estimation results from the sub-discrimination units 501-1 to 501-N. The integration discrimination unit 502 inputs the emotion estimation results from the sub-discrimination units 501-1 to 501-N to the integrator to generate a final emotion estimation result y ｓ Get a hat.

[0220] The integrated discrimination unit 502 calculates the final emotion estimation result y ｓ hat and the emotion label Y associated with the sth sample of training data. ｓ Based on the error between the signal set and the integrated signal, the integrated discrimination unit 502 performs backpropagation to optimize the model parameters of the integrated unit and the model parameters of the emotion estimation model for each signal set. The integrated discrimination unit 502 outputs the optimized model parameters.

[0221] In the second modified example of the seventh embodiment, it is necessary to provide the sub-discrimination unit 301 and the integrated discrimination unit 302 in the information processing device 2, as described in the sixth embodiment.

[0222] 9. Modified System Configuration The information processing device 2 may be configured by a single information processing device, or may be configured by a plurality of information processing devices.

[0223] 20 to 22 are block diagrams showing modified examples of the configuration of the information processing device 2. In FIG.

[0224] 20, the feature extraction unit 101 may be provided in an information processing device 601, and the emotion determination unit 102 may be provided in an information processing device 602. The information processing devices 601 and 602 are connected wirelessly or by wire.

[0225] 21 , for example, the single-channel feature extraction unit 111 may be provided in an information processing device 611, and the emotion determination unit 102 and the connectivity feature extraction unit 112 may be provided in an information processing device 612. The information processing devices 611 and 612 are connected wirelessly or by wire.

[0226] Since the computational complexity of the process of extracting connectivity features is thought to be large, it is possible to achieve more efficient and faster computation by separating the device that extracts single-channel features from the device that extracts connectivity features.

[0227] 22 , for example, the single-channel feature extraction unit 111 may be provided in an information processing device 621, the connectivity feature extraction unit 112 may be provided in an information processing device 622, and the emotion determination unit 102 may be provided in an information processing device 623. The information processing device 621 and the information processing device 622 are connected to the information processing device 623 wirelessly or via a wire.

[0228] <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0229] FIG. 23 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0230] A CPU (Central Processing Unit) 1001 , a ROM (Read Only Memory) 1002 , and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004 .

[0231] An input / output interface 1005 is also connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, etc., and an output unit 1007 including a display, a speaker, etc. are connected to the input / output interface 1005. In addition, a storage unit 1008 including a hard disk, a nonvolatile memory, etc., a communication unit 1009 including a network interface, etc., and a drive 1010 that drives removable media 1011 are also connected to the input / output interface 1005.

[0232] In a computer configured as described above, the CPU 1001 performs the above-described series of processes by, for example, loading a program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it.

[0233] The program executed by the CPU 1001 is provided, for example, by being recorded on a removable medium 1011 or via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and is installed in the storage unit 1008.

[0234] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0235] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device with multiple modules housed in a single housing, are both systems.

[0236] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0237] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0238] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0239] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0240] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0241] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0242] (1) An information processing method comprising: extracting a connectivity feature indicating a relationship between a plurality of biosignals included in a signal set, based on a plurality of biosignals obtained by measuring a user's bioreactions on each of a plurality of channels included in the signal set; and estimating the user's emotion based on the connectivity feature extracted for each of the signal sets that can be combined in one sample. (2) The information processing method described in (1), wherein extracting the connectivity feature comprises: calculating the connectivity feature based on the plurality of biosignals; determining signal quality for each of the biosignals; and correcting the biosignals or the connectivity feature based on the signal quality determination result. (3) The information processing method described in (2), wherein extracting the connectivity feature further comprises: determining a correction parameter used to correct the biosignals or the connectivity feature based on the signal quality determination result, and performing a time direction correction or a spatial direction correction on the biosignals or the connectivity feature using the correction parameter. (4) The information processing method according to (3), wherein the signal quality determination result includes at least one of a signal quality score that quantifies the signal quality of each of the plurality of biological signals and a noise label that indicates a type of noise superimposed on each of the plurality of biological signals. (5) The information processing method according to (4), wherein the connectivity feature at a time to be corrected and the connectivity feature at a time earlier than the time to be corrected are weighted and added based on the correction parameter to perform the time direction correction on the connectivity feature, or the biosignal measured at the time to be corrected and the biosignal measured at the earlier time are weighted and added based on the correction parameter to perform the time direction correction on the biosignal.(6) The information processing method according to (5), wherein the correction parameter is determined such that the higher the signal quality score of each of the plurality of biological signals at the time to be corrected, the greater the weight of the connectivity feature or the biological signal at the time to be corrected, and the lower the signal quality score of each of the plurality of biological signals at the time to be corrected, the greater the weight of the connectivity feature or the biological signal at the past time. (7) The information processing method according to (4), wherein the spatial direction correction is performed on the connectivity feature by performing a weighted addition of the connectivity feature for the signal set to be corrected and the connectivity feature for another signal set based on the correction parameter, or the spatial direction correction is performed on the biological signal by performing a weighted addition of the biosignal to be corrected and the other biosignal based on the correction parameter. (8) The information processing method according to (7), wherein the correction parameters are determined such that the lower the signal quality score of each of the plurality of biosignals included in the signal set to be corrected or the lower the signal quality score of the biosignal to be corrected, the greater the weight of the connectivity feature for the other signal sets or the other biosignals. (9) The information processing method according to any of (4) to (8), wherein the correction parameters are determined by referring to a table in which the correction parameters corresponding to the signal quality scores of each of the plurality of biosignals are recorded. (10) The information processing method according to (9), wherein the table to be referred to is switched based on whether the type of noise superimposed on each of the plurality of biosignals is the same. (11) The information processing method according to any of (1) to (10), wherein estimating the user's emotion includes: determining signal quality on a sample-by-sample basis; estimating the user's emotion based on the connectivity feature for each of the signal sets that can be combined in one sample; and correcting the estimation result of the user's emotion based on the determination result of the signal quality.(12) The information processing method according to any one of (1) to (10), wherein the sample includes a number of the biological signals sufficient to form a plurality of the signal sets within the sample, and estimating the user's emotion includes: determining signal quality for each signal set; estimating the user's emotion for each signal set based on the connectivity feature for the signal set; correcting the estimation result of the user's emotion for each signal set based on the determination result of the signal quality; and integrating the estimation result of the user's emotion for each signal set after correction. (13) The information processing method according to (1), wherein the plurality of biological signals include at least an electroencephalogram signal obtained by measuring electroencephalograms in a left region of the user's head and an electroencephalogram signal obtained by measuring electroencephalograms in a right region of the head. (14) The information processing method according to any one of (1) to (13), wherein the plurality of biological signals are acquired by a measuring device provided with a plurality of sensors corresponding to a plurality of channels so as to be able to measure potentials at a plurality of locations on the user's head. (15) The information processing method according to (14), wherein the measurement device is one of a head-mounted display, a headband, headphones, earphones, glasses, and a hat. (16) An information processing device comprising: a feature extraction unit that extracts connectivity features that indicate a relationship between a plurality of biosignals included in a signal set, based on a plurality of biosignals obtained by measuring a bioreaction of a user on each of a plurality of channels included in the signal set; and an emotion estimation unit that estimates the emotion of the user based on the connectivity features extracted for each of the signal sets that can be combined in one sample.(17) A learning method comprising: extracting connectivity features indicating a relationship between a plurality of biosignals included in a signal set, based on a plurality of biosignals obtained by measuring a bioreaction of a person on each of a plurality of channels included in the signal set; determining signal quality on a sample-by-sample basis; and performing learning using the connectivity features extracted for each of the signal sets that can be combined in the sample, the determination results of the signal quality on a sample-by-sample basis, and an emotion label that indicates the emotion of the person associated with the sample, to obtain model parameters used for emotion estimation. (18) The sample includes a number of the biosignals that can be combined into a plurality of the signal sets in the sample, and determining the signal quality on a sample-by-sample basis comprises: determining the signal quality on a signal set-by-signal basis; and calculating the signal quality on a sample-by-sample basis based on the determination results of the signal quality for each of the plurality of signal sets that can be combined in the sample. (19) The learning method according to (18), wherein determining the signal quality on a sample-by-sample basis further includes determining the signal quality for each of the biological signals, and determining the signal quality on a signal set basis based on a determination result of the signal quality of each of the plurality of biological signals included in the signal set. (20) The learning method according to any of (17) to (19), wherein the determination result of the signal quality on a sample-by-sample basis includes a signal quality score that quantifies the signal quality of the biological signals included in the sample, and learning is performed such that the higher the signal quality score, the higher the contribution of the sample to learning.

[0243] REFERENCE SIGNS LIST 1 Measurement device, 2 Information processing device, 101 Feature extraction unit, 102 Emotion discrimination unit, 111 Single-channel feature extraction unit, 112 Connectivity feature extraction unit, 151 Connectivity feature calculation unit, 152 Single-channel signal quality judgment unit, 153 Correction parameter determination unit, 154, 201 Correction unit, 251 Sample signal quality judgment unit, 252 Correction unit, 261 Single-channel signal quality judgment unit, 262 Inter-channel signal quality judgment unit, 263 Sample signal quality calculation unit, 301 Sub-discrimination unit, 302 Integrated discrimination unit, 311 Single-channel signal quality judgment unit, 312 Inter-channel signal quality judgment unit, 313 Correction unit, 401 Emotion estimation model learning unit, 411 Feature extraction unit, 412 Single-channel signal quality judgment unit, 413 Inter-channel signal quality determination unit, 414 Sample signal quality determination unit, 415 Learning unit, 431 Connectivity feature extraction unit, 451 Emotion estimation model sub-learning unit, 501 Sub-discrimination unit, 502 Integrated discrimination unit, 514 Classifier

Claims

1. An information processing method comprising: extracting connectivity features that indicate the relationship between multiple biosignals included in a signal set, based on multiple biosignals obtained by measuring a user's bioreactions on each of multiple channels included in the signal set; and estimating the user's emotions based on the connectivity features extracted for each of the signal sets that can be combined in one sample.

2. The information processing method of claim 1, wherein extracting the connectivity feature comprises: calculating the connectivity feature based on a plurality of the biological signals; determining the signal quality of each of the biological signals; and correcting the biological signals or the connectivity feature based on the result of the signal quality determination.

3. The information processing method according to claim 2, wherein extracting the connectivity feature further includes determining a correction parameter to be used for correcting the biological signal or the connectivity feature based on the result of the signal quality judgment, and performing a time-direction correction or a spatial-direction correction on the biological signal or the connectivity feature using the correction parameter.

4. The information processing method of claim 3, wherein the signal quality judgment result includes at least one of a signal quality score that quantifies the signal quality of each of the multiple biological signals, and a noise label that indicates the type of noise superimposed on each of the multiple biological signals.

5. The information processing method according to claim 4, wherein the connectivity feature at the time to be corrected and the connectivity feature at a time earlier than the time to be corrected are weighted and added based on the correction parameter, thereby correcting the connectivity feature in the time direction; or the biosignal measured at the time to be corrected and the biosignal measured at the earlier time are weighted and added based on the correction parameter, thereby correcting the biosignal in the time direction.

6. The information processing method according to claim 5, wherein the correction parameters are determined so that the higher the signal quality score of each of the plurality of biological signals at the time to be corrected, the greater the weight of the connectivity feature or the biological signal at the time to be corrected, and the lower the signal quality score of each of the plurality of biological signals at the time to be corrected, the greater the weight of the connectivity feature or the biological signal at the past time.

7. The information processing method of claim 4, wherein the spatial direction correction is performed on the connectivity feature by performing a weighted addition of the connectivity feature for the signal set to be corrected and the connectivity feature for another signal set based on the correction parameter, or the spatial direction correction is performed on the biosignal by performing a weighted addition of the biosignal to be corrected and the other biosignal based on the correction parameter.

8. The information processing method of claim 7, wherein the correction parameters are determined so that the lower the signal quality score of each of the plurality of biological signals included in the signal set to be corrected or the lower the signal quality score of the biological signal to be corrected, the greater the weight of the connectivity feature or other biological signal for the other signal sets.

9. The information processing method according to claim 4, wherein the correction parameters are determined by referring to a table in which the correction parameters corresponding to the signal quality scores of each of the plurality of biological signals are recorded.

10. The information processing method according to claim 9, wherein the table to be referenced is switched based on whether the type of noise superimposed on each of the plurality of biological signals is the same.

11. The information processing method of claim 1, wherein estimating the user's emotion includes: determining signal quality on a sample-by-sample basis; estimating the user's emotion based on the connectivity features for each of the signal sets that can be combined within one of the samples; and correcting the estimation result of the user's emotion based on the determination result of the signal quality.

12. The information processing method of claim 1, wherein the sample includes a number of the biological signals sufficient to form multiple signal sets within the sample, and estimating the user's emotion includes: determining signal quality for each signal set; estimating the user's emotion for each signal set based on the connectivity features for the signal sets; correcting the estimation results of the user's emotion for each signal set based on the determination results of the signal quality; and integrating the estimation results of the user's emotion for each signal set after the correction.

13. The information processing method according to claim 1, wherein the plurality of biological signals include at least an electroencephalogram signal obtained by measuring electroencephalograms in the left region of the user's head and an electroencephalogram signal obtained by measuring electroencephalograms in the right region of the head.

14. The information processing method according to claim 1, wherein the plurality of biosignals are acquired by a measuring device provided with a plurality of sensors corresponding to a plurality of channels so as to be able to measure electrical potentials at a plurality of locations on the user's head.

15. The information processing method according to claim 14, wherein the measurement device is one of a head-mounted display, a headband, headphones, earphones, glasses, and a hat.

16. An information processing device comprising: a feature extraction unit that extracts connectivity features that indicate the relationship between multiple biosignals included in a signal set, based on multiple biosignals obtained by measuring a user's bioreactions on each of multiple channels included in the signal set; and an emotion estimation unit that estimates the user's emotion based on the connectivity features extracted for each of the signal sets that can be combined in one sample.

17. A learning method comprising: extracting connectivity features indicating the relationship between multiple biosignals included in a signal set based on multiple biosignals obtained by measuring a person's bioreactions on each of multiple channels included in the signal set; determining signal quality on a sample-by-sample basis; and performing learning using the connectivity features extracted for each of the signal sets that can be combined within the sample, the determination results of the signal quality on a sample-by-sample basis, and emotion labels that indicate the person's emotion linked to the sample, to obtain model parameters used for emotion estimation.

18. The learning method described in claim 17, wherein the sample includes as many biological signals as there are possible combinations of the signal sets within the sample, and determining the signal quality on a sample-by-sample basis includes: determining the signal quality on a signal set-by-signal basis; and calculating the signal quality on a sample-by-sample basis based on the signal quality determination results for each of the possible combinations of the signal sets within the sample.

19. The learning method described in claim 18, wherein determining the signal quality on a sample-by-sample basis further includes determining the signal quality for each of the biological signals, and determining the signal quality on a signal set basis based on the determination results of the signal quality for each of the multiple biological signals included in the signal set.

20. The learning method described in claim 17, wherein the judgment result of the signal quality on a sample-by-sample basis includes a signal quality score that quantifies the signal quality of the biological signal contained in the sample, and learning is performed so that the higher the signal quality score, the higher the contribution of the sample during learning.

Images