Perceptualization device, perceptualization method, and perceptualization program

Abstract

This perceptualization device (10) predicts a change in brain wave of a person from the current time to a predetermined time in the future on the basis of brain wave data for a predetermined time until the current time and estimates the current internal state of the person using the predicted change in brain wave. The perceptualization device (10) calculates the value of a parameter used for perceptualizing the internal state of the person on the basis of the estimated current internal state and uses the calculated value of the parameter to output information (for example, image data, audio data, and tactile sense data) obtained by perceptualizing the internal state of the person.

Description

[Technical Field]

[0001] The present invention relates to a perceptualization device, a perceptualization method, and a perceptualization program for making the internal state of a human being perceptible. [Background technology]

[0002] <000000> Estimating a person's internal state—what they are thinking, feeling, and what state they are in—is crucial for human communication. For example, a technique has been proposed to estimate a person's internal state by translating recognition results obtained from brainwaves into linguistic information.

[0003] Furthermore, a technology has been proposed that estimates a person's emotional state from brainwaves and provides visual feedback of the estimated emotional state through an HMD (head-mounted display) (see Non-Patent Literature 1). In this technology, a person's emotional state is evaluated in two dimensions: Arousal and Valence, and the evaluation results are represented by a fractal (see Figure 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] N. Semertzidis, et al., “Neo-Noumena”, CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, p.1-4, [Retrieved November 9, 2022], Internet<URL:https: / / doi.org / 10.1145 / 3334480.<3383163> [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] However, since electroencephalograms (EEGs) are time-series data that reflect information processing within the human brain, analyzing EEGs over a predetermined period of time is necessary to accurately recognize a person's internal state from EEGs. As a result, delays are unavoidable in recognizing internal states from EEGs, and visualization based on EEG analysis can hinder smooth communication. [Means for solving the problem]

[0006] To solve the aforementioned problems, the present invention is characterized by comprising: a data acquisition unit that acquires a series of electroencephalogram (EEG) data; an estimation unit that predicts changes in EEG from a predetermined time to a predetermined time in advance based on the EEG data up to a predetermined time, and estimates the internal state of the person from whom the EEG data was acquired at the predetermined time using the predicted changes in EEG data; and a perceptibility unit that calculates the values ​​of parameters used to perceptualize the internal state at the predetermined time based on the estimated internal state at the predetermined time, and outputs perceptible information of the internal state at the predetermined time using the calculated values ​​of the parameters. [Effects of the Invention]

[0007] According to the present invention, the internal state of a person can be made perceptible without delay. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram illustrating the conventional technology. [Figure 2] Figure 2 shows an example of the configuration of the perceptualization device in each embodiment. [Figure 3] Figure 3 is a flowchart showing an example of the processing procedure performed by the perceptualization device of the first embodiment. [Figure 4] Figure 4 is a diagram illustrating the estimation of the current internal state by a perceptualization device. [Figure 5] Figure 5 is a flowchart showing an example of the processing procedure performed by the perceptualization device of the second embodiment. [Figure 6]Figure 6 illustrates an example of predicting the power spectral density centered on the current time using the perceptualization device of the second embodiment. [Figure 7] Figure 7 is a flowchart showing an example of the processing procedure performed by the perceptualization device of the third embodiment. [Figure 8] Figure 8 is a flowchart showing an example of the processing procedure performed by the perceptualization device of the fourth embodiment. [Figure 9] Figure 9 shows an example of a computer configuration for running a perceptualization program. [Modes for carrying out the invention]

[0009] Hereinafter, with reference to the drawings, embodiments for carrying out the present invention will be described in four parts, from the first to the fourth embodiment. The present invention is not limited to any of these embodiments.

[0010] The perceptualization device of each embodiment generates information (perceptualization information) for perceptualizing a person's internal state (hereinafter abbreviated as "internal state" as appropriate) based on human electroencephalogram data. Perceptualization information includes, for example, image data, audio data, and tactile data. For example, the perceptualization device displays the generated perceptualization information as an image using a display, plays audio using a speaker, or presents it as tactile data using a vibrator or Peltier element.

[0011] [overview] Next, an overview of the perceptualization device for each embodiment will be described. For example, when the perceptualization device acquires human brainwave data (past brainwave data) from an electroencephalograph, it estimates the changes in brainwaves over a predetermined period of time centered on the current time (the timing for perceptualizing the internal state) at predetermined intervals (for example, every u seconds) based on the analysis results of the acquired brainwave data.

[0012] Then, the perceptible visualization device calculates perceptible visualization parameters for perceptibly visualizing the internal state of a human using the estimated changes in brain waves. Thereafter, the perceptible visualization device generates and outputs perceptible visualization information corresponding to the perceptible visualization parameters. For example, when the perceptible visualization parameters are parameters for drawing geometric figures, the perceptible visualization device draws geometric figures using the perceptible visualization parameters and displays them on a display.

[0013] Here, the brain wave data is time-series data, and it is necessary to use brain wave data for a predetermined time for its analysis. Therefore, if the brain wave data obtained by the perceptible visualization device by a specific timing is directly used for analysis, only the internal state before that timing can be obtained. Therefore, the perceptible visualization device cannot, for example, perceptibly visualize the internal state at the current time.

[0014] Therefore, the perceptible visualization device estimates changes in brain waves for a predetermined time centered on the current time at every predetermined time (for example, every u seconds) based on the analysis result of the acquired brain wave data. Thereby, the perceptible visualization device can provide feedback on the internal state at the timing to be perceptibly visualized.

[0015] In addition, the perceptible visualization device estimates the perceptible visualization parameters in consideration of the execution time of a series of processes in the perceptible visualization of the internal state. Thereby, the perceptible visualization device can provide feedback on the internal state at the timing to be perceptibly visualized even in a situation where a delay occurs in the process and communication for the perceptible visualization of the internal state.

[0016] Furthermore, when the perceptible visualization device continuously provides feedback while interpolating the perceptible visualization parameters of the internal state calculated at every predetermined time, providing feedback based on the calculated perceptible visualization parameters of the internal state will be after the elapse of a predetermined time. For this reason, a delay occurs in the feedback of the internal state.

[0017] Here, the perceptible state device can perform feedback on the internal state without delay by estimating a perceptible parameter that reflects the internal state after a predetermined time at each predetermined time.

[0018] [Configuration Example] Next, a configuration example of the perceptible state device 10 of each embodiment will be described with reference to FIG. 2. The perceptible state device 10 includes, for example, an electroencephalogram data acquisition unit (data acquisition unit) 11, a storage unit 12, an internal state estimation unit (estimation unit) 13, and an internal state perceptible state unit 14.

[0019] Although not shown in the figure, the perceptible state device 10 includes a communication interface such as a NIC (Network Interface Card), an input / output interface, and the like. For example, the communication interface controls communication with an external device such as an electroencephalograph via a network. Also. The input / output interface outputs the perceptible state information of the internal state generated by the perceptible state device 10 to an external device. For example, the input / output interface outputs the perceptible state information (e.g., an image of a geometric figure) of the human internal state obtained from the electroencephalogram to a display.

[0020] The electroencephalogram data acquisition unit 11 acquires electroencephalogram data of the analysis target measured by an electroencephalograph or the like. The storage unit 12 stores data and programs referred to in the estimation and perceptible state of the internal state. For example, the storage unit 12 stores the electroencephalogram data acquired by the electroencephalogram data acquisition unit 11, the perceptible state parameter of the internal state estimated by the internal state estimation unit 13, and the like.

[0021] The internal state estimation unit 13 predicts the change in the electroencephalogram from a predetermined time point (e.g., the current time) to a predetermined time in the future based on the electroencephalogram data of a predetermined time up to the predetermined time point (e.g., the current time) acquired by the electroencephalogram data acquisition unit 11, and uses the predicted change in the electroencephalogram to estimate the internal state at the predetermined time point.

[0022] For example, the internal state estimation unit 13 performs noise reduction and other processing on a series of electroencephalogram (EEG) data acquired by the EEG data acquisition unit 11. Subsequently, the internal state estimation unit 13 performs EEG feature calculations, such as calculating the power spectral density, on the processed EEG data. Then, based on the calculated EEG feature quantities, the internal state estimation unit 13 estimates the changes in the EEG and estimates the current internal state from the estimated changes in the EEG.

[0023] The internal state perceptibility unit 14 calculates the values ​​of parameters used to perceptualize the internal state based on the current internal state estimated by the internal state estimation unit 13. Then, the internal state perceptibility unit 14 outputs information that perceptualizes the current internal state using the calculated parameter values.

[0024] For example, the internal state perceptibility unit 14 calculates multidimensional perceptibility parameters corresponding to the internal state from the internal state estimated by the internal state estimation unit 13. Then, the internal state perceptibility unit 14 draws geometric figures based on the calculated perceptibility parameters and displays them on the display.

[0025] Such a perceptualization device 10 makes a person's internal state perceptible without delay.

[0026] [First Embodiment] Next, the perceptualization device 10 of the first embodiment will be described. The perceptualization device 10 of the first embodiment predicts brainwave data for N seconds from the current time using brainwave data from n seconds ago to the present time (the timing for perceptualizing the internal state). Then, the perceptualization device 10 estimates the current internal state using the predicted brainwave data.

[0027] [Example of processing procedure] Using Figure 3, an example of the procedure by which the perceptualization device 10 of the first embodiment makes the internal state perceptible will be explained.

[0028] First, the electroencephalogram (EEG) data acquisition unit 11 of the perceptualization device 10 acquires EEG data (S11). The EEG data acquired here is time-series data, and the EEG data acquisition unit 11 sequentially acquires EEGs measured at an arbitrary sampling rate. The acquired EEG data is stored in the memory unit 12.

[0029] After S11, the internal state estimation unit 13 performs preprocessing on the electroencephalogram data from n seconds prior to the present time (S12).

[0030] Preprocessing involves, for example, filtering to remove various noise components contained in electroencephalogram (EEG) data. Filtering can include, for example, bandpass filtering to extract only signals in a specific frequency band from the EEG data, bandstop filtering to remove hum noise originating from AC power supplies, noise reduction filtering to remove impulsive noise associated with body movements, artifact removal filtering using independent component analysis (ICA), or a combination thereof.

[0031] The internal state estimation unit 13 may also store the pre-processed electroencephalogram (EEG) data in the storage unit 12. Furthermore, if the internal state estimation unit 13 performs the same processing on the same EEG data, it may omit the pre-processing by reading the stored pre-processed EEG data.

[0032] After S12, the internal state estimation unit 13 uses pre-processed electroencephalogram data from n seconds prior to the present time to predict electroencephalogram data for N seconds from the present time (S13).

[0033] Any method can be used to predict electroencephalogram (EEG) data, but for example, time series prediction techniques can be used.

[0034] For example, the perceptualization device 10 uses pre-recorded, pre-processed electroencephalogram (EEG) data to construct a neural network (EEG data predictor) that predicts EEG data for the following N seconds based on EEG data from the past n seconds. Then, the internal state estimation unit 13 uses the constructed EEG data predictor to predict EEG data for the N seconds from the current time based on pre-processed EEG data from n seconds ago to the current time.

[0035] Alternatively, the internal state estimation unit 13 may predict brainwave data for N seconds from the current time from preprocessed brainwave data from n seconds prior to the current time, as described below.

[0036] For example, the perceptualization device 10 uses pre-recorded, pre-processed electroencephalogram (EEG) data to train a recurrent neural network (RNN), such as an LSTM, which predicts the next EEG data from past EEG data. The internal state estimation unit 13 then uses the trained RNN to predict the EEG data.

[0037] For example, the perceptualization device 10 uses pre-recorded, pre-processed electroencephalogram (EEG) data to train an RNN such as an LSTM (Long Short-Term Memory) that predicts the next EEG data from past EEG data. The internal state estimation unit 13 then inputs the EEG data from the past n seconds into the RNN to predict the EEG data for the next timing. Next, the internal state estimation unit 13 inputs the predicted EEG data into the RNN to predict the EEG data for the next timing. The internal state estimation unit 13 repeats the above process to predict EEG data for N seconds.

[0038] After S13, the internal state estimation unit 13 calculates the power spectral density using the pre-processed electroencephalogram (EEG) data from n seconds prior to the present time and the EEG data for N seconds predicted in S13 (S14).

[0039] After S14, the internal state estimation unit 13 estimates the internal state using the power spectral density calculated in S14 (S15). For example, the internal state estimation unit 13 outputs a multidimensional vector showing the estimation result of the internal state.

[0040] Any method can be used to estimate the internal state, but for example, a large number of power spectral densities annotated with multidimensional vectors representing the internal state are prepared in advance. The perceptualization device 10 then uses these power spectral densities as training data to construct a neural network (internal state estimator) that estimates multidimensional vectors representing the internal state from the power spectral densities.

[0041] Then, the internal state estimation unit 13 inputs the power spectral density calculated in S14 to the internal state estimator to obtain a multidimensional vector representing the internal state (estimated result of the internal state).

[0042] Alternatively, the internal state estimation unit 13 may estimate the internal state by performing a similarity analysis of the representation between the power spectral density for each internal state collected in advance and the power spectral density calculated in S14. In other words, the internal state estimation unit 13 may estimate the internal state based on the similarity between the brain activity indicated by the power spectral density collected in advance and the brain activity indicated by the power spectral density calculated in S14.

[0043] For example, the internal state estimation unit 13 generates a similarity vector for each power spectral density collected in advance, showing the similarity to the power spectral density calculated in S14. The internal state estimation unit 13 then outputs the generated similarity vector as a multidimensional vector representing the internal state.

[0044] After S15, the internal state perceptibility unit 14 makes the internal state estimated in S15 perceptible (S16). For example, the internal state perceptibility unit 14 generates geometric figures, sounds, vibrations, etc., using the multidimensional vector representing the internal state estimated in S15 as a parameter.

[0045] Furthermore, the internal state perceptibility unit 14 may not directly perceptify the multidimensional vector obtained in S15, but may convert it into a lower-dimensional vector using, for example, a dimensionality reduction method such as principal component analysis or discriminant analysis, before perceptifying it.

[0046] By executing the above processing flow, the perceptualization device 10 can estimate the internal state based on frequency analysis of electroencephalogram (EEG) data over a predetermined time span, even when estimating the internal state based on EEG data centered on the current time. As a result, the perceptualization device 10 can estimate the current internal state (a predetermined time span centered on the current time) rather than the past internal state (see Figure 4).

[0047] [Second Embodiment] Next, the perceptualization device 10 of the second embodiment will be described. The perceptualization device 10 of the second embodiment predicts the power spectrum density of EEG data from the present to N seconds using the power spectrum density of EEG data from n seconds ago to the present. Then, the perceptualization device 10 estimates the current internal state using the predicted power spectrum density.

[0048] [Example of processing procedure] Using Figure 5, an example of the procedure by which the perceptualization device 10 of the second embodiment makes the internal state perceptible will be explained. The processes S21 and S22 in Figure 5 are the same as the processes S11 and S12 in Figure 3, so their explanation will be omitted, and the explanation will begin from S23 in Figure 5.

[0049] After S22 in Figure 5, the internal state estimation unit 13 of the perceptualization device 10 calculates the power spectral density using the pre-processed electroencephalogram data obtained in S22 from M seconds prior to the present time (S23). Subsequently, the internal state estimation unit 13 associates the time information with the calculated power spectral density and stores it in the memory unit 12.

[0050] After S23, the internal state estimation unit 13 predicts the power spectral density of the electroencephalogram data for a predetermined time centered on the current time, using time-series data of the power spectral density of the electroencephalogram data from M seconds prior to the current time (S24). Any method can be used to predict the power spectral density, but for example, the time-series prediction technique described above can be used.

[0051] For example, the perceptualization device 10 generates time-series data of power spectral density using pre-recorded and pre-processed electroencephalogram (EEG) data. The perceptualization device 10 then constructs a neural network (power spectral density predictor) that predicts the power spectral density centered on the current time from the time-series data of power spectral density of the EEG data over the past x seconds. Subsequently, the internal state estimation unit 13 uses the above power spectral density predictor to predict the power spectral density centered on the current time from the time-series data of power spectral density of the EEG data over the past x seconds.

[0052] Furthermore, the internal state estimation unit 13 may predict the power spectral density centered on the current time using the RNN described above.

[0053] For example, the perceptualization device 10 generates time-series data of power spectral density using pre-recorded, pre-processed electroencephalogram (EEG) data. The perceptualization device 10 then learns an RNN, such as an LSTM, that predicts the next power spectral density from past power spectral densities. Subsequently, the internal state estimation unit 13 predicts the power spectral density at the next timing by inputting the power spectral density of the EEG data for the past x seconds into the RNN. Next, the internal state estimation unit 13 predicts the power spectral density at yet another timing by inputting the predicted power spectral density into the RNN. The internal state estimation unit 13 predicts the power spectral density centered on the current time by repeating the above process.

[0054] Furthermore, the internal state estimation unit 13 may predict the power spectral density centered on the current time by performing a representational similarity analysis with time-series data of the power spectral density of pre-recorded electroencephalogram data. An example of predicting the power spectral density centered on the current time using representational similarity analysis will be explained with reference to Figure 6.

[0055] For example, the internal state estimation unit 13 generates time-series data of the power spectral density of pre-recorded electroencephalogram (EEG) data. For example, the internal state estimation unit 13 generates time-series data of the power spectral density with a calculation window of n seconds and a sliding window of m seconds.

[0056] The internal state estimation unit 13 then performs a representational similarity analysis between the time-series power spectral density data of the previously recorded electroencephalogram (EEG) data and the time-series power spectral density data of the past y seconds of EEG data.

[0057] Next, the internal state estimation unit 13 weights the time series data of power spectral density collected in advance, according to the degree of similarity between the time series data of power spectral density of the EEG data recorded in advance and the time series data of power spectral density of the EEG data from the past y seconds. Then, the internal state estimation unit 13 generates predicted time series data of power spectral density by adding the weighted time series data of power spectral density.

[0058] Subsequently, the internal state estimation unit 13 extracts data corresponding to the power spectral density centered on the current time from the predicted time series data of power spectral density.

[0059] Furthermore, the internal state estimation unit 13 may predict the power spectral density centered on the current time by weighting and adding only the portion of the power spectral density used in the expression similarity analysis that corresponds to the power spectral density centered on the current time, rather than weighting and adding the entire time-series data of the power spectral density of the previously recorded electroencephalogram data (see Figure 6).

[0060] Returning to the explanation of Figure 5, after S24 in Figure 5, the internal state estimation unit 13 estimates the internal state using the power spectral density centered on the current time predicted in S24 (S25). The estimation of the internal state in S25 is the same as S15 in Figure 3 described above, so the explanation is omitted. Also, S26 in Figure 5 is the same as S16 in Figure 3, so the explanation is omitted.

[0061] Generally, the sampling frequency of electroencephalogram (EEG) data is very high (i.e., the sampling period is short), so for example, predicting one second of EEG data requires a very large number of predictions. On the other hand, the sampling frequency of time-series data of power spectral density depends on the sliding window used when obtaining time-series data of power spectral density from EEG data. Therefore, the sampling period of time-series data of power spectral density is generally set to be significantly wider than the sampling period of EEG data. Consequently, as in the perceptualization device 10 of the second embodiment, by making the target of prediction the power spectral density of EEG data instead of EEG data, the amount of processing required for prediction can be reduced.

[0062] [Third Embodiment] Next, the perceptualization device 10 of the third embodiment will be described. The perceptualization device 10 of the third embodiment estimates the internal state from M seconds ago to the present using the power spectrum density of electroencephalogram data from M seconds ago to the present, and uses the estimated internal state from M seconds ago to the present to predict the current internal state (a predetermined time centered on the present).

[0063] [Example of processing procedure] Using Figure 7, an example of the procedure by which the perceptibility device 10 of the third embodiment makes the internal state perceptible will be explained. The processes S31 to S33 in Figure 7 are the same as the processes S21 to S23 in Figure 5, and the process S34 in Figure 7 is the same as S15 in Figure 3, so the explanation will be omitted, and the explanation will start from the process S35 in Figure 7. The internal state estimation unit 13 stores the past internal state (time-series data of the internal state) estimated in S34 of Figure 7 in the storage unit 12.

[0064] After S34 in Figure 7, the internal state estimation unit 13 uses the time-series data of the internal state stored in the memory unit 12 (time-series data of the internal state from M seconds ago to the present) to predict the current internal state (a predetermined time centered on the present) (S35). The process in S35 is the same as the process in S24 in Figure 5, except that the time-series data handled is changed from power spectral density to internal state, so the explanation is omitted. Also, S36 in Figure 7 is the same as S16 in Figure 3, so the explanation is omitted.

[0065] [Fourth Embodiment] Next, the perceptualization device 10 of the fourth embodiment will be described. The perceptualization device 10 of the fourth embodiment directly predicts the current internal state (a predetermined time period centered on the present) from the power spectrum density of electroencephalogram data from M seconds ago to the present.

[0066] [Example of processing procedure] Using Figure 8, an example of the procedure by which the perceptibility device 10 of the fourth embodiment makes the internal state perceptible will be explained. The processes S41 to S43 in Figure 8 are the same as the processes S21 to S23 in Figure 5, so the explanation will be omitted, and the explanation will start from S44 in Figure 8.

[0067] After S43 in Figure 8, the internal state estimation unit 13 of the perceptualization device 10 predicts the current internal state (a predetermined time period centered on the present) using the time-series data of the power spectral density of the electroencephalogram data stored in the memory unit 12 from M seconds ago to the present time (S44). The process in S44 can be the same as the process in S24 in Figure 5, but the prediction target is the internal state from the power spectral density. Also, S45 in Figure 8 is the same as S16 in Figure 3, so the explanation is omitted.

[0068] As in the perceptualization device 10 of the fourth embodiment, the amount of computation required to predict the current internal state can be reduced by directly predicting the current internal state from time-series data of power spectral density.

[0069] In the fourth embodiment, the perceptibility device 10 has different inputs (power spectral density) and outputs (internal states) in the prediction process. Therefore, the fourth embodiment requires the internal states to be determined before training the internal state predictor using a neural network. Consequently, the cost of training the internal state predictor may be high. However, since the internal state predictor only needs to be trained once in advance, the total computational cost can be reduced if the cost of calculations that need to be performed repeatedly can be reduced.

[0070] Each component shown in the diagram represents a functional concept and does not necessarily need to be physically configured as depicted. In other words, the specific forms of distribution and integration of each device are not limited to those shown; all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. Furthermore, each processing function performed by each device can be implemented, in whole or in any part, by a CPU and the program executed on that CPU, or by wired logic hardware.

[0071] Furthermore, among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0072] [program] The aforementioned perceptualization device 10 can be implemented by installing a program (perceptualization program) as packaged software or online software on a desired computer. For example, by having the above program run on an information processing device, the information processing device can function as the perceptualization device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), as well as terminals such as PDA (Personal Digital Assistant).

[0073] Figure 9 shows an example of a computer running a perceptibility program. Computer 1000 has, for example, memory 1010 and a CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0074] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0075] The hard disk drive 1090 stores, for example, the OS 1091, application program 1092, program module 1093, and program data 1094. That is, the program that defines each process executed by the perceptualization device 10 is implemented as a program module 1093 in which executable code for a computer is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing a process similar to the functional configuration in the perceptualization device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0076] Furthermore, the data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0077] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070. [Explanation of Symbols]

[0078] 10 Perceptualization Device 11. Electroencephalogram (EEG) data acquisition unit 12 Storage section 13. Internal state estimation unit 14. Internal state perceptibility unit

Claims

1. A data acquisition unit that acquires a series of electroencephalogram (EEG) data, An estimation unit that predicts changes in brain waves from a predetermined time to a predetermined time in advance based on the brain wave data up to a predetermined time, and estimates the internal state of the person from whom the brain wave data was acquired at the predetermined time using the predicted changes in brain waves, A perceptibility unit calculates the value of a parameter used to perceptualize the internal state at a predetermined time based on the estimated internal state at that time, and outputs information that perceptualizes the internal state at that time using the calculated parameter value. A perceptualization device characterized by comprising the following features.

2. The estimation unit, Based on the electroencephalogram (EEG) data up to a predetermined time, the EEG data from that predetermined time to a predetermined time in the future is predicted, the power spectrum density of the predicted EEG data and the EEG data up to the predetermined time is calculated, and the internal state at the predetermined time is estimated based on the calculated power spectrum density. The perceptualization device according to feature 1.

3. The estimation unit, The internal state prediction model, which was trained using data in which the internal state of a person indicated by the EEG data is assigned to the power spectral density of the EEG data, is used as training data. By inputting the calculated power spectral density into this model, the internal state at a predetermined time is estimated. The perceptualization device according to feature 2.

4. The estimation unit, The system calculates time-series data of the power spectral density of electroencephalogram (EEG) data up to the predetermined time point, predicts the power spectral density for a predetermined time period centered on the predetermined time point based on the calculated time-series data of the power spectral density, and estimates the internal state of the person at the predetermined time point based on the predicted power spectral density. The perceptualization device according to feature 1.

5. The estimation unit, By performing a similarity analysis of representations between pre-recorded historical power spectral density time series data and the calculated power spectral density time series data, the power spectral density for a predetermined time period centered on the predetermined time point is predicted. The perceptualization device according to feature 4.

6. The estimation unit, The system calculates time-series data of the human internal state from the electroencephalogram data up to the predetermined time point, and estimates the internal state at the predetermined time point based on the calculated time-series data of the internal state. The perceptualization device according to feature 1.

7. A method of perceptualization performed by a perceptualization device, The process of acquiring a series of electroencephalogram (EEG) data, A step of predicting changes in brain waves from a predetermined time to a predetermined time in advance based on the brain wave data up to a predetermined time, and using the predicted changes in brain waves to estimate the internal state of the person from whom the brain wave data was acquired at the predetermined time, A step of calculating the value of a parameter used to make the internal state at the predetermined time perceptible, based on the estimated internal state at the predetermined time, and outputting information that makes the internal state at the predetermined time perceptible, using the calculated value of the parameter. A method for making something perceptible, characterized by including the following:

8. A perceptualization program for causing a computer to function as a perceptualization device according to any one of claims 1 to 6.

Images