Information presentation device

Abstract

This information presentation device acquires information regarding the time when a stimulus is presented to a user, and biological information of the user. Thereafter, the information presentation device extracts biological information of the user at predetermined time intervals starting from the time when the stimulus was presented to the user. Then, the information presentation device calculates phase information on each of the extracted pieces of biological information, thereby obtaining time-series phase information. Thereafter, the information presentation device calculates a score indicating how close the time-series phase information is to the prior time-series phase information of the user stored in a storage unit. Then, the information presentation device outputs information indicating the calculated score.

Description

Information presentation device

[0001] This invention relates to an information presentation device used in neurofeedback.

[0002] There is a technology called neurofeedback that provides real-time feedback of brain information measured from a user. Neurofeedback has been reported to be useful in improving users' attention and motor skills (see Non-Patent Literature 1).

[0003] Neurofeedback often uses the amplitude of measured brain signals at specific frequencies, or phase synchronization between simultaneously measured signals, as feature quantities for brain information. Furthermore, neurofeedback methods aimed at synchronizing brain information from multiple individuals have also been proposed (see Patent Document 1).

[0004] Conventional neurofeedback systems acquire user brain information, for example, using an electroencephalograph (EEG), and record the user's brain activity during task execution. This brain information is analyzed in real time, and features such as the amplitude of specific frequencies in the brain's measured signals and inter-electrode phase synchronization during the same time period are fed back to the user. The user can then modulate their brain activity by spontaneously controlling the information they receive back. This enables improvements in specific user abilities and synchronization between brainwaves.

[0005] Patent No. 5317277

[0006] Brain Tech Evidence Book ver 2.0, [online], [Accessed September 19, 2024], Internet <URL: https: / / brains.link / braintech_guidebook>

[0007] However, conventional neurofeedback methods either present indicators of brain information measured in real time or compare them to pre-measured baseline values. Therefore, it has not been possible to provide real-time feedback on features where the baseline fluctuates significantly.

[0008] In particular, with regard to electroencephalogram (EEG) indicators, the phase information of EEG changes moment by moment, making it difficult to utilize for feedback. However, EEG phase information changes in response to the user's perceptual stimuli, suggesting potential for using it to provide feedback on brain activity related to the user's perception. Therefore, neurofeedback using EEG phase information is expected to be an effective training method for enhancing the user's perceptual abilities.

[0009] Therefore, in order to solve the aforementioned problems, the present invention aims to provide the user with feedback of the phase information of brain waves that changes moment by moment.

[0010] To solve the aforementioned problems, the present invention is characterized by comprising: an information acquisition unit that acquires the user's biological information extracted at predetermined time intervals based on the time when a stimulus was presented to the user; a biological information analysis unit that calculates the phase information of each of the acquired biological information to obtain time-specific phase information, and calculates a score indicating how close the time-specific phase information is to the distribution of the user's time-specific phase information stored in a memory unit in the past; and an output processing unit that outputs information indicating the score.

[0011] According to the present invention, it is possible to provide the user with feedback on the phase information of brain waves that changes moment by moment.

[0012] Figure 1 is a diagram illustrating the overview of the information presentation device. Figure 2 is a diagram showing an example of the configuration of the information presentation device. Figure 3 is a diagram showing an example of the calibration processing procedure performed by the information presentation device. Figure 4 is a diagram showing an example of the neurofeedback processing procedure performed by the information presentation device. Figure 5 is a diagram showing an example of a computer that runs the information presentation program.

[0013] The following describes embodiments for carrying out the present invention with reference to the drawings. The present invention is not limited to these embodiments.

[0014] [Overview] The system including the information presentation device 10 of this embodiment will be explained using Figure 1. The system acquires brain waves as the user's biological information and feeds back the moment-by-moment changing phase information of the brain waves to the user.

[0015] Here, the system accurately time-synchronizes brainwaves measured at different times to enable real-time feedback of the user's brainwave phase information, and also displays how closely the current brainwave phase information has approached the distribution of past brainwave phase information.

[0016] The system includes, for example, an information display device 10, a biological information measurement unit 20, and an output unit 40, as shown in Figure 1.

[0017] The bio-information measurement unit 20 measures the user's brainwaves and outputs them to the information display device 10. The brainwaves reflect, for example, the user's response to stimuli (events). The brainwaves also include feature quantities that change moment by moment (for example, phase).

[0018] The information presentation device 10 controls the timing of stimuli (events) presented to the user. The information presentation device 10 then records characteristic quantities of the brainwave (the user's brainwave measured by the bio-information measurement unit 20) according to the timing of when the stimuli are presented to the user.

[0019] Subsequently, the information display device 10 generates a feedback screen showing the results of comparing the user's brainwave features with the distribution of past brainwave features, and outputs it to the output unit 40. For example, the information display device 10 generates a feedback screen (see Figure 1) that shows, using the height of a bar, how closely the brainwave phase when a certain stimulus is presented to the user approaches the distribution of brainwave phases when the same stimulus was presented in the past, and outputs it to the output unit 40.

[0020] This allows users to see in real time the changes in their brainwaves when they perceive a stimulus on a feedback screen. As a result, for example, users can modulate their own brain activity by spontaneously controlling (manipulating) objects (e.g., bars) on the feedback screen.

[0021] [Configuration Example] Next, an example of the system configuration will be explained using Figure 2. The system comprises a biological information measurement unit 20, a stimulus presentation unit 30, an output unit 40, and an information presentation device 10. These are connected to each other via a wired or wireless network for communication.

[0022] [Biometric Information Measurement Unit] The biometric information measurement unit 20 measures the user's biometric information (e.g., electroencephalogram) in real time while the user is performing a task and outputs it to the information display device 10. This biometric information measurement unit 20 is implemented by, for example, an electroencephalograph. In addition to an electroencephalograph, the biometric information measurement unit 20 may also be other devices that measure the user's biometric information and movements, such as an eye movement measuring device (eye tracker), an electrocardiograph, or a respiratory measuring device.

[0023] [Stimulus Presentation Unit] The stimulus presentation unit 30 presents stimuli indicating the start and end times of the tasks performed by the user, based on control from the information presentation device 10. The stimulus presentation unit 30 presents and plays stimuli such as images, sounds, electrical stimuli, etc., at predetermined time intervals.

[0024] [Output Unit] The output unit 40 outputs the information output from the information display device 10. For example, the output unit 40 displays the feedback screen output from the information display device 10. This output unit 40 is implemented by, for example, a liquid crystal monitor, a speaker, a device that outputs electrical stimulation to the user, etc.

[0025] [Information display device] The information display device 10 comprises a biological information holding unit (information acquisition unit) 11, a biological information analysis unit 12, a biological information storage unit 13, and an output processing unit 14.

[0026] [Biometric Information Storage Unit] The biometric information storage unit 11 receives electroencephalogram (EEG) data (time-series data of EEG waveforms) output from the biometric information measurement unit 20 during the user's task performance at predetermined intervals. The biometric information storage unit 11 also acquires time information from the stimulus presentation unit 30 when a stimulus was presented to the user. The biometric information storage unit 11 then temporarily stores the EEG data from the user during the task performance and the time information from the stimulus presentation.

[0027] [Biological Information Analysis Unit] The biological information analysis unit 12 uses the electroencephalogram (EEG) data obtained from the biological information storage unit 11 during the user's task performance and time information from stimulus presentation to calculate time-specific feature quantities of the acquired EEG data. For example, the biological information analysis unit 12 uses the time when the stimulus was presented to the user as a reference, extracts EEG data at predetermined time intervals, and calculates the phase information of the extracted EEG data to obtain time-specific phase information. The biological information analysis unit 12 then calculates a score (feedback score) indicating how closely the time-specific phase information is similar to the distribution of the user's time-specific phase information stored in the biological information storage unit 13 in the past.

[0028] For example, when the biological information analysis unit 12 acquires electroencephalogram (EEG) data from the biological information storage unit 11, it preprocesses the acquired EEG data to reduce the influence of noise originating from non-brain information. Subsequently, the biological information analysis unit 12 calculates the phase of each frequency of the preprocessed EEG data using discrete Fourier transforms, Hilbert transforms, etc.

[0029] During calibration (details described later), the bio-information analysis unit 12 outputs time information and phase information from the stimulus presentation to the bio-information storage unit 13, thereby creating a set of phase information for each time period from the stimulus presentation.

[0030] Furthermore, during neurofeedback, the bio-information analysis unit 12 calculates a feedback score based on the statistical quantities (statistical information) of the set of phase information stored in the bio-information storage unit 13, and outputs it to the output processing unit 14. Note that the feature quantities of the electroencephalogram data calculated by the bio-information analysis unit 12 may be amplitude, waveform, etc., in addition to the phase of a specific frequency.

[0031] [Biological Information Storage Unit] The biological information storage unit 13 acquires and stores characteristic quantities (e.g., phase information) and temporal information of the electroencephalogram data output from the biological information analysis unit 12.

[0032] Also, during neurofeedback, the biological information storage unit 13 calculates the statistic of the set of stored feature amounts and outputs it to the biological information analysis unit 12. For example, the biological information storage unit 13 fits the set of feature amounts with matching time information from stimulus presentation to a predetermined probability distribution (for example, von Mises distribution), and calculates the statistic representing the distribution of the set of feature amounts. By fitting the set of feature amounts of the electroencephalogram data to a predetermined probability distribution by the biological information storage unit 13, it is possible to reduce the influence even when a deviation occurs in the time information.

[0033] Incidentally, the biological information storage unit 13 may update the set of feature amounts stored in the biological information storage unit 13 based on the feature amounts newly obtained by neurofeedback. Also, when there are different types of tasks performed by the user, the biological information storage unit 13 may hold the set of feature amounts for each type of task performed by the user.

[0034] [Output Processing Unit] The output processing unit 14 outputs the information indicating the feedback score output from the biological information analysis unit 12 to the output unit 40. For example, the output processing unit 14 generates a feedback screen with an object of a size corresponding to the feedback score and outputs it to the output unit 40.

[0035] For example, the output processing unit 14 generates a feedback screen (see FIG. 1) with a larger object size as the value of the feedback score approaches 0 (for example, as the measured phase of the user's electroencephalogram approaches the distribution of the past electroencephalogram phases of the user), and outputs it to the output unit 40.

[0036] Incidentally, the output processing unit 14 may generate and output a feedback screen in which the feedback score is converted into the color of an object, the position of a cursor. Also, the output processing unit 14 may convert the feedback score into information indicating the pitch of a sound, the intensity of an electrical stimulus, etc., and output it. Further, the output processing unit 14 may generate a feedback screen including an object (for example, a broken line in the feedback screen shown in FIG. 1) corresponding to the feedback score that the user should aim for, and output it to the output unit 40.

[0037] [Example of Processing Procedure] Next, an example of the processing procedure executed by the information presentation device 10 will be described using FIGS. 3 and 4. The information presentation device 10 first performs calibration for neurofeedback and then performs neurofeedback.

[0038] [Calibration] An example of the calibration processing procedure executed by the information presentation device 10 will be described using FIG. 3. Calibration is performed to create a set of feature quantities used for comparison when the information presentation device 10 performs neurofeedback.

[0039] First, the stimulus presentation unit 30 presents a plurality of stimuli for causing the user to perform a target task (S1). Then, the biological information measurement unit 20 measures the biological information of the user during task execution (S2). The biological information holding unit 11 temporarily holds the biological information of the user during task execution and the time information from stimulus presentation.

[0040] After that, the biological information analysis unit 12 calculates the feature quantity of a single trial from the biological information of the user during task execution acquired from the biological information holding unit 11 (S3). The biological information analysis unit 12 executes the process of S3 above for the number of trials and outputs the calculated feature quantity of each trial to the biological information storage unit 13.

[0041] After that, the biological information storage unit 13 calculates the statistic of the feature quantities of a plurality of trials (S4) and stores the calculated statistic in the biological information storage unit 13 (S5).

[0042] Note that the number of trials of the task for calibration is, for example, about 20 - 100. For example, by having the user perform about 20 - 100 trials of the task, the information presentation device 能够创建计算稳定概率分布所需的特征量集合。

[0043] Note that the number of trials of the task for calibration may be changed according to the types of feature quantities calculated by the information presentation device 10. Also, when the distribution of the target feature quantities is defined in advance, the information presentation device 10 may perform neurofeedback without performing calibration.

[0044] [Neurofeedback] Next, an example of the neurofeedback processing procedure performed by the information presentation device 10 will be explained using Figure 4.

[0045] First, the stimulus presentation unit 30 presents a stimulus once to prompt the user to perform the target task (S11). Then, the biological information measurement unit 20 measures the user's biological information at the time of stimulus presentation (S12). The biological information storage unit 11 temporarily stores the user's biological information at the time of task performance and time information from stimulus presentation.

[0046] Subsequently, the biological information analysis unit 12 calculates a feedback score (S13) based on the statistical quantities stored in the biological information storage unit 13 and the biological information (biological information measured in S12). For example, the biological information analysis unit 12 calculates a feedback score using the characteristic quantities of the biological information measured in S12 (for example, the time-dependent phase information of the electroencephalogram data) and the statistical quantities stored in the biological information storage unit 13.

[0047] After S13, the output processing unit 14 presents the user with information indicating the feedback score calculated in S13 (S14). For example, the output processing unit 14 generates a feedback screen with objects of a size corresponding to the feedback score calculated by the bio-information analysis unit 12, and outputs it to the output unit 40.

[0048] Subsequently, the biological information storage unit 13 corrects the statistical values ​​based on the acquired data (S15), and stores the corrected statistical values ​​in the biological information storage unit 13 (S16). Then, the process returns to S11.

[0049] For example, when the biological information storage unit 13 obtains new biological information features from the biological information analysis unit 12, it recalculates statistics using the new features and the set of features accumulated so far. Then, the biological information storage unit 13 updates the statistics stored in the biological information storage unit 13 with the recalculated statistics.

[0050] By performing the above processing, the information presentation device 10 can provide feedback to the user on feature quantities that change moment by moment while the user is performing the task, such as phase information of electroencephalogram data.

[0051] [Example of Embodiment] Next, an example of the information presentation device 10 will be described. Here, a case where the information presentation device 10 executes neurofeedback for modulating the user's olfactory ability will be described as an example.

[0052] First, the information presentation device 10 executes calibration in order to acquire a set of baseline feature amounts (phase information of electroencephalogram data). In the calibration, the stimulus presentation unit 30 presents an olfactory stimulus to the user 30 times. Then, the biological information measurement unit 20 measures the electroencephalogram data of the user's forehead at that time. The biological information analysis unit 12 performs processing of a band-pass filter for extracting a frequency of interest, a notch filter for reducing power noise, and a spatial filter for extracting a local signal as preprocessing of the measured electroencephalogram data.

[0053] Then, the biological information analysis unit 12 performs discrete Fourier transform on the electroencephalogram data cut out in a 500-ms window every 50 ms based on the timing of the olfactory stimulus presentation by the stimulus presentation unit 30, thereby obtaining the phase information θ t in the frequency band of interest.

[0054] And the biological information storage unit 13 receives from the biological information analysis unit 12 the time t based on the above stimulus and the phase information θ' t = [θ 1t , θ 2t , …, θ nt ] (the phase information θ 1t of time t in the first trial, the phase information θ 2t of time t in the second trial, …, and the phase information θ nt of time t in the nth trial nt nt ), and stores it in the biological information storage unit 13 as a set of phase information serving as a baseline.

[0055] In the neurofeedback, the biological information analysis unit 12 acquires electroencephalogram data in a 500-ms window every 50 ms based on the trigger output of the stimulus presentation. Then, the biological information analysis unit 12 performs the same processing as in the calibration in real time on the acquired electroencephalogram data, thereby obtaining the phase information θ t in the frequency band of interest.

[0056] Subsequently, the biological information analysis unit 12 uses the data acquired in the calibration session to determine the phase information θ, for example, based on the following equation (1). t Feedback score (FB score) t The feedback score is calculated as follows: The feedback score is a value that indicates how closely the phase information of the electroencephalogram (EEG) data obtained by neurofeedback is similar to the distribution of the phase information of the EEG data obtained by calibration.

[0057]

[0058] μ in equation (1) t and k t This is the set of baseline phase information θ' obtained during calibration. t =[θ 1t ,θ 2t ,…,θ nt This shows the average phase and concentration when the ] is fitted to a von Mises distribution. Note that the feedback score calculated by equation (1) is closer to 0, the closer the value is to the distribution of the electroencephalogram phase information obtained by neurofeedback and the electroencephalogram phase information obtained by calibration.

[0059] The biological information analysis unit 12 then outputs the calculated feedback score to the output processing unit 14.

[0060] The output processing unit 14 generates a feedback screen (see Figure 1) in which the height of the bar is increased the closer the output feedback score value is to 0, and outputs it to the output unit 40. The user trains to raise the bar while looking at the feedback screen displayed on the output unit 40. As a result, the user can spontaneously synchronize the phase of their brainwaves when a stimulus is presented, thereby improving their olfactory ability.

[0061] Furthermore, the baseline set of phase information stored in the bio-information memory unit 13 is updated even during neurofeedback. For example, the bio-information memory unit 13 updates its set of phase information with data from the most recent 30 trials for each trial during neurofeedback. The bio-information memory unit 13 then recalculates statistics using the updated set of phase information. As a result, the information presentation device 10 can provide neurofeedback to the user based on statistics that reflect the user's latest electroencephalogram (EEG) phase information.

[0062] The information presentation device 10 described above uses different baselines depending on the elapsed time since the user started performing the task when providing neurofeedback to the user. This allows the information presentation device 10 to use feature quantities that change moment by moment, such as phase information, for neurofeedback.

[0063] [System Configuration, etc.] Furthermore, the components of each part shown in the diagram are functional concepts and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown in the diagram, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. In addition, all or any part of the processing functions performed by each device can be realized by a CPU and the program executed on that CPU, or by hardware using wired logic.

[0064] Furthermore, among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0065] [Program] The information presentation device 10 described above can be implemented by installing a program (information presentation program) as packaged software or online software on a desired computer. For example, by having the above program run on an information processing device, the information processing device can be made to function as the information presentation device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), as well as terminals such as PDA (Personal Digital Assistant).

[0066] Figure 5 shows an example of a computer that executes an information presentation program. Computer 1000 has, for example, memory 1010 and CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0067] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0068] The hard disk drive 1090 stores, for example, the OS 1091, application program 1092, program module 1093, and program data 1094. That is, the program that defines each process executed by the information presentation device 10 is implemented as a program module 1093 in which executable code for a computer is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing a process similar to the functional configuration of the information presentation device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0069] Furthermore, the data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0070] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

[0071] 10 Information display device 11 Biological information storage unit 12 Biological information analysis unit 13 Biological information storage unit 14 Output processing unit 20 Biological information measurement unit 30 Stimulus presentation unit 40 Output unit

Claims

1. An information presentation device comprising: an information acquisition unit that acquires time information of when a stimulus was presented to a user and the user's biological information; a biological information analysis unit that, based on the time the stimulus was presented to the user, extracts the biological information at predetermined time intervals, calculates the phase information of each of the extracted biological information, thereby obtaining time-dependent phase information, and calculates a score indicating how closely the time-dependent phase information is similar to the distribution of the user's time-dependent phase information stored in a memory unit in the past; and an output processing unit that outputs information indicating the score.

2. The information presentation device according to claim 1, characterized in that the distribution of the user's time-based phase information stored in the memory unit in the past is represented by the average phase and concentration of the set of time-based phase information obtained by fitting the set of the user's time-based phase information in the past to a von Mises distribution.

3. After the information indicating the score is output, the biological information analysis unit updates the distribution of the user's past phase information for each time period, which is stored in the memory unit, using the phase information of the biological information for each time period, as described in claim 1.

Images