Processing method and program

Abstract

To provide new solution means for estimating the state of brain activities more quantitatively.SOLUTION: A processing method of the present invention includes: a step for acquiring a plurality of brain activity patterns indicating the state of brain activities of a subject over a predetermined time; a step for generating a spatial correlation matrix by calculating spatial correlation on each combination of the plurality of brain activity patterns; a step for calculating coordinates of a predetermined number of dimensions for each of the plurality of brain activity patterns by converting the spatial correlation matrix to a distance matrix by a multi-dimensional scaling method and subjecting the distance matrix to singular value decomposition; and a step for determining a brain state corresponding to each brain activity pattern on the basis of a degree of each spatial correlation between each of the plurality of brain activity patterns and a plurality of templates indicating each of the plurality of predetermined brain states.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present invention relates to a processing method and a program for more quantitatively estimating the state of brain activity.

Background Art

[0002] As a method for non-invasively measuring brain activity, typically, an electroencephalogram (hereinafter also abbreviated as "EEG") for observing the electrical activity of the brain, a magnetoencephalogram (hereinafter also abbreviated as "MEG"), and a functional magnetic resonance imaging method (hereinafter also abbreviated as "fMRI") for observing the state of blood flow in the brain are known.

[0003] A method of neurofeedback (hereinafter also abbreviated as "NF") training has been developed and proposed, which observes the neural activity of a subject based on information indicating the brain activity measured from the subject and guides the subject in a more favorable direction according to the observed neural activity (see, for example, Non-Patent Document 1 and Non-Patent Document 2, etc.).

[0004] As a visualization method for presenting information indicating brain activity, for example, Non-Patent Document 3 discloses an example of expressing the activity (EEG) of a specific brain region in the form of a level meter. Further, Non-Patent Document 4 discloses an example of expressing the activity (fMRI) of a specific brain region by the size of a flame. However, the techniques disclosed in Non-Patent Document 3 and Non-Patent Document 4 are not for NF training, but for healthy people to understand the state of their own brain activity.

[0005] Also, Non-Patent Document 5 discloses a technique for visualizing a specific waveform in order to detect abnormal electroencephalograms (for example, stroke, etc.). The technique disclosed in Non-Patent Document 5 is not for NF training, but for monitoring the brain state of patients.

[0006] Furthermore, Non-Patent Document 6 discloses a technique for mapping and presenting the electroencephalogram spectrum in a multimodal manner. The technique disclosed in Non-Patent Document 6 is intended for use as a relaxation tool. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Anastasiia Belinskaia et al, "Short-delay neuralfeedback facilitates training of the parietal alpha rhythm," Journal of Neural Engineering, Volume 17, Number 6, 16 December 2020 [Non-Patent Document 2] Laura Diaz Hernandez et al, "Towards Using Microstate-Neurofeedback for the Treatment of Psychotic Symptoms in Schizophrenia," A Feasibility Study in Healthy Participants, Brain Topogr 29, 308-321 (2016). https: / / doi.org / 10.1007 / s10548-015-0460-4, 19 November 2015 [Non-Patent Document 3] Madita Stirner et al, "An Investigation of Awareness and Metacognition in Neurofeedback with the Amygdala Electrical Fingerprint," Consciousness and Cognition, Volume 98, February 2022, 103264. https: / / doi.org / 10.1016 / j.concog.2021.103264 [Non-Patent Document 4] R. Christopher deCharms et al, "Control over brain activation and pain learned by using real-time functional MRI," PNAS, December 20, 2005, vol.102, no.51. https: / / www.pnas.org / doi / pdf / 10.1073 / pnas.0505210102 [Non-Patent Document 5] Gerold Baier et al, "Event-based sonification of EEG rhythms in real time," Clinical Neurophysiology 118 (2007) 1377-1386, [Non-Patent Document 6] Thilo Hinterberger et al, "The Sensorium: Psychophysiological Evaluation of Responses to a Multimodal Neurofeedback Environment," Appl Psychophysiol Biofeedback 41, 315-329 (2016). https: / / doi.org / 10.1007 / s10484-016-9332-2 [Overview of the project] [Problems that the invention aims to solve]

[0008] For example, when considering applications such as NF training, there is a need for technology that can evaluate the state of brain activity corresponding to cognitive function (whole brain state) in real time. The background technologies disclosed in the prior art documents mentioned above cannot meet this requirement.

[0009] This invention provides a novel solution for more quantitatively estimating the state of brain activity. [Means for solving the problem]

[0010] A processing method according to one embodiment of the present invention includes the steps of: acquiring multiple brain activity patterns indicating the state of a subject's brain activity over a predetermined period of time; generating a spatial correlation matrix by calculating the spatial correlation for each combination of the multiple brain activity patterns; converting the spatial correlation matrix into a distance matrix using multidimensional scaling and calculating coordinates of a predetermined number of dimensions for each of the multiple brain activity patterns by singular value decomposition of the distance matrix; and determining the brain state corresponding to each brain activity pattern based on the magnitude of the respective spatial correlations between each of the multiple brain activity patterns and a plurality of templates each representing a predetermined plurality of brain states.

[0011] The processing method may further include the step of representing the coordinates corresponding to each of several brain activity patterns and the corresponding brain states on the same coordinate system.

[0012] Multiple templates may include multiple pairs of templates with reversed polarity. For each pair of templates, the average value of the coordinates corresponding to one template and the coordinates corresponding to the other template may be determined as the center of the coordinates corresponding to each of the multiple brain activity patterns.

[0013] The processing method may further include the step of determining the coordinates corresponding to a new template based on the coordinates corresponding to a first template among multiple templates and the coordinates corresponding to a second template among multiple templates.

[0014] A processing method according to another embodiment of the present invention includes the steps of: obtaining a set of standard state patterns including a plurality of brain activity patterns indicating the brain activity states of one or more subjects and a plurality of templates each indicating a predetermined set of brain states; generating a first spatial correlation matrix by calculating the spatial correlation for each combination included in the set of standard state patterns; converting the first spatial correlation matrix into a distance matrix using multidimensional scaling and calculating a first set of coordinates corresponding to the elements included in the first spatial correlation matrix by singular value decomposition of the distance matrix; and indicating the brain activity states of the subjects. The method includes the steps of: obtaining a first brain activity pattern; generating a second spatial correlation matrix by calculating the spatial correlation for each combination included in the standard state pattern group and the first brain activity pattern; converting the second spatial correlation matrix into a distance matrix using multidimensional scaling and calculating a second set of coordinates corresponding to the elements included in the second spatial correlation matrix by singular value decomposition of the distance matrix; and estimating the coordinates corresponding to the first brain activity pattern in the first coordinate set based on the relative positional relationships of the coordinates corresponding to the first brain activity pattern in the second coordinate set.

[0015] The processing method may further include the step of displaying the coordinates corresponding to the first brain activity pattern in a two-dimensional or three-dimensional coordinate system.

[0016] The processing method may further include the steps of acquiring a first brain activity pattern, generating a second spatial correlation matrix, calculating a second set of coordinates, and estimating coordinates, repeating these steps. The display step may include displaying the sequentially estimated coordinates as a trajectory.

[0017] The display step may include the step of displaying the estimated coordinates using a spherical projector.

[0018] The processing method may further include: obtaining a plurality of brain activity patterns indicating the states of brain activities of a plurality of subjects; generating a third spatial correlation matrix by calculating the spatial correlation for each combination included in the obtained plurality of brain activity patterns; converting the third spatial correlation matrix into a distance matrix by multidimensional scaling and calculating a third group of coordinates corresponding to the elements included in the third spatial correlation matrix by performing singular value decomposition on the distance matrix; and extracting, as a plurality of brain activity patterns of a standard state pattern group, the brain activity patterns corresponding to a predetermined number of coordinates extracted from the third group of coordinates.

[0019] The predetermined number of coordinates may be extracted from the third group of coordinates according to a predetermined rule.

[0020] According to another embodiment of the present invention, a program for causing a computer to execute the above-described processing method is provided.

Advantages of the Invention

[0021] According to an embodiment of the present invention, means for more quantitatively estimating the state of brain activity can be realized.

Brief Description of the Drawings

[0022] [Figure 1] It is a schematic diagram showing a configuration example of a neurofeedback system according to this embodiment. [Figure 2] It is a schematic diagram showing the device configuration of a processing device included in the neurofeedback system according to this embodiment. [Figure 3] It is a diagram for explaining a processing example for estimating a brain state according to this embodiment. [Figure 4] It is a diagram showing an example of a vector indicating a brain state according to this embodiment. [Figure 5] It is a diagram for explaining a brain activity pattern and a template for estimating a brain state according to this embodiment. [Figure 6]This is a schematic diagram showing an example of a user interface screen provided by a processing apparatus according to this embodiment. [Figure 7] This figure shows an example of the evaluation results for the intrinsic dimensions included in the EEG. [Figure 8] This figure shows an example of a brain activity pattern used to determine the state space in the brain state estimation process according to this embodiment. [Figure 9] This figure shows an example of a spatial correlation matrix used to determine the state space in the brain state estimation process according to this embodiment. [Figure 10] This figure shows an example of a state space calculated from a predetermined number of brain activity patterns in the brain state estimation process according to this embodiment. [Figure 11] This figure shows an example of a neural manifold that appears in the state space determined by the brain state estimation process according to this embodiment. [Figure 12] This figure shows another example of a neural manifold appearing in the state space determined by the brain state estimation process according to this embodiment. [Figure 13] This figure shows an example of hyperalignment processing according to this embodiment. [Figure 14] This is a schematic diagram illustrating the hyperalignment processing procedure according to this embodiment. [Figure 15] This is a schematic diagram showing an example of a brain state according to this embodiment mapped onto a standard space. [Figure 16] This is a schematic diagram showing another example of mapping brain states according to this embodiment onto a standard space. [Figure 17] This is a schematic diagram showing an example of a discrete mapping of brain states according to this embodiment onto a standard space. [Figure 18] This flowchart shows an example of a processing procedure for generating a standard space according to this embodiment. [Figure 19] This flowchart shows an example of a hyperalignment processing procedure according to this embodiment. [Modes for carrying out the invention]

[0023] Embodiments of the present invention will be described in detail with reference to the drawings. Note that identical or corresponding parts in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[0024] [A. Brain state] In this embodiment, a "brain state" is estimated as an example of a state of brain activity. The estimated "brain state" refers to the state of brain activity determined by evaluating the entire brain of the subject. The "brain state" is estimated based on information obtained by measuring brain activity such as EEG, MEG, and fMRI.

[0025] Multiple brain states can be defined in advance. Below, we will use "microstates," as reported in previous studies, as possible brain states. A "microstate" refers to the smallest unit of brain activity observed regardless of the subject. Microstates represent the overall state dynamics of the brain.

[0026] Previous studies have repeatedly reported that there are four microstates, regardless of the number of channels, preprocessing procedures, or subjects. Therefore, in this embodiment as well, the brain state will be one of the four microstates. The four microstates included in the brain state are sometimes referred to as msA, msB, msC, and msD. However, the number of states included in the brain state is not limited to four, and any number of states can be set.

[0027] Each microstate can be considered a range of stable states (attractors). By considering that "brain states" transition between microstates, it becomes possible to observe the transitions of brain states labeled by microstates and to calculate the duration of brain states labeled by each microstate. Based on these features, applications such as biomarkers for mental disorders are expected.

[0028] For the sake of explanation, the following will mainly describe an example of a process that estimates brain state using EEG. However, the technical scope of the present invention is not limited to processes that estimate brain state using EEG, but includes processes that estimate brain state using any information obtained by measuring brain activity.

[0029] [B. System Configuration Example] Next, as an example of an application of the brain state estimation process according to this embodiment, we will describe an example configuration of a neurofeedback system for performing NF training.

[0030] Figure 1 is a schematic diagram showing an example configuration of a neurofeedback system 1 according to this embodiment. Referring to Figure 1, the neurofeedback system 1 estimates the brain state of subject 2 in real time using EEG measured through a cap 10 worn by subject 2. The neurofeedback system 1 guides subject 2 in an ideal direction by providing visual and / or auditory feedback to subject 2 according to the estimated brain state.

[0031] The neurofeedback system 1 includes, as its main components, a measurement circuit 40, a processing unit 100, a display 20, and a speaker 30.

[0032] The measurement circuit 40 corresponds to the measurement unit that measures the brainwaves of subject 2. More specifically, the measurement circuit 40 measures multiple channel components (electrical signals corresponding to EEG) as the brainwaves of subject 2 via a cap 10 containing multiple electrodes 12 attached to subject 2. The measurement circuit 40 includes, for example, a multiplexer 42, a noise filter 44, and an A / D (Analog to Digital) converter 46.

[0033] The multiplexer 42 sequentially selects multiple channels (electrodes 12) output from the cap 10 and electrically connects them to the noise filter 44. The noise filter 44 is a filter that allows only specific frequency components to pass through, removing noise components contained in the signal (electrical signal) appearing in the selected channel. For example, the passband can be set to 0.016 to 250 Hz.

[0034] The A / D converter 46 samples the electrical signal (analog signal) output from the noise filter 44 at predetermined intervals and outputs it as a digital signal.

[0035] The processing unit 100 estimates the brain state of subject 2 and outputs feedback corresponding to the estimated brain state.

[0036] The display 20 provides visual feedback to subject 2. The speaker 30 provides auditory feedback to subject 2.

[0037] Figure 2 is a schematic diagram showing the device configuration of a processing unit 100 included in a neurofeedback system 1 according to this embodiment. The processing unit 100 can typically employ a computer following a general-purpose architecture. Referring to Figure 2, the processing unit 100 includes, as its main components, a processor 102, memory 104, input unit 106, network controller 108, measurement interface 110, display controller 112, audio controller 114, and storage 120.

[0038] The processor 102 consists of arithmetic processing circuits such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), and executes the code contained in various programs stored in the storage 120 in a specified order to realize the various functions described later. The memory 104 consists of DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory), and holds the code of the programs executed by the processor 102 and various work data necessary for program execution.

[0039] The input unit 106 typically consists of a mouse or keyboard, and accepts user input.

[0040] The network controller 108 exchanges data with external devices (for example, a data server on the cloud). The network controller 108 is composed of any communication components such as wired LAN (Local Area Network), wireless LAN, USB (Universal Serial Bus), and Bluetooth (registered trademark).

[0041] The measurement interface 110 receives the EEG (digital signal) output from the measurement circuit 40.

[0042] The display controller 112 displays an image on the display 20 according to the processing result of the processor 102. The audio controller 114 outputs sound from the speaker 30 according to the processing result of the processor 102. A controller that outputs both images and sound simultaneously may also be used.

[0043] Storage 120 typically consists of a hard disk or SSD (Solid State Drive) and holds various programs executed by the processor 102, various data necessary for processing, and setting values. More specifically, storage 120 includes the OS (Operating System) 122, the measurement program 124, and the NF program 126.

[0044] The measurement program 124 includes code to cause the processor 102 to perform processing and visualization processing for estimating the brain state from the subject 2's EEG. The measurement program 124 may also include code to cause the processor 102 to perform processing for providing visual and / or auditory feedback to subject 2.

[0045] [C. Estimation of brain state] Next, we will explain an example of processing for estimating brain state.

[0046] Figure 3 illustrates an example of processing for estimating brain state according to this embodiment. Figure 3(A) shows an example of raw EEG signal 50 (time waveform) measured through a cap 10 worn by subject 2. As an example, Figure 3(A) shows an example of raw signal 50 (32 channels) measured at each of the 32 electrodes 12. That is, the cap 10 includes 32 electrodes 12 corresponding to regions of subject 2's head.

[0047] Figure 3(B) shows an example of vectorizing the set of time waveforms (hereinafter also referred to as "epoch 52") that occur in an interval (100 ms in this example) set in the raw EEG signal 50 in the spatial direction. The channel space 60 shown in Figure 3(B) has each channel of the measured EEG as a dimension. The vector 62 shown in Figure 3(B) corresponds to the epoch 52 shown in Figure 6(A). The vector 62 represents the brain state set with respect to the spatio-temporal dimension. Each of the vectors 62 represents the brain state at the moment corresponding to epoch 52.

[0048] The components of each dimension that make up vector 62 may be the average value at epoch 52, or they may be measured values ​​at any point in time included in epoch 52.

[0049] Figure 4 shows an example of a vector 62 representing a brain state according to this embodiment. Referring to Figure 4, the vector 62 generated from each epoch 52 of the EEG can be defined on the channel space 60.

[0050] By clustering a large number of vectors 62 defined on the channel space 60, one or more clusters 64 can be determined. Each of the clusters 64 determined using a sufficient number of vectors 62 represents a specific brain state.

[0051] In this embodiment, four microstates msA, msB, msC, and msD are adopted as brain states, and four corresponding clusters 64A, 64B, 64C, and 64D can be determined for each. The centroid coordinates of clusters 64A, 64B, 64C, and 64D, corresponding to each of the microstates msA, msB, msC, and msD, represent 32-dimensional vectors. Known clustering methods such as modified k-means clustering and hierarchical clustering can be used.

[0052] Vectors 62 belonging to the same cluster 64 have similar characteristics. The common characteristics of vectors 62 belonging to the same cluster 64 are also called eigenspatial patterns. As will be described later, the eigenspatial pattern 74 corresponds to a mapping of the signal intensity of each channel of the EEG according to the position of the corresponding electrode 12 (i.e., the brain region from which the information is collected).

[0053] Figure 5 illustrates the brain activity patterns and templates for estimating brain states according to this embodiment. Referring to Figure 5, the brain activity pattern 70 is a matrix that shows the state of brain activity for each epoch 52. In the example shown in Figure 5, a 7x7 matrix is ​​used, and the intensity of each matrix element indicates the magnitude of the set value (within a range of ±5).

[0054] The measured values ​​of each EEG channel are mapped to specific positions (row x column) in a matrix representing the brain activity pattern 70, depending on the position of the corresponding electrode 12. For example, the electrodes 12 of the cap 10 are positioned at the following locations: Fp1, Fp2, Fz, F3, F4, F7, F8, F9, F10, FC1, FC2, FC5, FC6, Cz, C3, C4, T7, T8, CP1, CP2, CP5, CP6, Pz, P3, P4, P7, P8, P9, P10, Oz, O1, and O2. The reference electrode and ground electrode may also be positioned at the FCz and Fpz locations, respectively.

[0055] Since the EEG measured has 32 channels, there will be matrix elements that are not mapped to channels. For matrix elements that are not mapped to channels, any value (for example, "0") may be set.

[0056] Brain activity patterns 70 may be generated every epoch 52 (for example, every 100 ms). A spatial correlation (similarity) is calculated between each of the generated brain activity patterns 70 and the pre-prepared templates 72A, 72B, 72C, and 72D (hereinafter sometimes collectively referred to as "template 72"). The microstate corresponding to the template 72 with the largest calculated spatial correlation is determined as the brain state represented by brain activity pattern 70.

[0057] Template 72 is a type of spatial filter that shows a peak dataset of GFP (global field power) representing the strength of the brain's electric field at each instant. Templates 72A, 72B, 72C, and 72D may be determined, for example, based on the centroid coordinates (32-dimensional vectors) of clusters 64A, 64B, 64C, and 64D defined on the channel space 60. That is, template 72 can be determined by mapping the centroid coordinates of cluster 64 to a matrix in a similar manner to that of the brain activity pattern 70.

[0058] Figure 5 shows the eigenspatial patterns 74A, 74B, 74C, and 74D, which correspond to templates 72A, 72B, 72C, and 72D. Eigenspatial patterns 74A, 74B, 74C, and 74D were generated by spatial interpolation of templates 72A, 72B, 72C, and 72D, and represent the continuous distribution of brain activity in each microstate.

[0059] [D. Presentation of brain state] Next, we will describe an example that presents an estimated brain state.

[0060] Figure 6 is a schematic diagram showing an example of a user interface screen provided by the processing device 100 according to this embodiment.

[0061] Referring to Figure 6(A), the processing unit 100 displays on the display 20 a brain state object 200 containing four brain states (microstates) and a current state object 210 indicating the estimated current brain state. As shown in Figure 6(A), the processing unit 100 may also present the subject 2 with a display (brain state object 200) containing multiple regions corresponding to multiple brain states. By providing a user interface screen like the one shown in Figure 6(A), the subject 2 can recognize at a glance which state their current brain state is.

[0062] Referring to Figure 6(B), the processing unit 100 may further display a target object 220 that indicates the target brain state. As shown in Figure 6(B), the processing unit 100 may display the estimated brain state of subject 2 (current brain state) in a first display mode (e.g., a circle as shown in Figure 6(B)) in association with multiple regions indicated by the brain state object 200, and display the target brain state in a second display mode (e.g., a rectangle as shown in Figure 6(B)). By providing a user interface screen as shown in Figure 6(B), subject 2 can recognize at a glance what state their current brain state is, as well as what brain state they should be in.

[0063] Referring to Figure 6(C), if the processing unit 100 cannot determine which brain state is present, the current state object 210 will not be displayed.

[0064] Furthermore, the fact that it is not possible to determine which brain state the subject 2 is in may be notified visually using text or objects, or auditorily using voice messages or sound effects.

[0065] In this case as well, Subject 2 attempts to focus their attention on achieving the target brain state.

[0066] [E. State space and neural manifolds] While the microstate method described above is very effective when comparing subjects, it is preferable to have a mechanism that can track the transitions in the overall state dynamics of the brain when observing the brain state of a specific subject.

[0067] Below, we will describe a state space that can represent the continuous transitions in the state dynamics of the entire brain.

[0068] In the state space according to this embodiment, instead of jumping between brain states, a smooth trajectory of state dynamics can be represented. Such a state space that can represent a smooth trajectory is sometimes called a neural manifold.

[0069] The inventors of this invention investigated the eigendary dimensions contained in the EEG in channel space 60. More specifically, they used principal component analysis (PCA) to investigate the eigendary dimensions contained in the 32-channel EEG.

[0070] Figure 7 shows an example of the evaluation results for the intrinsic dimensions included in the EEG. Figure 7(A) shows the evaluation results when using the raw EEG signal, and Figure 7(B) shows the evaluation results when normalized on a channel-by-channel basis.

[0071] Figures 7(A) and 7(B) suggest that a dimension of "3" is sufficient to adequately represent the dynamics of the EEG. In other words, by using a three-dimensional space, the global state dynamics can be represented quite intuitively. It should be noted that a two-dimensional space, obtained by projecting the three-dimensional space onto a specific plane, can also be used as a representation.

[0072] In the brain state estimation process according to this embodiment, a dimensionality reduction method is applied to compress the EEG channel space 60 (32 dimensions if there are 32 channels) into a sufficiently low-dimensional space. Any method such as principal component analysis (PCA) or independent component analysis (ICA) can be used as the dimensionality reduction method, but in this embodiment, multidimensional scaling (MDS) is used. According to multidimensional scaling, the space after dimensionality reduction (typically a simple sphere) becomes a smooth surface, and in this space, the state is geometrically differentiable.

[0073] Figure 8 shows an example of a brain activity pattern 76 used to determine the state space in the brain state estimation process according to this embodiment. Figure 9 shows an example of a spatial correlation matrix 80 used to determine the state space in the brain state estimation process according to this embodiment.

[0074] Referring to Figure 8, multiple brain activity patterns 76, which represent the state of the subject's brain activity, are acquired over a predetermined period of time. Each brain activity pattern 76 corresponds to a sampling point for the raw EEG signal 50, and represents the brain state at each sampling point. Similar to brain activity pattern 70, brain activity patterns 76 are maps of the measured values ​​of each channel of the EEG to their corresponding positions. The predetermined number can be set, for example, to 500 points (1 second at a sampling rate of 500 Hz).

[0075] For example, the processing unit 100 samples the measured values ​​of each channel of the EEG and maps the measured values ​​of each channel obtained through sampling to generate a brain activity pattern 76.

[0076] Next, a spatial correlation matrix 80 is generated by calculating the spatial correlation for each combination of the multiple brain activity patterns 76. That is, spatial correlations are calculated for all combinations of a predetermined number of brain activity patterns 76. By placing the calculated spatial correlations in their corresponding positions, a spatial correlation matrix 80 as shown in Figure 8 can be calculated. The spatial correlation matrix 80 shows the spatial similarity between the brain activity patterns 76 (each value takes the range of -1 to 1).

[0077] By focusing on features included in the spatial correlation matrix 80 (e.g., nonlinear waveforms and ripples), the time-series dynamics as trajectories can determine the space as a neural manifold. That is, the spatial correlation matrix 80 is transformed into a distance matrix using multidimensional scaling, and the distance matrix is ​​subjected to singular value decomposition (SVD) to calculate coordinates of a predetermined number of dimensions for each of the multiple brain activity patterns 76. As an example, the calculated coordinates may be 3-dimensional (XYZ coordinates).

[0078] The sets of coordinates corresponding to each of the multiple brain activity patterns 76 become the state space 90 and the neural manifold 92 that appear in the state space 90.

[0079] Figure 10 shows an example of a state space calculated from a predetermined number of brain activity patterns 76 in the brain state estimation process according to this embodiment. Referring to Figure 10, the state space 90 has coordinates (500 points) corresponding to each of the brain activity patterns 76 plotted, and can be considered to be continuous in time.

[0080] Thus, the state space 90 according to this embodiment can also be defined as a "spatial similarity space," and in the state space 90, trajectories that are continuous in time are also continuous in space and time.

[0081] By plotting as many brain activity patterns 76 as possible onto the state space 90, a spherical neural manifold emerges. That is, the brain activity patterns 76 can be considered to exist at any coordinate on the sphere determined by the state space 90, and they move along the sphere in time.

[0082] Figure 11 shows an example of a neural manifold 92 appearing in the state space 90 determined by the brain state estimation process according to this embodiment. Referring to Figure 11, the brain activity patterns 76 within the neural manifold 92 are distributed such that their norm is approximately 1. Therefore, a sphere (or spherical body) with radius 1 can be determined as the neural manifold 92. That is, the centroid (center) of the entire set of coordinates constituting the neural manifold 92 corresponds to the origin of the state space 90.

[0083] Furthermore, by determining the brain state (microstate) corresponding to the template 72 that shows the greatest spatial correlation with the brain activity pattern 76 as the brain state in that brain activity pattern 76 (winner-take-all method), the regions corresponding to microstates msA, msB, msC, and msD can be determined for the neural manifold 92, respectively.

[0084] In other words, the brain state (microstate) corresponding to each brain activity pattern is determined based on the magnitude of the spatial correlation between each of the multiple brain activity patterns 76 and the multiple templates 72 that each represent a predetermined set of brain states. At this time, for one brain activity pattern 76, the spatial correlation (i.e., four values) for the microstates msA, msB, msC, and msD is calculated, but the brain state (microstate) with the largest spatial correlation may be determined as the brain state for that brain activity pattern 76.

[0085] Furthermore, as shown in Figure 11, the coordinates (plots) corresponding to multiple brain activity patterns and the corresponding brain states (microstates) may be represented on the same coordinate system.

[0086] The template 72 can be determined based on a vector corresponding to the centroid coordinates of the cluster 64. For example, since polarity is ignored in the modified k-means method, it is preferable to prepare two sets of templates 72 with inverted polarities. For example, one set of templates 72 with the frontal region as positive and another set of templates 72 with the frontal region as negative can be prepared. When determining a region in the neural manifold 92, two sets of templates 72 are used. That is, the template 72 includes multiple pairs of templates with inverted polarities.

[0087] Here, we will explain the center point of neural manifold 92. Considering a set of microstates msA and msA' with opposite polarities, the coordinates corresponding to microstate msA and microstate msA' are positioned point-symmetrically with respect to the center point of the neural manifold. In other words, the average value of the coordinates corresponding to microstate msA and microstate msA' represents the center point of neural manifold 92. The same applies to other microstates.

[0088] In this way, for each of a pair of templates with opposite polarities, the average value of the coordinates corresponding to one template and the coordinates corresponding to the other template is determined as the center of the neural manifold 92.

[0089] Figure 12 shows another example of a neural manifold 92 appearing in the state space 90 determined by the brain state estimation process according to this embodiment. Figures 12(A) to (C) show the state space 90 drawn from different viewpoints.

[0090] The coordinates corresponding to the templates representing each microstate exist on the spherical neural manifold 92. Figures 12(A) to (C) show the state space 90 containing the coordinates corresponding to the four types of microstates.

[0091] Here, we can heuristically determine or estimate new microstates by utilizing the property that the centroid (center) of the entire set of coordinates constituting the neural manifold 92 corresponds to the origin of the state space 90. More specifically, by focusing on any combination of templates, the center (average) of the coordinates corresponding to each template can become a candidate for the coordinates corresponding to the new microstate.

[0092] As an example, Figure 12(C) shows a case where a landmark 78 that could be a candidate for a new microstate is placed at the center of the coordinates corresponding to microstate msA and the coordinates corresponding to microstate msB. In this case, the paired landmark 78 (i.e., the coordinates corresponding to the template with reversed polarity) is placed at a position symmetrical with respect to the origin of the state space 90.

[0093] Thus, the coordinates corresponding to a new microstate may be determined from the coordinates corresponding to any two pre-prepared microstates. In other words, the coordinates corresponding to a new template may be determined based on the coordinates corresponding to one template and the coordinates corresponding to another template among multiple templates.

[0094] By heuristically determining new templates, the number of pre-defined templates can be increased as needed. In other words, the required number of microstates can be prepared retrospectively.

[0095] As described above, by using the state space 90 according to this embodiment, the brain activity pattern 76 calculated from the EEG at each moment moves continuously on the spherical neural manifold 92, which becomes the trajectory of the state dynamics. When the sampling rate is sufficiently high (the sampling period is short), a continuous trajectory is shown, but when the sampling rate is low, the trajectory may jump, so it is preferable to set the sampling rate appropriately.

[0096] [F. Standard Space and Hyperalignment] The result of mapping the EEG measured from a subject to state space is unique to that subject. Therefore, when statistically evaluating state dynamics, it is necessary to determine the coordinates when mapped to a space standardized across subjects (hereinafter also referred to as the "standard space"). The process of arranging the result of mapping the EEG measured from a subject to state space into the standard space will be referred to as "hyperalignment" below.

[0097] Figure 13 shows an example of hyperalignment processing according to this embodiment. Referring to Figure 13, if the EEG is mapped to the state space 90 for each subject, the same brain state may be plotted at different coordinates.

[0098] Therefore, hyperalignment is used to realize a mapping to a standard space 94 that standardizes the unique characteristics of each subject.

[0099] Figure 14 is a schematic diagram showing the hyperalignment processing procedure according to this embodiment.

[0100] Referring to Figure 14(A), first, a basic state space (neural manifold 92) used for hyperalignment is generated. In generating the neural manifold 92, multiple brain activity patterns 76 representing the brain activity states of multiple subjects are obtained. The brain activity patterns 76 are obtained using EEG sampling data measured from the subjects.

[0101] Generating neural manifold 92 requires several minutes of EEG (resting state) from a few subjects. For example, measuring five subjects for one minute at a sampling rate of 100 Hz can collect a total of 30,000 brain activity patterns 76.

[0102] Then, a spatial correlation matrix is ​​generated by calculating the spatial correlation for each combination included in the multiple brain activity patterns 76 obtained. Furthermore, the spatial correlation matrix is ​​transformed into a distance matrix using multidimensional scaling, and the coordinate set corresponding to the elements included in the spatial correlation matrix is ​​calculated by singular value decomposition of the distance matrix. The calculated coordinate set constitutes the neural manifold 92. In this way, a neural manifold 92 is generated from a large number of collected brain activity patterns 76 using multidimensional scaling and singular value decomposition. The neural manifold 92 is a set of coordinates plotted on a sphere with radius 1.

[0103] Next, a predetermined number of grids 96 are extracted (sampled) from the generated neural manifold 92 according to a predetermined rule. That is, brain activity patterns 76 corresponding to a predetermined number of coordinates (grids 96) extracted from the calculated coordinate set (neural manifold 92) are extracted as part of the standard state pattern set.

[0104] The sampled grid 96 serves as a reference (landmark) for determining coordinates in hyperalignment. While a larger number of grid 96 results in higher hyperalignment accuracy, it also requires more processing time. Therefore, the number of sampled grid 96 is determined based on a balance between accuracy and processing time. For example, 50 grid 96 points may be sampled. It is preferable that the grid 96 be sampled spatially evenly across the neural manifold 92 (sphere).

[0105] Next, for the brain activity patterns 76 (e.g., 50 points) corresponding to the sampled grid 96 and the templates 72 (e.g., a set of two templates 72 with reversed polarity: 8 points), each point is plotted in the state space using multidimensional scaling and singular value decomposition. That is, a spatial correlation matrix is ​​generated by calculating the spatial correlation for each combination included in the standard state pattern set. Furthermore, the spatial correlation matrix is ​​transformed into a distance matrix using multidimensional scaling, and the coordinate set corresponding to the elements included in the spatial correlation matrix is ​​calculated by performing singular value decomposition on the distance matrix. The calculated coordinate set constitutes the standard space 94.

[0106] In standard space 94, the coordinates corresponding to grid 96 are not necessarily placed at the same coordinates as the original. However, the center point of the neural manifold 92, determined from each coordinate corresponding to template 72, is considered to remain virtually unchanged. Therefore, the center point of the neural manifold can be estimated based on each coordinate corresponding to template 72.

[0107] The brain activity patterns 76 and templates 72 corresponding to the grid 96 described above, plotted in state space, are used as the standard space 94. The set of brain activity patterns 76 and templates 72 used to generate the standard space 94 (for example, a total of 58 points) corresponds to a standard state pattern group that represents a standard state. That is, the standard state pattern group includes multiple brain activity patterns 76 that represent the brain activity states of one or more subjects, and multiple templates 72 that each represent a predetermined set of brain states (microstates).

[0108] Through the above process, a standard space 94 is prepared, consisting of a set of standard state patterns and a set of coordinates calculated from the set of standard state patterns. Using the standard space 94 and the set of standard state patterns, the coordinates in the state space (standard space 94) corresponding to the EEG measured by the subject are sequentially determined.

[0109] Referring to Figure 14(B), a new brain activity pattern 98, sampled from EEG data measured from the subject, is added to the standard state pattern group. That is, a brain activity pattern 98 representing the state of the subject's brain activity is obtained and added to the standard state pattern group.

[0110] Next, for the set to which the brain activity patterns 98 have been added, each point is plotted in the state space using multidimensional scaling and singular value decomposition to generate a modified standard space 94A. More specifically, a spatial correlation matrix is ​​generated by calculating the spatial correlation between each combination included in the standard state pattern group and the brain activity patterns 98 of the target subjects. Furthermore, the spatial correlation matrix is ​​transformed into a distance matrix using multidimensional scaling, and the coordinate set corresponding to the elements included in the spatial correlation matrix is ​​calculated by performing singular value decomposition on the distance matrix. The calculated coordinate set constitutes the modified standard space 94A.

[0111] Here, if the number of standard state patterns is sufficiently large compared to the number of additional brain activity patterns 98 (e.g., one), the modified standard space 94A will only shift slightly from the standard space 94. In other words, the relative positional relationships (e.g., distances between coordinates) of the coordinate groups corresponding to the standard state pattern groups are substantially maintained.

[0112] Therefore, a precise transformation can be performed based on the positional relationship between the standard space 94 and the modified standard space 94A to which the brain activity pattern 98 has been added. That is, the coordinates corresponding to the brain activity pattern 98 in the coordinate group constituting the standard space 94 are estimated based on the relative positional relationship of the coordinates corresponding to the brain activity pattern 98 in the coordinate group constituting the modified standard space 94A. In this way, the coordinates in the modified standard space 94A to which the brain activity pattern 98 has been added can be converted back to the coordinates in the original standard space 94.

[0113] As an example, in the modified standard space 94A, multiple coordinates in the vicinity of the coordinate corresponding to the brain activity pattern 98 are extracted. Alternatively, all coordinates within the modified standard space 94A may be extracted. The relative positional relationship between the extracted neighboring coordinates and the coordinate corresponding to the brain activity pattern 98 is determined. Then, in the standard space 94, the coordinate corresponding to each of the extracted neighboring coordinates is identified, and the relative positional relationship is applied to the identified coordinates to determine the coordinate corresponding to the brain activity pattern 98 in the standard space 94.

[0114] As another example, a mapping transformation formula is determined for a set of standard state patterns between the standard space 94 and the modified standard space 94A. Then, using the determined mapping transformation formula, the coordinates corresponding to the brain activity patterns 98 in the modified standard space 94A are mapped to the standard space 94, thereby determining the coordinates corresponding to the brain activity patterns 98 in the standard space 94.

[0115] In standard space 94, the coordinates corresponding to brain activity pattern 98 represent the coordinates that have been standardized for brain activity pattern 98.

[0116] Each time a brain activity pattern 98 is acquired by sampling EEG data measured from a subject, the process described above is repeatedly executed. That is, by repeatedly performing a series of processes such as acquiring the brain activity pattern 98, calculating the modified standard space 94A, and estimating the coordinates corresponding to the brain activity pattern 98, the trajectory of the subject's state dynamics can be represented in the standard space 94, as shown in Figure 14(C). In other words, the sequentially estimated coordinates are displayed as a trajectory.

[0117] According to the hyperalignment method of this embodiment, the trajectory of state dynamics can be represented on a common standard space 94 among subjects, allowing for more accurate comparisons and statistical processing between subjects.

[0118] Furthermore, according to the hyperalignment method of this embodiment, the coordinates in the standard space 94 can be determined each time a new brain activity pattern 98 (EEG sampling data) is acquired, making it possible to represent the subject's brain state in real time. By presenting such a brain state to the subject in real time, NF training can be performed more efficiently.

[0119] Furthermore, according to this embodiment of hyperalignment, it is possible to confirm the trajectory of state dynamics that represent the brain state of the subject, making it easy to verify the effects of NF training on the subject.

[0120] [Presentation to G. Users] Next, we will illustrate several methods for presenting the coordinates and trajectory calculated by this embodiment to the user.

[0121] As described above, hyperalignment according to this embodiment allows the EEG measured from the subject to be represented as coordinates and trajectories (temporal changes in coordinates) in a standard space 94. The standard space 94 is a spherical neural manifold, and the state dynamics move sequentially on a sphere with radius 1. Therefore, the coordinates in the standard space 94 corresponding to the subject's brain activity pattern 98 may be displayed in a two-dimensional or three-dimensional coordinate system.

[0122] Figure 15 is a schematic diagram showing an example of mapping brain states according to this embodiment onto a standard space. Referring to Figure 15, the movement of state dynamics may be represented by sequentially displaying the coordinates corresponding to each sampling point on a sphere representing the standard space.

[0123] Furthermore, the viewing direction of the sphere representing standard space may be arbitrarily changed. In the example shown in Figure 15, it is represented in three dimensions, but by placing the viewpoint on a specific axis, it can also be represented in two dimensions.

[0124] Figure 16 is a schematic diagram showing another example of mapping brain states according to this embodiment onto a standard space.

[0125] Figure 16(A) shows an example of a three-dimensional representation. As shown in Figure 16(A), a sphere representing the standard space may be represented as a mesh, and the subject's brain state and coordinates corresponding to the template may be represented in any way.

[0126] A spherical projector may be used to visualize the three-dimensional representation shown in Figure 16(A). That is, the estimated coordinates may be displayed using a spherical projector. A spherical projector is a spherical light-emitting body that can arbitrarily change the color of any region on its surface. Using such a spherical projector makes the transition of state dynamics easier to understand.

[0127] Figure 16(B) shows an example of a two-dimensional representation. As shown in Figure 16(B), by projecting the standard space onto a specific two-dimensional coordinate system, the movement of brain states in the two-dimensional coordinate system of interest can be represented. The regions of each microstate may also be displayed.

[0128] Figure 17 is a schematic diagram showing an example of a presentation in which brain states according to this embodiment are discretely mapped onto a standard space. Referring to Figure 17, state dynamics moving continuously across the standard space may be represented, or the regions corresponding to brain states may be discretely and appropriately selected, and the transitions between regions may be represented. Adopting a discrete representation as shown in Figure 17 is suitable for NF training and other applications where a target brain state is specified.

[0129] The presentation examples shown in Figures 15 to 17 are not the only ways in which the overall state dynamics and brain states can be represented.

[0130] [H. Processing Procedure] Next, an example of the processing procedure for estimating brain states according to this embodiment will be described.

[0131] Figure 18 is a flowchart showing an example of a processing procedure for generating a standard space according to this embodiment. Figure 19 is a flowchart showing an example of a hyperalignment processing procedure according to this embodiment. Each step shown in Figures 18 and 19 may be implemented, for example, by the processor 102 of the processing unit 100 executing the measurement program 124 and the NF program 126.

[0132] Referring to Figure 18, the processing unit 100 acquires raw EEG signals measured from multiple subjects (step S100). The processing unit 100 then generates multiple brain activity patterns from each raw EEG signal at a predetermined sampling rate (step S102). Furthermore, the processing unit 100 calculates the spatial correlation for each combination included in the generated multiple brain activity patterns (step S104) and generates a spatial correlation matrix from the calculated spatial correlations (step S106). The processing unit 100 calculates a set of coordinates corresponding to the elements included in the generated spatial correlation matrix using multidimensional scaling and singular value decomposition (step S108). The calculated set of coordinates becomes the neural manifold 92.

[0133] Next, the processing unit 100 samples a predetermined number of grids 96 from the generated neural manifold 92 (step S110). Then, the processing unit 100 generates a set of standard state patterns consisting of brain activity patterns and templates 72 corresponding to the sampled grids 96 (step S112). That is, brain activity patterns corresponding to the predetermined number of sampled grids 96 are extracted as part of the set of standard state patterns.

[0134] The processing unit 100 then calculates the spatial correlation for each combination included in the standard state pattern group (step S114) and generates a spatial correlation matrix from the calculated spatial correlations (step S116). The processing unit 100 calculates the coordinate group corresponding to the elements included in the generated spatial correlation matrix using multidimensional scaling and singular value decomposition (step S118). The calculated coordinate group becomes the standard space 94. Finally, the processing unit 100 stores the standard space 94 (coordinate group) and the standard state pattern group (step S120). Then the processing ends.

[0135] Referring to Figure 19, the processing unit 100 acquires a new brain activity pattern 98 by sampling EEG measured from the subject (step S200), and adds the acquired brain activity pattern 98 to the standard state pattern group (step S202). Then, the processing unit 100 calculates the spatial correlation for each combination included in the added brain activity pattern (step S204), and generates a spatial correlation matrix from the calculated spatial correlations (step S206). The processing unit 100 calculates a set of coordinates corresponding to the elements included in the generated spatial correlation matrix using multidimensional scaling and singular value decomposition (step S208). The calculated set of coordinates becomes the modified standard space 94A.

[0136] Next, the processing unit 100 identifies the coordinates corresponding to the new brain activity pattern 98 in the modified standard space 94A and their positional relationship (relative positional relationship) with other coordinates (step S210). Then, based on the identified positional relationship, the processing unit 100 estimates the coordinates corresponding to the new brain activity pattern 98 in the standard space 94 (step S212).

[0137] The processing unit 100 then outputs the coordinates of the standard space 94 corresponding to the new brain activity pattern 98 (step S214). A specific example of how to output coordinate information will be described later.

[0138] The processing unit 100 determines whether or not the observation of the subject is continuing (step S216). If the observation of the subject is continuing (YES in step S216), the process from step S200 onward is repeated.

[0139] If the observation of the subject is not continued (NO in step S216), the process ends.

[0140] [I. Others] In the explanation above, the case using EEG was used as an example, but it can also be applied to MEG and fMRI. When using MEG or fMRI, the same process can be performed by collecting data from multiple subjects and multiple samples and generating a spatial correlation matrix.

[0141] [J. Advantages] According to this embodiment, by mapping the subject's brain activity pattern to a relatively low-dimensional state space and representing it as state dynamics, it becomes possible to visualize not just discrete transitions between four brain states (microstates), but more intricate, continuous transitions and movements of brain states.

[0142] Furthermore, according to this embodiment, the brain activity patterns of subjects can be mapped onto a standard space. The standard space is a space that is standardized so that there are no differences between subjects, and by evaluating the coordinates and their movement (trajectory) projected onto the standard space, evaluations can be performed on multiple subjects using common criteria.

[0143] Furthermore, according to this embodiment, coordinates projected onto standard space can be represented in various forms. This allows for observation and evaluation of the subject's brain state from various perspectives. It also enables the provision of easily understandable information to subjects in NF training and other applications.

[0144] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the description of the embodiments above, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0145] 1 Neurofeedback system, 2 Subject, 10 Cap, 12 Electrodes, 20 Display, 30 Speaker, 40 Measurement circuit, 42 Multiplexer, 44 Noise filter, 46 A / D converter, 50 Raw signal, 52 Epoch, 60 Channel space, 62 Vector, 64, 64A, 64B, 64C, 64D Cluster, 70, 76, 98 Brain activity pattern, 72, 72A, 72B, 72C, 72D Template, 74, 74A, 74B, 74C, 74D Eigenspace pattern, 80 Spatial correlation matrix, 90 State space, 92 Neural manifold, 94 Standard space, 94A Modified standard space, 96 Grid, 100 Processing unit, 102 Processor, 104 Memory, 106 Input unit, 108 Network controller, 110 Measurement interface, 112 Display controller, 114 Voice controller, 120 storage, 124 measurement programs, 126 programs, 200 brain state objects, 210 current state objects, 220 target objects, msA, msB, msC, msD microstates.

Claims

1. A step of acquiring a brain activity pattern indicating the state of a subject's brain activity multiple times at predetermined intervals over a predetermined period of time, The steps include: generating a spatial correlation matrix by calculating spatial correlations for all combinations of the multiple brain activity patterns obtained; The steps include: converting the spatial correlation matrix into a distance matrix using multidimensional scaling, and calculating coordinates of a predetermined number of dimensions for each of the multiple brain activity patterns by singular value decomposition of the distance matrix; A processing method comprising the step of determining a brain state corresponding to each brain activity pattern based on the magnitude of the spatial correlation between each of the aforementioned plurality of brain activity patterns and a plurality of templates that each represent a predetermined plurality of brain states.

2. The processing method according to claim 1, further comprising the step of representing the coordinates corresponding to each of the plurality of brain activity patterns and the corresponding brain states on the same coordinate system.

3. The aforementioned multiple templates include multiple pairs of templates with reversed polarity, The processing method according to claim 1, wherein for each of the pair of templates, the average value of the coordinates corresponding to one template and the coordinates corresponding to the other template is determined as the center of the coordinates corresponding to each of the plurality of brain activity patterns.

4. The processing method according to claim 1, further comprising the step of determining coordinates corresponding to a new template based on coordinates corresponding to a first template among the plurality of templates and coordinates corresponding to a second template among the plurality of templates.

5. The process includes a step of acquiring a set of standard state patterns, which include multiple brain activity patterns representing the brain activity states of one or more subjects, and multiple templates representing predetermined brain states, wherein each brain activity pattern is acquired at predetermined intervals. The steps include generating a first spatial correlation matrix by calculating spatial correlations for all combinations included in the aforementioned standard state pattern group, The steps include: converting the first spatial correlation matrix into a distance matrix using multidimensional scaling, and calculating a first set of coordinates corresponding to the elements included in the first spatial correlation matrix by singular value decomposition of the distance matrix; The method includes a step of acquiring a first brain activity pattern that indicates the state of the subject's brain activity, wherein each first brain activity pattern is acquired at predetermined intervals. The steps include generating a second spatial correlation matrix by calculating spatial correlations for all combinations included in the standard state pattern group and the first brain activity pattern, The steps include: converting the second spatial correlation matrix into a distance matrix using multidimensional scaling, and calculating a second set of coordinates corresponding to the elements included in the second spatial correlation matrix by singular value decomposition of the distance matrix; A processing method comprising the steps of: estimating coordinates corresponding to the first brain activity pattern in the first coordinate group based on the relative positional relationship of coordinates corresponding to the first brain activity pattern in the second coordinate group.

6. The processing method according to claim 5, further comprising the step of displaying the coordinates corresponding to the first brain activity pattern in a two-dimensional or three-dimensional coordinate system.

7. The method further comprises repeating the steps of acquiring the first brain activity pattern, generating the second spatial correlation matrix, calculating the second set of coordinates, and estimating the coordinates. The processing method according to claim 6, wherein the display step includes the step of displaying the sequentially estimated coordinates as a trajectory.

8. The processing method according to claim 7, wherein the display step includes the step of displaying the estimated coordinates using a spherical projector.

9. The process includes a step of acquiring multiple brain activity patterns that show the state of brain activity of multiple subjects, wherein each brain activity pattern is acquired at predetermined intervals. The steps include: generating a third spatial correlation matrix by calculating spatial correlations for all combinations included in the multiple brain activity patterns obtained; The steps include: converting the third spatial correlation matrix into a distance matrix using multidimensional scaling, and calculating a third set of coordinates corresponding to the elements included in the third spatial correlation matrix by singular value decomposition of the distance matrix; The processing method according to claim 5, further comprising the step of extracting brain activity patterns corresponding to a predetermined number of coordinates extracted from the third group of coordinates as the plurality of brain activity patterns of the standard state pattern group.

10. The processing method according to claim 9, wherein the predetermined number of coordinates are extracted from the third group of coordinates according to a predetermined rule.

11. A program for causing a computer to execute the processing method described in any one of claims 1 to 10.

Images