Information processing apparatus, information processing method, and information processing program

Abstract

To provide an information processing apparatus, an information processing method, and an information processing program capable of improving accuracy in visualizing, from brain information, a mental image that is imagined in a brain but is not actually seen with eyes.SOLUTION: An information processing apparatus 1 comprises: a decoding unit for acquiring brain activity data related to mental data about a predetermined item being recalled in a brain, and decoding the acquired brain activity data into a plurality of types of Bayesian estimation data; a posterior distribution acquisition unit for acquiring a posterior distribution by Bayesian estimation based on the decoded Bayesian estimation data; and an output unit for outputting output data that is obtained by reconstructing the mental data by sampling using the Langevin dynamics method based on the posterior distribution.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to, for example, an information processing apparatus, an information processing method, and an information processing program that process brain information to estimate matters envisioned in the brain.

Background Art

[0002] In recent years, techniques for processing brain information to restore (reconstruct) an image seen with the eyes have been proposed. As an example of such a prediction method, a method of reconstructing an image seen by a subject (person) from brain signals measured by functional magnetic resonance imaging (fMRI) has been proposed (Non-Patent Document 1).

[0003] However, in the prior art, it is only possible to reconstruct an image (visual image) seen with the eyes, and visualization of mental data such as an image (mental image) envisioned in the brain that is not actually being seen with the eyes has not been realized.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In view of the above problems, an object of the present invention is to provide an information processing apparatus, an information processing method, and an information processing program capable of improving the accuracy in visualizing mental data envisioned in the brain that is not actually being seen with the eyes from brain information.

Means for Solving the Problem

[0006] The present invention is characterized by an information processing apparatus including: a data acquisition unit that acquires brain activity data when mental data (mental image) representing predetermined matters is recalled in the brain; a decoding unit that decodes the brain activity data acquired by the data acquisition unit into a plurality of types of data for Bayesian estimation; a posterior distribution acquisition unit that acquires a posterior distribution by Bayesian estimation based on the data for Bayesian estimation decoded by the decoding unit; and an output unit that samples using the Langevin dynamics method based on the posterior distribution acquired by the posterior distribution acquisition unit and outputs the reconstructed output data as estimated data of the mental data.

Advantages of the Invention

[0007] According to the present invention, it is possible to provide an information processing apparatus, an information processing method, and an information processing program that can improve the accuracy in visualizing mental data imagined in the brain that is not actually seen with the eyes from brain information.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0009] Hereinafter, an embodiment of the present invention will be described.

[0010] In recent years, as an example of a technology for processing brain information and restoring (reconstructing) an image (visual image) seen with the eyes, a conventional technology has been proposed that reconstructs an image seen by a subject (person) from brain signals of the subject measured by functional magnetic resonance imaging (fMRI). When local oxygen demand increases with the enhancement of neural activity, the relative amount of red blood cells containing oxygenated hemoglobin that flows in to supply oxygen to the same location increases. This state can be measured as an increase in the signal value of fMRI. That is, the activity state of nerves can be detected through the fluctuation of the fMRI signal value that reflects the change in local oxygen supply amount.

[0011] In the prior art, it has been successful to reconstruct to a certain degree of accuracy the visual images seen by humans from human brain activities. However, there have been few studies reporting success in reconstructing (visualizing) mental images that are imagined in the brain without actually seeing them with the eyes, and the types of images that can be visualized have been limited to simple ones such as human faces (H. Lee & Kuhl, 2016 ※1) and alphabet letters (Senden et al., 2019 ※2). ※1:Lee, H., & Kuhl, B. A. (2016). Reconstructing Perceived and Retrieved Faces from Activity Patterns in Lateral Parietal Cortex. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 36(22), 6069-6082. https: / / doi.org / 10.1523 / JNEUROSCI.4286-15.2016, ※2:Senden, M., Emmerling, T. C., van Hoof, R., Frost, M. A., & Goebel, R. (2019). Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception. Brain Structure & Function, 224(3), 1167-1183. https: / / doi.org / 10.1007 / s00429-019-01828-6

[0012] There are also studies that attempt to reconstruct mental images targeting both the images seen with the eyes and any natural images recalled in the brain (Shen et al., 2019 ※3, hereinafter sometimes referred to as "Basic Technology 1"). However, in this Basic Technology 1, the only thing that could be barely reconstructed from the mental image was the rough silhouette of the geometric shape. ※3:Shen, G., Horikawa, T., Majima, K., & Kamitani, Y. (2019). Deep image reconstruction from human brain activity. PLOS Computational Biology, 15(1), e1006633. https: / / doi.org / 10.1371 / journal.pcbi.1006633

[0013] As described above, currently, for the method of reconstructing the visual image being seen, it is only possible to reconstruct with a certain degree of accuracy. The reconstruction of mental data from brain information (especially including so-called natural images, voices, etc. that are not human faces or geometric shapes) has not been realized.

[0014] The present inventor has conducted intensive research to realize the visualization of mental data from brain information. Then, a new method (reconstruction method) leading to the visualization of mental data from brain information was invented, and an information processing apparatus, an information processing method, and an information processing program using this reconstruction method were invented.

[0015] Hereinafter, the reconstruction method of the present invention will be described. FIG. 1 is a flowchart of the information processing (reconstruction method) of the present embodiment. FIG. 2 is an illustrative diagram showing the flow of the information processing of the present invention. The reconstruction method of the present embodiment executes each step shown in FIGS. 1 and 2. In the following, the case where the mental data is a natural image (mental image) will be described as an example. Also, the flow described below shows only the minimum steps constituting the reconstruction method of the present invention, and it is not necessary to be limited to only the steps shown in FIG. 1.

[0016] First, when starting the reconstruction method, brain activity data is acquired when mental data representing a predetermined matter is recalled in the brain (step S1). Brain activity data is data obtained by recording the brain nerve activities of animals (including humans), and contains information necessary for reconstructing mental data (hereinafter referred to as brain information). As an example of brain activity data, there is data measured by various methods such as fMRI, electroencephalogram, cortical electroencephalogram, magnetoencephalogram, diffuse optical tomography, and calcium imaging.

[0017] In this embodiment, brain activity data measured using fMRI is used as brain information. Also, for comparison with the above-described basic technology 1, the same fMRI data set as that of basic technology 1 was used as brain activity data. This fMRI data set is composed of fMRI data from three subjects: subject 1 (male, 33 years old), subject 2 (male, 23 years old), and subject 3 (female, 23 years old). In this experiment, each subject viewed or recalled an image (image to be recalled) that was the target in each trial, and brain activity (3 seconds to 10 seconds) was measured using fMRI.

[0018] The fMRI data was divided into two sets: a training data set and a test data set. The training data set was used for training a decoder (a program for restoring a mental image), and the test data set was used for evaluation.

[0019] The training dataset consists of fMRI data measured while subjects viewed 1200 images with different contents. Each image viewed by the subjects was collected from the online image database ImageNet (Deng et al., 2009 ※4). Each image was presented to each subject 5 times. Thus, 6000 fMRI responses per subject were available as training data. ※4: Deng, J., Dong, W., Socher, R., Li, L.-J., Kai Li, & Li Fei-Fei. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248-255. https: / / doi.org / 10.1109 / CVPR.2009.5206848

[0020] The test dataset consists of fMRI data measured when the subject observed 50 natural images and 40 artificial shape images (geometric shape images), and fMRI data measured when the subject recalled 10 (included in the above 50) natural images and 15 (included in the above 40) artificial shape images. The 50 natural images are images collected from the above ImageNet and have no category overlap with the 1200 natural images used in the training dataset. The 40 artificial shape images are shapes composed of combinations (40 combinations) of 5 shapes and 8 colors (red, green, blue, cyan, magenta, yellow, white, black). Among these 50 natural images and 40 artificial shape images, 10 natural images and 15 artificial shape images were used as the recall target images. Before the fMRI experiment in which the subject recalls the images, the subject looked at 25 recall target images and memorized the hints of 25 words associated with each image. In the subsequent fMRI experiment, one of the word cues was presented in each trial, and the subject recalled the corresponding image. In the test dataset, each subject viewed each natural image 24 times and each artificial shape image 20 times, and each subject recalled each natural image and each artificial shape image 20 times.

[0021] Regarding the preprocessing of the fMRI dataset, in order to appropriately compare and evaluate the present invention and Basic Technique 1, the same fMRI preprocessing procedure as Basic Technique 1 was adopted. Also, according to the same procedure, the training data was used without performing trial averaging, and the trial-averaged test data was used for evaluation.

[0022] When brain activity data is acquired (step S1), the brain activity data is decoded (decoded) into multiple types of data for Bayesian estimation (step S2). Here, the brain activity data is decoded into multiple types of data for Bayesian estimation using supervised machine learning methods such as appropriate deep neural networks (DNNs) and linear regression respectively.

[0023] In this embodiment, the brain activity data is decoded into data related to the mental image to be reconstructed (image data) and data of the meaning evoked from the mental image to be reconstructed (meaning data).

[0024] In this embodiment, three pre-trained neural networks, namely VGG19 (Simonyan & Zisserman, 2014) for decoding brain activity data into image data, VQGAN (Esser et al., 2021) for decoding brain activity data into generative AI data, and the image encoder of CLIP (Radford et al., 2021) for decoding brain activity data into meaning data, were used as the DNN.

[0025] Also, in this embodiment, a pre-trained VGG19 model provided by PyTorch was used as the DNN for image data. Following the same procedure as in Basic Technique 1, the unit activation values included in the fMRI data were set as the decoding targets. In this embodiment, these 8 layers are called conv1, conv2, conv3, conv4, conv5, fc6, fc7, and fc8. The unit activation vector (image data of the first layer) for the input image to the DNN (Equation [1] below) is represented by Equation [2] below.

[0026]

Equation

[0027]

Equation

[0028] Also, in this embodiment, "VQGAN ImageNet(f = 16), 1024" was used as the DNN for generating AI data. This VQGAN model uses the latent vector z as input and outputs an image. The probability distribution of the output image given the latent vector z is represented by the following [Equation 3]. In this embodiment, since it can be combined with an image generation model that probabilistically generates an image, the probability distribution is used.

[0029]

Equation

[0030] Also, in this embodiment, as the DNN for semantic data, "ViT - B / 32", which is a model of the CLIP image encoder (hereinafter sometimes simply referred to as "CLIP"), was used. The output from the last layer is used as the target for decoding, and the output is represented by the following [Equation 4].

[0031]

Equation

[0032] Then, when decoding the brain activity data into the data for Bayesian estimation (step S2), a posterior distribution is obtained by Bayesian estimation based on the data for Bayesian estimation (in this embodiment, image data, semantic data, and generated AI data) (step S3). First, the log - likelihood function of the image data is expressed as in [Equation 5], and the log - likelihood function of the semantic data is expressed as in [Equation 6]. Note that the decoded data is represented by φ with a hat symbol.

[0033]

Equation

[0034]

Equation

[0035] Here, T is a parameter called "temperature" and is set to 10 -7 The function similarity(·,·) is a similarity metric for measuring the similarity between two input vectors. In this embodiment, the Pearson correlation coefficient was used as the similarity index. When the negative L2 norm was adopted as the similarity metric, the maximum likelihood estimation using this likelihood function is equivalent to the reconstruction method proposed in Basic Technique 1, and the reconstruction method in this embodiment is an extension of the reconstruction method of Basic Technique 1 to Bayesian estimation.

[0036] In [Equation 5], w_l (lowercase "L") is a parameter that controls the strength of the contribution from the l-th layer of VGG and is set to the reciprocal of the number of layers of VGG used. L is the total number of layers of VGG. Reconstruction was performed using each layer of VGG, a subset of the layers of VGG, or all VGG layers. The sum in [Equation 5] was obtained over all the layers used. In [Equation 6], λ_CLIP is a parameter that controls the strength of semantic assistance and is set to 0.1 and 0.25 for the reconstruction of the seen image and the reconstruction of the image, respectively. To make the likelihood function robust to perturbations of I, a random affine transformation was applied to I and the average similarity was evaluated with the likelihood function. The conditional probability distribution p_Perturb(I ’ │I) represents a random affine transformation.

[0037] Also, when a latent vector z ∈ R (14×14×256) (R is the set of real numbers) is given, the image generation model of VQGAN generates an image. The probability distribution of the generated image conditional on the latent vector z is represented as in the above [Equation 3].

[0038] Therefore, by preparing the distribution p(z) and constructing the joint distribution p(I,z) = p_VQGAN(I|z)p(z) and marginalizing the latent vector z, a prior image p(I) can be obtained as in [Equation 7]. Here, any analytically differentiable distribution p(z) can be used. In this embodiment, an uninformative distribution was used for p(z).

[0039] [Mathematics] From the above matters, the posterior distribution can be obtained by Bayesian estimation as shown in [Equation 8] below.

[0040] [Mathematics]

[0041] Subsequently, to obtain an image (reconstructed image) from the posterior distribution, sampling was performed using the Stochastic Gradient Langevin Dynamics algorithm (hereinafter referred to as the "SGLD algorithm"), and the reconstruction result was output (the reconstructed output data was output as the estimated data of the mental data) (Step S4). The posterior distribution shown in [Equation 8] can be rewritten as shown in the following [Equation 9].

[0042] [Mathematics]

[0043] Therefore, if z can be sampled from the following [Equation 10] using the SGLD algorithm, an image can be sampled from the posterior distribution by Bayesian estimation through p_VQGAN(I│z). The update rule of the algorithm is given as shown in the following [Equation 11].

[0044] [Mathematics]

[0045] [Mathematics]

[0046] Here, z t and ε t are the sample and learning rate at step t, and η t is the Gaussian distribution N(0, ε t) is a vector from which elements are sampled. In this study, the learning rate gradually decreases, and the order is given as follows in [Equation 12].

[0047]

Equation

[0048] Here, a was set to 0.00015, b was set to 0.15, and γ was set to 0.055. 500 iterations were performed, and a reconstructed image was generated using z_500 obtained from [Equation 11]. For fast convergence, the Adam algorithm was first run using an initial random latent vector. Next, the SGLD algorithm was run using the result of the Adam algorithm as the initial latent vector.

[0049] For each update of the SGLD algorithm, when approximately calculating the right side of [Equation 11], the logarithm of p(z|φ_VGG (1) ,…,φ_VGG (L) ,φ_CLIP) is expressed as follows in [Equation 13].

[0050]

Equation

[0051] Also, the first term of [Equation 11] can be rewritten as follows in [Equation 14] for approximate calculation, and using Jensen's inequality, it is approximated as follows in [Equation 15].

[0052]

Equation

[0053]

Equation

[0054] According to the present invention, a reconstructed image can be obtained by implementing the above-described reconstruction method. Next, as an embodiment of this invention, an example of an apparatus using the above-described reconstruction method will be described.

[0055] FIG. 3 is a block diagram showing an example of the configuration of the information processing apparatus 1. The information processing apparatus 1 is composed of a general-purpose computer (terminal), and is an information processing apparatus (brain information visualization apparatus) for acquiring brain activity data and reconstructing (visualizing) the matter envisioned in the brain.

[0056] As shown in FIG. 3, the information processing apparatus 1 includes an input unit 2, a display unit 3, a brain activity data acquisition unit 4, a result output unit 5, a control unit 6, and an auxiliary storage unit 7. Each of the input unit 2, the display unit 3, the brain activity data acquisition unit 4, the result output unit 5, and the auxiliary storage unit 7 is connected to the control unit 6.

[0057] The control unit 6 includes an arithmetic unit 61 and a main storage unit 62, and executes various arithmetic and control operations in the information processing apparatus 1. The arithmetic unit 61 is an arithmetic processing unit including a CPU or an MPU, etc. The main storage unit 62 includes a RAM (DRAM) and a ROM, etc. The RAM is used as a work area and a buffer area of the arithmetic unit 61. The ROM stores the startup program of the information processing apparatus 1 and default values for various information, etc.

[0058] The input unit 2 includes an input member that receives the operation input of the user of the information processing apparatus 1, and an input detection circuit interposed between the input member and the control unit 6. The input member is, for example, a touch panel or / and a hardware operation button or operation key. As the touch panel, any type such as a capacitance type, an electromagnetic induction type, a resistive film type, an infrared type, etc. can be used. The input detection circuit outputs an operation signal or operation data corresponding to the operation of each input member to the control unit 6.

[0059] The display unit 3 includes a display and a display control circuit interposed between the display and the control unit 6. As the display, for example, an LCD (liquid crystal display) or an organic EL display can be used. The display control circuit includes a GPU, a VRAM, and the like. Under the instruction of the control unit 6, the GPU generates display image data for displaying various screens on the display using the data for image generation stored in the RAM, and outputs the generated display image data to the display.

[0060] The brain activity data acquisition unit 4 is connected to a brain activity data detection device 10 capable of detecting appropriate brain information, and acquires the detection signal (detection data) transmitted from the brain activity data detection device 10 as brain activity data. As the brain activity data detection device 10, a known detection device can be used according to the type of brain information such as brain activity information, electroencephalogram, cortical electroencephalogram, information obtained from magnetoencephalogram, information obtained by calcium imaging, and information obtained by extracellular measurement.

[0061] The result output unit 5 outputs data (output data) corresponding to the items (images, voices, etc.) reconstructed from the acquired brain activity data. If the output data is visualized data, the result output unit 5 outputs it as image data and causes the display unit 3 or an external display device to display the output data. Also, if the output data is voice computer, the result output unit 5 outputs it as voice via voice output means such as a speaker.

[0062] The auxiliary storage unit 7 is composed of other non-volatile memories such as an HDD, an SSD, a flash memory, and an EEPROM, and stores a program for the control unit 6 (arithmetic unit 61) to control the operation of the information processing device 1 and various data. The various data stored in the auxiliary storage unit 7 are developed (read out) to the main storage unit 62 as necessary.

[0063] The auxiliary storage unit 7 stores at least a main processing program 71 for executing various operations of the information processing apparatus 1, a decoding program 72 for decoding brain activity data into a plurality of types of Bayesian estimation data, a posterior distribution acquisition program 73 for obtaining a posterior distribution by Bayesian estimation from the Bayesian estimation data, a reconstruction program 74 for sampling using the Langevin dynamics method based on the posterior distribution and outputting output data obtained by reconstructing mental data, and the like.

[0064] Further, the auxiliary storage unit 7 stores Bayesian estimation data 75 obtained by decrypting brain activity data, posterior distribution data 76 obtained by Bayesian estimation from the Bayesian estimation data, output data (reconstructed data) 77 obtained by reconstructing mental data, and the like.

[0065] With the information processing apparatus 1 having such a configuration, the reconstruction method of the present invention can be executed based on the brain activity data related to the input mental data, and output data obtained by reconstructing the mental data can be output.

[0066] FIG. 4 is a diagram comparing a target image and a reconstruction result when the mental image is a natural image. FIG. 5 is a diagram comparing a target image and a reconstruction result when the mental image is an image of a geometric shape. In FIG. 4, an image reconstructed using the method of basic technology 1 is also shown as the reconstructed image.

[0067] As shown in FIG. 4, in basic technology 1, the reconstruction result (reconstructed image) of the mental image becomes an overall blurred image, whereas in the reconstructed image of the reconstruction method of the present invention, an image is obtained in which the contour shape can be recognized to some extent. Thus, the accuracy of reconstructing the mental image, which is mental data related to a predetermined matter being recalled in the brain, has been improved.

[0068] Also, as shown in FIG. 5, when the mental image is an image of a geometric shape, by using the reconstruction method of the present invention, a reconstructed image closer to the target image could be obtained.

[0069] As described above, according to the present invention, it is possible to improve the accuracy when visualizing a mental image that is actually imagined in the brain without actually seeing it from brain information.

[0070] Further, according to the present invention, the data for Bayesian estimation includes image data related to mental data to be reconstructed, semantic data which is data of the meaning recalled from the mental data, and generated AI data which is output data related to the mental data generated by a generative AI. In this way, by using not only the image data which is the direct object of reconstruction but also the semantic data and the generated AI data for reconstruction, it is possible to further improve the accuracy when visualizing a mental image.

[0071] Further, according to the present invention, it is possible to suppress the reconstruction result (reconstructed image in the embodiment) from becoming an unclear result (for example, a blurred image) where it is completely unclear what it represents, and to make it a result (for example, an image where at least a part is clear) where at least something can be read.

[0072] The information processing apparatus of this invention corresponds to the information processing apparatus 1 of the above embodiment. Similarly hereinafter, the data acquisition means corresponds to the brain activity data acquisition unit 4, the decoding unit corresponds to the decoding program 72 and the control unit 6 that operates according to this, the posterior distribution acquisition unit corresponds to the posterior distribution acquisition program 73 and the control unit 6 that operates according to this, the output unit corresponds to the reconstruction program 74 and the control unit 6 that operates according to this, [Equation 1] corresponds to [Equation 16] to be described later, but this invention is not limited to this embodiment and can be other various embodiments. Also, the specific configurations etc. mentioned in the above embodiment are examples, and can be appropriately changed according to the actual product.

[0073] For example, in the above-described embodiment, the case where the mental image is a natural image is taken as an example, and neural networks for image recognition such as VGG and CLIP are used to decode brain activity data into data for Bayesian estimation for images, but it is not limited thereto. For example, known neural networks for image recognition such as AlexNet, GoogleNet, ResNet, DenseNet, and MobileNet can be used for decoding.

[0074] Also, VQGAN is used as the DNN for generating AI data, but it is not limited thereto. For example, known generative AIs such as BigGAN, VQVAE, and VQVAE-2 can be used.

[0075] In the above-described embodiment, the case where the type of matter (mental data) being recalled in the brain is an image (mental image) is taken as an example for explanation, but it is not limited thereto. For example, the mental data may be a voice being recalled in the brain. In this case, known neural networks for voice such as VGGish and Yamnet can be used to decode the brain activity data into data for Bayesian estimation for images. Similarly, as a generative model for generating sound, a neural network proposed in prior research (Iashin, & Rahtu, 2021) and the like can be used.

[0076] Note that the characters such as "VGG", "CLIP", and "VQGAN" within Numbers 2 to 11 and Numbers 13 to 15 shown in the above embodiment are characters indicating the names of neural networks used for convenience of explanation, but the characters themselves indicating the names of neural networks have no technical meaning, and the parts of φ_VGG(L) and φ_CLIP can be changed according to the image features to be used. Also, it is possible to change so as to combine the features obtained from any number of different multiple models. Therefore, the above [Number 8] can be generalized and expressed as the following [Number 16]. Note that φ_1,…,φ_K represent K types of image features decoded from brain signals.

[0077] [Number]

[0078] In addition, the present invention can also be provided as an information processing system configured by a plurality of computers with a configuration for exerting the functions of the above-described information processing apparatus 1 and including the plurality of computers.

[0079] Furthermore, the present invention can be provided not only as an information processing apparatus, but also as a method, a program, and a non-transitory tangible storage medium storing the program for detecting an event using the information processing apparatus. [Industrial Applicability]

[0080] This invention can be used in industries such as processing brain information to estimate matters envisioned in the brain. [Explanation of Signs]

[0081] 1... Information processing apparatus 4... Brain activity data acquisition unit 5... Result output unit 6... Control unit 61... Arithmetic unit 62... Main memory unit 7... Auxiliary storage unit 72... Decoding program 73... Posterior distribution acquisition program 74... Reconstruction program

Claims

1. A data acquisition unit that acquires brain activity data when mental data representing predetermined items is being recalled in the brain, A decoding unit that decodes the brain activity data acquired by the data acquisition unit into multiple types of data for Bayesian estimation, A posterior distribution acquisition unit that acquires a posterior distribution by Bayesian estimation based on the data for Bayesian estimation decoded by the decoding unit, An output unit that samples using the Langevin dynamics method based on the posterior distribution acquired by the posterior distribution acquisition unit and outputs the reconstructed output data as estimated data of the mental data. An information processing apparatus.

2. The data for Bayesian estimation has image data related to the mental data to be reconstructed and semantic data that is data on the meaning evoked from the mental data. The information processing apparatus according to Claim 1.

3. The mental data to be reconstructed is mental image data about an image imagined in the brain without actually seeing it with the eyes. The brain activity data is fMRI signals. The image data is data related to an image. The information processing apparatus according to Claim 2.

4. The posterior distribution acquisition unit acquires the posterior distribution by performing an operation using the following [Equation 1]. The information processing apparatus according to Claim 3. 【Number 1】

5. Acquire brain activity data when mental data representing predetermined items is being recalled in the brain, Decode the acquired brain activity data into multiple types of data for Bayesian estimation, Acquire a posterior distribution by Bayesian estimation based on the decoded data for Bayesian estimation, Sample using the Langevin dynamics method based on the acquired posterior distribution and output the reconstructed output data as estimated data of the mental data. An information processing method.

6. A computer, A data acquisition means for acquiring brain activity data when mental data representing predetermined items is being recalled in the brain, A decoding means for decoding the brain activity data acquired by the data acquisition means into multiple types of data for Bayesian estimation, A posterior distribution acquisition means for acquiring a posterior distribution by Bayesian estimation based on the data for Bayesian estimation decoded by the decoding means, Function as output means for sampling using the Langevin dynamics method based on the posterior distribution obtained by the posterior distribution acquisition means and outputting the reconstructed output data as estimated data of the mental data An information processing program.

Images