Sensory transmission system and sensory transmission method

Abstract

To provide a sensory transmission system and a sensory transmission method that can appropriately protect personal information of a subject.SOLUTION: A sensory transfer system includes: a first device that detects, when perceived by the first subject, brain activation information of a first subject; an estimation device that estimates, on the basis of the detected brain activation information of the first subject, recollected sensory information that a second subject different from the first subject recalls in response to a perception and encodes the recollected sensory information using the brain activation information of the first subject; and a second device that decodes the encoded recalled sensory information and provides a stimulus to the second subject so as to recall the decoded recalled sensory information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a sensory transmission system and a sensory transmission method. [Background technology]

[0002] In recent years, non-invasive measurement of brain activation information, such as functional magnetic resonance and near-infrared spectroscopy, has become possible. As technology advances, the technology for brain-machine interfaces, which are interfaces between the brain and the outside world, is becoming increasingly sophisticated. An example of the use of such technology is when a subject perceives Based on the detected brain activation information, A technology for converting recalled sensory information into a message has been disclosed (for example, Patent Document 1 reference). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-115913 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, the subject's recalled sensory information corresponds to so-called personal information. From the viewpoint of protecting personal information, it is necessary to handle the subject's recalled sensory information appropriately. can be done.

[0005] The present invention has been made in view of the above, and aims to appropriately protect the personal information of subjects. The object of the present invention is to provide a sensory transmission system and a sensory transmission method that are capable of the above. [Means for solving the problem]

[0006] The sensory transmission system according to the present invention is a sensory transmission system for transmitting a sensory information of a first subject when the first subject perceives the sensory information of the first subject. a first device for detecting brain activation information of the first subject; Based on the above, a second subject different from the first subject may recall a recollection of the sensory sensation in response to the perception. and encoding the recalled sensory information using the brain activation information of the first subject. a determination device for determining whether the encrypted recalled sensory information is to be recalled; and a second device for applying a stimulus to the second subject so as to stimulate the second subject.

[0007] The sensory transmission method according to the present invention is a method for transmitting a sensory information of a brain of a first subject when the first subject perceives the sensory information. detecting activation information within the brain of the first subject; and and estimating the recollected sensory information that a second subject different from the first subject recalls in response to the perception. encoding the recalled sensory information using brain activation information of the first subject; and decoding the encoded recalled sensory information to recall the decoded recalled sensory information. and administering the stimulus to a second subject. [Effects of the Invention]

[0008] According to the present invention, a sensory transmission system and and a method of transmitting sensations. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a sensation transfer system according to the first embodiment. [Figure 2] FIG. 2 is a functional block diagram illustrating an example of a sensation transmission system according to the first embodiment. [Figure 3]FIG. 3 is a diagram illustrating an example of a neural network. [Figure 4] FIG. 4 is a diagram schematically illustrating an example of the operation of the sensation transfer system according to the first embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of the operation of the sensation transfer system according to the first embodiment. [Figure 6] FIG. 6 is a diagram schematically illustrating another example of the operation of the sensation transfer system according to the second embodiment. [Figure 7] FIG. 7 is a diagram schematically illustrating another example of the operation of the sensation transmission system according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following embodiments are not limited to the above. This includes things that are possible and easy, or things that are substantially the same.

[0011] [First embodiment] FIG. 1 is a schematic diagram illustrating an example of a sensation transmission system 100 according to the first embodiment. 1 and 2 are functional block diagrams showing an example of a sensation transmission system 100. As described above, the sensation transfer system 100 includes a first device 10, an estimation device 20, and a second device 30. Prepare.

[0012] The first device 10 measures the brain activity of the first subject R1 when perceived by the first subject R1. The first device 10 includes a detection unit 11, a communication unit 12, a processing unit 13, and a stimulator. It has a stimulus applying unit 14 and a memory unit 15.

[0013] The detection unit 11 detects brain activation information. The brain activation information may include, for example, the subject's Oxygenated hemoglobin, deoxygenated hemoglobin, and total hemoglobin concentrations in cerebral blood flow The detection unit 11 may be, for example, an fMRI (functional Magnetron Imaging) device. Functional Magnetic Resonance Imaging (fNIR) S(functional Near-Infrared Spectroscopy: Measurement devices that perform measurements based on the principles of functional near-infrared spectroscopy, etc., and invasive electrodes are used. A measuring device that places a micromachine inside the blood vessels of the brain and measures the The detector 11 is not limited to the above-mentioned device, and may be any other device. The brain activation information may be obtained by measuring the brain of the first subject R1 for example by measuring the brain activity for several milliseconds. When partitioned by a three-dimensional matrix consisting of voxels below 1, In this embodiment, the detection unit 11 detects the activity of each of the 2. Brain activation information for estimating the recalled sensory information to be transmitted to subject R2, and the recalled sensory information The detection unit 11 detects the brain activation information for encoding the information separately. In order to detect brain activation information for encoding information, for example, the primary brain activity of the first subject R1 is detected. It is possible to detect brain activation information generated in the visual cortex. When detecting the brain activation information for encryption, the detection unit 11 detects, for example, the first subject R1 The stripes are monotone and have a specific direction (hereinafter referred to as monotone stripes). When an image is viewed, activation information in the brain can be detected in the primary visual cortex. Since the cerebral activation information is present on the surface of the brain, the detection unit 11 can obtain highly accurate information on activation of the brain. In addition, the primary visual cortex is able to easily recognize the above monotone striped pattern. When viewing a monotone striped image, most subjects recalled a similar sensation. become.

[0014] The communication unit 12 is an interface for performing wired or wireless communication. The brain activation information detected by the detection unit 11 is transmitted to the estimation device 20. The communication unit 12 For communicating with external devices via wireless LAN in accordance with the IEEE802.11 standard The communication unit 12 communicates with an external device under the control of the processing unit 22 and the like. The communication method is not limited to wireless LAN, and may be, for example, infrared. Wired communication method, Bluetooth (registered trademark) communication method, Wireless USB, etc. It can also include wireless communication methods. ), IEEE1394, Ethernet, or other wired connections may also be used.

[0015] The processing unit 13, the stimulus applying unit 14, and the storage unit 15 will be described later. In this case, the processing unit 13, the stimulus applying unit 14, and the storage unit 15 may not be provided.

[0016] The estimation device 20 includes a communication unit 21, a processing unit 22, and a storage unit 23. The communication unit 21 is an interface for performing wired or wireless communication. The communication unit 21 receives the brain activation information transmitted from the device 10. The communication unit 21 transmits the corresponding sensory information, which will be described later, estimated by the above-mentioned method to the second device 30. The communication unit 12 may have the same configuration as the communication unit 12 described above.

[0017] The processing unit 22 is a processor such as a CPU (Central Processing Unit). The device and RAM (Random Access Memory) or ROM (Read Only Memory) The processing unit 22 has a storage device such as a memory device (memory device). It performs various processes including processing.

[0018] The storage unit 23 stores various information. The storage unit 23 may be, for example, a hard disk drive, The storage unit 23 has a storage such as a solid state drive. An external storage medium such as a bubble disk may also be used.

[0019] The processing unit 22 calculates the first subject R1's brain activation information based on the detected brain activation information of the first subject R1. The system estimates the recalled sensory information (reference sensory information) about the sensations recalled in response to the perception. Sensory information may be information from at least one of the five senses, i.e., sight, hearing, touch, taste, and smell. It may be information about one sense, or it may be information about balance or other somatic senses. Specifically, when the recalled sensory information is information related to vision, the recalled sensory information may be The image data perceived by the first subject R1, but is not limited to this, and the image data itself Instead, it is image data sampled from that image data, or that image data Furthermore, the image data may be image data that has been subjected to filtering. In the case where the information is about the light entering the eyeball of the first subject R1, the information may be about the light entering the eyeball of the first subject R1. In this case, for example, information about light is acquired by contact lenses equipped with optical sensors. Alternatively, an artificial retina may be used to obtain information about light. The information about the light from the CCD sensor installed in the artificial retina can be used. If the information is auditory, the recalled sensory information is the audio signal perceived by the first subject R1. In addition, if the recalled sensory information is information about taste, The information is data showing indicators of multiple chemical substances that reproduce the taste perceived by the first subject R1. In addition, when the recalled sensory information is information related to the sense of touch, the recalled sensory information is 1. The entire body surface of subject R1 is laid out on a plane, and which part of the developed image is stimulated and to what extent. It is sufficient if the data indicates what happened. Regarding the first subject R1, what kind of recalled sensory information was recalled and what kind of To confirm whether the information is activated in the brain, we will conduct a preliminary experiment to confirm the correspondence relationship. For example, the brain activation information detected from the first subject R1 and The brain activation information is associated with the corresponding recalled sensory information and used as a set of learning data sets. , the first learning model can be generated by performing machine learning on the learning dataset. The first learning model can be stored in the storage unit 23, for example.

[0020] The processing unit 22 also calculates the reference sensory information of a second subject R1 different from the first subject R1 based on the estimated reference sensory information. Estimate the corresponding sensory information, which is the recalled sensory information corresponding to the reference sensory information for subject R2. The specific example of the corresponding sensory information may correspond to the specific example of the recalled sensory information. R2 is the subject to whom the sensations of the first subject R1 are transmitted. The relationship with the sensory information can be determined by conducting experiments in advance. For example, the corresponding sensory information between the first subject R1 and the second subject R2 is used as a set of learning data. The training dataset is used as a set, and a second learning model is generated by performing machine learning on the training dataset. The second learning model can be stored in the storage unit 23, for example.

[0021] The processing unit 22 also calculates the estimated corresponding sensory information by using the brain activation information of the first subject R1. When performing encryption, the processing unit 22 detects the image from the primary visual cortex of the first subject R1. More specifically, the processing unit 22 can use the brain activation information output from the processing unit 22, for example. When the first subject R1 looked at the above-mentioned monotone striped image, We can use brain activation information detected from the primary visual cortex. Since the visual cortex is located on the surface of the brain, the first device 10 can obtain highly accurate brain activation information. Furthermore, the primary visual cortex is able to easily recognize monotone striped patterns such as the one shown above. Therefore, when viewing a monotone striped image, most subjects recalled a similar sensation. This increases the reproducibility of encryption.

[0022] The processing unit 22 performs encryption using an encryption key generated using, for example, brain activation information. The processing unit 22 can perform the encryption using known encryption methods such as public key encryption, common key encryption, and key exchange. A cryptographic key can be generated using a key generation method. The cryptographic key is generated using brain activation information. In one embodiment, the processing unit 22 may detect, for example, a monotone striped image viewed by the first subject R1. In this case, the brain activation information detected from the primary visual cortex of the first subject R1 can be reproducibly and the quantized brain activation information is used to affect the factor information for generating the encryption key. In this case, the detection unit 11 of the first device 10 detects the signal to be transmitted to the second subject R2. In addition to the brain activation information used to estimate the recalled sensory information, we also The processing unit 22 detects brain activation information when the object is viewed. ,The estimated reference sensory information may be encrypted using a similar procedure.

[0023] The second device 30 provides a stimulus to the second subject R2 so as to recall the estimated corresponding sensory information. The second device 30 includes a detection unit 31, a communication unit 32, a processing unit 33, and a stimulus applying unit 34. , and a storage unit 35.

[0024] The detection unit 31 will be described later. In the first embodiment, the detection unit 31 is not provided. It's okay.

[0025] The communication unit 32 is an interface for performing wired or wireless communication. The communication unit 32 receives corresponding sensory information transmitted from the estimation device 20. The communication unit 32 transmits the detected brain activation information to the estimation device 20. The communication unit 12 may have the same configuration as the communication unit 12 described above.

[0026] The processing unit 33 decodes the corresponding sensory information received by the communication unit 32. For example, the processing unit 33 The processing unit 22 of the estimation device 20 decrypts the corresponding sensory information using a decryption key corresponding to the encryption key. The decryption key is generated in accordance with the encryption key generation method described above. Alternatively, the decryption key may be acquired in advance and stored in the storage unit 35, or the encryption key and the decryption key may be stored in the storage unit 35. The encryption key may be transmitted from the estimation device 20 to the second device 30 together with the encryption key. 33 receives the corresponding sensory information and the third learning model described below based on the decoded corresponding sensory information. It is possible to calculate stimulus image information corresponding to the corresponding sensory information.

[0027] The third learning model is based on stimulus image information for the second subject R2, which will be described later, and the stimulus image information. Based on this, the corresponding sensation is the recalled sensory information that the second subject R2 recalls when irradiated with electromagnetic waves. The data is then matched with the information to create a set of training data sets, and machine learning is performed on the training data sets. The third learning model is stored in the storage unit 35 of the second device 30, for example. It can be placed.

[0028] The stimulation unit 34 irradiates the target area of ​​the brain of the second subject R2 with an electromagnetic wave signal to stimulate the target area. By activating the second subject R2, the brain of the second subject R2 is stimulated. For example, a three-dimensional matrix consisting of voxels of a few millimeters or less is used to divide the image. The stimulus applying unit 34 applies electromagnetic waves to each voxel in the three-dimensional matrix. The electromagnetic wave can be irradiated based on stimulus image information indicating the intensity of the electromagnetic wave to be irradiated. A voxel in the three-dimensional matrix of stimulus image information can be used to represent, for example, a three-dimensional matrix of brain activation information. The dimensions and positions may correspond to those of the voxels in the 2-dimensional matrix. What thoughts would be produced by irradiating which voxels of the brain of subject R2 with electromagnetic waves of what intensity? To determine whether sensory information is recalled, we first conduct an experiment to determine the correspondence. It is possible.

[0029] The first, second and third learning models described above are, for example, VGG16. Generated using a neural network (convolutional neural network) represented as FIG. 3 is a diagram showing an example of a neural network. As shown, the neural network NW consists of 13 convolutional layers S1 and 5 pooling layers. The neural network has a neural network with a 3-layered fully connected layer S2 and a 3-layered fully connected layer S3. The information is processed in the convolutional layer S1 and the pooling layer S2 in order, and the processed results are combined in the fully connected layer S3. and output.

[0030] When generating the first learning model, the second learning model, and the third learning model, As shown in Fig. 1, a training data set containing the corresponding information (I1, I2) is used for the neural network. The correlations of the training data set are learned through machine learning such as deep learning. In other words, the neural network NW is optimized through learning to generate a learning model. For example, when one of the pieces of information that make up each learning data set is input, Students learn to solve problems that require one piece of information in a given situation.

[0031] When performing inference using the first, second, and third learning models, the bottom of Figure 3 As shown in the figure, each of the two pieces of information that make up the training data set is I1 is input to the neural network NW. The learning model extracts the input data based on the correlations of the information that make up the learning dataset. The other information I2 corresponding to the input information I1 is output. ,Generate a training model using a convolutional neural network represented by VGG16. Although examples are described, other types of neural networks may be used for learning. A training model may be generated.

[0032] Next, a method for transmitting sensation using the sensation transmission system 100 configured as described above will be described. FIG. 4 is a diagram illustrating an example of the operation of the sensation transmission system 100. As shown in the figure, the first subject R1 is made to perceive the reference sensory information. Let us take the example of subject R1 visually perceiving a cat's face and recalling it as visual information. In addition, the first subject R1 is asked to view a monotone striped image IM1 for encryption. To be perceived by the senses.

[0033] As shown in the upper part of FIG. 4, the detection unit 11 perceives the cat's face and recalls recalled sensory information 41. The detection unit 11 detects the brain activation information 42 of the first subject R1. The brain activation information 48 of the first subject R1 that is recalled by the pattern image IM1 is detected. The unit 12 transmits the brain activation information 42, 48 detected by the detection unit 11 to the estimation device 20. .

[0034] In the estimation device 20, the communication unit 21 receives the brain activation information 4 transmitted from the first device 10. 2, 48. The processing unit 22 receives the received brain activation information 42 and calculates the reference sensation. In this case, the processing unit 22 estimates the brain activation information 42 of the first subject R1 as The first learning model receives the activation information 42 and the reference sensory information. Based on the learning results of the correlation with 43, the reference sense corresponding to the input brain activation information 42 is generated. The processing unit 22 outputs the reference sensory information 43 as an estimation result. Get it.

[0035] The processing unit 22 calculates the reference sensory information 43 based on the estimated reference sensory information 43. The processing unit 22 estimates the corresponding sensory information 44 corresponding to the quasi-sensory information 43. The reference sensory information 43 is input to the second learning model. Based on the learning result of the correlation between the input reference sensory information 43 and the corresponding sensory information 44, The processing unit 22 outputs the corresponding sensory information 44 corresponding to the output sensory information 43. 4 as the estimation result. In addition, the processing unit 22 receives the brain activation information 42 separately. The communication unit 21 encrypts the corresponding sensory information 44 using the brain activation information 48. The converted corresponding sensory information 44 is transmitted to the second device 30.

[0036] In the second device 30, the communication unit 32 receives the corresponding sensory information 44 transmitted from the estimation device 20. The processing unit 33 decodes the received corresponding sensory information 44 and outputs the composite corresponding sensory information 44 is input to the third learning model stored in the storage unit 35. Based on the learning results of the correlation between the response sensory information 44 and the stimulus image information, the input response sensory information The stimulus applying unit 34 outputs the stimulus image information 45 corresponding to the stimulus image information 44. Based on the above, stimulation was given to the second subject R2 by irradiating the brain of the second subject R2 with electromagnetic waves. As a result, the second subject R2 to whom the stimulus is applied by the stimulus applying unit 34 receives the stimulus image information The second subject R2 recalls the corresponding sensory information 46 corresponding to the visual sensation of the cat's face. The information is recalled as corresponding sensory information 46.

[0037] On the other hand, there are differences in personality between the first subject R1 and the second subject R2, so brain activity may differ. The correspondence between the movement state and the brain activation information is different. As shown in the figure (indicated by the dashed line), the brain activation information 42 of the first subject R1 is directly transmitted to the second device 30. and transmits the brain activation information 42 of the second subject R2 to the second device 30. When the stimulus was given to the second subject R2, the second subject R2 recalled visual information different from the cat's face. In this case, the sensory information of the first subject R1 is likely to be recalled as 2 will not be properly communicated to

[0038] In contrast, in the sensation transfer system 100 of this embodiment, the estimation device 20 To estimate the corresponding sensory information 44 of the subject R2, the first subject R1 is sent to the second subject R2. This allows the visual information of the cat's face to be transmitted appropriately.

[0039] FIG. 5 is a flowchart showing an example of the operation of the sensation transfer system 100. As shown in the figure, in the sensation transmission system 100, the first device 10 transmits the sensation perceived by the first subject R1. In order to estimate the brain activation information, i.e., the recalled sensory information, of the first subject R1 in the case The brain activation information (hereinafter referred to as "for estimation") is detected (step S101). The first device 10 is a first subject for performing encryption (hereinafter referred to as "for encryption"). Next, the estimation device 20 separately detects the brain activation information of R1 (step S102). Based on the estimated brain activation information of the first subject R1, the first subject R1 recalls the perception. The reference sensory information is estimated, and a sensory information different from the first subject R1 is obtained based on the estimated reference sensory information. Estimate corresponding sensory information corresponding to the reference sensory information for the second subject R2 (step S 103). In addition, the estimation device 20 uses the brain activation information for encryption to estimate the corresponding sense. The second device 30 encrypts the information and transmits it to the second device 30 (step S104). The encoded corresponding sensory information is received and decoded (step S105), and the decoded corresponding sensory information is A stimulus is given to the second subject R2 to recall the information (step S106).

[0040] As described above, the sensation transmission system 100 according to this embodiment transmits the sensation perceived by the first subject R1. a first device (10) for detecting brain activation information of a first subject (R1) when the first subject (R1) is Based on the brain activation information of the first subject R1, a second subject R2 different from the first subject R1 The sensory information recalled by the first subject R1 is estimated using the brain activation information of the first subject R1. an estimation device 20 that encrypts the recalled sensory information and decrypts the encrypted recalled sensory information; and a second device 30 for providing a stimulus to the second subject R2 so as to recall the recalled sensory information. do.

[0041] The sensation transmission method according to this embodiment is a method for transmitting sensations to a first subject R1 when the first subject R1 perceives the sensations. Detecting the brain activation information of R1 and Based on this, the recollected sensory feelings that the second subject R2, who is different from the first subject R1, recalls in response to the perception are The information is estimated, and the brain activation information of the first subject R1 is used to encode the recalled sensory information. The second subject is instructed to decode the encoded recalled sensory information and recall the decoded recalled sensory information. and providing a stimulus to subject R2.

[0042] According to this configuration, when there is a difference in brain activity between the first subject R1 and the second subject R2, Even in this case, the first subject R1 can appropriately transmit recalled sensory information to the second subject R2. In addition, when the recalled sensory information is transmitted, the brain activation information of the first subject R1 is Since the recalled sensory information is more easily encrypted, the recalled sensory information, which is the subject's personal information, is properly preserved. It can be protected.

[0043] In the sensation transfer system 100 according to this embodiment, the estimation device 20 performs encryption. In this case, the brain activation information detected from the primary visual cortex of the first subject R1 is used. Therefore, this configuration allows us to obtain highly accurate information on brain activation. This can be done.

[0044] In the sensation transfer system 100 according to this embodiment, the estimation device 20 performs encryption. In this case, when the first subject R1 looks at the image of the monotone striped pattern, the first subject R1's primary We use the brain activation information detected from the visual cortex. Therefore, when viewing an image with a monotone stripe pattern, many subjects This will evoke a similar sensation, ensuring high reproducibility. .

[0045] In the sensory transmission system 100 according to this embodiment, the estimation device 20 According to this configuration, the data is encrypted using a cryptographic key generated using the brain activation information. By encrypting using the encryption key generated by the .

[0046] [Second embodiment] Next, a second embodiment will be described. In the first embodiment, the sensation transmission system 100 Let us take the example of transmitting sensation in one direction from the first subject R1 to the second subject R2. In contrast to this, in the second embodiment, the sensation transfer system 100 transfers the sensation from the second subject R2 to the The sensation is transmitted from the first subject R1 to the first subject R2. This configuration allows sensations to be transmitted bidirectionally between the subject R1 and the second subject R2.

[0047] The overall configuration of the sensation transmission system 100 is the same as that of the first embodiment. 1 and 2, the configuration of the sensation transmission system 100 will be described from the side of the second device 30. explain.

[0048] The second device 30 includes a detection unit 31, a communication unit 32, a processing unit 33, a stimulus applying unit 34, and a storage unit. The processing unit 33, the stimulus applying unit 34, and the storage unit 35 are the same as those in the first embodiment. The detecting unit 31 is the same as the detecting unit 11 in the first embodiment. In this embodiment, the detection unit 31 detects brain activation information of the first subject R2. 1, brain activation information for estimating the recalled sensory information to be transmitted to the The detection unit 31 separately detects the brain activation information for encrypting the recalled sensory information. For example, if the second subject R2 is wearing a monotone striped pattern, It is possible to detect brain activation information generated in the primary visual cortex when viewing an image. The communication unit 32 transmits the brain activation information detected by the detection unit 31 to the estimation device 20. .

[0049] The estimation device 20 includes a communication unit 21, a processing unit 22, and The communication unit 21 is capable of wired communication or wireless communication. In this embodiment, the communication unit 21 receives brain activation information transmitted from the second device 30, for example. The communication unit 21 transmits, for example, corresponding sensory information estimated by the processing unit 22 (to be described later) to the first device. Send to 10.

[0050] The processing unit 22 calculates the brain activation information of the second subject R2 based on the detected brain activation information of the second subject R2. The second step is to estimate the recalled sensory information (reference sensory information) about the sensations recalled in response to the perception. Regarding subject R2, what kind of brain activation occurred when what kind of sensory information was recalled? It is possible to determine the correspondence by conducting experiments in advance to determine whether the information is accurate. For example, the brain activation information detected from the second subject R2 and the The corresponding recalled sensory information is associated with the training data set, and the training data set is The fourth learning model can be generated by machine learning the data. For example, it can be stored in the storage unit 23.

[0051] Furthermore, the processing unit 22 calculates a reference value for the first subject R1 based on the estimated reference sensory information. Corresponding sensory information, which is recalled sensory information corresponding to the sensory information, is estimated. In this case, the processing unit 22 can be estimated based on the second learning model stored in the storage unit 23.

[0052] The processing unit 22 also calculates the estimated corresponding sensory information by using the brain activation information of the second subject R2. When performing encryption, the processing unit 22 detects the image from the primary visual cortex of the second subject R2. More specifically, the processing unit 22 can use the brain activation information output from the first realization. Similarly to the embodiment, when the second subject R2 views the monotone striped image, The brain activation information detected from the primary visual cortex of the first subject R2 can be used. As in the embodiment, the processing unit 22 encrypts the data using a cryptographic key generated using, for example, brain activation information. Encryption can be performed.

[0053] The first device 10 includes a detection unit 11, a communication unit 12, a processing unit 13, a stimulus applying unit 14, and a storage unit. The detecting unit 11 and the communicating unit 12 have the same configuration as in the first embodiment. The communication unit 12 receives the corresponding sensory information transmitted from the estimation device 20.

[0054] The processing unit 13 decodes the corresponding sensory information received by the communication unit 12. The processing unit 13 performs, for example, The processing unit 22 of the estimation device 20 decrypts the corresponding sensory information using a decryption key corresponding to the encryption key. The processing unit 13 estimates stimulus image information according to the decoded corresponding sensory information. The stimulus image information is information indicating the stimulus content to be applied to the first subject R1 by the stimulus application unit 14. be.

[0055] The stimulation unit 14 irradiates the target area of ​​the brain of the first subject R1 with an electromagnetic wave signal to stimulate the target area. By activating the first subject R1, a stimulus is given to the first subject R1. As with the stimulation unit 34, the brain of the first subject R1 is represented by a voxel consisting of, for example, several millimeters or less. The stimulation unit 14 divides the area into three voxels by a three-dimensional matrix, and irradiates each voxel with electromagnetic waves. indicates the intensity of the electromagnetic waves to be irradiated to which voxel in the three-dimensional matrix. The electromagnetic waves can be emitted based on the stimulus image information. What kind of sensory information is evoked by irradiating a voxel with electromagnetic waves of what intensity? The correspondence can be determined by conducting experiments in advance. For example, Stimulus image information for the first subject R1 and a case where electromagnetic waves are irradiated based on the stimulus image information and the sensory information recalled by the first subject R1 are associated with each other to form a set of learning data sets. A fifth learning model can be generated by performing machine learning on the learning dataset. The fifth learning model can be stored in the storage unit 15 of the first device 10, for example.

[0056] The fourth and fifth learning models mentioned above are the first to third learning models. Similarly, it can be generated using a neural network represented by, for example, VGG16. When generating the fourth and fifth learning models, the learning dataset is The correlation of the training data set is input into a neural network and is then analyzed using machine learning such as deep learning. In other words, the neural network is optimized by learning to generate a learning model. Note that this is not limited to convolutional neural networks represented by VGG16, but can also be used with other A variety of neural networks may be used to generate the learning model.

[0057] Next, a method for transmitting sensation using the sensation transmission system 100 configured as described above will be described. The following describes a method for transmitting sensation from the first subject R1 to the second subject R2. The method is the same as that of the first embodiment. The case of transmitting sensation to the subject R1 will be described.

[0058] FIG. 6 is a diagram schematically illustrating an example of the operation of the sensation transmission system 100. In this way, the second subject R2 is made to perceive the reference sensory information. This will be explained using an example in which a person 2 visually perceives a cat's face and recalls it as visual information. In the following example, the cat's face is the same as in the first embodiment. R2 is made to visually perceive the monotone striped image IM2 for encryption.

[0059] As shown in the upper part of FIG. 6, the detection unit 31 perceives the face of a cat and recalls recalled sensory information 51. The detection unit 11 detects brain activation information 52 of the second subject R2. The brain activation information 58 of the first subject R1 that is recalled by the pattern image IM2 is detected. The unit 32 transmits the brain activation information 52, 58 detected by the detection unit 31 to the estimation device 20. .

[0060] In the estimation device 20, the communication unit 21 receives the brain activation information 5 transmitted from the second device 30. The processing unit 22 receives the reference sensory information 52 based on the received brain activation information 52. In this case, the processing unit 22 estimates the brain activation information 52 of the second subject R2 as a fourth learning. The fourth learning model inputs the activation information 52 and the reference sensory information 53 into the learning model. Based on the learning results of the correlation, the reference sensory information corresponding to the input brain activation information 52 is The processing unit 22 acquires the output reference sensory information 53 as an estimation result. do.

[0061] The processing unit 22 calculates the reference sensory information 53 based on the estimated reference sensory information 53. The processing unit 22 estimates the corresponding sensory information 54 corresponding to the quasi-sensory information 53. The reference sensory information 53 is input to the second learning model. Based on the learning result of the correlation between the input reference sensory information 53 and the corresponding sensory information 54, The processing unit 22 outputs corresponding sensory information 54 corresponding to the output corresponding sensory information 53. 4 as the estimation result. In addition, the processing unit 22 receives the brain activation information 52 separately. The communication unit 21 encrypts the corresponding sensory information 54 using the brain activation information 58. The converted corresponding sensory information 54 is transmitted to the first device 10.

[0062] As shown in the lower part of FIG. 6, in the first device 10, the communication unit 12 receives the signal transmitted from the estimation device 20. The processing unit 13 receives the transmitted corresponding sensory information 54. The processing unit 13 decodes the received corresponding sensory information 54. The combined corresponding sensory information 54 is input to the fifth learning model stored in the storage unit 15. 5 The learning model shows that the correlation between corresponding sensory information54 and stimulus image information55 is learned based on the results of the learning. The stimulus image information 55 corresponding to the input corresponding sensory information 54 is output. 4 is to irradiate the brain of the first subject R1 with electromagnetic waves based on the output stimulus image information 55. As a result, the first subject R1 to whom the stimulus is applied by the stimulus applying unit 14 The subject R1 recalls the corresponding sensory information 56 corresponding to the stimulus image information 55. The subject R1 recalls the visual information of the cat's face as the corresponding sensory information 56. Visual information of a cat's face is transmitted from subject R2 to first subject R1.

[0063] As described above, in the sensory transmission system 100 according to this embodiment, the reference sensory information is This is the recalled sensory information recalled by the second subject R2 who detected the brain activation information. The sensory information recalled by the second subject R2 is directly associated with the sensory information recalled by the first subject R1. By doing so, it is possible to appropriately transmit recalled sensory information between subjects with differences in brain activity. In addition, when the recalled sensory information is transmitted, the recalled information is activated in the brain of the second subject R2. Since the sensory information is encrypted, the subject's personal information, including the sensory information, is appropriately protected. It is possible.

[0064] [Third embodiment] Next, a third embodiment will be described. In the first and second embodiments, The sensory information recalled by the subject who detected the activation information was explained as the reference sensory information. In contrast, in the third embodiment, extraction is performed based on corresponding recalled sensory information among multiple subjects. This section explains the case where the standard sensory information is used as the reference sensory information. The overall configuration of the system 100 is the same as that of the first embodiment.

[0065] In the estimation device 20, when the corresponding sensory information is estimated from the reference sensory information, the reference sensory information is Standard sensory information is used as the information. For example, standard sensory information is used when multiple detectors detect the same image. When the same perception is performed, such as viewing an image, the brain activation information detected in multiple subjects is Therefore, the standard sensory information is based on acquired memory, etc. There is little individual variation, and it becomes the sensory information that the average person recalls.

[0066] Standard sensory information is used as a learning result when learning corresponding recall sensory information among multiple subjects. For example, a specific subject (e.g., the first subject R1 or the second subject R2) can be extracted. R2) and standard sensory information (average of brain activation information of multiple subjects). Each of these is used as a training dataset, and machine learning is performed on the training dataset to Six learning models can be generated. When generating the sixth learning model, the learning dataset Standard sensory information is extracted from the multiple sensory information contained in the It may be possible to associate the information with each piece of brain activation information contained in the training data set. As a result, for example, the visual information recalled when seeing a certain object, or the information recalled when hearing a certain sound Recollected sensory information about hearing when touching an object, and recollected sensory information about touch when touching an object We investigated the brain activation patterns of individual subjects for different types of recalled sensory information, such as The sixth learning model is generated by machine learning the correspondence between the information and the standard sensory information. The model can be stored in the storage unit 35, for example.

[0067] Next, a method for transmitting sensation using the sensation transmission system 100 configured as described above will be described. FIG. 7 is a diagram illustrating an example of the operation of the sensation transmission system 100. For example, the recollected sensory information 61 when the first subject R1 perceives is transmitted to the second subject R2. In this case, the first device 10 acquires the brain activation information 62 of the first subject R1, and the estimation device 2 0. In addition, in the first device 10, the image IM3 of the monotone striped pattern is Brain activation information 68 of the first subject R1 is detected.

[0068] In the estimation device 20, the processing unit 22 receives the brain activation information 6 transmitted from the first device 10. 2 and the identification information of the second subject R2 to whom the recalled sensory information is to be transmitted are stored in the storage unit 35. The sixth learning model uses the target corresponding to the brain activation information 62. The quasi-sensory information 63 is calculated, and the second subject R2's imagination associated with the standard sensory information 63 is The sensory information is output as corresponding sensory information 64. The processing unit 22 outputs the brain activation information 62 and The brain activation information 68 received separately is used to encode the corresponding sensory information 64 that is output. The communication unit 21 transmits the encrypted corresponding sensory information 64 to the second device 30.

[0069] In the second device 30, the corresponding sense transmitted from the estimation device 20 is Receive and decode information 64, and obtain stimulus image information 65 based on the decoded corresponding sensory information 64. The stimulation applying unit 34 applies an electric current to the brain of the second subject R2 based on the output stimulation image information 65. The second subject R2 is stimulated by irradiating the magnetic wave. The stimulation is given by the stimulation applying unit 34. The second subject R2 then recalls corresponding sensory information 66 corresponding to the stimulus image information 65.

[0070] As described above, in the sensory transmission system 100 according to this embodiment, the reference sensory information is Standard sensory information 63 is extracted based on corresponding recalled sensory information among multiple subjects. In this configuration, the standard sense extracted based on the corresponding recalled sensory information among multiple subjects is Since sensory information 63 is used as the reference sensory information, it is possible to accurately recall sensory information among many subjects. In addition, when the recalled sensory information is transmitted, the brain of the first subject R1 Since the activation information encodes the recalled sensory information, the recalled sensory information, which is the subject's personal information, is This allows for appropriate protection of information.

[0071] The technical scope of the present invention is not limited to the above-described embodiment, and any modifications may be made without departing from the spirit of the present invention. For example, in the above embodiments, the visual sense of recollection is Although the explanation has been given using information as an example, it is not limited to this, and the brain activation information can be detected as a recall sensation. As long as it is information, it may be other recollected sensory information. It may also be detailed recall sensory information obtained by moving.

[0072] Furthermore, the sensation transfer system 100 described in the above embodiment is When recalling sensory information to a first subject R1 at a second time point that is later in time than the first time point, The first device 10 can be used in cases where the first subject R1 has reference sensory information at a first time point. The estimation device 20 stores the reference sensory information in the storage unit 25. Then, the estimation device 20 calculates the following based on the reference sensory information of the first subject R1 at the first time point: The corresponding sensory information of the first subject R1 at the second time point is estimated, and the estimation result is transmitted to the first device 10. In the first device 10, the stimulus providing unit 14 transmits the estimated corresponding sensory information. This gives a stimulus to the first subject R1 so that the sensation of the first subject R1 at the first time point is In this case, the sensory transmission system 10 0 is the difference between the first subject R1 in the past (first time point) and the first subject R1 at the time of stimulus application (second time point). In other words, the recollection sensory information is transmitted between the same person. However, if there is no risk of unauthorized use such as eavesdropping, The first device 20 may transmit the recalled sensory information to the first device 10 without encrypting it. This allows for rationalization in terms of the processing time and power consumption required for encryption and decryption. [Explanation of symbols]

[0073] I1, I2... information, R1... first subject, R2... second subject, S1... convolutional layer, S2... Pooling layer, S3...connection layer, IM1, IM2, IM3...image, NW...neural network Workpiece, 10...first device, 11, 31...detection unit, 12, 21, 32...communication unit, 13, 22 ,33...processing unit, 14,34...stimulation unit, 15,23,35...storage unit, 16...VGG, 20...Estimation device, 30...Second device, 41, 47, 51, 61...Recall sensory information, 42, 48 ,52,58,62,68…Intracerebral activation information, 43,53…Reference sensory information, 44,46, 54, 56, 64, 66... ​​Corresponding sensory information, 45, 55, 65... Stimulus image information, 63... Standard sensation Sensory information, 100...sensory transmission system

Claims

1. Based on the brain activation information of the first subject when the first subject perceives the sensory information, the sensory information that a second subject different from the first subject recalls in response to the perception is estimated, and the sensory information that is recalled is encoded using the brain activation information of the first subject. Estimation device.

2. When performing the encryption, the brain activation information detected from the primary visual cortex of the first subject is used. The estimation device according to claim 1 .

3. When the encryption is performed, the brain activation information detected from the primary visual cortex of the first subject when the first subject views a monotone image with a striped pattern having a predetermined direction is used. The estimation device according to claim 2 .

4. The encryption is performed using a cryptographic key generated using the brain activation information. The estimation device according to any one of claims 1 to 3.

Images