Brain information transmission system

Abstract

To appropriately grasp the processing capabilities of brain information on each of the users in the case of intention transmission on the basis of brain information among a plurality of users.SOLUTION: A brain information transmission system comprises a plurality of BMI devices that perform intention transmission on the basis of brain information between a plurality of users. The brain information transmission system comprises a processing section that calculates the processing capabilities of users' brain information on the basis of levels of brain activity of the users when the brains of the users are commonly stimulated.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a brain information transmission system.

Background Art

[0002] In recent years, technologies for non-invasively measuring brain activation information such as functional magnetic resonance imaging and near-infrared spectroscopy have been developed, and the technology of BMI (Brain-Machine Interface), which is an interface between the brain and the outside, is becoming realistic. As an example using such technology, a thought transmission support device that calculates information for specifying visual stimuli imagined by a subject based on information including feature amounts derived from brain surface signals and feature amounts derived from intracranial signals is disclosed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When performing thought transmission based on brain information among a plurality of users, if the amount of information per unit time is too much for the processing ability of the user's brain when inputting brain information to the user's brain, real-time thought transmission may not proceed smoothly.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a brain information transmission system capable of appropriately grasping the processing ability of each user's brain information when performing thought transmission based on brain information among a plurality of users.

Means for Solving the Problems

[0006] The brain information transmission system according to the present invention is a brain information transmission system including a plurality of BMI devices that perform intention transmission based on brain information among a plurality of users, and includes a processing unit that calculates the processing ability of the brain information of the user based on the activity level of the user's brain when a common stimulus is applied to the user's brain.

Effect of the Invention

[0007] According to the present invention, when performing intention transmission based on brain information among a plurality of users, it is possible to appropriately grasp the processing ability of the brain information of each user.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the brain information transmission system according to the present invention will be described with reference to the drawings. Note that the present invention is not limited by this embodiment. Further, the constituent elements in the following embodiments include those that can be replaced and are easy for those skilled in the art, or those that are substantially the same.

[0010] FIG. 1 is a schematic diagram showing an example of a brain information transmission system 100 according to the present embodiment. FIG. 2 is a functional block diagram showing an example of the brain information transmission system 100. As shown in FIGS. 1 and 2, the brain information transmission system 100 performs intention transmission based on brain information among a plurality of users. The intention transmission based on brain information includes digitizing and transmitting the intentions, emotions, and sensations of the users. The brain information transmission system 100 includes a plurality of BMI devices 10 and a control device 20.

[0011] The BMI device 10 is an interface between the brain and the outside using the technology of BMI (Brain-Machine Interface). The BMI device 10 has a communication unit 11, a processing unit 12, and a storage unit 13. The communication unit 11 transmits and receives information. The communication unit 11 transmits various information including brain information acquired from the user's brain and information on the processing ability for the user's brain information described later to the control device 20. The communication unit 11 receives various information including brain information of other users transmitted from the control device 20. The brain information includes brain activation information and stimulation information described later.

[0012] The processing unit 12 controls the communication operation in the communication unit 11. The processing unit 12 acquires the brain information detected by the detection unit 14. In the present embodiment, the detection unit 14 detects, for example, brain activation information as the brain information. Examples of the brain activation information include the concentration of oxyhemoglobin, deoxyhemoglobin, and total hemoglobin contained in the cerebral blood flow of the user. As the detection unit 14, for example, a measuring device that performs measurement based on the principle of fMRI (functional Magnetic Resonance Imaging), fNIRS (functional Near-Infrared Spectroscopy), etc., a measuring device using invasive electrodes, a measuring device that arranges micromachines in the blood vessels of the brain and performs measurement by the micromachines, etc. can be used. Note that the detection unit 14 is not limited to the above devices, and other types of devices may be used. The brain activation information can be shown as the magnitude of the activation degree for each voxel when the user's brain is partitioned by a three-dimensional matrix composed of voxels of several millimeters or less.

[0013] Based on the acquired brain activation information, the processing unit 12 generates stimulation information for applying a stimulation to the brain of another user by the stimulation unit 15. For example, the processing unit 12 analyzes the brain activation information, extracts the user's intention, emotion, and sensation as data, and generates stimulation information for applying a stimulation that causes another user to recall the intention, emotion, and sensation. In this case, the correspondence between the brain activation information and the intention, emotion, and sensation, and the correspondence between the intention, emotion, and sensation and the stimulation information may be stored in the storage unit 13. Further, the processing unit 12 supplies the stimulation information extracted from the brain information of another user received by the communication unit 11 to the stimulation unit 15.

[0014] The stimulation application unit 15 applies a stimulation to the user by irradiating an electromagnetic wave signal to a target part of the user's brain to activate the target part. The stimulation application unit 15 divides the user's brain by, for example, a three-dimensional matrix composed of voxels of several millimeters or less, and irradiates electromagnetic waves for each voxel. The stimulation information includes information indicating which voxel in the three-dimensional matrix is irradiated with an electromagnetic wave of what intensity. The voxels in the three-dimensional matrix of the stimulation information may, for example, correspond in dimensions, positions, etc. to the voxels in the three-dimensional matrix of the brain activation information. Regarding which voxel in the user's brain is irradiated with an electromagnetic wave of what intensity and what kind of intention, emotion, and sensation are generated, a correspondence relationship can be obtained in advance by conducting experiments. For example, the stimulation information for the user and the intention, emotion, and sensation generated in the user's brain when an electromagnetic wave is irradiated based on the stimulation information are associated as a set of learning datasets, and a learning model obtained by machine learning the learning datasets can be generated. The learning model can be stored, for example, in the storage unit 13 of the BMI device 10.

[0015] In addition, the stimulation application unit 15 can apply a common stimulation to the user's brain in order to measure the processing ability of the user's brain information described later. As an example, the stimulation application unit 15 can apply a stimulation to the user by an auditory stimulation pulse. FIG. 3 is a diagram showing an example of an auditory stimulation pulse applied to the user. In FIG. 3, the horizontal axis represents time and the vertical axis represents the signal value. By applying an auditory stimulation pulse as shown in FIG. 3, the part of the user's brain related to hearing is activated. In this case, the detection unit 14 can acquire the brain activation information of the user. The stimulation application unit 15 may, for example, periodically apply an auditory stimulation pulse during a period when intention transmission is performed among a plurality of users. In this case, the processing ability of the user's brain information is periodically obtained.

[0016] When measuring the processing ability of the user, the processing unit 12 calculates the activity level of the user based on the brain activation information acquired from the detection unit 14. FIG. 4 is a diagram showing an example of the activity level of each user when the auditory stimulation pulses shown in FIG. 3 are given to different users. In FIG. 4, the horizontal axis represents time and the vertical axis represents the activity level. Note that the horizontal axis (time axis) of FIG. 4 coincides with the horizontal axis (time axis) of FIG. 3. As shown in FIG. 4, the processing unit 12 sets the relaxed state without being stimulated as the steady state of the user. The processing unit 12 can calculate the processing ability of the user based on the reference value S of the activity level in the steady state and the peak value of the activity level when the auditory stimulation is applied. In the example of User 1 shown in FIG. 4(a), the peak value of the activity level is A1, in the example of User 2 shown in FIG. 4(b), the peak value of the activity level is A2, and in the example of User 3 shown in FIG. 4(c), the peak value of the activity level is A3. Such peak values of the activity level can be obtained in advance as values indicating the processing ability and stored in the storage unit 16. In this case, the processing unit 12 can easily calculate the processing ability corresponding to the activity level by using the data stored in the storage unit 16. Hereinafter, an example of the determination of the processing ability in the processing unit 12 will be described. For example, User 1 and User 2 have an increasing activity level at the same timing. When calculating the processing ability based on the peak value of the activity level, since the peak value A1 of User 1 is larger than the peak value A2 of User 2, it can be determined that User 1 has a higher processing ability.

[0017] In addition, the processing unit 12 can calculate the processing ability of the user based on the speed of the start time of the increase in activity. For users 1 and 2, user 3 has a lower peak value of activity, while the activity increases at an earlier timing on the time axis. Specifically, the activity increases with a small pulse before the maximum amplitude of the auditory stimulation pulse. From this, when calculating the processing ability based on the speed of the reaction to the stimulation, it can be determined that user 3 has a higher processing ability than users 1 and 2. Note that the processing ability may be calculated based on either the peak value or the speed of the start time of the increase, or it may be calculated based on an index calculated from both. Also, the common stimulation for measuring the processing ability of the user's brain information is not limited to auditory stimulation, and may be other stimuli such as visual stimulation.

[0018] The processing unit 12 causes the communication unit 11 to transmit the obtained value of the processing ability. The processing unit 12 can cause the obtained processing ability to be transmitted to the control device 20 by, for example, the communication unit 11. In this case, in the control device 20, the transmission speed of the brain information is set based on the processing ability for each user, as will be described later. The set transmission speed is transmitted from the control device 20 to each BMI device 10. Also, the processing unit 12 may cause the obtained processing ability to be broadcast to each BMI device 10 by, for example, the communication unit 11. When broadcasting the processing ability, the processing unit 12 can use H.323, which is a protocol for real-time audio and video communication. In this case, the processing ability of the brain information of each user can be broadcast using a TCS (Terminal Capability Set), which is a communication control message for mutual confirmation of the capabilities of the terminals of each BMI device 10.

[0019] The processing unit 12 controls the transmission operation of brain information to be transmitted from the communication unit 11 so as to correspond to the set value of the transmission speed transmitted from the control device 20. The processing unit 12 can control the transmission speed, for example, by inserting invalid information into the brain information to be transmitted and received. When invalid information is inserted into and transmitted and received for the brain information, the amount of information of the brain information transmitted and received per unit time decreases. Therefore, it can be easily set to the transmission speed of a user with low processing power. The processing unit 12 causes the communication unit 11 to transmit the generated stimulation information to the control device 20 as brain information.

[0020] The storage unit 13 stores various types of information. The storage unit 13 has a storage such as a hard disk drive or a solid state drive, for example. Note that an external storage medium such as a removable disk may be used as the storage unit 13. The storage unit 13 stores, for example, the processing power of the brain information of each user, which is transmitted from the control device 20 or broadcast from the communication unit 11 in each user's BMI device 10.

[0021] The control device 20 controls a plurality of BMI devices 10. The control device 20 includes a communication unit 21, a communication control unit 22, a setting unit 23, and a storage unit 24. The communication unit 21 is an interface that performs wired communication or wireless communication. The communication unit 21 receives the brain information transmitted from each BMI device 10 and the value of the processing power of the user. The communication unit 21 transmits the received brain information to the corresponding BMI device 10.

[0022] The communication control unit 22 controls the communication operation in the communication unit 21.

[0023] The setting unit 23 causes the brain information received by the communication unit 21 to be transferred to another BMI device 10 different from the BMI device 10 that is the transmission source.

[0024] The setting unit 23 sets the transmission speed of brain information between the BMI devices 10 based on the values of the processing capabilities of each user received by the communication unit 21. A user with a relatively high value of the processing capability of brain information is presumed to be able to process a relatively large amount of brain information per unit time. Also, a user with a relatively low processing capability of brain information is presumed to be able to process a relatively small amount of brain information per unit time. When inputting brain information to a user's brain, if the amount of information per unit time is too large compared to the processing capability, real-time processing may not proceed smoothly. In the present embodiment, the setting unit 23 can set the transmission speed so that the amount of brain information transmitted per unit time is the same, thereby setting a transmission speed at which all users can process smoothly in real time. The setting unit 23 may set the transmission speed corresponding to the user with the lowest value of the processing capability, or may set a transmission speed lower than the transmission speed corresponding to the user with the lowest value of the processing capability.

[0025] The storage unit 24 stores various information. The storage unit 24 has a storage such as a hard disk drive, a solid state drive, etc. For the storage unit 24, an external storage medium such as a removable disk may be used.

[0026] Next, an example of the operation of the brain information transmission system 100 configured as described above will be described. In the following example, the case where intention transmission based on brain information is performed between two users, the first user A and the second user B, will be described as an example. Note that the same description is possible for the case where intention transmission based on brain information is performed between three or more users.

[0027] First, in each BMI device 10, the processing unit 12 acquires the brain activity levels of each user when a common stimulus is applied to the first user A and the second user B. The processing unit 12 can perform the application of the stimulus to the first user A and the second user B in advance. Based on the acquired activity levels, the processing unit 12 calculates the processing ability of the user's brain information. The processing unit 12 causes the communication unit 11 to transmit the calculated processing ability value, which is the obtained result, to the control device 20. Note that the processing unit 12 may also broadcast and distribute the calculated processing ability value, which is the obtained result, to other BMI devices 10 via the communication unit 11.

[0028] In the control device 20, the communication unit 21 receives the processing ability values acquired from each BMI device 10. The setting unit 23 sets the transmission speed of brain information between the two BMI devices 10 based on the received processing ability values. In this case, the setting unit 23 may set it to the transmission speed corresponding to the user with the lowest processing ability value, or may set it to a transmission speed lower than the transmission speed corresponding to the user with the lowest processing ability value. The setting unit 23 causes the communication unit 21 to transmit the set value of the transmission speed to each BMI device 10.

[0029] The processing unit 12 generates stimulus information to be transmitted to the control device 20 based on the set value of the transmission speed transmitted from the control device 20. FIG. 5 is a diagram schematically showing an example of the stimulus information generated in the processing unit 12. As shown in FIG. 5, the processing unit 12 packetizes the stimulus information for each sense such as vision, audition, touch, taste, smell, and somatosensation. The size of each packet can be a preset value. For example, a vision packet includes a packet header, vision data, and invalid data. The packets for other senses can have a similar configuration.

[0030] The processing unit 12 can generate the stimulation information of a plurality of sensations as one piece of information by time-division multiplexing the packets generated for each of the plurality of sensations. When multiplexing the stimulation information, the processing unit 12 can use a technique similar to the multiplexing standard when packetizing, for example, an elementary stream of MPEG (Moving Picture Experts Group). In this case, the processing unit 12 preferably performs multiplexing in, for example, the program stream format, but may also perform multiplexing in the transport stream format.

[0031] The processing unit 12 synchronizes the stimulation information packetized for each sensation with a signal such as a reference clock. At this time, the processing unit 12 adjusts the amount of information of the stimulation information transmitted per unit time based on the set value of the transmission rate received from the control device 20. FIG. 6 is a diagram schematically showing an example of the control of the transmission operation of brain information. FIG. 6(a) shows an example of the transmission rates and transmission times of the first user A and the second user B. As shown in FIG. 6(a), it is assumed that the first user A has a processing capacity corresponding to the transmission rate F A and the second user B has a processing capacity corresponding to the transmission rate F B . Let the transmission rates F A , F B be calculated based on the transmission times T A , T B when transmitting N packets P of the same amount of information. Here, T A >T B . That is, regarding the transmission rates, F A <F B , and it is assumed that the second user B has a higher processing capacity than the first user A. In this case, in the control device 20, the transmission rate is set so that the amount of brain information transmitted per unit time is the same according to the processing capacity of the first user A, which has a relatively low processing capacity.

[0032] FIG. 6(b) shows an example of the control of the transmission operation when the set value of the transmission rate is F A . As shown in FIG. 6(b), when the set value of the transmission rate is F AIn the case where it is, in the BMI device 10 used by the first user A, brain information can be transmitted without inserting invalid information. Also, in the BMI device 10 used by the second user B, invalid information such as null packets NP is inserted so that the transmission speed becomes F A In the BMI device 10 used by the second user B, as shown in the middle section (B1) of FIG. 6(b), invalid data may be inserted as invalid information within one packet. Also, as shown in the lower section (B2) of FIG. 6(b), null packets may be inserted as packet unit invalid information between a plurality of packets P.

[0033] FIG. 6(c) shows an example of the control of the transmission operation when the set value of the transmission speed is F A lower than F C In the case where it is, as shown in FIG. 6(c), when the set value of the transmission speed is F A lower than F C in the BMI device 10 used by the first user A and the BMI device 10 used by the second user B, invalid information such as null packets NP and invalid data is inserted so that the transmission speed becomes F C respectively.

[0034] The processing unit 12 can control the transmission speed, for example, by inserting invalid information such as null packets between packets P of the brain information to be transmitted. When invalid information is inserted into the brain information for transmission and reception, the amount of brain information transmitted and received per unit time decreases. Therefore, it can be easily set to the transmission speed of a user with low processing power.

[0035] The processing unit 12 only needs to adjust the transmission speed of the entire brain information to be transmitted. If the transmission speed of the entire brain information becomes an appropriate transmission speed, the processing unit 12 can appropriately set, for example, the timing of inserting invalid information packets and the amount of invalid information per packet. The processing unit 12 causes the generated brain information to be transmitted from the communication unit 11 to the control device 20.

[0036] In the control device 20, brain information is received by the communication unit 21. The received brain information is transferred by the communication control unit 22 to other BMI devices 10.

[0037] In other BMI devices 10, the brain information transferred by the control device 20 is received by the communication unit 11. As a result, the brain information is transmitted between the BMI devices 10 at a transmission speed corresponding to the set value of the transmission speed set in the control device 20. The processing unit 12 supplies the received brain information to the stimulation unit 15. The stimulation unit 15 applies a stimulation corresponding to the supplied brain information to the second user B. The second user B can experience the intention, emotion, sensation, etc. recalled by the first user A by the stimulation from the stimulation unit 15.

[0038] FIG. 7 is a flowchart showing the flow of operations in the brain information transmission system 100. As shown in FIG. 7, the processing unit 12 of each BMI device 10 calculates the processing ability of the user's brain information based on the brain activity of the user when a common stimulation is applied to the user's brain (step S101).

[0039] Next, the setting unit 23 of the control device 20 sets the transmission speed of the brain information between the plurality of BMI devices 10 based on the processing ability of the user's brain information (step S102).

[0040] Next, in each BMI device 10, the processing unit 12 controls the transmission operation according to the set transmission speed, for example, by generating brain information with inserted invalid information and transmitting the generated brain information (step S103).

[0041] As described above, the brain information transmission system 100 according to the present embodiment is a brain information transmission system 100 including a plurality of BMI devices 10 that perform intention transmission based on brain information among a plurality of users, and includes a processing unit 12 that calculates the processing ability of the user's brain information based on the brain activity of the user when a common stimulation is applied to the user's brain.

[0042] According to this configuration, in order to calculate the processing ability of the user's brain information based on the activity of the user's brain when a common stimulus is applied to the user's brain, when performing information transmission based on brain information among a plurality of users, the processing ability of each user's brain information can be appropriately grasped.

[0043] The brain information transmission system 100 according to the present embodiment further includes a setting unit 23 that sets the transmission speed of brain information among a plurality of BMI devices 10 based on the processing ability of the user's brain information. According to this configuration, since the transmission speed of brain information among a plurality of BMI devices 10 is set based on the processing ability of the user's brain information, even when there is a difference in the processing ability of brain information among users, real-time information transmission can be smoothly performed.

[0044] In the brain information transmission system 100 according to the present embodiment, the plurality of BMI devices 10 insert invalid information into the brain information according to the transmission speed set by the setting unit 23. According to this configuration, when there is a difference in the processing ability of brain information among users, the transmission speed can be easily adjusted.

[0045] In the brain information transmission system 100 according to the present embodiment, the brain information is information about a plurality of sensations that stimulate the user's brain, and the plurality of BMI devices 10 multiplex and synchronize the information from the plurality of sensations. According to this configuration, the brain information including the information from the plurality of sensations can be efficiently transmitted.

[0046] In the brain information transmission system 100 according to the present embodiment, the plurality of BMI devices 10 acquire the processing capabilities of the respective users from each other at the start of information transmission. According to this configuration, each BMI device 10 can acquire the processing capabilities of other users.

[0047] The technical scope of the present invention is not limited to the above embodiments, and appropriate modifications can be made without departing from the spirit of the present invention. For example, in the above embodiments, when calculating the processing ability of the user's brain information, an example of applying an auditory stimulus to the user was given for explanation, but it is not limited to this. For example, as the stimulus applied to the user, a stimulus corresponding to the sensation that the user recalls may be applied to calculate the processing ability.

[0048] Also, in the above embodiments, the case where each BMI device 10 calculates the processing ability was taken as an example for explanation, but it is not limited to this. For example, the control device 20 may transmit a common stimulus to each BMI device 10, each BMI device 10 may acquire the activity level and transmit it to the control device 20, and based on the activity level transmitted from the BMI device 10, the control device 20 may calculate the processing ability of each user.

[0049] Also, in the above embodiments, when adjusting the transmission speed, the case of inserting invalid information into the brain information was taken as an example for explanation, but it is not limited to this, and the transmission speed of the entire brain information may be adjusted by transmitting only a part of the brain information. FIG. 8 is a diagram for explaining another example of the operation of adjusting the transmission speed. FIGS. 8(a) and 8(b) show the relationship between the activity level of the brain information and time, where the vertical axis represents the activity level and the horizontal axis represents time.

[0050] As shown in FIG. 8(a), the BMI device 10 can extract and transmit, for example, a part of the brain information where the activity level value is equal to or greater than the threshold S1 (the hatched part in FIG. 8(a)). In other words, the BMI device 10 can thin out and transmit a part of the brain information where the activity level value is less than the threshold S1. It can be said that the part with a high activity level in the brain information is the part with a high intensity of sensations, emotions, etc. recalled by the user. Therefore, by extracting or thinning out information using the threshold S1 of the activity level, it is possible to transmit a rough sensation with fine parts omitted while suppressing the transmission amount of the brain information and share it with other users.

[0051] Also, as shown in FIG. 8(b), the BMI device 10 can extract and transmit, for example, a portion of brain information where the activity value is equal to or less than the threshold value S2 (the hatched portion in FIG. 8(b)). In other words, the BMI device 10 can thin out and transmit a portion of brain information where the activity value exceeds the threshold value S2. In the example shown in FIG. 8(b), the change in amplitude equal to or greater than the threshold value S2 is smoothed, but the configuration is not limited to this, and the whole may be smoothed. In this way, by extracting or thinning out information using the activity threshold value S2, it is possible to transmit a feeling that suppresses the amount of transmitted brain information and omits hypersensitive reactions, and share it with other users.

[0052] In the above, as an example of the transmission operation of transmitting only a part of brain information according to the set transmission speed, a configuration of extracting or thinning out information using the activity threshold value of brain information has been described, but it is not limited to this, and information may be extracted or thinned out in other modes.

[0053] Also, in the above embodiment, the stimulation unit 15 may evoke thoughts, emotions, and sensations by a method other than irradiating an electromagnetic wave signal to the target site of the user's brain to activate the target site. Stimulation information such as visual, auditory, tactile, gustatory, olfactory, and somatosensory information may be evoked by a device that presents each sensation. For example, it may be a display that presents an image corresponding to visual stimulation information, a speaker that presents a voice corresponding to auditory stimulation information, and the like. An auditory stimulation pulse for measuring the processing ability of the user's brain information may be applied as a stimulus to the user's ear from a sound emitting device such as a speaker or headphones.

Explanation of Reference Numerals

[0054] NP… Null Packet, P… Packet, 10… BMI Device, 11, 21… Communication Unit, 12… Processing Unit, 13, 16, 24… Storage Unit, 14… Detection Unit, 15… Stimulation Unit, 20… Control Device, 22… Communication Control Unit, 23… Setting Unit, 100… Brain Information Transmission System

Claims

1. A brain information transmission system comprising a plurality of BMI devices for performing intention transmission based on brain information among a plurality of users, the system comprising a processing unit configured to calculate the processing ability of the brain information of the user based on the brain activity of the user when a common stimulus is applied to the brain of the user. Brain information transmission system.

2. The system further comprising a setting unit configured to set the transmission speed of the brain information among the plurality of BMI devices based on the processing ability of the brain information of the user. The brain information transmission system according to claim 1.

3. The plurality of BMI devices insert invalid information into the brain information according to the transmission speed set by the setting unit. The brain information transmission system according to claim 2.

4. The plurality of BMI devices transmit only a part of the brain information according to the transmission speed set by the setting unit. The brain information transmission system according to claim 2.

5. The brain information is information about a plurality of sensations that stimulate the user's brain, and the plurality of BMI devices multiplex and synchronize the information based on the plurality of sensations. The brain information transmission system according to any one of claims 2 to 4.

Images