Information processing device, method for controlling information processing device, and program

Abstract

The purpose of the present invention is to provide an information processing device capable of easily extracting brain wave data corresponding to a human activity designated by a user from collected brain wave data.　An information processing device 1 comprises: a brain wave database 90 as an information storage unit that stores brain wave-related information in which a human activity, intracranial brain wave data synchronized with the human activity, and an intracerebral region where the intracranial brain wave data has been detected are associated with one another; an input processing unit 34 that receives input of a search condition including at least one of a human activity, intracranial brain wave data, and an intracerebral region; an information search unit 35 that searches the brain wave database 90 for corresponding brain wave-related information on the basis of the search condition input to the input processing unit 34; and an output processing unit 36 that outputs the brain wave-related information retrieved by the information search unit 35.

Description

Information processing apparatus, control method for information processing apparatus, and program ,

[0006] ,

[0005] ,

[0001] The present invention relates to an information processing apparatus, a control method for the information processing apparatus, and a program.

[0002] Conventionally, in order to investigate the relationship between the state of a subject and brain waves, a technique for measuring brain wave data by arranging electrodes inside the head of the subject is known. For example, an epileptic patient may perform a predetermined action (human activity) such as flapping or stiffening the right hand when an epileptic seizure occurs. The brain waves when an epileptic patient performs a predetermined action are measured and used for clinical diagnosis. As a technique related to this type of technique, for example, Patent Document 1 has been proposed.

[0003] Patent Document 1 relates to an apparatus for determining epilepsy. Patent Document 1 describes an epilepsy determination apparatus including an acquisition unit that acquires brain wave data of a subject, a generation unit that cuts out the brain wave data at a predetermined time width and generates a plurality of images representing the brain waves, and a determination unit that inputs the plurality of images into a learned model and causes the learned model to determine whether an epileptic seizure brain wave appears in any of the plurality of images.

[0004] Japanese Unexamined Patent Application Publication No. 2019-162237

[0005] By the way, since intracranial brain wave data contains more information related to brain functions than scalp brain wave data, application to fields other than the treatment of epilepsy is expected. For example, brain wave data can be utilized in various fields such as prosthetic hands and prosthetic feet that operate corresponding to the brain wave data of the wearer. However, analysis of brain wave data requires collection of a huge amount of data. Also, even if it is known that the brain waves measured by a specific electrode have changed when the action of raising the right hand is performed, if the position of the electrode and the brain waves are not associated, it cannot be determined whether the intracranial brain wave data corresponds to a specific action. In addition, the work of extracting necessary data from the huge amount of intracranial brain wave data collected is very laborious. There was room for improvement in the conventional technology in terms of utilization of brain wave data. <0000This invention has been made in view of the above circumstances, and aims to provide an information processing device, a control method for the information processing device, and a program that can easily extract brainwave data corresponding to the activity of a person specified by the user from collected brainwave data.

[0007] To achieve the above objective, one aspect of the present invention is an information processing device comprising: an information storage unit that stores brainwave-related information associating human activity, intracranial brainwave data synchronized with the human activity, and brain regions where the intracranial brainwave data was detected; an input processing unit that receives input of a search condition including at least one of the human activity, the intracranial brainwave data, and the brain region; an information retrieval unit that searches the information storage unit for the corresponding brainwave-related information based on the search condition input to the input processing unit; and an output processing unit that outputs the brainwave-related information retrieved by the information retrieval unit.

[0008] Furthermore, one aspect of the present invention is an information processing device comprising an electroencephalogram-related information creation unit that creates electroencephalogram-related information relating to the human activity, the intracranial electroencephalogram data synchronized with the human activity, and the brain region where the intracranial electroencephalogram data was detected, based on activity information relating to the human activity, intracranial electroencephalogram data associated with the activity information, and brain region information relating to the brain region corresponding to the placement position of electrode members placed inside the skull, and registers this information in an information storage unit.

[0009] Furthermore, one aspect of the present invention is a control method for an information processing device that stores electroencephalogram-related information relating human activity, intracranial electroencephalogram data synchronized with the human activity, and brain regions where the intracranial electroencephalogram data was detected, the control method for an information processing device comprising: an input processing step of receiving input of a search condition including at least one of the human activity, the intracranial electroencephalogram data, and the brain region; an information retrieval step of searching for the corresponding electroencephalogram-related information from the electroencephalogram-related information stored in the information processing device based on the search condition input in the input processing step; and an output processing step of outputting the electroencephalogram-related information retrieved in the information retrieval step.

[0010] Furthermore, one aspect of the present invention is a program for causing a computer to execute the following as processing for an information processing device that stores brainwave-related information relating human activity, intracranial brainwave data synchronized with the human activity, and brain regions where the intracranial brainwave data was detected: an input processing step that accepts input of a search condition including at least one of the human activity, the intracranial brainwave data, and the brain region; an information retrieval step that searches for the corresponding brainwave-related information stored in the information processing device based on the search condition input in the input processing step; and an output processing step that outputs the brainwave-related information retrieved in the information retrieval step.

[0011] According to the present invention, it is possible to provide an information processing device, a control method for the information processing device, and a program that can easily extract brainwave data corresponding to the activity of a person specified by the user from collected brainwave data.

[0012] This figure shows an electroencephalogram (EEG) data retrieval system to which an information processing device according to one embodiment of the present invention is applied. This is a schematic diagram showing the configuration of an EEG detection unit that measures EEG data to be registered in the information processing device according to this embodiment. This is a block diagram showing the hardware configuration of the information processing device according to this embodiment. This is a functional block diagram showing an example of the functional configuration of the information processing device according to this embodiment. This is a flowchart showing an example of the data registration process flow by the information processing device according to this embodiment. This is a table showing an example of EEG-related information to be registered in an EEG database. This is a flowchart showing an example of the information retrieval process flow by the information processing device according to this embodiment. This figure shows an example of search condition input items displayed in menu format. This figure shows an example of a registration screen for cranial EEG data and MRI data. This figure shows an example of a confirmation screen for registered video data and intracranial EEG data.

[0013] The following describes one embodiment of the present invention with reference to the drawings. <System Configuration> First, the overall system configuration will be described. Figure 1 is a diagram showing an electroencephalogram (EEG) data retrieval system 100 to which an information processing device 1 according to one embodiment of the present invention is applied. The EEG data retrieval system 100 provides a comprehensive brain function database that includes disease information and brain images, focusing on the simultaneous recording of human activity and high-quality intracranial EEG data from the deep and superficial parts of the brain. In this specification, "human activity" includes, for example, movements, actions, conversations, facial expressions, thoughts, etc. Of these, movements are included as one aspect of action in a broad sense, but in this specification, the term is used not only in this broad sense but also in a narrow sense where each is considered an individual aspect.

[0014] The electroencephalogram (EEG) data retrieval system 100 is implemented by an information processing device 1 that transmits and receives various information to and from a retrieval terminal 3 via a communication network N such as the Internet. The information processing device 1 is a computer that provides various information related to intracranial EEG data to the retrieval terminal 3 and functions as a server.

[0015] The electroencephalogram (EEG) data acquisition device 2 is a computer used by the data acquirer to acquire intracranial EEG data registered in the information processing device 1. The EEG data acquisition device 2 is composed of, for example, a personal computer or a tablet. The EEG data acquisition device 2 aggregates intracranial EEG data acquired from the EEG detection unit 51 and video data acquired by the imaging unit 52.

[0016] The electroencephalogram (EEG) detection unit 51 is a device that detects brain waves from within the subject's skull. The EEG detection unit 51 is, for example, an electrode that can be placed at any position within a blood vessel. The EEG detection unit 51 acquires intracranial EEG data in a time series of 1000 Hz to 2000 Hz. The EEG detection unit 51 is placed at multiple locations on the subject's head. The intracranial EEG data acquired by each of the multiple EEG detection units 51 is registered in the EEG data acquisition device 2 in association with its placement location. For example, the intracranial EEG data from the EEG detection unit 51 placed at the first location is acquired as intracranial EEG data for brain region A corresponding to the placement location. Similarly, the intracranial EEG data from the EEG detection unit 51 placed at the second location is acquired as intracranial EEG data for brain region B corresponding to the placement location. Similarly, the intracranial EEG data from the EEG detection unit 51 placed at the third location is acquired as intracranial EEG data for brain region C corresponding to the placement location. Although Figure 1 shows three electroencephalogram (EEG) detection units 51 as an example, the number of EEG detection units 51 can be set arbitrarily, and the number of brain regions acquired will also correspond to the position of the EEG detection units 51.

[0017] Figure 2 is a schematic diagram showing the configuration of an electroencephalogram (EEG) detection unit 51 that measures intracranial EEG data registered in the information processing device 1 according to this embodiment. As shown in Figure 2, the EEG detection unit 51 is formed in the shape of a flexible, elongated string. The first end 51a of the EEG detection unit 51 is placed inside a biological blood vessel (typically a cerebral blood vessel), and the second end 51b is electrically connected to a measuring instrument or oscillator, etc. (not shown). The EEG detection unit 51 is inserted into a cerebral vein, for example, through a catheter used in endovascular surgery.

[0018] The electroencephalogram (EEG) detection unit 51 comprises a wire member 510 and an electrode member 511. The wire member 510 extends from the first end 51a to the second end 51b of the EEG detection unit 51 and is made of a conductive metal. The surface of the wire member 510 is coated with an insulating film and a biocompatible coating. The electrode member 511 is provided around the wire member 10 on the first end 51a side and is electrically connected to the wire member 510. The electrode member 511 is made of a conductive metal such as Pt (platinum) and is exposed on the surface to detect weak currents such as brain waves generated in the body of humans or animals, or to transmit currents emitted by an oscillator into the body.

[0019] In this embodiment, the electroencephalogram detection unit 51 has a simple configuration in which an electrode member 511 is placed at the end of a wire member 510, and is implanted inside a blood vessel. By implanting the electroencephalogram detection unit 51 in a deep cerebral vein, high-quality electroencephalograms can be obtained minimally invasively. Intracranial electroencephalogram data may be transmitted by wire only to a measuring instrument or oscillator (not shown), or it may be transmitted by outputting from inside the skin to the outside using short-range wireless communication such as Bluetooth®, or it may be transmitted by a combination of wireless and wired methods.

[0020] The electroencephalogram (EEG) detection unit 51 is implanted in the subject for a predetermined period, such as 1-2 weeks or 2-4 weeks, and the subject's intracranial EEG data is acquired in real time 24 hours a day. During this predetermined period, the subject will perform various physical movements and experience various emotional changes. When such physical movements or emotional changes occur, the system can collect data on which areas of the brain produce what kind of intracranial EEG data for the predetermined period. The EEG detection unit 51 is retrieved from the subject's brain after the predetermined period has elapsed. Changes in the subject's emotions can be detected, for example, by changes in the subject's emotions, pulse rate, etc.

[0021] Furthermore, the electroencephalogram (EEG) detection unit 51 only needs to be able to acquire intracranial EEG data from the subject, and the intracranial EEG data detected by the EEG detection unit 51 is not limited to data acquired by the EEG detection unit 51 placed inside a blood vessel. For example, a configuration may be adopted in which a stent is placed at the tip of a wire, and an electrode is placed at the tip of this stent, and the subject's head is opened surgically and the EEG detection unit is implanted inside the skull to acquire intracranial EEG data. Alternatively, the EEG detection unit 51 may be configured to acquire intracranial EEG data using intracranial electrodes (subdural electrodes, deep electrodes) placed inside the skull.

[0022] The imaging unit 52 is a video camera that images the subject during the acquisition of intracranial electroencephalogram (EEG) data. Imaging by the imaging unit 52 continues while intracranial EEG data is being acquired. The video data also includes audio information such as the subject's speech.

[0023] The electroencephalogram (EEG) data acquisition device 2 registers the intracranial EEG data acquired from each of the multiple EEG detection units 51 in correspondence with the time of the video data.

[0024] The search terminal 3 is an information processing device used by users to search for intracranial electroencephalogram (EEG) data. The search terminal 3 consists of a personal computer, tablet, smartphone, etc. The search terminal 3 may exchange various types of information with the information processing device 1 using a pre-installed program, or it may exchange various types of information through a web browser.

[0025] <Hardware Configuration> Next, an example of the hardware comprising the information processing device 1 will be described. Figure 3 is a block diagram showing the hardware configuration of the information processing device 1 according to this embodiment. The information processing device 1 includes a CPU (Central Processing Unit) 11 as a processor, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20.

[0026] The CPU 11 executes various processes according to the program recorded in the ROM 12 or the program loaded from the storage unit 18 into the RAM 13. The RAM 13 also stores data necessary for the CPU 11 to execute various processes. The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output interface 15 is also connected to this bus 14.

[0027] The input / output interface 15 is connected to an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20. The output unit 16 consists of a display, speakers, etc., and outputs various information as images and sounds. The input unit 17 consists of a keyboard, mouse, etc., and inputs various information. The storage unit 18 consists of a hard disk, DRAM (Dynamic Random Access Memory), etc., and stores various data. The communication unit 19 communicates with other devices via a network, including the Internet.

[0028] A removable media 21, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately mounted on the drive 20. Programs read from the removable media 21 by the drive 20 are installed in the storage unit 18 as needed. The removable media 21 can also store various data stored in the storage unit 18, just like the storage unit 18.

[0029] The hardware configuration described here is merely an example. The computer described in this embodiment, including the information processing device 1, may have the same configuration as that shown in Figure 3, or it may have a different configuration. Furthermore, the computer may consist of two or more computers. The electroencephalogram data acquisition device 2 and the search terminal 3 in Figure 1 are, for example, personal computers having a configuration similar to the hardware configuration shown in Figure 3.

[0030] <Functional Configuration> Next, the functional configuration of the information processing device 1 will be described. Figure 4 is a functional block diagram showing an example of the functional configuration of the information processing device 1 according to this embodiment.

[0031] As shown in Figure 4, the information processing device 1 includes a data acquisition unit 31, an analysis processing unit 32, an electroencephalogram-related information creation unit 33, an input processing unit 34, an information retrieval unit 35, and an output processing unit 36, all of which are functional units implemented on a processor (CPU 11). Furthermore, the information processing device 1 has an electroencephalogram database 90 for registering various types of information related to brain waves. The electroencephalogram database 90 may be physically located outside the information processing device 1, and various types of information may be read via a communication network N.

[0032] <Data Registration Process> Referring to Figure 5, the data acquisition unit 31, analysis processing unit 32, and brainwave-related information creation unit 33, which function to register brainwave-related information in the brainwave database 90, will be described. Figure 5 is a flowchart showing an example of the data registration process flow by the information processing device 1 according to this embodiment.

[0033] In step S1, the data acquisition unit 31 acquires information about brain waves for registration in the brainwave database 90. The data acquisition unit 31 may acquire information about brain waves by receiving it from the brainwave data acquisition device 2 via a communication network N such as the Internet, or it may acquire information about brain waves that is input via a recording medium such as memory, hard disk, CD, or DVD.

[0034] The information related to electroencephalography (EEG) includes intracranial EEG data for each region detected by the EEG detection unit 51, and video data of the subject corresponding to the intracranial EEG data. Furthermore, the EEG information may also include brain imaging data such as CT (radiography) and MRI (magnetic resonance imaging).

[0035] Furthermore, the electroencephalogram (EEG) information may include type information indicating the circumstances and purpose of acquiring the EEG data. This type information could include, for example, the symptoms of the subject's illness, such as epilepsy, intracranial hemorrhage, depression, or Parkinson's disease. However, the type information is not limited to disease symptoms; it could also specify tests for creating prosthetic arms or legs that operate in response to EEG, or control tests for exoskeleton robots and prosthetic limbs based on intravascular electrodes. For example, a prosthetic leg control test could be conducted to demonstrate a proof-of-concept for achieving walking assistance by autonomously controlling the exoskeleton using higher-order commands (walk initiation and cessation) decoded from intravascular electrodes. While walking involves a complex mechanism of coordinated peripheral movements, it can be said that it is fundamentally governed by motor intention (higher-order motor commands). This enables new research that acquires motor intention from deep brain electrodes and uses it to control robots.

[0036] In step S2, the analysis processing unit 32 performs image analysis to analyze human movements (activities) from the video data. The analysis processing unit 32 analyzes human movements at predetermined times in the video data and automatically determines tags that indicate what kind of movements the person is performing at those predetermined times. Tags are movement information (activity information) related to human movements, and are expressed as text that describes human movements in words or sentences. For example, if a subject with symptoms of tantrums is flapping their right hand, the analysis processing unit 32 sets the text "movement of raising and lowering the right hand" as a tag and associates it with the predetermined time. Also, if a healthy subject is holding an apple in one hand, the analysis processing unit 32 sets text such as "movement of raising the right hand" or "movement of lifting an object with the right hand" as a tag and associates it with the predetermined time. Furthermore, image analysis may be performed to analyze changes in human emotions (changes in facial expressions, etc.) from the video data. For example, the analysis processing unit 32 may set text such as "facial expression showing pain" as a tag and associate it with the predetermined time to represent changes in emotions when a person feels pain.

[0037] For example, machine learning can be used for image analysis in the analysis processing unit 32. The analysis processing unit 32 can be configured to perform image analysis using a learning model constructed by supervising learning on a dataset in which training video data of a specific human action (e.g., raising the right hand) is used as raw data, and the specific human action is labeled (e.g., raising the right hand). The dataset may also include type information.

[0038] In step S3, the EEG-related information creation unit 33 creates EEG-related information by associating a tag, which is video information related to a person's movements, with multiple intracranial EEG data corresponding to the video data from which the tag was obtained, and brain region information corresponding to the placement positions of the electrode members 511 that acquired the intracranial EEG data. Here, the video data from which the tag was obtained is the input data that the analysis processing unit 32 used to set the tag. The multiple intracranial EEG data corresponding to the video data from which the tag was obtained are intracranial EEG data from multiple locations acquired by the EEG detection unit 51 corresponding to the imaging time of the video data, which is also the input data. In addition, if brain image data is available, the EEG-related information creation unit 33 also associates it with the EEG-related information so that it is included in the EEG-related information.

[0039] In step S4, the EEG-related information creation unit 33 processes the creation of the EEG-related information and registers it in the EEG database 90. The EEG-related information may be created in arbitrary divisions, such as predetermined time units or measurement experiments of predetermined movements. Multiple EEG-related information records for one to two weeks are registered in the EEG database 90. This makes it possible to extract EEG-related information linking human movements, intracranial EEG data, and brain regions from the vast amount of data for one to two weeks.

[0040] Figure 6 is a table showing an example of EEG-related information registered in the EEG database. In the example in Figure 6, intracranial EEG data tagged with "raising the right hand," intracranial EEG data tagged with "raising the left hand," and intracranial EEG data tagged with "lifting an object" are shown as examples of EEG-related information. As mentioned above, the intracranial EEG data tagged with "raising the right hand" is registered by distinguishing brain regions, such as brain region A, brain region B, and brain region C, corresponding to the placement positions of multiple EEG detection units 51. In addition, brain image data of the subject from whom the intracranial EEG data for "raising the right hand" was acquired, and classification information indicating that it was acquired from a subject with epilepsy symptoms are set. Similarly, for "raising the right hand," intracranial EEG data for each brain region is shown along with brain image data and classification information.

[0041] Furthermore, for the action of "lifting an object," intracranial electroencephalogram (EEG) data for each brain region is shown along with information indicating the type of test. The test may also involve measuring event-related potentials (ERPs: brain waves that occur in relation to perception and cognitive processing) in response to specific human actions. For example, this could include tests to investigate the relationship between actions performed by healthy subjects and their brain waves, tests to create a prosthetic arm that operates based on the wearer's brain waves, or tests to create a robot that operates based on the wearer's brain waves. The test may also involve asking a subject to "imagine an apple." Thus, human activity is not limited to outwardly visible actions, behaviors, conversations, facial expressions, etc., but also includes thoughts that change intracranial EEG data. For example, by conducting a test where each subject is asked to imagine an apple 100 times, 10,000 sets of intracranial EEG data on thought patterns when imagining an apple can be obtained, which can be used to elucidate the workings of the brain.

[0042] Note that the electroencephalogram-related information shown in FIG. 6 is merely an example. For example, some information such as classification information may be omitted. Also, the storage location and link destination of the video data, the date and time when the intracranial electroencephalogram data was acquired, etc. may be included. Also, it is conceivable that the subject may reject the registration of face information. In that case, while preventing the contamination of face information in the database, after performing a process of replacing the face information with plot information (coordinate information) or the like, the video data may be registered in the electroencephalogram database 90. Note that the replacement with plot information or the like is not limited to face information, and for example, it may be video data in which the movements of the subject's arms, legs, etc. are imaged. That is, the video data to be replaced with plot information or the like includes all parts of the subject and their movements, actions, expressions, etc.

[0043] <Search Process> As described above, electroencephalogram-related information is registered in the electroencephalogram database 90 by the series of processes shown in FIG. 5. Next, referring to FIG. 7, the input processing unit 34, the information search unit 35, and the output processing unit 36, which are functional units for searching for and calling electroencephalogram-related information from the electroencephalogram database 90, will be described. FIG. 7 is a flowchart showing an example of the flow of information search processing by the information processing apparatus 1 according to the present embodiment.

[0044] In step S11, the input processing unit 34 executes a process of receiving an input of search conditions from a user who wants to examine data. The user inputs search conditions, for example, via the search terminal 3 through a communication network N such as the Internet. Also, the user may directly input search conditions from the input unit 17 without using a communication network such as the Internet.

[0045] The search criteria include at least one of the following: human movement (activity), intracranial electroencephalogram (EEG) data, and brain regions. In this embodiment, the input processing unit 34 can also include type information in the search criteria in addition to human movement, intracranial EEG data, and brain regions. Furthermore, the search criteria may include information such as the date and time the data was measured. In this embodiment, the input processing unit 34 displays a search criteria input screen for entering search criteria and accepts the information entered by the user on this search criteria input screen as search criteria. In addition, the search criteria may include information such as the pathology of epilepsy, the gender and age of the subject.

[0046] Figure 8 shows an example of search condition input items displayed in a menu format. The screen in Figure 8 displays a list of search conditions in a menu format. In the example in Figure 8, as a method for specifying human actions as search conditions, an input box 201 for entering words or sentences is set up, along with an input item 202 that is pre-registered for specifying human actions. Input item 202 displays multiple items such as "raise right hand" and "raise left hand," and each item can be selected. When "raise right hand" or "raise left hand" is selected as a search condition, a search for EEG-related information corresponding to this tag is performed. In addition, as a method for specifying EEG data as a search condition, an input box 204 for entering words or sentences is set up, along with an input item 205 that is pre-registered for specifying brain regions. Furthermore, as a method for specifying type information, an input box 207 for entering words or sentences is set up, along with an input item 208 that is pre-registered for specifying types such as epilepsy and intracerebral hemorrhage. Furthermore, the method for specifying EEG data as search criteria is not limited to inputting words or sentences; for example, an illustrated diagram simulating the waveform of EEG data could be displayed in a menu format.

[0047] Furthermore, setting units 203, 206, and 209 are provided that can set any one of the search methods of AND, OR, or NOT for each of human motion, intracranial electroencephalogram data, and type information. The setting unit 203 is a part that sets the search method for human motion, the setting unit 206 is a part that sets the search method for electroencephalogram data, and the setting unit 209 is a part that sets the search method for types. For example, when the user wants data when raising the right hand normally rather than during an epileptic seizure, the user checks "epilepsy" and selects the NOT search method for the type information setting unit 209, enabling the extraction of electroencephalogram-related information on the motion of raising the right hand of a healthy person from the electroencephalogram database 90. Note that the example in FIG. 8 is merely an example, and the screen output by the input processing unit 34 is not limited to the example in FIG. 8.

[0048] Returning to FIG. 7, in step S12, the information search unit 35 executes a process of searching for corresponding electroencephalogram-related information from the electroencephalogram database 90 based on the search conditions input by the input processing unit 34. Through this search process, the corresponding electroencephalogram-related information is extracted from the electroencephalogram database 90. Personal information (personal data display unit 211 in FIG. 9 described later) input when registering intracranial electroencephalogram data in the information processing device 1 is data displayed at the time of registration and is processed so as not to be included in the search results of the information search unit 35.

[0049] In step S13, the output processing unit 36 executes a process of outputting the electroencephalogram-related information searched in the electroencephalogram database 90. The output processing unit 36 transmits the search results to the search terminal 3 that input the search conditions via the communication network N, for example. Also, when the output processing unit 36 directly receives the search conditions through the input unit 17, it displays the search results on the output unit 16. Information to be downloaded can be specified from the search result screen. The information that can be downloaded includes, in addition to intracranial electroencephalogram data for each intracranial region, brain image data such as CT (radiation) and MRI (magnetic field).

[0050] Furthermore, the position of the electroencephalogram (EEG) detection unit 51 within the head may be displayed in a three-dimensional image using brain image data. When the user specifies the position of the EEG detection unit 51 displayed in the three-dimensional image, intracranial EEG data at the specified position may be extracted through a search process.

[0051] <Example of Data Registration Screen> Next, we will explain an example of a screen displayed when registering cranial electroencephalogram (EEG) data to the information processing device 1. The example screen described below is intended to be displayed on a terminal used when registering intracranial EEG data, such as the EEG data acquisition device 2.

[0052] Figure 9 shows an example of a registration screen for cranial electroencephalogram (EEG) data and MRI data. The screen shown in Figure 9 includes a personal data display unit 211 for the subject, a disease information display unit 213, an MRI data confirmation unit 214, a video / EEG data confirmation unit 215, etc. The personal data display unit 211 displays the subject's name, ID, blood type, age, gender, weight, height, etc. The disease information display unit 213 displays information about the subject's disease in text format. The MRI data confirmation unit 214 displays a list of operation units for checking the upload date and time and details of the MRI data. The video / EEG data confirmation unit 215 displays a list of operation units for checking the upload date and time and details of video data synchronized with intracranial EEG data.

[0053] Figure 10 shows an example of a confirmation screen for registered video data and intracranial electroencephalogram (EEG) data. The screen shown in Figure 10 includes a video display unit 221, an intracranial EEG data display unit 222, and a motion analysis unit 223. The video display unit 221 is a video playback unit that can play back video data of a subject. The intracranial EEG data display unit 222 displays time-series data of intracranial EEG data for each of multiple brain regions synchronized with the video data. The intracranial EEG data display unit 222 displays a time bar 222a when the video is played back. The time bar 222a is a linear indicator that moves from left to right in the figure in synchronization with the video playback speed. The motion analysis unit 223 includes a motion interval display unit that shows the subject's motion items and their occurrence intervals. In the motion interval display unit, the occurrence intervals of motions are displayed in color, for example (shaded area in the figure), so that they can be distinguished from the non-occurrence intervals. The user can view the intervals in which the actions occur and the intracranial electroencephalogram data synchronized with those intervals in chronological order as the time bar 222a moves.

[0054] As described above, the information processing device 1 of this embodiment includes an electroencephalogram database 90 as an information storage unit that stores electroencephalogram-related information associating human movement (activity), intracranial electroencephalogram data synchronized with the human movement, and brain regions where the intracranial electroencephalogram data was detected; an input processing unit 34 that receives input of search conditions including at least one of human movement, intracranial electroencephalogram data, and brain regions; an information retrieval unit 35 that searches for the corresponding electroencephalogram-related information from the electroencephalogram database 90 based on the search conditions input to the input processing unit 34; and an output processing unit 36 ​​that outputs the electroencephalogram-related information retrieved by the information retrieval unit 35.

[0055] Furthermore, the information processing device 1 of this embodiment includes an electroencephalogram-related information creation unit 33 that creates electroencephalogram-related information relating to the human movement, the intracranial electroencephalogram data synchronized with the human movement, and the brain region where the intracranial electroencephalogram data was detected, based on motion information (activity information) relating to human movement, intracranial electroencephalogram data associated with the motion information, and brain region information relating to brain regions corresponding to the placement positions of electrode members placed inside the skull, and registers this information in an electroencephalogram database 90 as an information storage unit.

[0056] Furthermore, the control method for the information processing device 1 of this embodiment is a control method for an information processing device that stores brainwave-related information relating human actions (activities), intracranial brainwave data synchronized with the human actions, and brain regions where intracranial brainwave data was detected, and includes: an input processing step that accepts input of a search condition including at least one of human actions, intracranial brainwave data, and brain regions; an information retrieval step that searches for the corresponding brainwave-related information from the brainwave-related information stored in the information processing device based on the search condition input in the input processing step; and an output processing step that outputs the brainwave-related information retrieved in the information retrieval step.

[0057] Furthermore, the program of this embodiment causes a computer to execute the following as processing for an information processing device that stores brainwave-related information associating human movement (activity), intracranial brainwave data synchronized with the human movement, and brain regions where the intracranial brainwave data was detected: an input processing step that accepts input of a search condition including at least one of human movement, intracranial brainwave data, and brain regions; an information retrieval step that searches for the corresponding brainwave-related information stored in the information processing device based on the search condition entered in the input processing step; and an output processing step that outputs the brainwave-related information retrieved in the information retrieval step.

[0058] As described above, with the information processing device 1, the control method for the information processing device 1, and the program configured, intracranial electroencephalogram (EEG) data according to the user's purpose can be extracted from the EEG database 90, which aggregates EEG data under video surveillance, through a search process. As in this embodiment, by continuously imaging the subject with the imaging unit 52 for 1 to 2 weeks along with measuring intracranial EEG data, intracranial EEG data synchronized with a specific action (activity), such as raising the right hand, is registered in the EEG database 90 for each time the right hand is raised during the measurement period. If a large number of subjects can be secured, it becomes possible to register a vast amount of intracranial EEG data for a specific action. Users who want to investigate the relationship between a specific action and EEG can extract only the necessary intracranial EEG data from the vast amount of intracranial EEG data by using the specific action as a search condition. It is also possible to extract only 10 seconds before and after the action of raising the right hand, or to extract only the EEG during a seizure in occipital lobe epilepsy.

[0059] The EEG database 90 can be used, for example, to develop new communication support devices for patients with impaired consciousness, develop programs to predict epileptic seizures, establish neurological monitoring methods, develop minimally invasive and inexpensive biomarkers effective for the early diagnosis of dementia, and establish diagnostic and treatment methods based on comparisons of intracranial EEG data between healthy individuals and patients with depression. By utilizing the EEG database 90, for example, in healthy elderly individuals, those with mild cognitive impairment, Alzheimer's disease, Lewy body dementia, and epilepsy cases, it becomes possible to evaluate the symptoms of epilepsy, then perform conventional EEG using scalp electrodes and EEG using intravascular / intracranial electrodes, and compare the findings of both tests to examine the effectiveness of EEG using intravascular / intracranial electrodes, including the diagnostic accuracy of epilepsy. Furthermore, the EEG database 90 can be used to identify EEG findings associated with the results of neuropsychological tests that evaluate memory, attention function, visuospatial cognition, verbal fluency, analogy, etc., as well as molecular imaging tests such as head MRI, cerebral blood flow SPECT, and amyloid PET, targeting healthy elderly individuals, those with mild cognitive impairment, Alzheimer's disease, Lewy body dementia, etc., and can be used to examine its usefulness for early diagnosis and evaluation of treatment effectiveness.

[0060] Furthermore, the electroencephalogram (EEG) database 90 can also be used to determine the effectiveness of continuing treatment for a subject and to predict the patient's prognosis by inserting the EEG detection unit 51 into the head of a subject who is unconscious due to intracranial hemorrhage or the like, and acquiring intracranial EEG data as EEG information. More specifically, it is expected that EEG information will be utilized in prognosis prediction by predicting DCI (delayed cerebral ischemic attack) after severe subarachnoid hemorrhage, selection of the optimal life-saving method by neuromonitoring for emergency transport patients with head injuries, improvement of the quality of life (QOL) of epilepsy patients through implantable brain stimulators and epilepsy prediction apps, establishment of new EEG diagnostic methods for patients with mental illnesses, and revision of EEG diagnostic criteria.

[0061] Furthermore, the information processing device 1 of this embodiment further includes an electroencephalogram-related information creation unit 33 that creates electroencephalogram-related information based on motion information (activity information) related to human movement, intracranial electroencephalogram data synchronized with the motion information, and intracranial region information corresponding to the placement positions of electrode members 511 placed inside the skull, and registers it in an electroencephalogram database (information storage unit) 90.

[0062] This automatically creates EEG-related information, which associates motion information related to human movement, intracranial EEG data, and intracranial region information, and registers it in the EEG database 90, thereby streamlining the data registration process in the EEG database 90.

[0063] Furthermore, the information processing device 1 of this embodiment further includes an analysis processing unit 32 that creates motion information by performing image analysis on video data captured of human motion.

[0064] As a result, motion information is automatically created (tagged) through image analysis of video data, making the registration process of searchable EEG-related information in the EEG database 90 even more efficient.

[0065] Furthermore, the input processing unit 34 of this embodiment displays a list of search conditions in a menu format. This allows for easy and quick filtering of intracranial electroencephalogram (EEG) data using the search conditions displayed in the menu format. Also, because it is displayed in a menu format, the frequency of unintended intracranial EEG data being extracted by the search process due to user input errors can be reduced.

[0066] Furthermore, in this embodiment, the electroencephalogram-related information registered in the electroencephalogram database 90 includes type information indicating the circumstances under which a person's movements were acquired. When the input processing unit 34 receives type information as a search condition, the information retrieval unit searches for electroencephalogram-related information corresponding to the input type information.

[0067] This allows users studying cases to extract intracranial electroencephalogram (EEG) data from specific cases, such as epilepsy patients or depression patients, through a search process. Furthermore, by distinguishing and registering frontal lobe epilepsy and occipital lobe epilepsy, it becomes easy to extract only the EEG data from frontal lobe epileptic seizures through a search process. Additionally, users creating prosthetic limbs or robots can easily investigate the relationship between specific movements, such as raising the right hand, and EEG.

[0068] Furthermore, in this embodiment, intracranial electroencephalogram data registered in the electroencephalogram database 90 is acquired by an electroencephalogram detection unit 51 which includes an electrode member 511 placed inside a blood vessel and a conductive wire member 510 that is electrically connected to the electrode member 511.

[0069] In conventional electroencephalography (EEG) systems, where a net with electrodes for EEG detection is worn on the subject's head (hereinafter referred to as the conventional type), the electrodes are often located far from the cells that emit EEGs, causing damage to the skull, skin, hair, etc. As a result, the conventional type results in reduced EEG amplitude and voltage. Furthermore, because the electrodes, acting as antennas, cover a wide area in the conventional type, it is difficult to pinpoint the area where EEGs are being emitted. Another method known to investigate the causes of epilepsy, etc., involves surgically cutting the skull and placing an EEG machine, but this method is highly invasive and has significant side effects. In contrast, with the configuration of the EEG detection unit 51 as in this embodiment, where the electrode member 511 connected to the tip of the wire member 510 is placed inside a blood vessel, the electrodes are placed inside a blood vessel, resulting in lower invasiveness and fewer side effects. Moreover, the EEG detection unit 51 in this embodiment can accurately detect EEGs, and because the area covered by a single electrode member 511 is narrower than in the conventional type, it is possible to accurately pinpoint the area.

[0070] While there may be only one electrode member 511 at the tip of the wire member 510, it is also possible to configure the device to have multiple electrode members 511 spaced apart (for example, 5 to 10). This allows for the identification of areas where intracranial electroencephalogram (EEG) data is detected based on the potential between the electrodes, at intervals of, for example, several millimeters to tens of millimeters. Furthermore, since the EEG detection unit 51 is a short-term implant, it can lower the psychological barrier for the subjects and is also suitable in terms of increasing the number of experiments (N).

[0071] Although one embodiment of the present invention has been described above, the present invention is not limited to the embodiments described above, and any modifications, improvements, etc. that can achieve the objectives of the present invention are included in the present invention.

[0072] For example, information about the electroencephalogram (EEG) detection device used to acquire EEG data may be stored, and the EEG data may be searched based on that device information. Information about the EEG detection device may include, for example, the type of EEG detection device (intravascular electrodes, intracranial electrodes, network electrodes, etc.) as well as the model name. In addition, brain regions used as search criteria may be displayed on the screen as an illustrated diagram (area map) simulating the inside of the skull, and any brain region may be selected using mouse operations.

[0073] Furthermore, the series of processes described above can be executed by hardware or by software. In other words, the functional configuration described above is merely illustrative and not particularly limiting. That is, it is sufficient that the information processing device 1 is equipped with a function that can execute the series of processes described above as a whole, and the type of functional block used to realize this function is not particularly limited to the example above. Also, the location of the functional block is not particularly limited and can be arbitrary. For example, the functional block of the information processing device 1 may be transferred to another device, etc. Conversely, the functional block of another device may be transferred to a server, etc. Also, a single functional block may be composed of hardware alone, software alone, or a combination of both.

[0074] When a series of processes are executed by software, the programs that make up that software are installed on a computer or other device from a network or storage medium. The computer may be a computer built into dedicated hardware. Alternatively, the computer may be a computer capable of performing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0075] Such recording media containing programs may consist not only of removable media (not shown) distributed separately from the main device to provide the programs, but also of recording media provided pre-installed in the main device. Since programs can be distributed via a network, the recording media may be installed on or accessible from a computer connected to or capable of connecting to a network.

[0076] In this specification, the step of describing a program to be recorded on a recording medium includes not only processes that are performed chronologically in that order, but also processes that are not necessarily performed chronologically, but are executed in parallel or individually. Furthermore, in this specification, the term "system" refers to an overall system composed of multiple devices, means, etc.

[0077] 1. Information processing device 2. Electroencephalogram (EEG) data acquisition device 3. Search terminal 11. CPU 12. ROM 13. RAM 31. Data acquisition unit 32. Analysis processing unit 33. EEG-related information creation unit 34. Input processing unit 35. Information retrieval unit 36. Output processing unit 90. EEG database 100. EEG data search system

Claims

1. An information processing device comprising: an information storage unit that stores brainwave-related information associating human activity, intracranial electroencephalogram data synchronized with the human activity, and brain regions where the intracranial electroencephalogram data was detected; an input processing unit that receives input of a search condition including at least one of the human activity, the intracranial electroencephalogram data, and the brain region; an information retrieval unit that searches the information storage unit for the corresponding brainwave-related information based on the search condition input to the input processing unit; and an output processing unit that outputs the brainwave-related information retrieved by the information retrieval unit.

2. An information processing device comprising an electroencephalogram-related information creation unit that creates electroencephalogram-related information relating to the human activity, the intracranial electroencephalogram data synchronized with the human activity, and the brain region where the intracranial electroencephalogram data was detected, based on activity information relating to the human activity, intracranial electroencephalogram data associated with the activity information, and brain region information relating to the brain region corresponding to the placement position of electrode members placed inside the skull, and registers this information in an information storage unit.

3. The information processing apparatus according to claim 2, further comprising an analysis processing unit that creates the activity information by performing image analysis on video data of the person's activity.

4. The information processing apparatus according to claim 1, wherein the input processing unit displays the list of search conditions in a menu format.

5. The information processing apparatus according to claim 1, wherein the electroencephalogram-related information registered in the information storage unit includes type information indicating the circumstances at the time the person's activity was acquired, and the information retrieval unit retrieves the electroencephalogram-related information corresponding to the input type information when the input processing unit receives the type information as a search condition.

6. The information processing apparatus according to claim 1 or 2, wherein the intracranial electroencephalogram data registered in the information storage unit is acquired by an electroencephalogram detection unit which includes an electrode member placed in a blood vessel and a conductive wire member electrically connected to the electrode member.

7. A method for controlling an information processing device that stores electroencephalogram (EEG) related information relating human activity, intracranial EEG data synchronized with the human activity, and brain regions where the intracranial EEG data was detected, comprising: an input processing step of receiving input of a search condition including at least one of the human activity, the intracranial EEG data, and the brain regions; an information retrieval step of searching for the corresponding EEG related information from the EEG related information stored in the information processing device based on the search condition input in the input processing step; and an output processing step of outputting the EEG related information retrieved in the information retrieval step.

8. A program for causing a computer to execute the following as processing for an information processing device that stores brainwave-related information associating human activity, intracranial brainwave data synchronized with the human activity, and brain regions where the intracranial brainwave data was detected: an input processing step that accepts input of a search condition including at least one of the human activity, the intracranial brainwave data, and the brain region; an information retrieval step that searches for the corresponding brainwave-related information from the brainwave-related information stored in the information processing device based on the search condition input in the input processing step; and an output processing step that outputs the brainwave-related information retrieved in the information retrieval step.

Images