Information processing apparatus, information processing system, information processing method, and computer program

Abstract

To more accurately associate brain wave information with an event that a user imagines when the brain wave information is output.SOLUTION: An information processing apparatus comprises: an output control unit that outputs character strings related to events to be recorded so that a user, whose brain wave information is to be recorded, can recognize the character strings; and a brain wave information recording control unit that acquires the brain wave information of the user in a period according to the output of the character strings by the output control unit, and records the brain wave information in association with the events to be recorded.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a computer program. [Background technology]

[0002] As a new user interface, technology using human electroencephalograms (e.g., brain-computer interface (BCI)) has been proposed. For example, Patent Document 1 discloses an electroencephalogram interface device that detects electroencephalogram signals having a frequency and controls equipment. In this way, it has been proposed that by detecting information related to electroencephalogram signals (hereinafter referred to as "electroencephalogram information") that is generated when a user imagines a certain event, an event corresponding to the electroencephalogram information can be estimated in an information processing device, and information processing can be performed according to the event. For example, one type of event is the operation of a device, or the manipulation, deformation, or movement of graphic information displayed on a screen. In this case, the content of the operation imagined by the user is estimated by the device via the electroencephalogram information, and an operation corresponding to the operation content is realized. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-178350 Summary of the Invention [Problem to be solved by the invention]

[0004] However, it has been difficult to accurately correlate detected EEG information with the events truly envisioned by the user who output the EEG information. This is because the EEG information of the events envisioned by the user is often unstable and contains a high error rate. As a result, the information processing device may recognize the events as different from the actual events envisioned. To improve this accuracy, there has also been a challenge in providing effective user training to stably output events and EEG information.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and provides a technology that enables electroencephalogram information to be more accurately associated with events that a user imagines when the electroencephalogram information is output. The present invention also provides a technology that includes user training for enabling a user to stably generate a specific electroencephalogram pattern and to more accurately associate the events that the user imagines with the electroencephalogram. [Means for solving the problem]

[0006] One aspect of the present invention is an information processing device that includes an output control unit that outputs a string of characters related to an event to be recorded in a manner that is recognizable to a user whose electroencephalogram information is to be recorded, and an electroencephalogram information recording control unit that acquires the user's electroencephalogram information during a period corresponding to the output of the string by the output control unit and records it in association with the event to be recorded.

[0007] In one aspect of the present invention, in the information processing device described above, the character string is a character string representing an onomatopoeia corresponding to the event.

[0008] In one aspect of the present invention, in the information processing device described above, the output control unit displays the character string on an image display device, and further displays an image related to the event on the image display device.

[0009] One aspect of the present invention is the above-mentioned information processing device, further comprising a learning unit that acquires a trained model by performing a learning process using learning information that associates the user's training electroencephalogram information, obtained in response to the output of a string related to the event to be recorded, with the event to be recorded.

[0010] One aspect of the present invention is the above-mentioned information processing device, further comprising a judgment unit that uses judgment-use EEG information obtained from the user and the trained model to judge the event that the user imagined when the judgment-use EEG information was obtained.

[0011] One aspect of the present invention is the above-mentioned information processing device, further comprising an output control unit that indicates an event in a manner that is recognizable to the user, wherein the judgment unit judges the event imagined by the user using judgmental electroencephalogram information obtained when the user imagines the event after the instruction is output by the output control unit, and the output control unit outputs information indicating the judgment result by the judgment unit in a manner that is recognizable to the user.

[0012] One aspect of the present invention is an information processing system comprising: an output control unit that outputs a string of characters related to an event to be recorded in a manner that is recognizable by a user whose EEG information is to be recorded; an EEG information recording control unit that acquires the user's EEG information during a period corresponding to the output of the string of characters by the output control unit and records the information in association with the event to be recorded; a memory unit that stores a trained model obtained by performing a learning process using training information that associates the user's training EEG information obtained in response to the output of the string of characters related to the event to be recorded with the event to be recorded; and a judgment unit that uses judgment EEG information obtained from the user and the trained model to judge the event that the user imagined when the judgment EEG information was obtained.

[0013] One aspect of the present invention is the above-mentioned information processing system, which further includes a learning processing unit that acquires the trained model by performing a learning process using learning information that associates the user's training electroencephalogram information, obtained in response to the output of a string related to the event to be recorded, with the event to be recorded.

[0014] One aspect of the present invention is an information processing method having an output control step of outputting a string of characters related to an event to be recorded in a manner that is recognizable to a user of the electroencephalogram information to be recorded, and an electroencephalogram information recording control step of acquiring the user's electroencephalogram information during a period corresponding to the output of the string of characters in the output control step and recording the information in association with the event to be recorded.

[0015] One aspect of the present invention is a computer program for causing a computer to function as an information processing device that includes an output control unit that outputs a string of characters related to an event to be recorded in a manner that is recognizable by a user whose EEG information is to be recorded, and an EEG information recording control unit that acquires the user's EEG information during a period corresponding to the output of the string by the output control unit and records it in association with the event to be recorded. [Effects of the Invention]

[0016] According to the present invention, it is possible to more accurately associate electroencephalogram information with the event that the user imagines when the electroencephalogram information is output. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a schematic block diagram showing the system configuration of an information processing system 100 according to the present invention. [Figure 2] 2 is a schematic block diagram showing a specific example of the functional configuration of the terminal device 10. FIG. [Figure 3] FIG. 10 is a diagram showing a specific example of training information. [Figure 4] FIG. 10 is a diagram showing a specific example of the correspondence between training information and electroencephalogram information. [Figure 5] FIG. 10 is a diagram showing a first mode of display. [Figure 6] FIG. 10 is a diagram showing a second mode of display. [Figure 7] 2 is a schematic block diagram showing a specific example of the functional configuration of the learning device 20. FIG. [Figure 8] 10 is a flowchart showing a specific example of processing by the learning device 20. [Figure 9] 2 is a schematic block diagram showing a specific example of the functional configuration of a determination device 30. FIG. [Figure 10] 10 is a flowchart showing a specific example of processing by the determination device 30. [Figure 11] FIG. 2 is a diagram illustrating an outline of an example of the hardware configuration of an information processing device 90 applied to the present embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of the configuration of a training assistance device 40. [Figure 13] FIG. 10 is a diagram showing a first display example when training output is performed using an image display device. [Figure 14] FIG. 10 is a diagram showing a second display example when training output is performed using an image display device. [Figure 15] FIG. 1 is a diagram showing an example of the configuration of a video game device 50. [Figure 16] FIG. 2 is a diagram illustrating an example of the configuration of an information processing device 60. [Figure 17] 10 is a diagram showing a specific example of a screen (third display mode) displayed in a modified example of the terminal device 10. FIG. [Figure 18] FIG. 10 is a diagram illustrating a modified example of the terminal device 10. [Figure 19] FIG. 10 is a diagram illustrating a modified example of the terminal device 10. [Figure 20] FIG. 10 is a diagram showing a modified example of the determination device 30. [Figure 21] FIG. 10 is a diagram illustrating a modified example of the terminal device 10. [Figure 22] FIG. 10 is a diagram showing experimental results. [Figure 23] FIG. 10 is a diagram showing experimental results. DETAILED DESCRIPTION OF THE INVENTION

[0018] 1 is a schematic block diagram showing the system configuration of an information processing system 100 according to the present invention. The information processing system 100 mainly performs three processes (acquisition process, learning process, and determination process).

[0019] In the acquisition process, information (electroencephalogram information) regarding the electroencephalogram signals generated when the user visualizes a certain event is acquired in association with each event. In this process, the user does not simply visualize the event, but visualizes an onomatopoeia corresponding to the event. By engaging in this onomatopoeia-based behavior, the user can achieve a training effect of outputting more stable electroencephalogram information (with less variation between actions) for a certain event, thereby enabling the acquisition of stable electroencephalogram information. The acquisition of such electroencephalogram information is performed by the terminal device 10. Japanese onomatopoeia is diverse and can express a wide range of images, movements, mental and physical states, internal perceptions, and more, enriching linguistic expression. Furthermore, onomatopoeia is widespread not only in Japanese but in languages ​​around the world, making it an effective learning tool for generating stable electroencephalogram information.

[0020] In the learning process, a trained model is obtained by performing a learning process using multiple pieces of information acquired in the acquisition process (information associating events with electroencephalogram information). In this trained model, electroencephalogram information and events are used as explanatory variables and objective variables, respectively. Such a learning process is performed by the learning device 20. The electroencephalogram information used in the learning process in this way is also called training electroencephalogram information.

[0021] In the determination process, inference is performed using the trained model obtained by the learning process. New brain wave information is acquired from the user, and by using the acquired brain wave information as an explanatory variable, the event envisioned by the user is determined as a target variable from the trained model. Further information processing may be performed according to the event determined by the determination process. This type of determination process is performed by the determination device 30. The brain wave information used in this determination process is also referred to as determination brain wave information.

[0022] The information processing system 100 includes a terminal device 10, a learning device 20, and a determination device 30. The terminal device 10, the learning device 20, and the determination device 30 are communicably connected via a network 70. The network 70 may be a network using wireless communication or a network using wired communication. The network 70 may be configured using, for example, the Internet or a local area network (LAN). The network 70 may also be configured by combining multiple networks. Note that, although an example will be described in which the devices are configured to be able to communicate with each other so as to facilitate the exchange of information between them, the devices do not necessarily need to be configured to be able to communicate with each other when implemented.

[0023] 2 is a schematic block diagram showing a specific example of the functional configuration of the terminal device 10. The terminal device 10 is configured using information equipment such as a smartphone, tablet, personal computer, dedicated device, etc. The terminal device 10 includes a communication unit 11, an input unit 12, an output unit 13, an electroencephalogram information acquisition unit 14, a storage unit 15, and a control unit 16.

[0024] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 communicates data with other devices via the network 70 in accordance with the control of the control unit 16. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.

[0025] The input unit 12 is configured using existing input devices such as a keyboard, a pointing device (mouse, tablet, etc.), buttons, a touch panel, etc. The input unit 12 is operated by a user when inputting user instructions to the terminal device 10. The input unit 12 may be an interface for connecting the input device to the terminal device 10. In this case, the input unit 12 inputs an input signal generated in the input device in response to a user input to the terminal device 10. The input unit 12 may be configured using a microphone and a voice recognition device. In this case, the input unit 12 acquires an acoustic signal generated by the user's speech, performs voice recognition on the words spoken by the user, and inputs character string information of the recognition result to the terminal device 10. The voice recognition process may be performed by the control unit 16. The input unit 12 may be configured in any way as long as it is capable of inputting user instructions to the terminal device 10.

[0026] The output unit 13 outputs information in a form that can be recognized by the user. The output unit 13 may be, for example, an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 13 may be an interface for connecting an image display device to the terminal device 10. In this case, the output unit 13 generates a video signal for displaying image data and outputs the video signal to the image display device connected to the output unit 13. The output unit 13 may be a device for outputting sound, such as a speaker. The output unit 13 may be an interface for connecting an audio output device, such as a speaker or headphones, to the terminal device 10. In this case, the output unit 13 generates an audio signal for reproducing audio data and outputs the audio signal to the audio output device connected to the output unit 13. The output unit 13 may be configured as a touch panel integrated with the input unit 12.

[0027] The electroencephalogram information acquisition unit 14 acquires information relating to the electroencephalogram signal of the user (electroencephalogram information). The electroencephalogram information may be time-series signal information indicating the electroencephalogram signal, or information indicating the feature quantity of the electroencephalogram signal. For example, the electroencephalogram information may be information indicating each frequency component of the electroencephalogram. The electroencephalogram information acquisition unit 14 may have a plurality of electrodes attached to the user's head, for example, and may measure the brain's action potential with each electrode.

[0028] The storage unit 15 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 15 stores data used by the control unit 16. The storage unit 15 stores data required when the control unit 16 performs processing. The storage unit 15 functions as a training information storage unit 151 and an electroencephalogram information storage unit 152.

[0029] The training information storage unit 151 stores training information. FIG. 3 is a diagram showing a specific example of training information. The training information includes events for which electroencephalogram information is scheduled to be acquired, associated with onomatopoeia related to the events. In the example of FIG. 3, the onomatopoeia "kuru kuru" (round and round) is associated with the event of "spinning." Furthermore, the onomatopoeia "tat tat tat tat" is associated with the event of "running forward," and the onomatopoeia "tot tottot" is associated with the event of "running backward." However, the associations between events and onomatopoeia shown in FIG. 3 are merely specific examples and are not limited to these. For example, the onomatopoeia "guruguru" (round and round) may be associated with the event of "spinning," or any other onomatopoeia such as "kurunkurun" (round and round), "gurun" (round and round), or "kyurun kyurun" (round and round). Furthermore, multiple onomatopoeia may be associated with one event.

[0030] The electroencephalogram information storage unit 152 stores multiple pieces of electroencephalogram information. FIG. 4 is a diagram showing a specific example of the correspondence between training information and electroencephalogram information. The electroencephalogram information storage unit 152 stores one or multiple pieces of electroencephalogram information in association with each event. The electroencephalogram information stored in the electroencephalogram information storage unit 152 is electroencephalogram information acquired by the electroencephalogram information acquisition unit 14 of the terminal device 10.

[0031] The control unit 16 is configured using a processor such as a CPU (Central Processing Unit) and a memory (main storage device). The control unit 16 functions when the processor executes a program. Note that all or part of the functions of the control unit 16 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor storage devices (e.g., SSDs: Solid State Drives), as well as storage devices such as hard disks and semiconductor storage devices built into computer systems. The program may be transmitted via a telecommunications line.

[0032] The control unit 16 may execute, for example, an application installed on its own device (terminal device 10). A specific example of such an application is an application provided to the terminal device 10 as a dedicated application for the information processing system 100. Another specific example of such an application is a web browser application. Such an application may be pre-installed on the terminal device 10 or may be downloaded each time a determination process is executed. For example, when implemented as a web browser application, the terminal device 10 may download and execute the application from a device specified by a specific web server (for example, the web server itself or another server) in response to the terminal device 10 connecting to the web server. The control unit 16 operates according to the program of the application being executed. The control unit 16 functions as an output control unit 161 and an electroencephalogram information recording control unit 162 by executing a program.

[0033] The output control unit 161 controls the output of the output unit 13. The output control unit 161 outputs from the output unit 13 to prompt the user to imagine onomatopoeia related to the event of the electroencephalogram recording. For example, when the output unit 13 displays an image, it displays a string of onomatopoeia related to the event of the electroencephalogram recording. When the output unit 13 outputs audio, it outputs audio indicating the string of onomatopoeia related to the event of the electroencephalogram recording.

[0034] FIG. 5 is a diagram showing a first mode of display. As shown in FIG. 5, the onomatopoeia character string "Kuru Kuru" corresponding to the event to be recorded is displayed in the onomatopoeia display area 131 on the screen of the output unit 13. The output control unit 161 displays (outputs) the onomatopoeia character string corresponding to the event to be recorded in this manner. The event to be recorded may be selected, for example, by the user, or may be selected by the output control unit 161 or the electroencephalogram information recording control unit 162. The output control unit 161 continues to output the onomatopoeia for a predetermined period. For example, the output control unit 161 continues to output the onomatopoeia for a period during which the electroencephalogram information recording control unit 162 is acquiring electroencephalogram information. For example, the output control unit 161 continues to output the onomatopoeia for a predetermined portion of a period during which the electroencephalogram information recording control unit 162 is acquiring electroencephalogram information. The predetermined portion of the period may be, for example, a period including the beginning of the period during which the electroencephalogram information is acquired (e.g., the first half). That is, during the period in which the electroencephalogram information recording control section 162 is acquiring electroencephalogram information, the onomatopoeia is output during a period in which the user can recite in his or her head the string of onomatopoeia that corresponds to the event.

[0035] FIG. 6 is a diagram illustrating a second display mode. As shown in FIG. 6, in the second display mode, an event image display mode 132 is provided in addition to an onomatopoeia display mode 131 on the screen of the output unit 13. The event image display mode 132 displays an image related to the event to be recorded (hereinafter referred to as an "event image"). The event image display mode 132 may display a still image or a moving image as the event image. The event image may be, for example, an image that expresses the event to be recorded. For example, in the example of FIG. 6, since the event to be recorded is "rotating," a moving image of an object rotating is displayed in the event image display mode 132. The event image corresponding to the event "running forward" may be, for example, a moving image of a character running forward, or a still image that expresses such an event. The output control unit 161 continues to output onomatopoeia and event images for a predetermined period of time. For example, the output control unit 161 continues to output onomatopoeia and event images for a period of time during which the electroencephalogram information recording control unit 162 acquires electroencephalogram information. By displaying an event image in addition to the onomatopoeia, the user can visualize the event more accurately and easily. This allows for more stable EEG information to be acquired for the event being recorded, and when the EEG information is output, it is possible to more accurately match the event visualized by the user with the EEG information.

[0036] The output control unit 161 may further output a character string representing the onomatopoeia in a manner other than a display while the display is being performed as shown in Fig. 5 or 6. For example, the output control unit 161 may further output the character string of the onomatopoeia by voice while the display is being performed as shown in Fig. 5 or 6.

[0037] The electroencephalogram information recording control unit 162 records the electroencephalogram information acquired by the electroencephalogram information acquisition unit 14 during the period when the onomatopoeia is being output by the output control unit 161 in the electroencephalogram information storage unit 152. At this time, the electroencephalogram information recording control unit 162 records the electroencephalogram information in association with an event corresponding to the onomatopoeia being output.

[0038] 7 is a schematic block diagram showing a specific example of the functional configuration of learning device 20. Learning device 20 is configured using an information processing device such as a personal computer or a server device. Learning device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0039] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 communicates data with other devices via the network 70 in accordance with the control of the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.

[0040] The storage unit 22 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as, for example, a teacher data storage unit 221 and a trained model storage unit 222.

[0041] The teacher data storage unit 221 stores teacher data used in the learning process executed by the learning device 20. The teacher data stored in the teacher data storage unit 221 is electroencephalogram information obtained by the terminal device 10. In other words, the teacher data is data that associates a specific event with electroencephalogram information obtained from a user who imagines that event.

[0042] For example, only EEG information obtained from a specific (single) user to be trained may be used as training data. By performing a training process using such training data, a trained model specialized for that specific user can be obtained, making it possible to accurately determine an event (objective variable) for that user's EEG information (explanatory variable).

[0043] Electroencephalogram information obtained by the terminal device 10 from multiple users who share a specific common attribute may be used as training data. By performing a learning process using such training data, a trained model specialized for users who share the specific common attribute can be obtained, making it possible to accurately determine an event (objective variable) based on the electroencephalogram information (explanatory variable) of users who share the attribute. Specific examples of such attributes include nationality, gender, age (generation), medical history, etc.

[0044] The electroencephalogram information obtained from an unspecified number of users by the terminal device 10 may be used as training data. By performing a learning process using such training data, a trained model common to various users can be obtained, and it becomes possible to accurately determine an event (objective variable) for the electroencephalogram information (explanatory variable) of various users.

[0045] The trained model storage unit 222 stores a trained model obtained by a learning process using the training data stored in the training data storage unit 221.

[0046] The control unit 23 is configured using a processor such as a CPU and a memory. The control unit 23 functions as an information control unit 231 and a learning control unit 232 by the processor executing a program. All or part of the functions of the control unit 23 may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., an SSD), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0047] The information control unit 231 controls the input and output of information. For example, the information control unit 231 acquires teacher data from another device (an information processing device or a storage medium) and records it in the teacher data storage unit 221. More specifically, the information control unit 231 acquires electroencephalogram information (associated with an event) generated by the terminal device 10. The electroencephalogram information may be received from the terminal device 10 via the network 70, for example, or may be acquired via another information processing device or another storage medium. For example, the information control unit 231 may transmit the trained model stored in the trained model storage unit 222 to another device (for example, the determination device 30) or may record it in a storage medium.

[0048] The learning control unit 232 executes a learning process using the teacher data stored in the teacher data storage unit 221. Specific examples of such learning processes include supervised learning for classification, such as support vector machines, random forests, and neural networks. The learning control unit 232 generates a trained model for outputting the event that the user envisioned when the electroencephalogram information was being output, based on the input electroencephalogram information, for example, by performing supervised learning. The learning control unit 232 records the generated trained model in the trained model storage unit 222. The trained model obtained by the learning control unit 232 may be transmitted to the determination device 30 and recorded in the determination model storage unit 321 of the determination device 30.

[0049] 8 is a flowchart showing a specific example of processing by the learning device 20. First, the information control unit 231 acquires training data (step S101). The training data may be input by a user, acquired via communication from another information device, or acquired from a recording medium connected to the learning device 20, for example. The learning control unit 232 executes a learning process using the data and records a trained model in the trained model storage unit 222 (step S102).

[0050] 9 is a schematic block diagram showing a specific example of the functional configuration of the determination device 30. For example, the determination device 30 is configured using information devices such as a smartphone, tablet, personal computer, dedicated device, etc. The determination device 30 includes a communication unit 31, an input unit 32, an output unit 33, an electroencephalogram information acquisition unit 34, a storage unit 35, and a control unit 36.

[0051] The communication unit 31 is a communication device. The communication unit 31 may be configured as, for example, a network interface. The communication unit 31 communicates data with other devices via the network 70 in accordance with the control of the control unit 36. The communication unit 31 may be a device that performs wireless communication or a device that performs wired communication.

[0052] The input unit 32 is configured using existing input devices such as a keyboard, a pointing device (mouse, tablet, etc.), buttons, a touch panel, etc. The input unit 32 is operated by a user when inputting user instructions to the determination device 30. The input unit 32 may be an interface for connecting the input device to the determination device 30. In this case, the input unit 32 inputs an input signal generated in the input device in response to a user input to the determination device 30. The input unit 32 may be configured using a microphone and a voice recognition device. In this case, the input unit 32 acquires an acoustic signal generated by the user's speech, performs voice recognition on the words spoken by the user, and inputs character string information of the recognition result to the determination device 30. The voice recognition process may be performed by the control unit 36. The input unit 32 may be configured in any way as long as it is capable of inputting user instructions to the determination device 30.

[0053] The output unit 33 outputs information in a form that can be recognized by the user. The output unit 33 may be, for example, an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 33 may be an interface for connecting an image display device to the determination device 30. In this case, the output unit 33 generates a video signal for displaying image data and outputs the video signal to the image display device connected to the output unit 33. The output unit 33 may be a device for outputting sound, such as a speaker. The output unit 33 may be an interface for connecting an audio output device, such as a speaker or headphones, to the determination device 30. In this case, the output unit 33 generates an audio signal for reproducing audio data and outputs the audio signal to the audio output device connected to the output unit 33. The output unit 33 may be configured as a touch panel integrated with the input unit 32.

[0054] The electroencephalogram information acquisition unit 34 acquires information relating to the electroencephalogram signal of the user (electroencephalogram information). The electroencephalogram information may be time-series signal information indicating the electroencephalogram signal, or information indicating the feature quantity of the electroencephalogram signal. For example, the electroencephalogram information may be information indicating each frequency component of the electroencephalogram. The electroencephalogram information acquisition unit 34 may have a plurality of electrodes attached to the user's head, and may measure the brain's action potential with each electrode.

[0055] The storage unit 35 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 35 stores data used by the control unit 36. The storage unit 35 may function as a determination model storage unit 351, for example.

[0056] The determination model storage unit 351 stores a determination model used when the determination unit 362 performs a determination process. The determination model may be configured using information of a trained model generated in advance by a learning process, for example. Such a learning process may be executed by another device (for example, the learning device 20). The determination model does not necessarily have to be generated by a learning process. The determination model may be configured using, for example, a lookup table that associates feature amounts of electroencephalogram information obtained by the terminal device 10 with events corresponding to the feature amounts, or may be configured in another manner.

[0057] The control unit 36 ​​is configured using a processor such as a CPU and a memory. The control unit 36 ​​functions as an information control unit 361 and a determination unit 363 by the processor executing a program. All or part of the functions of the control unit 36 ​​may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., an SSD), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0058] The information control unit 361 acquires electroencephalogram information via the electroencephalogram information acquisition unit 34. The information control unit 361 may acquire electroencephalogram information from another device. The information control unit 361 may control the determination device 30 based on the determination result obtained by the determination unit 362. For example, if the determination result by the determination unit 362 is "rotate," the information control unit 361 may rotate the character by using "rotate" as operation information for the character in an application program running on the control unit 36. That is, the information control unit 361 may use the determination result of the determination unit 362 as input information (operation information) by the user. The information control unit 361 may transmit the determination result to another device such as the terminal device 10. Such exchange of information between the information control unit 361 and another device may be performed, for example, by communication via the communication unit 31.

[0059] The determination unit 363 performs a determination process using the determination model stored in the determination model storage unit 351 and the electroencephalogram information newly obtained by the electroencephalogram information acquisition unit 34. The determination process determines an event corresponding to the electroencephalogram information. Such an event corresponding to the electroencephalogram information is determined to be an event imagined by the user wearing the electroencephalogram information acquisition unit 34.

[0060] 10 is a flowchart showing a specific example of processing by the determination device 30. First, the information control unit 361 acquires electroencephalogram information from the electroencephalogram information acquisition unit 34 (step S201). The determination unit 362 performs a determination process using the acquired electroencephalogram information (step S202). The determination unit 362 outputs information indicating the determination result (step S203). For example, the determination unit 362 outputs information indicating the determination result (information indicating the event) to the information control unit 361.

[0061] In the information processing system 100 configured in this manner, when acquiring electroencephalogram information generated when a user visualizes a certain event, onomatopoeia corresponding to the event is output to the user, allowing the user to visualize the event using the onomatopoeia. In this manner, electroencephalogram information generated when the user visualizes the event using the onomatopoeia is acquired. The electroencephalogram information acquired in this manner can then be recorded in association with the event, and the electroencephalogram information can be used as training data for the learning process. As a result, the acquired trained model can accurately determine the event corresponding to the electroencephalogram information.

[0062] FIG. 11 is a diagram illustrating an example of the hardware configuration of an information processing device 90 applied to this embodiment. The information processing device 90 includes a processor 91, a main memory device 92, a communication interface 93, an auxiliary memory device 94, an input / output interface 95, and an internal bus 96. The processor 91, the main memory device 92, the communication interface 93, the auxiliary memory device 94, and the input / output interface 95 are communicably connected to each other via the internal bus 96. The information processing device 90 may be applied to, for example, the terminal device 10, the learning device 20, and the determination device 30. In this case, for example, the communication units 11, 21, and 31 may be configured using the communication interface 93. For example, the memory units 15, 22, and 35 may be configured using the auxiliary memory device 94. Furthermore, the control units 16, 23, and 36 may be configured using the processor 91 and the main memory device 92.

[0063] (First application example) A first application example of this embodiment will be described. The determination device 30 of this embodiment may be used, for example, for training a user to control their electroencephalogram. In this case, the determination device 30 may be configured, for example, as a training support device 40. FIG. 12 is a diagram showing an example configuration of the training support device 40. The training support device 40 is configured using information devices such as a smartphone, a tablet, a personal computer, or a dedicated device.

[0064] The training support device 40 includes a communication unit 41, an input unit 42, an output unit 43, an electroencephalogram information acquisition unit 44, a storage unit 45, and a control unit 46. The communication unit 41, the input unit 42, the output unit 43, the electroencephalogram information acquisition unit 44, the storage unit 45, and the control unit 46 included in the training support device 40 may be configured in the same hardware configuration as the communication unit 31, the input unit 32, the output unit 33, the electroencephalogram information acquisition unit 34, the storage unit 35, and the control unit 36 ​​of the determination device 30, respectively. The control unit 46 of the training support device 40 may function as an output control unit 463 in addition to an information control unit 461 (information control unit 361) and a determination unit 462 (determination unit 362) by executing a program.

[0065] The output control unit 463 outputs a training output via the output unit 43. The training output outputs instructions for an event to be imagined by the trainee (user) (hereinafter referred to as an "instructed event") and an event determined by the determination unit 462 based on the electroencephalogram information currently acquired by the electroencephalogram information acquisition unit 44 (hereinafter referred to as a "determined event").

[0066] 13 is a diagram showing a first display example when training output is performed using an image display device. The screen of the output unit 43 is provided with an instructed event display area 431, an event image display area 432, and a determined event display area 433. The instructed event display area 431 displays a character string indicating the instructed event. The event image display area 432 displays an image (event image) related to the instructed event. The event image display area 432 may display a still image or a moving image as the event image. The determined event display area 433 displays a character string indicating the determined event. By checking the actual determination result in the determined event display area 433 while trying to visualize the event indicated in the instructed event, the user can train to output an electroencephalogram corresponding to the event corresponding to the instructed event (i.e., the event that the user is trying to visualize).

[0067] 14 is a diagram showing a second display example when training output is performed using an image display device. In the second display example, in addition to the display in the first display example, onomatopoeia corresponding to the commanded event is displayed in the onomatopoeia display area 434. The user can visualize the commanded event while looking at the onomatopoeia corresponding to the displayed commanded event, and therefore, can train to more accurately output electroencephalograms corresponding to the event corresponding to the commanded event (i.e., the event that the user is trying to visualize).

[0068] (Second application example) A second application example of this embodiment will be described. The determination device 30 of this embodiment may be used, for example, for operating a video game using electroencephalogram control by a user. In this case, the determination device 30 may be configured as, for example, a video game device 50. FIG. 15 is a diagram showing an example configuration of the video game device 50. The video game device 50 is configured using information devices such as a smartphone, a tablet, a personal computer, or a dedicated device.

[0069] Video game device 50 includes a communication unit 51, an input unit 52, an output unit 53, an electroencephalogram information acquisition unit 54, a storage unit 55, and a control unit 56. The communication unit 51, the input unit 52, the output unit 53, the electroencephalogram information acquisition unit 54, the storage unit 55, and the control unit 56 included in video game device 50 may be configured in the same hardware configuration as communication unit 31, input unit 32, output unit 33, electroencephalogram information acquisition unit 34, the storage unit 35, and the control unit 36 ​​of determination device 30, respectively. By executing a program, control unit 56 of video game device 50 may function as a video game control unit 563 in addition to an information control unit 561 (information control unit 361) and a determination unit 562 (determination unit 362).

[0070] The determination unit 562 passes information indicating the determination result to the video game control unit 563 as operation information. Specific examples of events determined by the determination unit 562 include events indicating actions, etc. taken by an operable character (avatar) in a video game provided by the video game control unit 563. Specific examples of events indicating actions, etc. and onomatopoeia that may be used include, for example, walking (teku teku), running (tat tat), jumping (pyon), and changing direction (guruguru). Other specific examples of events indicating actions, etc. include special abilities (magic, special move, unscientific ability, mystical ability, etc.). Specific examples of such events and onomatopoeia that may be used include generating flames (meramera), generating lightning or electricity (biribiri), generating ice (hiyahiya), etc.

[0071] The video game control unit 563 provides a video game to the user. The video game control unit 563 outputs images and sounds in accordance with the video game program, and progresses the game scenario. The video game control unit 563 progresses the game by using the determination result output from the determination unit 562 as part of the operation information. For example, the video game control unit 563 uses the determination result as information indicating an action to be taken by a character, and causes the character to perform that action.

[0072] When generating a judgment model to be used in the judgment unit 562, the above-described combination of events and onomatopoeia may be used as training information stored in the training information storage unit 151 of the terminal device 10. That is, combinations of events and onomatopoeia used in a predetermined video game may be stored in the training information storage unit 151, and training may be carried out based on the combinations. A trained model obtained by executing a learning process based on the training information thus obtained may be stored in the judgment model storage unit 551 and used in the processing of the judgment unit 562. In this way, the direct involvement of inner speech using onomatopoeia in magic generation, etc., can provide the user with a new video game playing experience and enhance the sense of immersion in the video game.

[0073] (Third application example) A third application example of this embodiment will be described. The determination device 30 of this embodiment may be used, for example, for user interface operation (UI operation: operation of the information processing device to be operated) using electroencephalogram control by the user. In this case, the determination device 30 may be configured as, for example, an information processing device 60. FIG. 16 is a diagram showing an example of the configuration of the information processing device 60. The information processing device 60 is configured using information devices such as a smartphone, a tablet, a personal computer, or a dedicated device.

[0074] The information processing device 60 includes a communication unit 61, an input unit 62, an output unit 63, an electroencephalogram information acquisition unit 64, a storage unit 65, and a control unit 66. The communication unit 61, the input unit 62, the output unit 63, the electroencephalogram information acquisition unit 64, the storage unit 65, and the control unit 66 included in the information processing device 60 may be configured in the same hardware configuration as the communication unit 31, the input unit 32, the output unit 33, the electroencephalogram information acquisition unit 34, the storage unit 35, and the control unit 36 ​​of the determination device 30, respectively. The control unit 66 of the information processing device 60 may function as an operation control unit 663 in addition to an information control unit 661 (information control unit 361) and a determination unit 662 (determination unit 362) by executing a program.

[0075] The determination unit 562 passes information indicating the determination result to the operation control unit 663 as operation information. A specific example of an event determined by the determination unit 562 is an event indicating an operation of a user interface provided by the operation control unit 663. Specific examples of events and onomatopoeia indicating an operation may include, for example, a cursor movement operation (swift), a selection with a cursor (click), a scrolling operation (swoosh), a zoom-in operation (squeak), and a zoom-out operation (whoosh). Such operations by the operation control unit 663 may be performed in an augmented reality environment (XR environment). Specific examples of events and onomatopoeia indicating an operation in an augmented reality environment may include, for example, an operation of pulling an object in an XR environment (pushing), and a selection operation of an application related to the XR space (click).

[0076] The operation control unit 663 operates a specific operation target. The operation target may be a program such as an application program (for example, a program that provides an XR space) or an operating system running on the information processing device 60, or may be an external device (such as a home appliance or a short-range wireless connection device) connected wirelessly or via a wire to the information processing device 60. The operation control unit 663 operates the operation target by using the determination result output from the determination unit 662 as part of the operation information.

[0077] When generating a judgment model to be used in such a judgment unit 662, a combination of the events and onomatopoeia as described above may be used as training information stored in the training information storage unit 151 of the terminal device 10. That is, a combination of events and onomatopoeia used in operating a specific operation target may be stored in the training information storage unit 151, and training may be carried out based on the combination. A trained model obtained by executing a learning process based on the training information obtained in this manner may be stored in the judgment model storage unit 651 and used for processing by the judgment unit 662.

[0078] In this way, by using onomatopoeia as inner speech for operation, users can intuitively interact within the XR environment. The onomatopoeia-based EEG control method has the potential to go beyond conventional UI operation and realize new, more intuitive and efficient device operation interactions for users. This could be a promising technology for improving user experience in today's diverse digital devices and XR environments.

[0079] (Variation) Fig. 17 is a diagram showing a specific example of a screen (third display mode) displayed in a modified example of the terminal device 10. As shown in Fig. 17, in the third display mode, an onomatopoeia input area 133 is provided on the screen of the output unit 13. At this time, the event image display area 132 may also be displayed, or the event image display area 132 may be configured not to be displayed.

[0080] In a modified example of the terminal device 10, the training information stored in the training information storage unit 151 does not initially need to contain onomatopoeia. The output control unit 161 of the terminal device 10 outputs the event to be recorded, rather than outputting an onomatopoeia corresponding to the event to be recorded, at the output of the output unit 13. In the example of FIG. 17, a character string indicating the event, such as "rotate," is displayed in the upper left. In response, the user inputs the onomatopoeia they envision in response to the displayed event in the onomatopoeia input area 133. In the above example, the onomatopoeia "kurukuru" (spinning) was defined for the event "rotate." However, in a modified example, the user can select and define their own onomatopoeia corresponding to the event. The user can then visualize the event using the onomatopoeia (for example, by reading it aloud in their head once or multiple times).

[0081] The electroencephalogram information recording control unit 162 records in the electroencephalogram information storage unit 152 the onomatopoeia input in the onomatopoeia input area 133, the event to be recorded, and the electroencephalogram information obtained from the user in association with each other.

[0082] In this embodiment, the terminal device 10 and the learning device 20 are configured as separate devices, but they may also be configured as an integrated device. FIG. 18 is a diagram showing a modified example of the terminal device 10 configured in this manner. As shown in FIG. 18, the storage unit 15 of the terminal device 10 also functions as a trained model storage unit 222. The control unit 16 of the terminal device 10 also functions as an information control unit 231 and a learning control unit 232. In this configuration, the information stored in the electroencephalogram information storage unit 152 may be used as training data for learning.

[0083] In this embodiment, the terminal device 10 and the determination device 30 are configured as separate devices, but they may also be configured as an integrated device. FIG. 19 is a diagram showing a modified example of the terminal device 10 configured in this manner. As shown in FIG. 19, the storage unit 15 of the terminal device 10 also functions as a determination model storage unit 351. The control unit 16 of the terminal device 10 also functions as an information control unit 361 and a determination unit 362.

[0084] In this embodiment, the learning device 20 and the determination device 30 are configured as separate devices, but they may also be configured as an integrated device. FIG. 20 is a diagram showing a modified example of the determination device 30 configured in this manner. The storage unit 35 of the determination device 30 shown in FIG. 20 also functions as a teacher data storage unit 221. The control unit 36 ​​of the determination device 30 shown in FIG. 20 also functions as a learning control unit 232. In this configuration, a trained model is recorded in the determination model storage unit 351.

[0085] In this embodiment, the terminal device 10, the learning device 20, and the determination device 30 are configured as separate devices, but they may also be configured as an integrated device. FIG. 21 is a diagram showing a modified example of the terminal device 10 configured in this manner. The storage unit 15 of the terminal device 10 shown in FIG. 21 also functions as a trained model storage unit 222. The control unit 16 of the terminal device 10 also functions as an information control unit 231, a learning control unit 232, an information control unit 361, and a determination unit 362. In this configuration, the information stored in the electroencephalogram information storage unit 152 may be used as training data for learning. Furthermore, the trained model recorded in the trained model storage unit 222 may be used as a determination model for determination. For example, when the terminal device 10 shown in FIG. 21 is used as the above-mentioned video game device 50, the training information storage unit 151 and the determination unit 362 function as described above. Furthermore, the terminal device 10 may be configured to further include a video game control unit 563. In this case, the terminal device 10 may first function as the terminal device 10 to acquire electroencephalogram information for each event of the user of the video game, then function as the learning device 20 to generate a trained model based on the acquired electroencephalogram information, and then function as the determination device 30 (video game device 50) to provide a video game based on the determination results using the trained model. Similarly, the terminal device 10 may be used as the information processing device 60 described above.

[0086] Learning device 20 may be implemented using multiple information processing devices. For example, learning device 20 may be implemented using a device such as a cloud. For example, in learning device 20, memory unit 22 and control unit 23 may be implemented in different information processing devices. For example, memory unit 22 of learning device 20 may be distributed and implemented in multiple information processing devices.

[0087] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0088] (Experimental results) The following describes the results of an experiment on the effect of combining the events described in this embodiment with onomatopoeia. The usefulness of training using onomatopoeia was investigated by imagining a specific event under the following two conditions: (1) visual imagery only, and (2) multimodal, consisting of visual imagery and speech imagery using onomatopoeia.

[0089] The participants were eight people (five men and three women), aged between 21 and 25 years old. The EEG acquisition unit used a device equipped with 16 sensors (two of which were reference sensors) that recorded electrical signals from the scalp. Each sensor was attached with saline-soaked felt to improve conductivity. The sensor positions used the international 10-20 system. The sampling frequency was set to 128 Hz to acquire EEG data. Information from the headset was transmitted to a computer via Bluetooth, and the EEG data was used to detect the user's facial expressions, mental state, and intentional thoughts. The user's EEG data was saved and used to develop unique algorithms. For example, during the training phase, the software learned the training data and developed a detection algorithm tailored to each action. The software then analyzed the user's EEG signals in real time, classifying each frame into one of the learned actions.

[0090] The experiment began after an overall explanation of the experiment was given to the participants. This process was important to ensure participants' understanding and increase the reliability of the experiment. In particular, because differences in knowledge among participants and distrust of the technology could affect performance, we carefully explained the EEG equipment and the contents of the experiment. The specific flow of the experiment is as follows.

[0091] (1) Receive an explanation about the experiment. (2) Wear the EEG information acquisition unit and check for proper fit. (3) Practice mental imagery. (4) Conduct training. (5) Conduct exercises. (6) Answer a questionnaire using a specified subjective evaluation method (NASA-TLX: an evaluation method for quantifying workload perception). (7) Answer an oral free-form questionnaire.

[0092] Through these procedures, we collected detailed information about participants' brain waves, scores on a predetermined subjective evaluation method, and their impressions of the experiment. In this experiment, participants performed a task of manipulating a cube on a screen. Participants manipulated the cube to cause two events: to stand still and to rotate. In the "Neutral" mode, participants imagined the cube standing still in a relaxed state, and in the "Rotate" mode, they imagined rotating the cube. Training was conducted for each event (each action), and the software acquired training data and learned from it. A test was then conducted to classify these actions.

[0093] In each training session, participants watched a 10-second video and performed a similar mental image. The software gradually learned from each session, improving its ability to classify actions. Participants completed 20 sessions for each action to strengthen specific neural patterns.

[0094] In the exercise, participants were tested to classify neutral and rotated actions. The test screen displayed instructions for the action the participant was imagining, along with the action detected by the headset. In other words, participants received real-time visual feedback while imagining the instructed action. The action instructions changed every 10 seconds, with a short break after 20 attempts. This trial was repeated five times, for a total of 100 classification attempts. The action classification was considered correct when the headset detected the instructed action more than half of the time within the 10 seconds.

[0095] Figures 22 and 23 show the experimental results. The test scores (correctness rates) for the exercises were calculated and used as user performance indicators. The exercise tests assessed the extent to which participants could classify neutral and rotated actions. In other words, the correctness rate indicates the extent to which participants could consistently generate specific EEG patterns. A nonparametric Wilcoxon signed-rank test was performed on the correctness rates for each condition. This analysis method can obtain appropriate results even with a small sample size. The results showed that the correctness rate was significantly higher in the multimodal condition than in the visual imagery-only condition (p<0.05). The boxplot in Figure 22 compares the correctness rates under the two conditions. These results demonstrate that the method of this embodiment is useful for improving user performance in training methods such as MI-BCI.

[0096] We also investigated the impact of multimodal imagery, consisting of visual imagery and onomatopoeic speech imagery, on cognitive load. To assess cognitive load, we conducted subjective evaluations using the NASA-TLX. The boxplot shown in Figure 23 compares NASA-TLX scores under the two conditions. Since a higher score on the NASA-TLX indicates a greater cognitive load, this figure shows that cognitive load tends to be lower under the multimodal condition. Furthermore, we performed a Wilcoxon signed-rank test on the NASA-TLX scores. The results showed a significant difference in scores between the visual imagery-only condition and the multimodal condition (p<.05). This suggests that the multimodal approach of visual imagery and onomatopoeic speech imagery can reduce the cognitive load during image generation.

[0097] At the end of the experiment, an oral free-form questionnaire was conducted. The purpose of this questionnaire was to investigate the ease of image generation and the effect on fatigue when image generation was performed under the two conditions. The following opinions were frequently expressed regarding image generation under the two conditions:

[0098] · Particular images were more likely to be repeated in the multimodal condition. The multimodal condition required less thinking. - The multimodal condition caused less fatigue. The onomatopoeia "Kuru Kuru" perfectly matched the image of a rotating cube.

[0099] These comments indicate that the multimodal approach of visual imagery and onomatopoeic speech imagery had a positive effect on brainwave control. In particular, repeating onomatopoeic speech imagery made it easier to generate specific images, reducing the mental burden. However, two participants responded that "the ease of image generation was the same under the two conditions." This comment came from participants who had low accuracy rates under both conditions. In other words, participants who were unable to satisfactorily control their brainwaves under either condition were unable to realize any difference in the ease of image generation. These comments suggest that the multimodal approach of visual imagery and onomatopoeic speech imagery is useful for many participants, but not for all. [Explanation of symbols]

[0100] 100...information processing system, 10...terminal device, 11...communication unit, 12...input unit, 13...output unit, 14...EEG information acquisition unit, 15...storage unit, 151...training information storage unit, 152...EEG information storage unit, 16...control unit, 161...output control unit, 162...EEG information recording control unit, 20...learning device, 21...communication unit, 22...storage unit, 221...teacher data storage unit, 222...trained model storage unit, 23...control unit, 231...information control unit, 232...learning control unit, 30...determination device, 31...communication unit, 32...input unit, 33...output unit, 34...EEG information acquisition unit, 35...storage unit, 351...determination model storage unit, 36...control unit, 361...information control unit, 362...determination unit

Claims

1. an output control unit that outputs a character string related to the event to be recorded in a manner that is recognizable by a user whose electroencephalogram information is to be recorded; An information processing device comprising: an electroencephalogram information recording control unit that acquires the user's electroencephalogram information during a period corresponding to the output of the string by the output control unit and records it in association with the event to be recorded.

2. The information processing device according to claim 1 , wherein the character string is a character string representing an onomatopoeia corresponding to the event.

3. The information processing device according to claim 1 , wherein the output control unit displays the character string on an image display device, and further displays an image related to the event on the image display device.

4. 2. The information processing device according to claim 1, further comprising a learning unit that acquires a trained model by performing a learning process using learning information that associates the user's training electroencephalogram information obtained in response to the output of a character string related to the event to be recorded with the event to be recorded.

5. The information processing device according to claim 4, further comprising a judgment unit that uses the electroencephalogram information for judgment obtained from the user and the trained model to judge the event that the user imagined when the electroencephalogram information for judgment was obtained.

6. an output control unit that indicates an event in a manner perceptible to the user; the determination unit determines the event imagined by the user using electroencephalogram information for determination obtained when the user imagines the event after the instruction is output by the output control unit; The information processing device according to claim 5 , wherein the output control unit outputs information indicating the determination result by the determination unit in a manner that can be recognized by the user.

7. an output control unit that outputs a character string related to the event to be recorded in a manner that is recognizable by a user whose electroencephalogram information is to be recorded; an electroencephalogram information recording control unit that acquires electroencephalogram information of the user during a period corresponding to the character string being output by the output control unit and records the information in association with the event to be recorded; a storage unit that stores a trained model obtained by performing a training process using training information that associates the user's training electroencephalogram information, obtained in response to the output of a character string related to the event to be recorded, with the event to be recorded; and An information processing system comprising: a judgment unit that uses judgment-use electroencephalogram information obtained from the user and the trained model to judge the event that the user imagined when the judgment-use electroencephalogram information was obtained.

8. The information processing system of claim 7 further comprises a learning processing unit that acquires the trained model by performing a learning process using learning information that associates the user's training electroencephalogram information obtained in response to the output of a character string related to the event to be recorded with the event to be recorded.

9. an output control step of outputting a character string related to the event to be recorded in a manner that is recognizable by the user of the subject of the electroencephalogram information recording; an electroencephalogram information recording control step for acquiring the user's electroencephalogram information during a period corresponding to the output of the character string in the output control step and recording the information in association with the event to be recorded.

10. an output control unit that outputs a character string related to the event to be recorded in a manner that is recognizable by a user whose electroencephalogram information is to be recorded; A computer program for causing a computer to function as an information processing device comprising: an electroencephalogram information recording control unit that acquires the user's electroencephalogram information during a period corresponding to the output of the string by the output control unit and records it in association with the event to be recorded.

Images