Operation control device, operation control method, and program
The operation control device addresses unintended EEG commands in BMIs by using image commands and passwords to verify and execute relevant brain signals, enhancing command accuracy and control.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- JVC KENWOOD CORP
- Filing Date
- 2021-09-14
- Publication Date
- 2026-05-19
AI Technical Summary
Conventional brain-machine interfaces (BMIs) are susceptible to issuing unintended commands due to irrelevant stimuli, such as verbal cues or visual feedback, leading to inaccurate electroencephalogram (EEG) commands.
An operation control device that includes a brain information acquisition unit, a storage unit for image commands, a determination unit to verify the relevance of acquired brain information, and an execution unit to process appropriate commands, using image commands and passwords to ensure accurate command execution.
Enables execution of appropriate EEG commands without interference from irrelevant stimuli, ensuring accurate and controlled operation of devices like robots.
Smart Images
Figure 0007861375000001 
Figure 0007861375000002 
Figure 0007861375000003
Abstract
Description
Technical Field
[0001] The present invention relates to an operation control device, an operation control method, and a program.
Background Art
[0002] In recent years, technologies for measuring brain information have been developed, and a brain-machine interface (BMI), which is an interface that directly connects the brain and a machine, has become realistic. As such a technology, for example, there is one described in Patent Document 1. Patent Document 1 obtains a plurality of steady-state visual evoked potential signals from a user when the user receives a stimulus and supplies a system command signal.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] In a conventional BMI, when executing an electroencephalogram command, if there is an irrelevant stimulus, there is a risk that the stimulus will appear in the electroencephalogram and be executed. For example, in the case of a forward command where when thinking "forward" to a robot, the robot moves forward, if someone around says "forward", the brain may spontaneously recall "forward" and issue a forward command. Also, when operating while watching the movement of the robot, if the state of the robot moving forward is displayed on the monitor, the brain of the operator who sees it will continue to spontaneously recall "forward", so a forward command will be issued without permission.
[0005] The present invention has been made in view of the above, and an object thereof is to be able to execute an appropriate electroencephalogram command without being affected by an irrelevant stimulus. [Means for solving the problem]
[0006] To solve the above-mentioned problems and achieve the objective, the operation control device according to the present invention comprises: a brain information acquisition unit that acquires the user's brain information; a storage unit that stores the content of operations for operating an object as an image command based on an image system; a determination unit that determines whether the brain information acquired by the brain information acquisition unit corresponds to the brain information stored in the storage unit; and an execution unit that executes processing of the content of operations corresponding to the image command when the determination unit determines that it has acquired brain information corresponding to the image command.
[0007] The operation control method according to the present invention includes the steps of acquiring user brain information, storing operation content for operating an object as an image command based on an image system, and determining whether the acquired brain information corresponds to the stored image.
[0008] The program according to the present invention causes a computer to perform the following steps: acquire user brain information; store the content of operations for operating on an object as an image command based on an image system; and determine whether the acquired brain information corresponds to the stored image. [Effects of the Invention]
[0009] According to the present invention, the effect is achieved that appropriate electroencephalogram (EEG) commands can be executed without being affected by irrelevant stimuli. [Brief explanation of the drawing]
[0010] [Figure 1] Figure 1 is a block diagram showing an example configuration of an operation control device according to the first embodiment. [Figure 2] Figure 2 is a flowchart showing the processing flow in the operation control device according to the first embodiment. [Figure 3]Figure 3 is a flowchart showing the flow of the command set selection process. [Figure 4] Figure 4 is a block diagram showing an example configuration of the operation control device according to the second embodiment. [Figure 5] Figure 5 is a flowchart showing the flow of the mode switching process in the operation control device according to the second embodiment. [Modes for carrying out the invention]
[0011] Embodiments of the operation control device, operation control method, and program according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.
[0012] <First Embodiment> [Operation control device] Figure 1 is a block diagram showing an example configuration of an operation control device according to the first embodiment.
[0013] In the first embodiment, as shown in Figure 1, the operation control device 10 controls the operation of the object 50 based on the user's brain information. The operation control device 10 includes an electroencephalogram (EEG) acquisition unit (brain information acquisition unit) 11, an EEG decoder 12, a command buffer 13, a password buffer 14, a storage unit 15, a processing unit 16, a display control unit 17, a display unit 18, and a communication unit 19.
[0014] The object to be operated 50 is, for example, a robot, but is not limited to a robot; it can be any device that operates via communication. The object to be operated 50, although not shown in the diagram, is equipped with a drive unit, a control unit that controls the drive unit, and a communication unit that can receive control signals. When the communication unit receives a control signal, the control unit drives the drive unit, thereby operating the robot.
[0015] The electroencephalogram (EEG) acquisition unit 11 can be attached to the user's head. The EEG decoder 12, processing unit 16, display control unit 17, display unit 18, and communication unit 19 may be worn by the user or placed in a predetermined location without being worn. Furthermore, the EEG acquisition unit 11, EEG decoder 12, processing unit 16, display control unit 17, display unit 18, and communication unit 19 may be provided as a single unit or as separate units.
[0016] The electroencephalogram (EEG) acquisition unit 11 acquires brain waves, which are the brain information of the user. The EEG acquisition unit 11 has, for example, an electrical sensor (e.g., an electrode) that detects brain waves, which are generated from weak electrical currents flowing through the neural network of the brain. The EEG acquisition unit 11 detects the potential (electrical signal) of weak electrical currents when the user receives external stimuli or when the user has thoughts or other thoughts. Note that the brain information acquisition unit is not limited to the EEG acquisition unit 11. The brain information acquisition unit may, for example, acquire blood flow due to brain activity, which is the brain information of the user, by, for example, near-infrared light measurement. The EEG acquisition unit 11 is connected to the EEG decoder 12 and transmits the brain waves, which are the brain information acquired from the user, to the EEG decoder 12.
[0017] The electroencephalogram decoder 12 reconstructs the electrical signals of the user's brainwaves acquired by the brainwave acquisition unit 11 into the user's thought information. In this case, the relationship between multiple electrical signals of the user's brainwaves and the user's thought information is linked in advance. For example, machine learning such as deep learning is used to link the electrical signals of the brainwaves with the user's thought information.
[0018] The command buffer 13 temporarily stores the first command determined by the determination unit 21 based on the user's thinking information. The first command is an image command set based on the image system for the operation content to operate the operation target 50. The command for the operation content to operate the operation target 50 is brain information in the left brain, and the brain information is a command based on the language system (character information). The command for the operation content to operate the operation target 50 is a command to activate the operation target 50, and for example, commands such as "forward", "backward", "stop", "accelerate", "decelerate", "right turn", "left turn", etc.
[0019] On the other hand, the image command is brain information in the right brain, and the brain information is a command based on the image system. The image command is, for example, "dog", "cat", "sea", "sound of a clock", "sound of a bell", etc. The image command is set to link to the command for the operation content. That is, image commands such as "dog", "cat", "sea", "sound of a clock", "sound of a bell" as the first command are set corresponding to "forward", "backward", "stop", "accelerate", "decelerate", "right turn", "left turn" as the operation content of the operation target 50.
[0020] In this case, a command set is set in which the first image command corresponding to the first operation of the operation target 50 and the second image command corresponding to the second operation paired with the first operation are paired. Here, if the first operation is "activation", the second operation is "stop" paired with the first operation. Also, if the first operation is "right turn", the second operation is "left turn" paired with the first operation. Further, if the first operation is "forward", the second operation is "backward" paired with the first operation.
[0021] Also, the first image command and the second image command are set as a plurality of command sets with different brain internal semantic spaces. The brain internal semantic space visualizes how multiple semantic categories are represented with what distance relationships in the brain. For example, set the first action as "operation" and the second action as "stop", set the first image command corresponding to the first action as "dog", and set the second image command corresponding to the second action as "cat". Here, since the first image command "dog" and the second image command "cat" are similar semantic categories of animals, the brain internal semantic space is close. On the other hand, set the first image command corresponding to the first action "operation" as "dog", and set the second image command corresponding to the second action "stop" as "sea". Here, since the first image command "dog" and the second image command "sea" are different semantic categories of animals and nature, the brain internal semantic space is far.
[0022] That is, the pair of the first action "operation" - the first image command "dog" and the second action "stop" - the second image command "cat" is set as a short-distance command set. On the other hand, the pair of the first action "operation" - the first image command "dog" and the second action "stop" - the second image command "sea" is set as a long-distance command set. Also, the pair of the first action "right turn" - the first image command "tiger" and the second action "left turn" - the second image command "lion" is set as a short-distance command set. On the other hand, the pair of the first action "right turn" - the first image command "tiger" and the second action "left turn" - the second image command "sky" is set as a long-distance command set.
[0023] For example, a user who is adept at differentiating between the first and second image commands for the operation of the object 50 will use the short-range command set. On the other hand, a user who is not adept at differentiating between the first and second image commands for the operation of the object 50 will use the long-range command set. Note that the number of command sets is not limited to just two types, a short-range command set and a long-range command set; for example, there may be three types including a medium-range command set, or even four or more types may be set.
[0024] Then, a command set consisting of a first image command and a second image command for the operation content of the object to be operated 50 is set in advance and stored in the storage unit 15. The command buffer 13 can send and receive various types of data with the processing unit 16.
[0025] The password buffer 14 temporarily stores the password, which is the second command determined by the determination unit 21 based on the user's thought information. The password, which is the second command, is the command used to decide whether to execute the first command. The password corresponding to the command is set in advance by the user and stored in the storage unit 15. Alternatively, the password may be set by the processing unit 16, stored in the storage unit 15, and also notified to the user in advance. The password is a combination of one or more strings, images of objects, abstract concepts, etc. A single common password may be set for different first commands, or multiple individual passwords may be set for different first commands. By setting an individual password for each first command, such as "forward," "backward," "stop," "accelerate," "deceleration," "turn right," and "turn left," it is possible to avoid accidentally issuing the next first command. The password buffer 14 can send and receive various types of data with the processing unit 16.
[0026] The memory unit 15 stores multiple first commands and multiple second commands, which are passwords. Here, the multiple first commands are image commands representing the first and second actions for the operation on the object 50, and are stored as a command set. The memory unit 15 may also store learning data obtained by machine learning the relationship between the user's brainwave electrical signals and thought information. The second commands are pre-associated with the first commands.
[0027] The command buffer 13 and password buffer 14 are recording units such as RAM (Random Access Memory) and flash memory, which are semiconductor memory elements provided by the processing unit 16. The storage unit 15 consists of a memory card, SSD (Solid State Drive), external storage device, etc.
[0028] The processing unit 16 is an arithmetic processing unit (control unit) composed of, for example, a CPU (Central Processing Unit). The processing unit 16 loads the stored program into memory and executes the instructions contained in the program. The processing unit 16 includes internal memory (not shown), which is used for temporary storage of data in the processing unit 16. The processing unit 16 has the functions of a determination unit 21, an execution unit 22, and a command set switching unit 23. The processing unit 16 also transmits the processing results to the display control unit 17 and the communication unit 19.
[0029] The determination unit 21 determines whether the user's thought information acquired by the brainwave acquisition unit 11 and reconstructed by the brainwave decoder 12 corresponds to two different first and second commands (passwords). In this case, the determination unit 21 determines whether the user's thought information acquired by the brainwave acquisition unit 11 and reconstructed by the brainwave decoder 12 corresponds to the brain information of the first command, which is an image.
[0030] Specifically, the determination unit 21 compares the user's thought information reconstructed by the EEG decoder 12 with the first command stored in the memory unit 15. At this time, if the determination unit 21 finds that the user's thought information matches the first command, it determines that the user's thought information is the first command. In this case, the determination unit 21 stores the command it has determined to be the first command in the command buffer 13. On the other hand, if the user's thought information does not match the first command, the determination unit 21 determines that the user's thought information is not the first command. In this case, the determination unit 21 does not store the command it has determined to be not the first command in the command buffer 13. The determination unit 21's judgment of a match between the user's thought information reconstructed by the EEG decoder 12 and the first command is based on the ability to determine that the user's thought information represents the first command, even if it is not a perfect match.
[0031] Furthermore, the determination unit 21 compares the user's thought information reconstructed by the EEG decoder 12 with the password stored in the memory unit 15. At this time, if the user's thought information matches the password, the determination unit 21 determines that the user's thought information is a password. When the determination unit 21 determines that the user's thought information is a password, it stores the password in the password buffer 14. On the other hand, if the user's thought information does not match the password stored in the memory unit 15, the determination unit 21 determines that the user's thought information is not a password. When the determination unit 21 determines that the user's thought information is not a password, it does not store the password in the password buffer 14.
[0032] Furthermore, the determination unit 21 determines whether or not the command stored in the command buffer 13 and the password stored in the password buffer 14 are in a corresponding relationship.
[0033] When the execution unit 22 determines that the determination unit 21 has acquired brainwaves corresponding to the first command and brainwaves corresponding to the password, it executes the processing of the operation content linked to the first command. Specifically, when the execution unit 22 determines that the first user thought information acquired by the determination unit 21 is the first command, and the next user thought information acquired by the determination unit 21 is the password, it executes the operation content of the object 50 linked to the image command as the first command.
[0034] However, the execution unit 22 cancels the acquired first command after a predetermined waiting time, such as 3 seconds, has elapsed since the determination unit 21 determined that the first command had been acquired. Also, the execution unit 22 cancels the acquired first command if the processing unit 16 acquires a cancel command as a thought of the user after the determination unit 21 determined that the first command had been acquired. The cancel command is predetermined, and the relationship between the electrical signals of the user's brainwaves and the user's thought information (cancel command) is linked in advance and stored in the memory unit 15. A cancel command is a symbol or concept different from the first command, such as "×" or "cancel".
[0035] The command set switching unit 23 selects and switches to an appropriate command set from among multiple command sets stored in the memory unit 15 based on the brain information acquired by the electroencephalogram (EEG) acquisition unit 11. Specifically, the memory unit 15 stores multiple command sets, such as a short-range command set with a similar brain semantic space and a long-range command set with a different brain semantic space. The command set switching unit 23 selects a command set suitable for the user from among the multiple command sets stored in the memory unit 15. In this example, the command set switching unit 23 is configured to select an appropriate command set from among multiple command sets based on brain information, but this configuration is not limited to this. For example, the command set switching unit 23 may present the user with multiple command sets stored in the memory unit 15 and switch to the command set selected by the user.
[0036] The display control unit 17 is connected to the display unit 18. The display control unit 17 transmits the processing results sent from the processing unit 16 to the display unit 18 for display. The display control unit 17 displays the operating status of the operation control device 10 and the thought instructions for the user, which are the processing results of the processing unit 16, on the display unit 18.
[0037] The display unit 18 displays the operating status of the operation control device 10 and thought instructions for the user transmitted from the display control unit 17. The display unit 18 presents the user with necessary information. The display unit 18 is a display including, for example, a liquid crystal display (LCD) or an organic electro-luminescence (OLED) display.
[0038] The communication unit 19 is capable of wireless communication with the object to be operated 50. The communication unit 19 transmits the processing information processed by the processing unit 16 to the object to be operated 50. Specifically, when the determination unit 21 determines that it has acquired the brainwaves corresponding to the first command and the brainwaves corresponding to the password, the execution unit 22 generates a control signal corresponding to the processing of the first command and sends it to the communication unit 19. The communication unit 19 transmits the control signal corresponding to the processing of the first command to the object to be operated 50. The object to be operated 50 receives the first command transmitted by the communication unit 19 and operates in response to the processing of the first command.
[0039] [Operation control method] Here, we will explain the operation control method by the operation control device 10. Figure 2 is a flowchart showing the processing flow in the operation control device according to the first embodiment, and Figure 3 is a flowchart showing the command set selection processing flow.
[0040] As shown in Figures 1 and 2, in step S11, the command set switching unit 23 of the processing unit 16 selects the optimal command set from among the multiple command sets stored in the memory unit 15 based on the user's brain information. In this case, the command set switching unit 23 may select the optimal command set based on the user's brain information, or it may select a command set based on a selection instruction from the user.
[0041] For example, as shown in Figure 3, in step S31, the command set switching unit 23 causes the display control unit 17 to display "Say 'dog' in your head" on the display unit 18. In step S32, the command set switching unit 23 stores the brain activation map A, which is the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12, in the memory unit 15. In step S33, the command set switching unit 23 causes the display control unit 17 to display "Say 'cat' in your head" on the display unit 18. In step S34, the command set switching unit 23 stores the brain activation map B, which is the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12, in the memory unit 15.
[0042] In step S35, the command set switching unit 23 determines whether there is a pre-set difference between brain activation map A and brain activation map B. If the command set switching unit 23 determines that there is a difference between brain activation map A and brain activation map B (Yes), in step S36, it sets to the first command set (for example, the close-range command set) and terminates. On the other hand, if the command set switching unit 23 determines that there is no difference between brain activation map A and brain activation map B (No), in step S37, the command set switching unit 23 has the display control unit 17 display "Say 'sea' in your head" on the display unit 18. In step S38, the command set switching unit 23 stores the brain activation map C, which is the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12, in the memory unit 15.
[0043] In step S39, the command set switching unit 23 determines whether there is a pre-set difference between brain activation map A and brain activation map C. If the command set switching unit 23 determines that there is a difference between brain activation map A and brain activation map C (Yes), in step S40, it sets to the second command set (for example, the mid-to-short range command set) and terminates. On the other hand, if the command set switching unit 23 determines that there is no difference between brain activation map A and brain activation map C (No), in step S41, the command set switching unit 23 has the display control unit 17 display "Imagine the sound of a bell in your head" on the display unit 18. In step S42, the command set switching unit 23 stores the brain activation map D, which is the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12, in the memory unit 15. In step S43, the command set switching unit 23 has the display control unit 17 display "Imagine the sound of a bell in your head" on the display unit 18. In step S44, the command set switching unit 23 stores the brain activation map E, which is the user's thought information acquired by the brainwave acquisition unit 11 and restored by the brainwave decoder 12, in the memory unit 15.
[0044] In step S45, the command set switching unit 23 determines whether there is a pre-set difference between brain activation map D and brain activation map E. If the command set switching unit 23 determines that there is a difference between brain activation map D and brain activation map E (Yes), in step S46, it sets to the third command set (for example, the long-distance command set) and terminates. On the other hand, if the command set switching unit 23 determines that there is no difference between brain activation map D and brain activation map E (No), it performs the same processing as in steps S41 to S45 and sets a command set where the brain semantic space is farther away.
[0045] Returning to Figures 1 and 2, once the appropriate command set has been selected by the user, in step S12, the processing unit 16 controls the display control unit 17 to display an instruction on the display unit 18 prompting the user to recall the first command. The display control unit 17 displays a message on the display unit 18 instructing the user to recall the first command, such as "Please recall the first command".
[0046] In step S13, the determination unit 21 determines whether the user's thought information acquired by the electroencephalogram acquisition unit 11 and reconstructed by the electroencephalogram decoder 12 corresponds to the first command. Here, the determination unit 21 compares the user's thought information reconstructed by the electroencephalogram decoder 12 with the first command stored in the memory unit 15. If the determination unit 21 determines that the user's thought information does not match the first command (No), it exits this routine, determining that the user's thought information is not the first command. On the other hand, if the determination unit 21 determines that the user's thought information matches the first command stored in the memory unit 15 (Yes), it determines that the user's thought information is the first command, stores the first command in the command buffer 13, and proceeds to step S14.
[0047] In step S14, the processing unit 16 determines whether a timeout has occurred after a predetermined waiting time has elapsed since the determination unit 21 determined that the first command had been acquired. If the processing unit 16 determines that a timeout has occurred (Yes), it cancels the acquired first command in step S20. Then, in step S21, the processing unit 16 controls the display control unit 17 to display to the user on the display unit 18 that the first command has been canceled. The display control unit 17 displays a timeout message on the display unit 18, for example, "Timeout."
[0048] If the processing unit 16 determines that a timeout has not occurred (No), it determines in step S15 whether or not it has detected an EEG corresponding to the cancel command. If the processing unit 16 determines that it has detected an EEG corresponding to the cancel command (Yes), it cancels the acquired first command in step S22. Then, in step S23, the processing unit 16 controls the display control unit 17 to display to the user on the display unit 18 that the first command has been canceled. The display control unit 17 displays a message on the display unit 18 indicating that the first command has been canceled, for example, "The first command will be canceled."
[0049] On the other hand, if the processing unit 16 determines (No) that it has not detected an EEG corresponding to the cancel command, in step S15, the determination unit 21 determines whether or not it has detected an EEG corresponding to the second command (password). Here, the determination unit 21 compares the user's thought information recovered by the EEG decoder 12 with the password stored in the memory unit 15. If it determines (No) that the user's thought information does not match the password, it returns to step S13 and continues processing.
[0050] On the other hand, if the determination unit 21 determines (Yes) that the user's thought information matches a password stored in the memory unit 15, it stores the password that it determined to be the user's thought information as a password in the password buffer 14. Furthermore, in step S17, the determination unit 21 determines whether the first command, which was determined in step S13 and stored in the command buffer 13, and the password, which was determined in step S16 and stored in the password buffer 14, are a first command and password that have been pre-associated. If the determination unit 21 determines (No) that the first command and password are not associated, it returns to step S14. If the determination unit 21 determines (Yes) that the first command and password are associated, it proceeds to step S18.
[0051] In step S18, the execution unit 22 executes the processing corresponding to the first command. That is, the execution unit 22 transmits a control signal corresponding to the processing of the first command to the object to be operated 50 via the communication unit 19, thereby activating the object to be operated 50. Then, in step S19, the processing unit 16 controls the display control unit 17 to display to the user that the first command has been executed on the display unit 18. The display control unit 17 displays a message on the display unit 18 indicating that the first command has been executed, for example, "Acceleration command executed".
[0052] <Second Embodiment> Figure 4 is a block diagram showing an example configuration of the operation control device according to the second embodiment. Components having the same functions as those in the first embodiment described above are denoted by the same reference numerals, and detailed descriptions are omitted.
[0053] In the second embodiment, as shown in Figure 4, the operation control device 10 controls the operation of the object 50 based on the user's brain information. The operation control device 10 includes an electroencephalogram (EEG) acquisition unit (brain information acquisition unit) 11, an EEG decoder 12, a command buffer 13, a password buffer 14, a storage unit 15, a processing unit 16, a display control unit 17, a display unit 18, and a communication unit 19. The EEG acquisition unit 11, EEG decoder 12, command buffer 13, password buffer 14, storage unit 15, display control unit 17, display unit 18, and communication unit 19 are the same as in the first embodiment.
[0054] The processing unit 16 has the functions of a determination unit 21, an execution unit 22, a command set switching unit 23, and a mode switching unit 24. The determination unit 21, the execution unit 22, and the command set switching unit 23 are the same as in the first embodiment.
[0055] The mode switching unit 24 switches between execution mode and autistic mode (non-execution mode). Execution mode is a mode in which the execution unit 22 processes the operation content corresponding to the first command (image command). On the other hand, autistic mode is a mode in which the execution unit 22 does not process the operation content corresponding to the first command (image command). The mode switching unit 24 switches between execution mode and autistic mode based on the user's thought information acquired by the electroencephalogram acquisition unit 11 and reconstructed by the electroencephalogram decoder 12.
[0056] Specifically, the execution mode is a mode that enables communication with the outside world, and in this embodiment, it enables the transmission of signals from the processing unit 16 to the communication unit 19. In execution mode, for example, a WAN (Wide Area Network) such as the Internet is connected. Therefore, the user can operate the object to be operated 50 in execution mode. On the other hand, the closed mode blocks the transmission of signals from the processing unit 16 to the communication unit 19. In closed mode, for example, a WAN (Wide Area Network) such as the Internet is blocked. Therefore, the user cannot operate the object to be operated 50 in closed mode. That is, in closed mode, for example, only the human body surface current interface is enabled, and the user can switch between execution mode and closed mode using the mode switching unit 24.
[0057] Figure 5 is a flowchart showing the flow of the mode switching process in the operation control device according to the second embodiment.
[0058] As shown in Figures 4 and 5, in step S51, the processing unit 16 determines whether the user's thought information acquired by the EEG acquisition unit 11 and reconstructed by the EEG decoder 12 is a mode switching command. A mode switching command is, for example, a command such as "Switch modes". If the processing unit 16 determines that the user's thought information is a mode switching command (Yes), then in step S52, the mode switching unit 24 switches the mode. That is, if the current mode is execution mode, the mode switching unit 24 switches to autism mode, and if it is autism mode, it switches back to execution mode. At this time, the mode switching unit 24 controls the display control unit 17 to display to the user that the mode has been switched on the display unit 18. For example, if the mode has been switched to execution mode, the display control unit 17 displays a message such as "Switched to execution mode" on the display unit 18, and if the mode has been switched to autism mode, it displays a message such as "Switched to autism mode" on the display unit 18 to inform the user that the mode has been switched.
[0059] On the other hand, in step S51, if the processing unit 16 determines that the user's thought information is not a mode switching command (No), then in step S53, it determines whether the execution mode is currently selected. If the processing unit 16 determines that the execution mode is currently selected (Yes), then in step S54, the processing unit 16 connects with the communication unit 19 and enters an external communication connection state. Then, in step S55, it starts processing in the execution mode.
[0060] On the other hand, in step S53, if the processing unit 16 determines that no execution mode is currently selected (No), in step S56, the processing unit 16 disconnects from the communication unit 19 and puts the system into an external communication blocking state. Then, in step S57, it starts processing in self-closed mode.
[0061] The processing for the execution mode is the same as in the first embodiment, so its explanation will be omitted. The processing for the autistic mode is basically the mode switching process shown in Figure 5.
[0062] [effect] In this embodiment, the system includes an electroencephalogram (EEG) acquisition unit (brain information acquisition unit) 11 that acquires the user's brain information, a storage unit 15 that stores the operation content for operating the object to be operated on 50 as an image command based on an image system, a determination unit 21 that determines whether the brain information acquired by the EEG acquisition unit 11 corresponds to the brain information stored in the storage unit 15, and an execution unit 22 that executes the processing of the operation content corresponding to the image command when the determination unit 21 determines that it has acquired brain information corresponding to the image command.
[0063] Therefore, when brain information corresponding to an image command is acquired, the operation content corresponding to the image command is processed, thereby activating the target object 50. In other words, commands that represent the operation content are replaced with image commands, and when brain information corresponding to the image command is acquired, the processing of the operation content corresponding to the image command is executed. As a result, the operation content corresponding to the image command is activated more safely and accurately, and appropriate EEG commands can be executed without being affected by unrelated stimuli.
[0064] Furthermore, in this embodiment, the memory unit 15 stores a first image command corresponding to a first action of the object to be operated 50 and a second image command corresponding to a second action paired with the first action as multiple command sets, and has a command set switching unit 23 that switches to a command set from among the multiple command sets based on brain information acquired by the electroencephalogram acquisition unit 11.Therefore, a command set suitable for the user can be selected according to the user's aptitude, and an appropriate electroencephalogram command can be executed.
[0065] Furthermore, in this embodiment, there is a mode switching unit 24 that switches between an execution mode in which the execution unit 22 processes the operation content corresponding to the first command (image command) and an autistic mode (non-execution mode) in which the execution unit 22 does not process the operation content corresponding to the first command (image command), based on brain information acquired by the electroencephalogram acquisition unit 11.Therefore, in the autistic mode, for example, external communication is blocked, so the user's brain information is not leaked to the outside, and external communication is connected and appropriate electroencephalogram commands can be executed only in the execution mode in which the user wants to operate the object 50.
[0066] Although the operation control device 10 according to the present invention has been described so far, it may be implemented in various other forms besides those described above.
[0067] Each component of the illustrated operation control device 10 is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage conditions of each device.
[0068] The configuration of the operation control device 10 is realized, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks realized by the cooperation of hardware or software. That is, these functional blocks can be realized in various forms by hardware alone, software alone, or a combination thereof.
[0069] The above-described components include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the above-described components can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the components are possible without departing from the spirit of the present invention. [Explanation of symbols]
[0070] 10 Operation control device 11. Electroencephalogram (EEG) Acquisition Unit (Brain Information Acquisition Unit) 12. EEG Decoder 13 Command Buffer 14 Password buffer 15 Storage section 16 Processing Unit 17 Display Control Unit 18 Display 19 Communications Department 21 Judgment section 22 Execution Department 23 Command Set Switching Section 24 Mode switching section 50 Objects to be operated on
Claims
1. A brain information acquisition unit that acquires the user's brain information, A storage unit that stores the operation details for manipulating an object as image commands based on an image system, A determination unit that determines whether the brain information acquired by the brain information acquisition unit is brain information corresponding to an image stored in the memory unit, When the determination unit determines that it has acquired brain information corresponding to the image command, the execution unit executes the processing of the operation content corresponding to the image command. Equipped with, The storage unit stores multiple command sets, which include a first image command that corresponds to a first operation of the object being operated on, and a second image command that corresponds to a second operation that is paired with the first operation. The system includes a command set switching unit that switches the command set from among the plurality of command sets based on the brain information acquired by the brain information acquisition unit, The memory unit stores multiple command sets in which the distance in the brain between the semantic category of the first image command and the semantic category of the second image command is different. Operation control device.
2. The system includes a mode switching unit that switches between an execution mode in which the execution unit processes the operation content corresponding to the image command and a non-execution mode in which the execution unit does not process the operation content corresponding to the image command, based on brain information acquired by the brain information acquisition unit. The operation control device according to claim 1.
3. Steps to acquire the user's brain information, The steps include: storing the operation details for manipulating the target object as image commands based on an image system; The steps include determining whether the acquired brain information corresponds to the stored image, The steps include storing a set of multiple command sets, each containing a first image command which corresponds to a first action of the object being operated on, and a second image command which corresponds to a second action that is paired with the first action. The steps include switching between the command sets from the aforementioned multiple command sets based on the acquired brain information, The steps include storing multiple command sets in which the distance in the brain between the semantic category of the first image command and the semantic category of the second image command differs, An operation control method including
4. Steps to acquire the user's brain information, The steps include: storing the operation details for manipulating the target object as image commands based on an image system; The steps include determining whether the acquired brain information corresponds to the stored image, The steps include storing a set of multiple command sets, each containing a first image command which corresponds to a first action of the object being operated on, and a second image command which corresponds to a second action that is paired with the first action. The steps include switching between the command sets from the aforementioned multiple command sets based on the acquired brain information, The steps include storing multiple command sets in which the distance in the brain between the semantic category of the first image command and the semantic category of the second image command differs, A program that causes a computer to execute something.