Operation control device, operation control method and program

Abstract

To provide an operation control device capable of executing an appropriate brain wave command without being affected by an irrelevant stimulus.SOLUTION: An operation control device includes: a brain information acquisition unit that acquires brain information of a user; a determination unit that determines whether or not the brain information acquired by the brain information acquisition unit is brain information which corresponds to a first command meaning an operation content for operating an operation target and a second command representing an action of operating an operation member; and an execution unit that executes processing corresponding to the first command when the determination unit determines that brain information corresponding to the first command and brain information corresponding to the second command are acquired.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an operation control device, an operation control method, and a program. [Background technology]

[0002] In recent years, advances in technology for measuring brain information have made it possible to realize a brain-machine interface (BMI), which is an interface that directly connects the brain and machines. One such technology is described in Patent Document 1, for example. Patent Document 1 discloses a system that obtains multiple steady-state visual evoked response potential signals from a user when the user receives a stimulus and supplies the signals to the system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-117957 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional BMIs, when an unrelated stimulus is received when executing an EEG command, that stimulus may appear in the EEG and cause the command to be executed. For example, if you think to a robot "move forward," and the command is to make the robot move forward, just having someone nearby say "move forward" will cause the brain to automatically recall "move forward" and issue the command. Also, when operating a robot while watching its movements, if the image of the robot moving forward is displayed on a monitor, the operator's brain will automatically continue to recall "move forward," causing the command to be issued automatically.

[0005] The present invention has been made in view of the above, and has as its object to make it possible to execute appropriate electroencephalogram commands without being influenced by irrelevant stimuli. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the operation control device of the present invention comprises a brain information acquisition unit that acquires brain information of a user, a determination unit that determines whether the brain information acquired by the brain information acquisition unit is brain information corresponding to a first command that indicates the operation content for operating an operation object and a second command that expresses the action of operating an operation member, and an execution unit that executes processing corresponding to the first command when it determines that the determination unit has acquired brain information corresponding to the first command and brain information corresponding to the second command.

[0007] The operation control method of the present invention includes the steps of acquiring brain information of a user, determining whether the acquired brain information corresponds to a first command that indicates the operation content for operating an operation object and a second command that expresses the action of operating an operation member, and executing processing corresponding to the first command when it is determined that brain information corresponding to the first command and brain information corresponding to the second command have been acquired.

[0008] The program of the present invention causes a computer to execute the steps of acquiring brain information of a user, determining whether the acquired brain information corresponds to a first command that indicates the operation content for operating an object to be operated and a second command that expresses the action of operating an operating member, and executing processing corresponding to the first command when it is determined that brain information corresponding to the first command and brain information corresponding to the second command have been acquired. [Effects of the Invention]

[0009] According to the present invention, it is possible to effectively execute appropriate electroencephalogram commands without being influenced by irrelevant stimuli. [Brief explanation of the drawings]

[0010] [Figure 1]FIG. 1 is a block diagram showing an example of the configuration of an operation control device according to the first embodiment. [Figure 2] FIG. 2 is a flowchart showing the flow of processing in the operation control device according to the first embodiment. [Figure 3] FIG. 3 is a flowchart showing the flow of processing in the operation control device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An operation control device, an operation control method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0012] First Embodiment [Operation control device] FIG. 1 is a block diagram showing an example of the configuration of an operation control device according to the first embodiment.

[0013] 1, in the first embodiment, the operation control device 10 controls the operation of an operation target object 50 based on brain information of a user. The operation control device 10 includes an electroencephalogram acquiring unit (brain information acquiring unit) 11, an electroencephalogram decoder 12, an execution command buffer 13, an image command buffer 14, a storage unit 15, a processing unit 16, a display control unit 17, a display unit 18, and a communication unit 19.

[0014] The operation object 50 is, for example, a robot, but is not limited to a robot and may be any device that operates through communication. Although not shown, the operation object 50 is provided with a drive unit, a control unit that controls the drive unit, and a communication unit that can receive control signals, etc. When the communication unit of the operation object 50 receives a control signal, the control unit drives the drive unit to operate the robot.

[0015] The electroencephalogram acquiring unit 11 can be worn on the user's head. The electroencephalogram decoder 12, processing unit 16, display control unit 17, display unit 18, and communication unit 19 may be worn by the user, or may be placed in predetermined positions without being worn. Furthermore, the electroencephalogram acquiring unit 11, electroencephalogram decoder 12, processing unit 16, display control unit 17, display unit 18, and communication unit 19 may be provided integrally or separately.

[0016] The brain wave acquisition unit 11 acquires brain waves, which are brain information of the user. The brain wave acquisition unit 11 has, for example, an electric sensor (e.g., an electrode) that detects brain waves generated from a weak current flowing in the neural network of the brain. The brain wave acquisition unit 11 detects the potential (electrical signal) of a weak current when the user receives an external stimulus or based on the user's thoughts, such as their intentions. Note that the brain information acquisition unit is not limited to the brain wave acquisition unit 11. The brain information acquisition unit may also acquire, for example, blood flow due to brain activity, which is brain information of the user, by, for example, near-infrared light measurement. The brain wave acquisition unit 11 is connected to the brain wave decoder 12 and transmits brain waves, which are brain information acquired from the user, to the brain wave decoder 12.

[0017] The electroencephalogram decoder 12 restores the user's electrical electroencephalogram signals acquired by the electroencephalogram acquisition unit 11 into the user's thought information. In this case, the relationship between the user's electrical electroencephalogram signals and the user's thought information is linked in advance. In this case, the relationship between the electroencephalogram signals and the user's thought information is linked using machine learning such as deep learning.

[0018] The execution command buffer 13 temporarily stores the first command determined by the determination unit 21 based on the user's thought information. The first command is a command based on brain information associated with a language system (character information). The first command is a command for operating the operation object 50, such as commands such as "forward," "backward," "stop," "accelerate," "decelerate," "turn right," and "turn left." In this case, the relationship between the first command and the operation of the operation object 50 is linked in advance. The execution command buffer 13 is capable of sending and receiving various data to and from the processing unit 16.

[0019] The first command is not limited to a concept indicating an instructed action, but may be one to which a numerical value or the like is added to complement the concept indicating the instructed action. For example, the electroencephalogram decoder 103 may restore the user's thought information based on the electrical signal acquired by the electroencephalogram acquisition unit 11, and at the same time, detect the intensity of the electroencephalogram as a decimal number ranging from 0 to 1, and add the intensity (numerical value) to the user's thought information to set the first command. In this case, the numerical value of the first command is a count of the maximum torque output of the drive device provided in the operation target object 50.

[0020] The image command buffer 14 temporarily stores the second command determined by the determination unit 21 based on the user's thought information. Image system The second command is a command based on brain information associated with the first command. The second command is a command for determining whether to execute the first command. The second command corresponding to the first command is set in advance by the user and stored in the storage unit 15. The second command may be set by the processing unit 16, stored in the storage unit 15, and notified to the user in advance. The image command buffer 14 is capable of transmitting and receiving various data to and from the processing unit 16.

[0021] The second command is an image command that represents the user's action of operating an operating member. The operating member has a movable part that can be operated by the user, such as a lever, pedal, handle, or dial. The movable part can move linearly or rotate, or may move in other directions. The operating member is one that can be operated with a part of the user's body, such as a hand or foot. The image command as the second command represents the user's action of moving the movable part of the operating member, for example, from the "ON" position to the "OFF" position, or from the "OFF" position to the "ON" position. The image command as the second command represents the user's action of moving the movable part of the operating member, for example, from the "Level 0" position to the "Level 1" position, or from the "Level 1" position to the "Level 0" position.

[0022] Here, the image command for moving the movable part of the operating member from the "ON" position to the "OFF" position or from the "Level 1" position to the "Level 0" position corresponds to the first command "Stop." On the other hand, the image command for moving the movable part of the operating member from the "OFF" position to the "ON" position or from the "Level 1" position to the "Level 0" position corresponds to the first commands "Forward" and "Backward."

[0023] The storage unit 15 stores a plurality of first commands (execution commands) and a plurality of second commands (image commands). The storage unit 15 may also store learning data obtained by machine learning the association between the electrical signals of the user's electroencephalograms and thought information. The second commands are associated with the first commands in advance.

[0024] The execution command buffer 13 and the image command buffer 14 are, for example, recording units such as semiconductor memory elements, such as RAM (Random Access Memory) and flash memory, provided in the processing unit 16. The storage unit 15 is configured with a memory card, SSD (Solid State Drive), external storage device, or the like.

[0025] The processing unit 16 is an arithmetic processing device (control device) configured with, for example, a CPU (Central Processing Unit). The processing unit 16 loads a stored program into memory and executes instructions included in the program. The processing unit 16 includes an internal memory (not shown), which is used for temporary storage of data in the processing unit 16, etc. The processing unit 16 has a determination unit 21 and an execution unit 22 as functions. The processing unit 16 also transmits processing results to the display control unit 17 and the communication unit 19.

[0026] The judgment unit 21 judges whether the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to two different first commands (execution commands) and second commands (image commands).

[0027] Specifically, the determination unit 21 compares the user's thought information restored by the electroencephalogram decoder 12 with the first command (execution command) stored in the storage unit 15. When the user's thought information matches the first command, the determination unit 21 determines that the user's thought information is the first command. When this happens, the determination unit 21 stores the command determined to be the first command in the execution command buffer 13. On the other hand, when the user's thought information does not match the first command, the determination unit 21 determines that the user's thought information is not the first command. When this happens, the determination unit 21 does not store the command determined to not be the first command in the execution command buffer 13. When the determination unit 21 determines whether the user's thought information restored by the electroencephalogram decoder 12 matches the first command, it determines that there is a match even if it is not a perfect match, as long as it can determine that the user's thought information means the first command.

[0028] The determination unit 21 also compares the user's thought information restored by the electroencephalogram decoder 12 with the second command (image command) stored in the memory unit 15. At this time, if the user's thought information matches the second command, the determination unit 21 determines that the user's thought information is the second command. If the determination unit 21 determines that the user's thought information is the second command, the determination unit 21 stores the second command in the image command buffer 14. On the other hand, if the user's thought information does not match the second command stored in the memory unit 15, the determination unit 21 determines that the user's thought information is not the second command. If the determination unit 21 determines that the user's thought information is not the second command, the determination unit 21 does not store the second command in the image command buffer 14.

[0029] The second command is an image command that represents the action of the user operating the operating member, and the operating member is a virtual object that the user imagines in his or her mind. Therefore, the determination unit 21 may determine that the user's thought information matches the operating member that represents the second command and its operating direction, even if they do not match completely.

[0030] Furthermore, the determination unit 21 determines whether the first command (execution command) stored in the execution command buffer 13 and the second command (image command) stored in the image command buffer 14 are associated with each other. In this case, when the determination unit 21 determines that the user's thought information is an electroencephalogram corresponding to the first command (execution command), it determines whether the user's thought information is an electroencephalogram corresponding to the second command (image command). However, when the determination unit 21 determines that the user's thought information is an electroencephalogram corresponding to the second command (image command), it may also determine whether the user's thought information is an electroencephalogram corresponding to the first command (execution command).

[0031] The second command may be of any type of image, and may be selectable from a plurality of images. For example, when the first command is "forward," the operating member as the second command may be any of a lever, pedal, handle, dial, etc., and the determination unit 21 may determine that the second command is associated with the first command if the user's thought information matches any of the operating members.

[0032] When the determination unit 21 determines that it has acquired an electroencephalogram corresponding to the first command and an electroencephalogram corresponding to the second command, the execution unit 22 executes a process corresponding to the first command. Specifically, when the determination unit 21 determines that the user's thought information acquired first is the first command and the determination unit 21 determines that the user's thought information acquired next is the second command associated with the first command, the execution unit 22 executes a process corresponding to the execution command as the first command.

[0033] However, the execution unit 22 cancels the acquired first command when a preset waiting time, for example, three seconds, has elapsed since the determination unit 21 determined that the first command has been acquired. Furthermore, the execution unit 22 cancels the acquired first command when the processing unit 16 acquires a cancel command as the user's thought after the determination unit 21 determines that the first command has been acquired. The cancel command is preset, and the relationship between the user's electrical electroencephalogram signal and the user's thought information (cancel command) is linked in advance and stored in the storage unit 15. The cancel command is, for example, a symbol or concept different from the first command, such as "x" or "cancel."

[0034] The display control unit 17 is connected to the display unit 18. The display control unit 17 transmits the processing results transmitted from the processing unit 16 to the display unit 18 and causes it to be displayed. The display control unit 17 causes the display unit 18 to display the processing results of the processing unit 16, such as the operating state of the operation control device 10 and thought instructions for the user.

[0035] The display unit 18 displays the operating status of the operation control device 10 and thought instructions for the user transmitted from the display control unit 17. The display unit 18 presents necessary information to the user. The display unit 18 is a display including, for example, a liquid crystal display (LCD) or an organic electro-luminescence (EL) display.

[0036] The communication unit 19 is capable of wireless communication with the operation target 50. The communication unit 19 transmits processing information processed by the processing unit 16 to the operation target 50. Specifically, when the determination unit 21 determines that an electroencephalogram corresponding to the first command and an electroencephalogram corresponding to the second command have been acquired, the execution unit 22 generates a control signal corresponding to the processing of the first command and transmits it to the communication unit 19. The communication unit 19 transmits the control signal corresponding to the processing of the first command to the operation target 50. Upon receiving the first command transmitted by the communication unit 19, the operation target 50 operates in response to the processing of the first command.

[0037] [Operation control method] Here, a description will be given of an operation control method by the operation control device 10. Fig. 2 is a flowchart showing the flow of processing in the operation control device according to the first embodiment.

[0038] 1 and 2, in step S11, the processing unit 16 controls the display control unit 17 to display an instruction to the user to recall a first command (execution command) on the display unit 18. The display control unit 17 displays a message on the display unit 18 instructing the user to recall the first command, which indicates the operation content, such as "Please recall the first command."

[0039] In step S12, the determination unit 21 determines whether the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to the first command (execution command). Here, the determination unit 21 compares the user's thought information restored by the electroencephalogram decoder 12 with the first command stored in the memory unit 15. If the determination unit 21 determines that the user's thought information does not match the first command (No), it determines that the user's thought information is not the first command and exits this routine. On the other hand, if the determination unit 21 determines that the user's thought information matches the first command stored in the memory unit 15 (Yes), it determines that the user's thought information is the first command, and the determination unit 21 stores the first command in the execution command buffer 13 and proceeds to step S13.

[0040] In step S13, the processing unit 16 controls the display control unit 17 to display an instruction on the display unit 18 to remind the user of a second command (image command). The display control unit 17 displays a message on the display unit 18 instructing the user to recall a first command, which indicates the operation content, such as "Please recall the second command." Here, the user recalls the second command in response to the display on the display unit 18. For example, when moving the operation target object 50 forward, the user imagines the action of moving the movable part of the operation member from the "OFF" position to the "ON" position.

[0041] In step S14, processing unit 16 determines whether a timeout has occurred by the lapse of a preset waiting time since determination unit 21 determined that the first command was acquired. If processing unit 16 determines that a timeout has occurred (Yes), in step S20, processing unit 16 cancels the acquired first command. Then, in step S21, processing unit 16 controls display control unit 17 to display to the user on display unit 18 that the first command has been canceled. Display control unit 17 displays a timeout message, such as "Timeout," on display unit 18.

[0042] If the processing unit 16 determines that a timeout has not occurred (No), then in step S15 it determines whether or not an electroencephalogram corresponding to a cancel command has been detected. If the processing unit 16 determines that an electroencephalogram corresponding to a cancel command has been detected (Yes), then in step S22 it cancels the acquired first command. Then, in step S23, the processing unit 16 controls the display control unit 17 to display to the user on the display unit 18 that the first command has been canceled. The display control unit 17 displays a message on the display unit 18 indicating that the first command has been canceled, such as "First command is being canceled."

[0043] On the other hand, if the processing unit 16 determines that it has not detected an electroencephalogram corresponding to a cancel command (No), then in step S16 the judgment unit 21 determines whether it has detected an electroencephalogram corresponding to a second command (image command). That is, the judgment unit 21 determines whether it has detected an electroencephalogram representing the image of the user moving the movable part of the operation member from the "OFF" position to the "ON" position, for example. Here, the judgment unit 21 compares the user's thought information restored by the electroencephalogram decoder 12 with the second command stored in the memory unit 15, and if it determines that the user's thought information does not match the second command (No), the process returns to step S14 and continues processing.

[0044] On the other hand, if the determination unit 21 determines that the user's thought information matches the second command stored in the memory unit 15 (Yes), it stores the second command determined as the user's thought information to be the second command in the image command buffer 14. That is, when the determination unit 21 detects brain information representing the image of the user moving the movable part of the operation member from the "OFF" position to the "ON" position, it stores the detected second command in the image command buffer 14. Furthermore, in step S17, the determination unit 21 determines whether the first command determined in step S12 and stored in the execution command buffer 13 and the second command determined in step S16 and stored in the image command buffer 14 are the first command and the second command that are previously associated with each other. If the determination unit 21 determines that the first command and the second command are not associated with each other (No), the process returns to step S14. If the determination unit 21 determines that the first command and the second command are associated with each other (Yes), the process proceeds to step S18.

[0045] In step S18, the execution unit 22 executes processing corresponding to the first command. That is, the execution unit 22 transmits a control signal corresponding to the processing of the first command to the operation object 50 via the communication unit 19, and operates the operation object 50. Then, in step S19, the processing unit 16 controls the display control unit 17 to display to the user on the display unit 18 that the first command has been executed. The display control unit 17 displays a message indicating that the first command has been executed, such as "Acceleration command has been executed," on the display unit 18.

[0046] Second Embodiment Fig. 3 is a flowchart showing the flow of processing in the operation control device according to the second embodiment. The basic configuration of the second embodiment is the same as that of the first embodiment described above, and will be described using Fig. 1. Components having the same functions as those of the first embodiment described above are given the same reference numerals, and detailed description thereof will be omitted.

[0047] 1, in the second embodiment, the operation control device 10 controls the operation of an operation target object 50 based on brain information of a user. The operation control device 10 includes an electroencephalogram acquisition unit 11, an electroencephalogram decoder 12, an execution command buffer 13, an image command buffer 14, a storage unit 15, a processing unit 16, a display control unit 17, a display unit 18, and a communication unit 19. Here, the electroencephalogram acquisition unit 11, the electroencephalogram decoder 12, the execution command buffer 13, the image command buffer 14, the display control unit 17, the display unit 18, and the communication unit 19 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0048] The storage unit 15 stores a plurality of first commands and a plurality of second commands (image commands). As described above, the first command is an execution command that indicates the operation content of the operation target object 50. The second command is a command for determining whether to execute the first command.

[0049] The second command is an image command that represents the user's action of operating the operating member. The image command as the second command represents the action of operating the movable part of the operating member by a predetermined amount or more. The image command as the second command represents the user's action of moving the movable part of the operating member, for example, from the "0" position to the "forward" position, or from the "0" position to the "backward" position. In this case, the direction of movement of the movable part from the "0" position to the "forward" position is opposite to the direction of movement from the "0" position to the "backward" position. A dead zone (play) is set between the "0" position and the "forward" position, and between the "0" position and the "backward" position.

[0050] That is, when the movable part is moved from the "0" position to the "forward" position, the image command is the movement of the operating member when the movable part is moved by more than 25% of the distance from the "0" position to the "forward" position. Also, when the movable part is moved from the "0" position to the "rear" position, the image command is the movement of the operating member when the movable part is moved by more than 25% of the distance from the "0" position to the "rear" position. Here, the image command for moving the movable part of the operating member from the "0" position to the "forward" position is associated with the first command "forward." On the other hand, the image command for moving the movable part of the operating member from the "0" position to the "rear" position is associated with the first command "rearward."

[0051] When moving the movable part from the "forward" position to the "0" position, the image command is the movement of the operating member when the movable part is moved by 25% or more of the distance from the "forward" position to the "0" position. When moving the movable part from the "rear" position to the "0" position, the image command is the movement of the operating member when the movable part is moved by 25% or more of the distance from the "rear" position to the "0" position. Here, the image command for moving the movable part of the operating member from the "forward" position to the "0" position and the image command for moving the movable part of the operating member from the "rear" position to the "0" position are associated with the first command "stop."

[0052] In this case, the amount of dead zone (play) as the second command (image command), i.e., the predetermined amount by which the movable part of the operating member is moved, may be set appropriately according to the form of the operating member. Also, the amount of dead zone (play) as the second command (image command), i.e., the predetermined amount by which the movable part of the operating member is moved, is notified to the user in advance.

[0053] The distance over which the movable portion of the operating member is moved may be detected as the amount of movement or the speed of movement of the operation target object 50.

[0054] The processing unit 16 has as its functions a determination unit 21 and an execution unit 22. The determination unit 21 determines whether the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to two different commands: a first command (execution command) and a second command (image command). The determination unit 21 determines whether the electroencephalogram acquired by the electroencephalogram acquisition unit 11 is an electroencephalogram corresponding to the second command, which expresses an action of operating the operating member by a predetermined distance or more (for example, 25% of the total distance).

[0055] Here, a description will be given of an operation control method by the operation control device 10. Fig. 3 is a flowchart showing the flow of processing in the operation control device according to the second embodiment.

[0056] As shown in FIGS. 1 and 3, the processes of steps S11 to S15 and steps S20 to S23 are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0057] In step S16, the determination unit 21 determines whether or not it has detected brain waves corresponding to the second command (image command). That is, the determination unit 21 determines whether or not it has detected brain information representing the image of the user moving the movable part of the operation member from the "OFF" position to the "ON" position, for example. Here, the determination unit 21 compares the user's thought information restored by the brain wave decoder 12 with the second command stored in the memory unit 15, and if it determines that the user's thought information does not match the second command (No), it returns to step S14 and continues processing.

[0058] On the other hand, if the determination unit 21 determines that the user's thought information matches the second command stored in the memory unit 15 (Yes), the process proceeds to step S31. In step S31, the determination unit 21 determines whether the image detected as the second command (image command) is an image of an action in which the movable part of the operation member moves by a predetermined amount. Here, if the determination unit 21 determines that the image detected as the second command (image command) is not an image of an action in which the movable part of the operation member moves by a predetermined amount (No), the process returns to step S14 and continues processing.

[0059] At this time, the processing unit 16 may control the display control unit 17 to display to the user on the display unit 18 a message that the movement of the operating member is insufficient. The display control unit 17 may display on the display unit 18 a message that reminds the user of the second command, such as "Please imagine moving the operating member by more than a predetermined amount."

[0060] On the other hand, when the determination unit 21 determines that the image detected as the second command (image command) is an image of an action of moving the movable part of the operation member by a predetermined amount (Yes), it stores the second command determined as the user's thought information to be the second command in the image command buffer 14. That is, when the determination unit 21 detects brain information of an image of an action of the user moving the movable part of the operation member from the "OFF" position to the "ON" position by more than a predetermined amount, it stores the detected second command in the image command buffer 14.

[0061] In step S17, the determination unit 21 determines whether the first command determined in step S12 and stored in the execution command buffer 13 and the second command determined in steps S16 and S17 and stored in the image command buffer 14 are the first command and the second command that have been previously associated with each other. If the determination unit 21 determines that the first command and the second command are not associated with each other (No), the process returns to step S14. If the determination unit 21 determines that the first command and the second command are associated with each other (Yes), the process proceeds to step S18.

[0062] In step S18, the execution unit 22 executes processing corresponding to the first command. That is, the execution unit 22 transmits a control signal corresponding to the processing of the first command to the operation object 50 via the communication unit 19, and operates the operation object 50. Then, in step S19, the processing unit 16 controls the display control unit 17 to display to the user on the display unit 18 that the first command has been executed. The display control unit 17 displays a message indicating that the first command has been executed, such as "Acceleration command has been executed," on the display unit 18.

[0063] [effect] In this embodiment, the device includes an electroencephalogram acquisition unit (brain information acquisition unit) 11 that acquires the user's brain information, a determination unit 21 that determines whether the brain information acquired by the electroencephalogram acquisition unit 11 is brain information corresponding to a first command that indicates the operation content for operating the operation object 50 and a second command that expresses the action of operating the operation member, and an execution unit 22 that executes processing corresponding to the first command when the determination unit 21 determines that it has acquired brain information corresponding to the first command and brain information corresponding to the second command.

[0064] Therefore, when brain information corresponding to the first command and brain information corresponding to the second command are acquired, a process corresponding to the first command is executed to operate the operation target object 50. In other words, even if brain information corresponding to the first command indicating the operation content is acquired, the process corresponding to the first command is not executed unless brain information corresponding to the second command indicating the intention to execute the operation content is acquired. Therefore, the content and execution timing of the first command are activated more safely and accurately, and an appropriate brain wave command can be executed without being influenced by unrelated stimuli.

[0065] Furthermore, the determination unit 21 determines whether the second command is an image command that expresses an action of operating an operating member, and therefore can accurately detect the user's intention to operate the operation target object 50.

[0066] In this embodiment, the first command is a command based on a language system that means the operation content for operating the operation object 50, and the second command is a command based on an image system that means the intention to execute the operation content of the operation object 50. Therefore, by acquiring brain information corresponding to the second command, the user can confirm the intention to execute the operation content of the operation object 50, and an appropriate electroencephalogram command can be executed.

[0067] In this embodiment, the determination unit 21 determines whether the electroencephalogram acquired by the electroencephalogram acquisition unit 11 is brain information corresponding to a second command representing an action of operating the operation member by a predetermined amount or more. Therefore, by accurately detecting the user's intention to operate the operation target object 50, an appropriate electroencephalogram command can be executed.

[0068] The operation control device 10 according to the present invention has been described above, but it may be embodied in various different forms other than the above-described embodiment.

[0069] Each component of the illustrated operation control device 10 is a functional concept and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of each device is not limited to that shown in the figure, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.

[0070] The configuration of the operation control device 10 is realized, for example, as software, by a program loaded into memory. In the above embodiment, the functional blocks are described as being realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms, using only hardware, only software, or a combination of both.

[0071] The above-described components include those that can be easily imagined by a person skilled in the art and those that are substantially the same. Furthermore, the above-described configurations can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the configurations are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0072] 10 Operation control device 11. Brain wave acquisition unit (brain information acquisition unit) 12 Brainwave Decoder 13 Execution Command Buffer 14 Image Command Buffer 15 Storage section 16 Processing section 17 Display control unit 18 Display 19 Communications Department 50 Operational Object

Claims

1. a brain information acquisition unit that acquires brain information of a user; a determination unit that determines whether the brain information acquired by the brain information acquisition unit is brain information corresponding to a first command that indicates an operation content for operating an operation object and a second command that indicates an action for operating an operation member, and determines whether the first command and the second command are first commands and second commands that are associated with each other in advance; an execution unit that executes a process corresponding to the first command when the determination unit determines that brain information corresponding to the first command and brain information corresponding to the second command have been acquired and that the first command and the second command are the first command and the second command that are associated with each other in advance; Equipped with the first command is a command based on a language system that means an operation content for operating an operation object, and the second command is a command based on an image system that means an intention to execute the operation content of the operation object. Operation control device.

2. the determination unit determines whether the brain information acquired by the brain information acquisition unit corresponds to a second command representing an action of operating the operation member by a predetermined amount or more. The operation control device according to claim 1 .

3. acquiring brain information of a user; a step of determining whether the acquired brain information corresponds to a first command based on a language system that means an operation content for operating an operation object and a second command based on an image system that expresses an action of operating an operation member; determining whether the first command and the second command are first commands and second commands that are associated with each other in advance; determining that brain information corresponding to the first command and brain information corresponding to the second command have been acquired, and executing a process corresponding to the first command when determining that the first command and the second command are the first command and the second command that are associated with each other in advance; An operation control method including:

4. acquiring brain information of a user; a step of determining whether the acquired brain information corresponds to a first command based on a language system that means an operation content for operating an operation object and a second command based on an image system that expresses an action of operating an operation member; determining whether the first command and the second command are first commands and second commands that are associated with each other in advance; determining that brain information corresponding to the first command and brain information corresponding to the second command have been acquired, and executing a process corresponding to the first command when determining that the first command and the second command are the first command and the second command that are associated with each other in advance; A program that causes a computer to execute the following.

Images