Operation control device, operation control method, and program

Abstract

The present invention comprises: a brain information acquisition unit that acquires brain information pertaining to a user; a sleep determination unit that determines whether the user is in a sleep state on the basis of the brain information which has been acquired by the brain information acquisition unit; and a mode switching unit that switches between a first mode in which, when the sleep determination unit has determined that the user is in an awake state, an operation target is operated on the basis of the brain information pertaining to the user and a second mode in which, when the sleep determination unit has determined that the user is in the sleep state, the operation target is operated on the basis of information other than the brain information pertaining to the user.

Description

Operation control device, operation control method and program

[0001] The present disclosure relates to an operation control device, an operation control method, and a program.

[0002] In recent years, technology for measuring brain activation information has advanced, and brain-machine interface technology, which is an interface between the brain and the outside, is becoming a reality. Patent Document 1 describes a robot that is provided with a first sensor that measures the electric field generated by brain activity and a second sensor that detects the state of cerebral blood flow, and that analyzes the brain activity of an operator based on signals indicating the electric field in the brain obtained by each sensor, and operates the robot.

[0003] Japanese Patent Application Laid-Open No. 2006-289565

[0004] However, Patent Document 1 does not describe a subject operating a robot while asleep.

[0005] The present disclosure has been made in consideration of the above, and aims to provide an operation control device, an operation control method, and a program that enable a user to operate an operation object even while sleeping.

[0006] In order to solve the above-mentioned problems and achieve the object, the operation control device according to the present disclosure includes a brain information acquisition unit that acquires brain information of a user, a sleep determination unit that determines whether the user is asleep based on the brain information acquired by the brain information acquisition unit, and a mode switching unit that switches between a first mode in which an operation object is operated based on the brain information of the user when the sleep determination unit determines that the user is asleep, and a second mode in which the operation object is operated based on information other than the brain information of the user when the sleep determination unit determines that the user is asleep.

[0007] The operation control method according to the present disclosure includes the steps of acquiring brain information of a user, determining whether the user is asleep based on the acquired brain information, and switching between a first mode in which an operation object is operated based on the brain information of the user when it is determined that the user is awake, and a second mode in which the operation object is operated based on information other than the brain information of the user when it is determined that the user is asleep.

[0008] The program according to the present disclosure causes a computer to execute the steps of acquiring brain information of a user, determining whether the user is asleep based on the acquired brain information, and switching between a first mode in which an object to be operated is operated based on the brain information of the user when it is determined that the user is awake, and a second mode in which the object to be operated is operated based on information other than the brain information of the user when it is determined that the user is asleep.

[0009] According to the present disclosure, it is possible to achieve an effect of enabling a user to operate an operation object even while sleeping.

[0010] Fig. 1 is a block diagram showing the configuration of an operation control device according to this embodiment. Fig. 2 is a schematic diagram showing details of a switching mode between an awake state and a sleep state by the operation control device. Fig. 3 is a schematic diagram showing details of a switching mode in the sleep state by the operation control device. Fig. 4 is a flowchart showing an operation control method according to this embodiment.

[0011] Hereinafter, embodiments of an operation control device, an operation control method, and a program according to the present disclosure will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.

[0012] <Operation Control Device> FIG. 1 is a block diagram showing the configuration of an operation control device according to this embodiment.

[0013] 1, the operation control device 10 controls the operation of an operation target object 100 based on brain information of a user. The operation control device 10 includes an input unit 11, a measurement unit 12, a stimulation unit 13, a conversion unit 14, a control unit 15, a storage unit 16, and a communication unit 17.

[0014] The input unit 11 is connected to the control unit 15. The input unit 11 can be operated by a user and can input various signals to the control unit 15. For example, the input unit 11 inputs to the control unit 15 a start signal for starting operation control for executing an operation of the operation target object 100 using brain information, and an end signal for ending operation control for executing an operation of the operation target object 100 using brain information. The input unit 11 can be realized by, for example, a touch panel, a button, a switch, a keyboard, etc.

[0015] The measurement unit 12 and the stimulation unit 13 are worn by the user. For example, the measurement unit 12 and the stimulation unit 13 are worn on the user's head. The measurement unit 12 has a frontal lobe electrode 21 and a motor cortex electrode 22. The stimulation unit 13 has a visual cortex electrode 31 and an auditory cortex electrode 32.

[0016] The measurement unit 12 acquires brain waves, which are brain information of the user. The measurement unit 12 has an electric sensor (e.g., an electrode) that detects brain waves generated from a weak current flowing through the neural network of the brain, for example, using an invasive electrode. The measurement unit 12 detects the potential (electrical signal) of the weak current when the user receives an external stimulus or based on the user's thoughts, such as their intentions.

[0017] The stimulation unit 13 stimulates the user by applying brain waves, which are brain information. The stimulation unit 13 has an electric sensor (e.g., an electrode) that applies brain waves to the brain's neural network as a weak current using, for example, an invasive electrode. The stimulation unit 13 applies to the user a weak current potential (electrical signal) based on an external event.

[0018] In the measurement unit 12, the frontal lobe electrode 21 is provided at a position corresponding to the user's frontal lobe. The frontal lobe electrode 21 acquires electrical electroencephalogram signals as brain information corresponding to the user's frontal lobe. The motor cortex electrode 22 is provided at a position corresponding to the motor cortex of the user's brain. The motor cortex electrode 22 acquires electrical electroencephalogram signals as brain information corresponding to the user's motor cortex.

[0019] In the stimulation unit 13, the visual area electrode 31 is provided at a position corresponding to the visual area of ​​the user's brain. The visual area electrode 31 applies electrical brain wave signals, which are brain information corresponding to the visual area, to the user as stimulation. The auditory area electrode 32 is provided at a position corresponding to the auditory area of ​​the user's brain. The auditory area electrode 32 applies electrical brain wave signals, which are brain information corresponding to the auditory area, to the user as stimulation.

[0020] The conversion unit 14 has a frontal lobe decoder 41, a movement decoder 42, a video encoder 51, and an audio encoder 52. The conversion unit 14 may be configured independently, or may be attached to the user's head together with the measurement unit 12 and the stimulation unit 13, or may be integrated with the control unit 15 and provided separately from the measurement unit 12 and the stimulation unit 13.

[0021] The frontal lobe decoder 41 is connected to the frontal lobe electrode 21 in the measurement unit 12. The frontal lobe decoder 41 restores the user's electrical electroencephalogram signals input from the frontal lobe electrode 21 to the user's thought information. The movement decoder 42 is connected to the motor cortex electrode 22 in the measurement unit 12. The movement decoder 42 restores the user's electrical electroencephalogram signals input from the motor cortex electrode 22 to the user's thought information. In this case, the relationship between the multiple electrical electroencephalogram signals of the user and the user's thought information is linked in advance. For example, the relationship between the electrical electroencephalogram signals and the user's thought information is linked using machine learning such as deep learning.

[0022] The video encoder 51 is connected to the visual area electrode 31 in the stimulation unit 13. The video encoder 51 converts the user's thought information into an electrical signal of the user's electroencephalogram and outputs it to the visual area electrode 31. The audio encoder 52 is connected to the auditory area electrode 32 in the stimulation unit 13. The audio encoder 52 converts the user's thought information into an electrical signal of the user's electroencephalogram and outputs it to the auditory area electrode 32. In this case, the relationship between the multiple electrical signals of the user's electroencephalogram and the user's thought information is linked in advance. For example, the relationship between the electrical signal of the electroencephalogram and the user's thought information is linked using machine learning such as deep learning.

[0023] The control unit 15 is connected to the input unit 11 and the conversion unit 14. The control unit 15 receives various types of information from the input unit 11 and the conversion unit 14 and outputs various types of information to the conversion unit 14. The control unit 15 has a brain information acquisition unit 61, a sleep determination unit 62, a mode switching unit 63, and an output unit 64. The control unit 15 is configured by an arithmetic circuit such as a CPU (Central Processing Unit), for example.

[0024] The brain information acquisition unit 61 acquires brain information of the user. The brain information acquisition unit 61 acquires the user's electrical brain wave signals detected by the frontal lobe electrodes 21 of the measurement unit 12 and the user's thought information converted from the user's electrical brain wave signals by the frontal lobe decoder 41. The brain information acquisition unit 61 also acquires the user's electrical brain wave signals detected by the motor cortex electrodes 22 of the measurement unit 12 and the user's thought information converted from the user's electrical brain wave signals by the motor decoder 42.

[0025] The sleep determination unit 62 determines the user's sleep state based on the brain information acquired by the brain information acquisition unit 61. In this case, the sleep determination unit 62 determines the user's sleep state based on either or both of the user's brain waves acquired by the frontal lobe electrode 21 and the user's brain waves acquired by the motor cortex electrode 22. Specifically, the sleep determination unit 62 determines whether the user is asleep or awake based on the user's brain information. Furthermore, when the sleep determination unit 62 determines that the user is asleep, it determines whether the user is in REM sleep or non-REM sleep based on the user's brain information. Furthermore, when the sleep determination unit 62 determines that the user is in REM sleep, it determines whether the user is in a lucid dream state in which the user's activity level is higher than a preset threshold based on the user's brain information.

[0026] In this case, whether the user is asleep is determined based on the user's brain waves. For example, when the user is awake, the detected user's brain waves have a waveform frequency component of significant rhythmic alpha waves of 8 to 13 Hz as a reference value, and slow delta waves (0.5 Hz to 3.0 Hz) and theta waves (4 Hz to 7.0 Hz) lower than this reference value are detected. In this case, if both alpha waves and delta waves are higher than a preset threshold, the user is determined to be asleep. On the other hand, if at least one of the alpha waves and delta waves is lower than the threshold, the user is determined to be awake.

[0027] A user's brain waves are waveforms composed of many overlapping waves in different frequency bands. When a user closes their eyes and begins to fall asleep, visual information is blocked, and the components of brain activity related to vision weaken. As the user gradually loses awareness of surrounding sounds, the brain waves related to hearing also weaken, and as sleep deepens, the components of the brain waves gradually become simpler. Sleep in which the user loses consciousness and brain waves become simpler is called non-REM sleep, which is divided into three stages: stage 1 (non-REM sleep 1: N1), in which the high-frequency alpha waves seen during wakefulness decrease and low-amplitude theta waves (4 Hz to 8 Hz) appear; stage 2 (non-REM sleep 2: N2), in which K-complexes and spindle waves appear; and stage 3, in which delta waves in the low-frequency band (0.5 Hz to 2 Hz) increase and account for more than 20% of the assessment interval (30 seconds).

[0028] On the other hand, REM sleep is a sleep in which the body is asleep but the brain is active. REM sleep is named after the initials REM, which stands for rapid eye movement (REM), in which the eyes move from side to side under closed eyelids. While the muscles are in their most relaxed state, the brain is partially active and dreams are often experienced. It is believed that information processing, such as memory consolidation, occurs during REM sleep. Immediately after falling asleep, the user typically experiences non-REM sleep, which gradually transitions from shallow non-REM sleep to deep non-REM sleep, followed by short periods of REM sleep. Non-REM and REM sleep then alternate. Therefore, by acquiring and analyzing the user's brain waves, it is possible to determine whether the user is in non-REM sleep or REM sleep.

[0029] REM sleep is an unconscious simulation, and lucid dreaming is also a simulation, but it activates the brain while the user is conscious. During non-REM sleep, the user's cerebral cortex loses its connection with other parts of the brain, which is thought to result in less complex and boring dreams. On the other hand, during REM sleep, the cerebral cortex becomes more active and begins to connect with other parts of the brain. When a user is lucid dreaming, the dorsolateral prefrontal cortex begins to become active, allowing the user to clearly recognize themselves even in their dreams and create their own story. Therefore, when the sleep determination unit 62 determines that the user is in a REM sleep state and the activity level of the dorsolateral prefrontal cortex is higher than a threshold, the sleep determination unit 62 determines that the user is in a lucid dream state.

[0030] The sleep determination unit 62 determines whether the user is asleep or awake based on the brain information acquired by the brain information acquisition unit 61. Here, the brain information is not limited to the above-mentioned electroencephalogram, and may be determined using, for example, cerebral blood flow.

[0031] The mode switching unit 63 switches the mode based on the sleep state (awake state) of the user determined by the sleep determination unit 62. Here, the switching modes are an awake mode (first mode) and a sleep mode (second mode), and specifically, an awake mode, a lucid dream mode (sleep mode), and an automatic mode (sleep mode) are set.

[0032] The mode switching unit 63 selects and switches to the awake mode when the sleep determination unit 62 determines that the user is not asleep but is awake. The awake mode is a so-called brain wave control mode in which the operation target object 100 is operated based on brain information from the user's frontal lobe.

[0033] The mode switching unit 63 selects and switches between a sleep mode, i.e., a lucid dream mode or an automatic mode, when the sleep determination unit 62 determines that the user is asleep. At this time, the mode switching unit 63 switches to the lucid dream mode when it determines that the user is in a REM sleep state and that the brain activity level is higher than a preset threshold. The lucid dream mode is a mode in which the operation object 100 is operated based on information from the user's motor cortex.

[0034] On the other hand, when the sleep determination unit 62 determines that the user is not in REM sleep, the mode switching unit 63 switches to the automatic mode. Furthermore, even if the user is in REM sleep, when the sleep determination unit 62 determines that the brain activity level is equal to or lower than a threshold, the mode switching unit 63 switches to the automatic mode. The automatic mode is a mode in which the operation target object 100 is operated based on a preset program.

[0035] Details of the awake mode, lucid dream mode, and automatic mode will be described later.

[0036] The output unit 64 outputs an operation signal when operating the operation object 100 based on the awake mode, lucid dream mode, or automatic mode switched by the mode switching unit 63. The output unit 64 also outputs a video signal and an audio signal input from the operation object 100 to the video encoder 51 and the audio encoder 52 of the conversion unit 14.

[0037] The storage unit 16 stores a program for the control unit 15 to control the operation. The storage unit 16 is an external storage device such as a hard disk drive (HDD) or a memory. The storage unit 16 stores threshold values ​​and the like that the control unit 15 uses in various determination processes.

[0038] The communication unit 17 is connected to the control unit 15. The communication unit 105 is capable of transmitting and receiving various types of information to and from the operation target object 100 based on command signals from the control unit 15.

[0039] The control unit 15 may also include an eyeball detection unit (not shown) that detects the user's eyeball movement. The eyeball detection unit may use an optical or electrical method. Specifically, methods such as the corneal reflex method, double Purkinje method, scleral reflex method, search coil method, and electrooculography are known. For example, the eyeball is positively charged on the corneal side and negatively charged on the retina side. Therefore, if electrodes are placed at the base of the nose and the outer corner of the eye and the potential difference is measured, when the eyeball moves outward, the electrode at the outer corner of the eye moves in the positive direction and the electrode at the base of the nose moves in the negative direction. The angle of rotation of the eyeball can be calculated from the potential difference between these two electrodes. Here, the user's eyeball movement also includes information about the user's motor area.

[0040] The operation target object 100 is, for example, a robot, but is not limited to a robot and may be any device or the like that operates via communication. The operation target object 100 can be connected to the operation control device 10. The operation target object 100 has a camera 101, a microphone 102, a drive unit 103, a control unit 104, and a communication unit 105. The camera 101 acquires video of the surroundings of the operation target object 100. The microphone 102 acquires sound of the surroundings of the operation target object 100. The drive unit 103 drives the operation target object 100. Here, when the operation target object 100 has limbs, the drive unit 103 operates the limbs. Furthermore, when the operation target object 100 is a vehicle or an aircraft, the drive unit 103 is a wheel, crawler, swing wing, or the like. The control unit 104 is connected to and control the camera 101, the microphone 102, the drive unit 103, and the communication unit 105. The communication unit 105 is capable of transmitting and receiving various types of information to and from the control unit 15 via the communication unit 17 .

[0041] The control unit 104 outputs the video captured by the camera 101 and the sound captured by the microphone 102 to the operation control device 10 via the communication unit 105. The control unit 104 also controls the driving of the driving unit 103 based on a signal input from the operation control device 10 via the communication unit 105.

[0042] <Switching Mode> FIG. 2 is a schematic diagram showing details of a switching mode between the awake state and the sleep state by the operation control device, and FIG. 3 is a schematic diagram showing details of a switching mode in the sleep state by the operation control device.

[0043] 1 and 2 , the mode switching unit 63 selects and switches between the awake mode and the sleep mode depending on the user's sleep state (awake state). When the user's brain waves include multiple waveforms, the mode switching unit 63 determines that the user is in an awake state and switches to the awake mode. The awake mode is an electroencephalogram control mode, and the output unit 64 operates the operation target object 100 based on brain information of the user's frontal lobe acquired by the frontal lobe electrode 21 of the measurement unit 12 and converted by the frontal lobe decoder 41.

[0044] The mode switching unit 63 determines that the user is asleep when the user's brain waves do not include multiple waveforms, and switches to the sleep mode. The sleep mode is an automatic mode, and the output unit 64 operates the operation target object 100 based on a program stored in the storage unit 16. Note that the storage unit 16 is connected to the control unit 15 (mode switching unit 63), but may also be connected to the control unit 104 of the operation target object 100.

[0045] The processing of the mode switching unit 63 is not limited to the above. As shown in FIGS. 1 and 3 , the mode switching unit 63 selects and switches one mode from among the awake mode, the lucid dream mode (sleep mode), and the automatic mode (sleep mode) depending on the user's sleep state (wakeful state). The mode switching unit 63 determines that the user is in an awake state when the user's brain waves include multiple waveforms, and switches to the awake mode. In the wakefulness mode, the output unit 64 operates the operation target object 100 based on the brain information of the user's frontal lobe acquired by the frontal lobe electrode 21 of the measurement unit 12 and converted by the frontal lobe decoder 41.

[0046] When the user is in REM sleep and the activity level is higher than a threshold, the mode switching unit 63 determines that the user is in a lucid dream state and switches to the lucid dream mode. In the lucid dream mode, the output unit 64 moves the operation target object 100 based on the information on the user's motor cortex acquired by the motor cortex electrode 22 of the measurement unit 12 and converted by the movement decoder 42.

[0047] Note that when the mode switching unit 63 switches to the lucid dream mode, the output unit 64 may be configured to operate the operation object 100 in the lucid dream mode based on the detection result of the eyeball detection unit. In other words, the output unit 64 operates the operation object 100 based on the user's eyeball movement, which is information on the user's motor cortex acquired by the eyeball detection unit.

[0048] When the user is in a non-REM sleep state, or when the user is in a REM sleep state but the activity level is equal to or lower than a threshold, the mode switching unit 63 determines that the user is in a non-REM sleep state and switches to the automatic mode. In the automatic mode, the output unit 64 operates the operation target object 100 based on a program stored in the storage unit 16.

[0049] <Operation Control Method> FIG. 4 is a flowchart showing the operation control method according to this embodiment.

[0050] As shown in FIGS. 1 and 4 , in step S11, the brain information acquisition unit 61 acquires the user's brain wave electrical signals and the user's thought information as the user's brain information. In step S12, the sleep determination unit 62 determines whether the user is asleep (awake) based on the user's brain information. If the sleep determination unit 62 determines that the user is not asleep but is awake (Yes), the process proceeds to step S13. In step S13, the mode switching unit 63 switches to the awake mode (brain wave control mode) because the sleep determination unit 62 has determined that the user is awake. In step S14, the output unit 64 executes the brain wave control mode. That is, the output unit 64 operates the operation target object 100 based on the brain information of the user's frontal lobe. For example, the output unit 64 continuously outputs the user's brain information from the communication units 17 and 107 to the control unit 104. The control unit 104 operates the operation target object 100 based on the received user's brain information.

[0051] If the sleep determination unit 62 determines in step S12 that the user is asleep but not awake (No), the process proceeds to step S15. In step S15, the sleep determination unit 62 determines whether the user is in REM sleep. If the sleep determination unit 62 determines that the user is not in REM sleep (No), the process proceeds to step S16. In step S16, the mode switching unit 63 switches to automatic mode, which is one of the sleep modes, because the sleep determination unit 62 has determined that the user is not in REM sleep, i.e., in non-REM sleep. In step S14, the output unit 64 executes the automatic mode. That is, the output unit 64 operates the operation target 100 based on a preset program. For example, the output unit 64 outputs a program for the automatic mode stored in the storage unit 16 from the communication units 17 and 107 to the control unit 104. The control unit 104 operates the operation target 100 based on the received program for the automatic mode. The output unit 64 may cut off communication by the communication units 17 and 107 after transmitting the program for the automatic mode.

[0052] In step S15, if the sleep determination unit 62 determines that the user is in a REM sleep state (Yes), the process proceeds to step S17. In step S17, the sleep determination unit 62 determines whether the user's brain activity level is higher than a threshold value. Here, if the sleep determination unit 62 determines that the user's brain activity level is equal to or lower than the threshold value (No), the process proceeds to step S16 and performs the same processing as described above. On the other hand, if the sleep determination unit 62 determines that the user's brain activity level is higher than the threshold value (Yes), the process proceeds to step S18. In step S18, the mode switching unit 63 switches to a lucid dream mode, which is one of the sleep modes, because the sleep determination unit 62 has determined that the user is in a lucid dream state in which the user's brain activity level is higher than the threshold value. In step S14, the output unit 64 executes the lucid dream mode. That is, the output unit 64 operates the operation object 100 based on information about the user's motor cortex. For example, the output unit 64 continuously outputs information on the user's motor cortex from the communication units 17 and 107 to the control unit 104. The control unit 104 operates the operation target object 100 based on the received information on the user's motor cortex.

[0053] In the above-described embodiment, the stimulation unit 13 is configured with the visual area electrode 31 and the auditory area electrode 32, but is not limited to this configuration. For example, instead of the visual area electrode 31 and the video encoder 51, a device that projects an image onto the retina from a contact lens worn by the user may be used. Also, instead of the auditory area electrode 32 and the audio encoder 52, a device that reproduces audio from headphones worn by the user may be used.

[0054] In the above-described embodiment, the visual area electrode 31 inputs brain waves to the user's visual area, but brain waves may be input to the entire user's brain. Also, while the auditory area electrode 32 inputs brain waves to the user's auditory area, brain waves may be input to the entire user's brain.

[0055] In the above-described embodiment, the motor cortex electrode 22 acquires brain waves from the user's motor cortex, but brain waves may be acquired from the user's entire brain. Also, the motor cortex electrode 22 acquires brain waves from the user's motor cortex, and the movement decoder 42 decodes the brain waves and outputs them to the operation object 100. However, the operation object 100 may have a speaker, and the movement decoder 42 may decode the user's brain waves into sound, which is then reproduced from the operation object 100.

[0056] Furthermore, in the above-described embodiment, the sleep determination unit 62 determines the user's sleep state based on the user's brain information, but the user's sleep state may also be determined based on a decrease in correlation between the motor cortex signal acquired from the motor cortex electrode 22 and the myoelectric signal acquired from a muscle electrode (not shown).

[0057] Furthermore, in the above-described embodiment, when the user's sleep state is determined based on the user's brain information, the user may be notified that they are dreaming by providing a specific stimulus, and may be encouraged to control their dreams by transitioning to a lucid dream.

[0058] The output unit 64 outputs an operation signal for operating the operation target 100 based on each mode, and this processing may be performed periodically (e.g., every 0.01 seconds). The output unit 64 outputs a video signal and an audio signal input from the operation target 100 to the video encoder 51 and the audio encoder 52 of the conversion unit 14, and this processing may also be performed periodically (e.g., every 0.01 seconds).

[0059] [Effects of the Present Embodiment] The operation control device of the present embodiment includes a brain information acquisition unit 61 that acquires brain information of the user, a sleep determination unit 62 that determines whether the user is asleep or not based on the brain information acquired by the brain information acquisition unit 61, and a mode switching unit 63 that switches between a first mode in which the operation object 100 is operated based on the user's brain information when the sleep determination unit 62 determines that the user is awake, and a second mode in which the operation object 100 is operated based on information other than the user's brain information when the sleep determination unit 62 determines that the user is asleep.

[0060] Therefore, the object to be operated 100 is operated based on appropriate information according to the user's state of wakefulness and sleep, and even if the user is asleep, the object to be operated 100 can be operated using the user's sleep time.

[0061] The second mode of the operation control device of this embodiment has an automatic mode in which the operation object is operated based on a preset program.Therefore, when the user is asleep, the automatic mode is switched to and the operation object 100 is operated based on a preset program, so that even if the user is asleep, the operation object 100 can be operated using the user's sleeping time.

[0062] The operation control device of this embodiment has a storage unit 16 that stores a program for executing the automatic mode, and the storage unit 16 is connected to the mode switching unit 63 or the operation target object 100. Therefore, the automatic mode can be quickly executed.

[0063] The operation control device of this embodiment has an output unit 64 that outputs an operation signal to the operation target 100 based on the sleep mode or automatic mode switched by the mode switching unit 63, and the output unit 64 is capable of outputting an operation signal to the operation target 100 when the sleep mode is being executed, and blocks output of the operation signal to the operation target 100 when the automatic mode is being executed. This makes it possible to simplify the processing between the operation control device and the operation target 100.

[0064] For example, the sleep mode is an automatic mode in which the output unit 64 operates the operation target 100 based on a program stored in the storage unit 16, but this is not limiting. In the sleep mode, the output unit 64 may output an operation signal that reduces the power consumption of the operation target 100 or an operation signal that stops the operation target 100, thereby operating the operation target 100. Furthermore, in the sleep mode, the output unit 64 may output a signal that specifies another operation control device. When a signal that specifies another operation control device is input to the operation target 100, the communication unit 105 may be capable of transmitting and receiving various information to and from a control unit of the other operation control device via the communication unit of the other operation control device.

[0065] The second mode of the manipulation control device of this embodiment has a sleep mode (lucid dream mode) in which the manipulation object is operated based on information from the user's motor cortex. Therefore, when the user is asleep, the device switches to the sleep mode and operates the manipulation object 100 based on information from the user's motor cortex. Even when the user is asleep, the user's brain is connected to the manipulation object 100, and the manipulation object 100 can be operated using the user's sleep time without interfering with the user's brain activity in the awake state.

[0066] In the operation control device of this embodiment, the mode switching unit 63 switches to a sleep mode (lucid dream mode) when the sleep determination unit 62 determines that the user is asleep and that the activity level is higher than a preset threshold, and switches to an automatic mode in which the operation object 100 is operated based on a preset program when the sleep determination unit 62 determines that the user is asleep and that the activity level is equal to or lower than the preset threshold, and the output unit 64 outputs an operation signal to the operation object 100 based on the sleep mode or the automatic mode switched by the mode switching unit 63. Therefore, when the user's brain activity level is higher than the threshold, the mode is switched to the lucid dream mode, but when the user's brain activity level is equal to or lower than the threshold, the mode is switched to the automatic mode in which the operation object 100 is operated based on the program, so that the operation object 100 can be operated appropriately when the user is asleep.

[0067] In the operation control device of this embodiment, the mode switching unit 63 switches to an awake mode in which the operation object 100 is operated based on brain information from the user's frontal lobe when the sleep determination unit 62 determines that the user is not in an asleep state, and the output unit 64 outputs an operation signal to the operation object 100 based on the awake mode switched to by the mode switching unit 63. Therefore, when the user is not in an asleep state, the mode is switched to the awake mode and the operation object 100 is operated based on brain information from the user's frontal lobe, so that the operation object 100 can be operated appropriately when the user is in an awake state.

[0068] The operation control device according to the present disclosure has been described above, but it may be implemented in various different forms other than the above-described embodiment.

[0069] The components of the illustrated operation control device are conceptual functional components and do not necessarily have to be physically configured as shown in the drawings. In other words, the specific form of each device is not limited to that shown in the drawings, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.

[0070] The configuration of the operation control device is realized, for example, as software, by a program loaded into memory. In the above embodiment, the functional blocks are described as being realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms, using only hardware, only software, or a combination of both.

[0071] The above-described components include those that can be easily imagined by a person skilled in the art and those that are substantially the same. Furthermore, the above-described configurations can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the configurations are possible within the scope of the gist of the present invention.

[0072] The information processing device, information processing method, and program of the present disclosure can be applied to technology for controlling a user's brain activity.

[0073] REFERENCE SIGNS LIST 10 Operation control device 11 Input unit 12 Measurement unit 13 Stimulation unit 14 Conversion unit 15 Control unit 16 Memory unit 17 Communication unit 21 Frontal lobe electrode 22 Motor cortex electrode 31 Visual cortex electrode 32 Auditory cortex electrode 41 Frontal lobe decoder 42 Movement decoder 51 Video encoder 52 Audio encoder 61 Brain information acquisition unit 62 Sleep determination unit 63 Mode switching unit 64 Output unit 100 Operation object 101 Camera 102 Microphone 103 Drive unit 104 Control unit

Claims

1. a brain information acquisition unit that acquires brain information of a user; a sleep determination unit that determines whether the user is asleep based on the brain information acquired by the brain information acquisition unit; a mode switching unit that switches between a first mode in which an operation object is operated based on brain information of the user when the sleep determination unit determines that the user is in an awake state, and a second mode in which the operation object is operated based on information other than the brain information of the user when the sleep determination unit determines that the user is in a sleeping state; Equipped with the second mode includes a sleep mode in which the operation object is operated based on information of the user's motor cortex; the mode switching unit switches to the sleep mode when the sleep determination unit determines that the user is in a REM sleep state and that the activity level of the dorsolateral prefrontal cortex is higher than a preset threshold, and switches to an automatic mode in which the operation object is operated based on a preset program when the sleep determination unit determines that the user is in a REM sleep state and that the activity level of the dorsolateral prefrontal cortex is equal to or lower than a preset threshold. Operation control device.

2. (delete)

3. a storage unit in which a program for executing the second mode is stored, the storage unit being connected to the mode switching unit or the operation object; The operation control device according to claim 1 .

4. an output unit that outputs an operation signal to the operation object based on the first mode or the second mode switched by the mode switching unit, wherein the output unit is capable of outputting the operation signal to the operation object when the first mode is being executed and blocks output of the operation signal to the operation object when the second mode is being executed; The operation control device according to claim 1 .

5. (delete)

6. (delete)

7. the mode switching unit switches to an awake mode in which the operation object is operated based on brain information from a frontal lobe of the user when the sleep determination unit determines that the user is in an awake state. The operation control device according to claim 1 .

8. acquiring brain information of a user; determining whether the user is asleep based on the acquired brain information; switching between a first mode in which an operation object is operated based on brain information of the user when it is determined that the user is in an awake state, and a second mode in which the operation object is operated based on information other than the brain information of the user when it is determined that the user is in a sleep state; the second mode includes a sleep mode in which the operation object is operated based on information of the user's motor cortex; the mode switching unit switches to the sleep mode when the sleep determination unit determines that the user is in a REM sleep state and that the activity level of the dorsolateral prefrontal cortex is higher than a preset threshold, and switches to an automatic mode in which the operation object is operated based on a preset program when the sleep determination unit determines that the user is in a REM sleep state and that the activity level of the dorsolateral prefrontal cortex is equal to or lower than a preset threshold; An operation control method including:

9. acquiring brain information of a user; determining whether the user is asleep based on the acquired brain information; switching between a first mode in which an operation object is operated based on brain information of the user when it is determined that the user is in an awake state, and a second mode in which the operation object is operated based on information other than the brain information of the user when it is determined that the user is in a sleep state; the second mode includes a sleep mode in which the operation object is operated based on information of the user's motor cortex; the mode switching unit switches to the sleep mode when the sleep determination unit determines that the user is in a REM sleep state and that the activity level of the dorsolateral prefrontal cortex is higher than a preset threshold, and switches to an automatic mode in which the operation object is operated based on a preset program when the sleep determination unit determines that the user is in a REM sleep state and that the activity level of the dorsolateral prefrontal cortex is equal to or lower than a preset threshold; A program that causes a computer to execute the following.