Information Processing Method

Abstract

To prevent equipment from being operated using biological information from a user in a case where the user has no intention of operating the equipment.SOLUTION: A computer executes an information processing method which includes determining whether to execute a specific process in accordance with biological information in a predetermined first time period including a point of time at which the biological information associated with the specific process is measured from a user, a predetermined second time period based on and before the point of time at which the biological information associated with the specific process is measured from the user, or a predetermined third time period based on and after the point of time at which the biological information associated with the specific process is measured from the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing method. [Background technology]

[0002] It is conceivable that devices could be operated using biological information such as brain waves.

[0003] Cited Document 1 describes an apparatus that detects an electroencephalogram biosignal from a user's electroencephalogram, detects a surface myoelectric potential biosignal from the user's surface myoelectric potential, and calculates a control signal based on both biosignals.

[0004] Cited Document 2 describes a device that acquires the brain waves of a user and selectively operates a plurality of operated devices in accordance with the analysis results obtained by analyzing the brain waves. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-211705 A [Patent Document 2] JP 2016-67922 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, even if the user does not intend to operate the device, biological information indicating the operation of the device may be generated and measured from the user. In this case, a situation may occur in which the device is operated regardless of the user's intention.

[0007] An object of the present invention is to prevent a device from being operated using a user's biometric information when the user has no intention of operating the device. [Means for solving the problem]

[0008] The invention according to claim 1 provides a computer that: Biometric information indicating the operation of a specific device Associated with a particular action The above A first predetermined time period including the time before and after the time when the biometric information is measured from the user, and the biometric information associated with the specific process is measured from the user as a reference. Current A second predetermined time period before the time point, or a time point when the biological information associated with the specific process is measured from the user is used as a reference. Current In a third predetermined time period after that time I , When specific biometric information different from the biometric information associated with the specific process, i.e., information not supposed to be measured under the circumstances in which the biometric information associated with the specific process is measured, is not measured from the user, or when biometric information indicating the operation content of the specific device is measured for less than a predetermined time even if it is measured from the user, a process of operating the specific device according to the biometric information indicating the operation content of the specific device is executed, and when the above-mentioned case is not met and when the biometric information is measured for more than the predetermined time, a process of not operating the specific device is executed. decision of Performing a process that includes The present invention is characterized in that it is configured as follows: , an information processing method. The invention of claim 2 is the information processing method of claim 1, characterized in that a computer is configured to execute a process including deciding to execute a process of operating the specific device when specific biometric information different from the biometric information associated with the specific processing, i.e., information that is not expected to be measured under the circumstances in which the biometric information associated with the specific processing is measured, is not measured from the user during a predetermined first time period including before and after the time point when the biometric information associated with the specific processing, which is biometric information indicating the operation content of a specific device, and a process of not operating the specific device in accordance with the biometric information indicating the operation content of the specific device when measured from the user, in a predetermined second time period before the time point when the biometric information associated with the specific processing is measured from the user, and in a predetermined third time period after the time point when the biometric information associated with the specific processing is measured from the user. Effect of the Invention

[0009] Claim 1 and claim 2 According to the present invention, it is possible to prevent a device from being operated using a user's biometric information when the user has no intention of operating the device. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a configuration of an information processing device according to an embodiment of the present invention. [Diagram 3] FIG. 4 is a flowchart showing a process performed by the information processing device according to the embodiment. [Figure 4] FIG. 13 illustrates a management table. [Diagram 5] FIG. 13 illustrates a management table. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An information processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 shows an example of the configuration of the information processing system according to this embodiment.

[0012] The information processing system according to the present embodiment includes an information processing device 10, a plurality of biological information measuring devices, one or more environmental information measuring devices 14, and one or more devices 16. In the example shown in FIG. 1, the information processing system includes biological information measuring devices 12A and 12B, but this is merely an example. In the following, when there is no need to distinguish between the biological information measuring devices 12A and 12B, the biological information measuring devices 12A and 12B will be referred to as "biological information measuring devices 12". Note that the number of each device shown in FIG. 1 is merely an example, and the number of each device is not limited to the number of each device shown in FIG. In addition, the information processing system may include other devices (for example, external devices such as a server) other than the devices shown in FIG. 1.

[0013] The information processing device 10, the bioinformation measuring device 12, the environmental information measuring device 14, and the device 16 are configured to communicate with other devices. The communication may be wired communication using a cable, or wireless communication. That is, each device may be physically connected to other devices by a cable and transmit and receive information to each other, or may transmit and receive information to each other by wireless communication. For example, short-range wireless communication or Wi-Fi (registered trademark) is used as the wireless communication. Wireless communication of other standards may be used. For example, short-range wireless communication is Bluetooth (registered trademark), RFID (Radio Frequency Identifier), NFC, etc. Each device may communicate with other devices, other sensors, etc. via a communication path such as a LAN (Local Area Network) or the Internet.

[0014] The information processing device 10 is, for example, a personal computer (hereinafter referred to as "PC"), a tablet PC, a smartphone, a mobile phone, or other device. The information processing device 10 may be a terminal device that a user can carry (for example, a tablet PC, a smartphone, a mobile phone, etc.), or may be a device that is installed on a table or the like and used. The information processing device 10 may be a smart speaker having a communication function, a microphone, and a speaker. The information processing device 10 may be installed indoors (for example, a floor, ceiling, or table of a room) or outdoors. The information processing device 10 may also be a mobile device (for example, a self-propelled device).

[0015] The biological information measuring devices 12 have sensors, electrodes, etc., and are configured to measure biological information of a user. Each biological information measuring device 12 may measure different types of biological information. Of course, some or all of the biological information measuring devices 12 may be configured to measure the same type of biological information. Furthermore, each biological information measuring device 12 may be configured to measure one type of biological information, or may be configured to measure multiple types of biological information.

[0016] The vital information measuring device 12 transmits vital information measured by the own device to the information processing device 10. The vital information measuring device 12 may transmit the vital information to the information processing device 10 every time it measures the vital information, may store the vital information and transmit the vital information to the information processing device 10 at predetermined time intervals, or may transmit the vital information to the information processing device 10 at a timing designated by a user. In addition, the vital information measuring device 12 may receive vital information measured by another vital information measuring device 12 from the other vital information measuring device 12, and transmit the vital information measured by the own device and the vital information measured by the other vital information measuring device 12 to the information processing device 10.

[0017] The biological information measuring device 12 may analyze biological information measured by its own device or another biological information measuring device, and transmit information indicating the analysis result to the information processing device 10. For example, the biological information measuring device 12 may include a processor, and the processor may analyze the biological information. Of course, the analysis may be performed by the information processing device 10.

[0018] The biological information measuring device 12 may include a battery and be driven by power supplied from the battery, or may be driven by power supplied from the information processing device 10.

[0019] The biological information measuring device 12 may be a wearable device in which the entire biological information measuring device 12 is worn by a user to measure biological information. For example, the biological information measuring device 12 may be a device worn on the user's head, a hearable device worn on the user's ear, a device worn on the user's arm, hand, wrist, finger, etc. (for example, a wristwatch-type device, etc.), a device hung around the user's neck, or a device worn on the user's torso, leg, etc.

[0020] Biometric information is various physiological information and anatomical information emitted from a user who is a living organism. The concept of biometric information includes, for example, information indicating brain activity (e.g., brain waves, cerebral blood flow, brain magnetic field signals, etc.), pulse rate, blood pressure, heart rate, electrocardiogram waveforms, electromyogram waveforms, eye movement, body temperature, sweat rate, gaze, voice, and user movement. These are merely examples of biometric information, and other physiological information and anatomical information may be used as biometric information. The biometric information measuring device 12 may measure one of these pieces of biometric information, or may measure multiple pieces of biometric information.

[0021] The concept of bioinformation also includes bioelectric potential information that indicates an electric potential measured from a living organism. The concept of bioelectric potential information includes, for example, an electroencephalogram that is a measurement result of a microcurrent generated with brain activity, an electrocardiogram that is a measurement result of a microcurrent generated with heartbeat, an electromyogram that is a measurement result of a microcurrent generated with muscle activity, and a skin potential that is a measurement result of a microcurrent generated on the skin. These are merely examples of bioelectric potential information, and other bioelectric potential information may also be used.

[0022] The information processing device 10 receives biometric information from the biometric information measuring device 12, and performs the following operations: analyzing the biometric information, storing the biometric information, outputting the biometric information, storing information indicating the analysis results of the biometric information, and outputting information indicating the analysis results of the biometric information. Of course, the analysis of the biometric information may be performed by the biometric information measuring device 12. Outputting the biometric information means, for example, displaying the biometric information or outputting the biometric information as audio information. Outputting information indicating the analysis results of the biometric information means, for example, displaying information indicating the analysis results or outputting the analysis results as audio information. The information processing device 10 may transmit the biometric information or information indicating the analysis results to another device.

[0023] The information processing device 10 may include one or more biological information measuring devices 12. That is, one or more biological information measuring devices 12 may be incorporated into the information processing device 10 to configure one device. The entire information processing device 10 including one or more biological information measuring devices 12 may be worn by a user to measure biological information. That is, the information processing device 10 may be a wearable device. For example, the information processing device 10 may be a device worn on the user's head, a hearable device worn on the user's ear, a device worn on the user's arm, hand, wrist, finger, etc. (for example, a wristwatch-type device), a device hung around the user's neck, or a device worn on the user's torso, leg, etc.

[0024] The information processing device 10 and the biological information measuring device 12 may be separate devices. For example, the information processing device 10 may be a smart speaker, and the biological information measuring device 12 may be a wearable device worn by a user.

[0025] The environmental information measuring device 14 is configured to measure environmental information related to a user or the environment surrounding the environmental information measuring device 14. For example, the environmental information measuring device 14 is a camera which is an imaging device, a microphone which collects sound, a temperature sensor which measures temperature, a humidity sensor which measures humidity, an odor sensor which measures odor, an illuminance sensor which measures brightness, an infrared sensor, a human presence sensor, or the like. One or more of these sensors may be included in the information processing system as the environmental information measuring device 14.

[0026] For example, the surroundings of the information processing device 10 or other places are photographed by a camera, and image data representing the surroundings or other places is generated. The image data may be moving image data or still image data. The image data photographed by the camera corresponds to an example of environmental information related to the environment included in the shooting range of the camera. Moreover, image data representing a user, which is generated by photographing the user by the camera, corresponds to an example of the biometric information of the user. For example, the movement of the user and the body shape of the user detected from the image data correspond to an example of the biometric information of the user. In that sense, the camera corresponds to an example of the biometric information measuring device 12 that measures the biometric information of the user.

[0027] Furthermore, sounds around the microphone (e.g., human voices and other sounds) are input to the microphone, and sound data is generated by the microphone. The sound data representing the sounds input to the microphone corresponds to an example of environmental information relating to the environment around the microphone. Furthermore, the sound data representing the user's voice input to the microphone corresponds to an example of the biometric information of the user. In that sense, the microphone corresponds to an example of a biometric information measuring device 12 that measures the biometric information of the user.

[0028] Moreover, data measured by a temperature sensor, a humidity sensor, an odor sensor, an illuminance sensor, an infrared sensor, a human presence sensor, and the like correspond to an example of environmental information. Moreover, data measured from a user by these sensors corresponds to an example of bioinformation of the user. In that sense, these sensors correspond to an example of a bioinformation measuring device 12 that measures the bioinformation of the user.

[0029] The environmental information may include information indicating the size of the room in which the user is located, information indicating the size of the room in which the device 16 is installed, information indicating the number of windows in the room, etc.

[0030] One or more environment information measuring devices 14 may be included in the information processing device 10.

[0031] The device 16 is, for example, a PC, a tablet PC, a smartphone, a mobile phone, a robot (for example, a humanoid robot, a non-human animal-type robot, a cleaning robot, and other robots), a display device such as a projector or a liquid crystal display, a recording device, a playback device, an imaging device such as a camera, a refrigerator, a rice cooker, a microwave oven, a coffee maker, a vacuum cleaner, a washing machine, an air conditioner, a lighting device, a clock, a surveillance camera, an automobile, a two-wheeled vehicle, an aircraft (for example, an unmanned aerial vehicle (for example, a drone)), a game machine, a gas range, a warm water washing toilet seat, a ventilation fan, a doorbell, a front door monitor, an elevator, a door, a window, or various sensing devices (for example, a temperature sensor, a humidity sensor, a voltage sensor, a current sensor, etc.). The category of the concept of the device 16 may include devices in general. For example, information devices, video devices, audio devices, and other devices may also be included in the category of the device 16 according to this embodiment.

[0032] The device 16 has a communication device which is a communication interface, a storage device which stores data, and a processor which controls the operation of the device 16. The device 16 may have a user interface. The device 16 may transmit device identification information for identifying the device 16 to the information processing device 10. The device identification information is, for example, the ID, name, model number, or address (e.g., MAC address, IP address, etc.) of the device 16.

[0033] The configuration of the information processing device 10 will be described in detail below.

[0034] The information processing device 10 includes, for example, a communication device 18, a UI 20, a storage device 22, and a processor 24. The information processing device 10 may include components other than these.

[0035] The communication device 18 is a communication interface and has a function of transmitting data to other devices and a function of receiving data transmitted from other devices. The communication device 18 may have a wireless communication function or a wired communication function. The communication device 18 may communicate with other devices by using, for example, short-distance wireless communication, or may communicate with other devices via a communication path such as a LAN or the Internet. For example, the communication device 18 receives biological information transmitted from the biological information measuring device 12. The communication device 18 may transmit control information for controlling the operation of the biological information measuring device 12 to the biological information measuring device 12. The communication device 18 also receives environmental information transmitted from the environmental information measuring device 14. The communication device 18 may transmit control information for controlling the operation of the environmental information measuring device 14 to the environmental information measuring device 14. The communication device 18 also transmits control information for controlling the operation of the device 16 to the device 16. The communication device 18 may receive information transmitted from the device 16.

[0036] The UI20 is a user interface and includes a display device and an operation device. The display device is a liquid crystal display, an EL display, or the like. The operation device is a keyboard, an input key, an operation panel, or the like. The UI20 may be a UI such as a touch panel that combines a display device and an operation device. The UI20 may also include a microphone, or a speaker that emits sound.

[0037] The storage device 22 is a device constituting one or more storage areas for storing various data. The storage device 22 is, for example, a hard disk drive, various memories (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more storage devices 22 are included in the information processing device 10.

[0038] The storage device 22 stores management information. The management information is information for identifying the operation content of the device 16 based on the biometric information measured from the user. For example, predetermined reference biometric information and operation information indicating the operation content of the device 16 are linked in advance and registered in the management information. The reference biometric information may be biometric information assumed to be generated by a user performing an operation linked to the reference biometric information, or biometric information assumed to be generated by a user requesting the execution of the operation. The reference biometric information can also be said to be biometric information corresponding to the operation content of the device 16. The reference biometric information and operation information may be linked to each other and registered in the management information for each user.

[0039] The management information may register operation information related to the power supply operation contents of device 16, operation information related to the function level operation contents of device 16, or both operation information related to the power supply operation contents and operation information related to the function level operation contents.

[0040] The power operation content of the device 16 is an operation to turn on or off the power of the device 16. The operation information regarding the power operation content is information indicating an operation to turn on or off the power of the device 16. It can be said that the biometric information linked to the operation information regarding the power operation content is biometric information corresponding to the power on or off of the device 16. For each user, the operation information regarding the power operation content of the device 16 and the reference biometric information may be linked and registered in the management information.

[0041] The operation content of the function level of the device 16 is an operation to set the function level of the device 16. The operation information related to the operation content of the function level is information indicating an operation to set the function level of the device 16. It can be said that the biometric information linked to the operation information related to the operation content of the function level is biometric information corresponding to the function level of the device 16. For each user, the operation information related to the operation content of the function level of the device 16 and the reference biometric information may be linked and registered in the management information.

[0042] The function level is, for example, a level related to the performance or output of the device 16. To explain by giving specific examples, examples of the device level include the set temperature of an air conditioner, the air volume of the air conditioner, the air direction of the air conditioner, the presence or absence of a dehumidification function of the air conditioner, the brightness of a display device, the brightness of a lighting device, the volume of a speaker, the moving speed of a self-propelled device (e.g., a robot or a self-propelled vacuum cleaner), the setting values ​​of devices such as an imaging device, a recording device, and a playback device, the setting values ​​of devices such as a refrigerator, a rice cooker, and a microwave oven, and the setting values ​​of various sensing devices. These are merely examples, and other setting values ​​may be the function level.

[0043] The processor 24 is configured to obtain biometric information of the user and output instructions to operate the device 16 according to the biometric information.

[0044] For example, when biological information is measured by the biological information measuring device 12, the biological information is transmitted from the biological information measuring device 12 to the information processing device 10. The processor 24 receives the biological information and determines the operation content of the device 16 based on the biological information. The processor 24 operates the device 16 by transmitting control information including information indicating the determined operation content to the device 16. Having received the control information, the device 16 operates according to the control information.

[0045] For example, a brain-machine interface may be constructed by a bioinformation measuring device 12 that measures brain activity and an information processing device 10. The brain-machine interface may be invasive or non-invasive. In this case, the processor 24 operates the device 16 based on the user's brain activity (e.g., brain waves, etc.). In order to operate the device 16, the processor 24 may extract characteristic components from the brain waves and operate the device 16 based on the extracted components. In order to extract characteristic components from the brain waves, a fast Fourier transform (FFT), a wavelet transform (WT), a time frequency distribution (TFD), an eigenvector method (EM), an autoregressive method (ARM), etc. may be used. In addition, as a method for linking the brain waves and the operation content of the device 16 using the feature vector obtained by extracting features, for example, Independent Component Analysis (ICA), k-means, Support Vector Machine (SVM), convolutional neural network, etc. may be used.

[0046] The processor 24 may be configured to receive device identification information transmitted from the device 16 and identify the device 16. For example, the processor 24 transmits a request for device identification information to the device 16, and acquires the device identification information transmitted from the device 16 in response to the request. The device 16 that has become capable of communicating with the information processing device 10 by being connected to the information processing device 10 or the like may transmit device identification information to the information processing device 10, and the processor 24 may receive the device identification information thus transmitted. The device identification information may be, for example, information indicating the name of the device 16, information indicating the model number of the device 16, or information indicating the address of the device 16 (for example, a MAC address or an IP address).

[0047] Moreover, the processor 24 is configured to control the operation of each unit of the information processing device 10. The processor 24 may include a memory.

[0048] Furthermore, when biometric information indicating the operation of device 16, which is not expected to be generated from the user under a specific situation, is measured from the user, processor 24 does not operate device 16 according to the biometric information. On the other hand, when biometric information indicating the operation of device 16 under a situation other than the specific situation is measured from the user, processor 24 operates device 16 according to the biometric information.

[0049] For example, the specific situation is a situation in which other specific biometric information is measured from the user, which is different from the biometric information indicating the operation content of the device 16. The biometric information indicating the operation content and the specific biometric information may be the same type of biometric information or different types of biometric information, and may be measured by the same biometric information measuring device 12 or different biometric information measuring devices 12.

[0050] The relationship between the biological information indicating the operation content of the device 16 and the time at which specific biological information is measured will be described below.

[0051] (Time Relationship 1)

[0052] For example, when biometric information indicating the operation content of device 16 is measured from a user, if specific biometric information is measured from the user, processor 24 may not operate device 16 according to the biometric information indicating the operation content. On the other hand, when biometric information indicating the operation content of device 16 is measured from a user, if specific biometric information is not measured from the user, processor 24 operates device 16 according to the biometric information indicating the operation content.

[0053] (Time Relationship 2)

[0054] As another example, when specific biometric information is measured from a user during a first predetermined time period including before and after the time when biometric information indicating the operation content of device 16 is measured from the user, processor 24 may not operate device 16 according to the biometric information indicating the operation content. On the other hand, when specific biometric information is not measured from the user during the first time period, processor 24 operates device 16 according to the biometric information indicating the operation content.

[0055] (Time Relationship 3)

[0056] As yet another example, when specific biometric information is measured from the user during a second predetermined time period prior to the time point when biometric information indicating the operation content of device 16 is measured from the user, processor 24 may not operate device 16 according to the biometric information indicating the operation content. On the other hand, when specific biometric information is not measured from the user during the second time period, processor 24 operates device 16 according to the biometric information indicating the operation content.

[0057] (Time Relationship 4)

[0058] As yet another example, when specific biometric information is measured from the user during a predetermined third time period after the time point when biometric information indicating the operation content of device 16 is measured from the user, processor 24 may not operate device 16 according to the biometric information indicating the operation content. On the other hand, when specific biometric information is not measured from the user during the third time period, processor 24 operates device 16 according to the biometric information indicating the operation content.

[0059] When specific biometric information is measured from the user at the time when biometric information indicating the operation content of device 16 is measured from the user, or in the first time zone, the second time zone, or the third time zone, processor 24 may not operate device 16 according to the biometric information indicating the operation content. On the other hand, when specific biometric information is not measured from the user at the time when biometric information indicating the operation content of device 16 is measured from the user, or in the first time zone, the second time zone, or the third time zone, processor 24 may operate device 16 according to the biometric information indicating the operation content.

[0060] When specific biometric information is measured for a predetermined time or longer, the processor 24 may not operate the device 16 according to the biometric information indicating the operation content. Even when specific biometric information is measured, if the specific biometric information is measured for less than a predetermined time, the processor 24 may operate the device 16 according to the biometric information indicating the operation content.

[0061] The specific biological information is biological information that is not expected to be measured under the circumstances in which biological information indicating the operation content of the device 16 is measured. For example, biological information measured from the user during sleep, in a stage before sleep (for example, a stage in which consciousness is hazy), or during meditation corresponds to an example of the specific biological information. The specific biological information is, for example, information indicating at least one of the following measured from the user during sleep, in a stage before sleep, or during meditation: electroencephalogram, blood flow of electroencephalogram, brain magnetic field signal, pulse rate, blood pressure, heart rate, electrocardiogram waveform, electromyogram waveform, eye movement, body temperature, amount of sweat, blood flow, respiratory rate, and user movement.

[0062] Examples will now be described.

[0063] Example 1

[0064] For example, the specific biological information is information indicating a body temperature equal to or lower than a threshold. The threshold is, for example, determined in advance. The threshold may be changed by the user. The threshold may also be determined for each user. When the body temperature measured from the user is equal to or lower than the threshold, the user's state is presumed to be asleep, in a stage just before sleep, or meditating. When the user's body temperature is equal to or lower than the threshold, the processor 24 does not operate the device 16 according to the biological information indicating the operation content. When the user's body temperature exceeds the threshold, the processor 24 operates the device 16 according to the biological information indicating the operation content. The processor 24 may determine whether the user's state is asleep, in a stage just before sleep, or meditating based on biological information other than the body temperature and the body temperature.

[0065] Example 2

[0066] As another example, the specific biological information is information indicating a blood pressure equal to or lower than a threshold. The threshold is, for example, determined in advance. The threshold may be changed by the user. The threshold may also be determined for each user. When the blood pressure measured from the user is equal to or lower than the threshold, the user's state is presumed to be asleep, in a stage just before sleep, or meditating. When the user's blood pressure is equal to or lower than the threshold, the processor 24 does not operate the device 16 according to the biological information indicating the operation content. When the user's blood pressure exceeds the threshold, the processor 24 operates the device 16 according to the biological information indicating the operation content. The processor 24 may determine whether the user's state is asleep, in a stage just before sleep, or meditating based on biological information other than blood pressure and blood pressure.

[0067] Example 3

[0068] As yet another example, the specific biometric information is information about the user's eyes.

[0069] (Example 3-1)

[0070] For example, the information on the user's eyes is information indicating that the user's eyes are closed. When biometric information indicating that the user's eyes are closed is measured, the user's state is inferred to be asleep, in a stage before sleep, or meditating. When biometric information indicating that the user's eyes are closed is measured, the processor 24 does not operate the device 16 according to the biometric information indicating the operation content. When biometric information indicating that the user's eyes are open is measured, the processor 24 operates the device 16 according to the biometric information indicating the operation content. When biometric information indicating that the user's eyes are closed is not measured, the processor 24 may operate the device 16 according to the biometric information indicating the operation content.

[0071] The biometric information indicating that the user's eyes are closed is, for example, image data (e.g., still image data or moving image data) representing the user's eyes. For example, the user's eyes are photographed by a photographing device such as a camera, and image data generated by the photographing is transmitted from the photographing device to the information processing device 10. The processor 24 applies image analysis processing to the image data to determine whether the user's eyes shown in the image data are closed or not. The photographing device corresponds to an example of the biometric information measuring device 12.

[0072] (Example 3-2)

[0073] As another example, the information on the user's eyes is the number of blinks the user makes per unit time, and is information indicating the number of blinks equal to or less than a threshold. The threshold is, for example, predetermined. The threshold may be changed by the user. The threshold may also be determined for each user. If the number of blinks per unit time measured from the user is equal to or less than the threshold, the user's state is inferred to be asleep, in a stage before sleep, or meditating. If the number of blinks per unit time is equal to or less than the threshold, the processor 24 does not operate the device 16 according to the biometric information indicating the operation content. If the number of blinks per unit time exceeds the threshold, the processor 24 operates the device 16 according to the biometric information indicating the operation content. For example, the processor 24 counts the number of blinks of the user based on video data representing the user's eyes. The video data is generated by photographing the user's eyes with a photographing device such as a camera.

[0074] (Example 3-3)

[0075] As yet another example, the information on the user's eyes is information indicating that the pupils of the user's eyes are constricted. The constriction of the pupils may be determined for each user. When the pupils of the user's eyes are constricted, it is assumed that the user is in a daze. When the pupils of the user's eyes are constricted, the processor 24 does not operate the device 16 according to the biometric information indicating the operation content. When the pupils of the user's eyes are not constricted, the processor 24 operates the device 16 according to the biometric information indicating the operation content. For example, the processor 24 determines whether the pupils of the user's eyes are constricted based on image data (e.g., still image data or video image data) representing the user's eyes. The image data is generated by photographing the user's eyes with a photographing device such as a camera.

[0076] The processor 24 may determine whether the user is asleep, in a pre-sleep stage, or meditating based on information about the user's eyes and other biological information.

[0077] Example 4

[0078] The specific situation may be a situation in which the biometric information measured from the user satisfies a specific condition. For example, the specific condition is a condition for determining whether the user is asleep, in a stage just before sleep, or in meditation, and the situation in which the biometric information measured from the user satisfies the specific condition is that the user is asleep, in a stage just before sleep, or in meditation. The biometric information may be biometric information indicating the operation content, or may be biometric information not indicating the operation content.

[0079] For example, the biological information is brain waves. Specifically, the biological information is brain waves in a specific frequency band. The brain waves in the specific frequency band are, for example, alpha waves. Alpha waves having an amplitude equal to or greater than a threshold are biological information that satisfy a specific condition. The threshold is, for example, predetermined. The threshold may be changed by the user. The threshold may also be determined for each user. For example, when the amplitude of alpha waves measured from the user is equal to or greater than a threshold, it is estimated that the user is asleep, in a stage just before sleep, or meditating. When the amplitude of alpha waves measured from the user is equal to or greater than a threshold, the processor 24 does not operate the device 16 according to the biological information indicating the operation content. When the amplitude of alpha waves measured from the user is less than a threshold, the processor 24 operates the device 16 according to the biological information indicating the operation content.

[0080] Example 5

[0081] As another example, in a case where the bio-information indicating the operation content is expressed by a waveform, if the amplitude of the waveform is equal to or greater than the threshold, the processor 24 may operate the device 16 according to the bio-information. On the other hand, if the amplitude of the waveform is less than the threshold, the processor 24 does not operate the device 16 according to the bio-information. The threshold is, for example, determined in advance. The threshold may be changed by the user. Also, the threshold may be determined for each user.

[0082] Example 6

[0083] When the amount of the user's movement is equal to or greater than a threshold, the processor 24 operates the device 16 according to the bio-information indicating the operation content, and when the amount of the user's movement is less than the threshold, the processor 24 may not operate the device 16 according to the bio-information indicating the operation content. The amount of the user's movement is measured by the bio-information measuring device 12 such as a gyro sensor or an acceleration sensor. The threshold is, for example, predetermined. The threshold may be changed by the user. The threshold may also be determined for each user. When the amount of the user's movement is less than the threshold, it is assumed that the user is asleep, in a stage before sleep, or meditating. For example, the amount of movement of the user's body parts such as the head, arms, hands, and feet is measured, and the processor 24 determines whether the user is asleep, in a stage before sleep, or meditating based on the amount of movement.

[0084] Example 7

[0085] The specific situation may be a specific environment that the user is in. The processor 24 determines, based on the environmental information measured by the environmental information measuring device 14, whether the environment in which the user is in corresponds to the specific environment.

[0086] An environment having a specific relationship with the operation content indicated by the biometric information is a specific environment. For example, the specific environment is an environment in which the operation content indicated by the biometric information measured from the user is not expected to be performed, or an environment in which the user is not expected to request to perform the operation content indicated by the biometric information measured from the user.

[0087] To explain this with a specific example, even if the environment in which the user is placed is not one in which the heating of the air conditioner should be turned on, if the biological information measured from the user indicates operation content to turn on the heating of the air conditioner, the processor 24 does not turn on the heating of the air conditioner. For example, if the room temperature, which is environmental information, is equal to or higher than a threshold for determining whether or not to turn on the heating, and if the biological information measured from the user indicates operation content to turn on the heating of the air conditioner, the processor 24 does not turn on the heating of the air conditioner. On the other hand, if the room temperature is below the threshold, and the biological information measured from the user indicates operation content to turn on the heating of the air conditioner, the processor 24 turns on the heating of the air conditioner.

[0088] As another example, even if the environment in which the user is placed is not one in which the cooling of the air conditioner should be turned on, if the biological information measured from the user indicates the operation to turn on the cooling of the air conditioner, the processor 24 does not turn on the cooling of the air conditioner. For example, if the room temperature, which is environmental information, is less than a threshold for determining whether or not to turn on the cooling, and the biological information measured from the user indicates the operation to turn on the cooling of the air conditioner, the processor 24 does not turn on the cooling of the air conditioner. On the other hand, if the room temperature is equal to or higher than the threshold, and the biological information measured from the user indicates the operation to turn on the cooling of the air conditioner, the processor 24 turns on the cooling of the air conditioner.

[0089] Hereinafter, the flow of processing by the information processing device 10 will be described with reference to Fig. 3. Fig. 3 shows a flowchart showing processing by the information processing device 10.

[0090] Biometric information of a user is measured by the biological information measuring device 12. The biological information measured by the biological information measuring device 12 is transmitted to the information processing device 10, and the processor 24 acquires the biological information (S01).

[0091] The biological information acquired in step S01 is biological information for operating the device 16. For example, the biological information measuring device 12 that measures biological information for operating the device 16 is determined in advance, and the biological information measured by the biological information measuring device 12 is used as biological information for operating the device 16. For convenience of explanation, the biological information for operating the device 16 will be referred to as "first biological information" below. For example, brain waves are used as the first biological information for operating the device 16. In this case, the biological information measuring device 12 that measures brain waves is determined as the device that measures the first biological information for operating the device 16. The processor 24 acquires the brain waves measured by the biological information measuring device 12 that measures brain waves as the first biological information.

[0092] In addition, other biological information than the first biological information is measured by the biological information measuring device 12. The other biological information measured by the biological information measuring device 12 is transmitted to the information processing device 10, and the processor 24 acquires the other biological information (S02).

[0093] The other biological information acquired in step S02 is biological information for determining whether or not the first biological information acquired in step S01 is adopted as biological information for operating the device 16. For example, a biological information measuring device 12 for measuring other biological information for making the determination is determined in advance, and biological information measured by the biological information measuring device 12 is used as the other biological information for making the determination. For convenience, the biological information for determining whether or not the first biological information is adopted as biological information for operating the device 16 will be referred to as "second biological information" below. The second biological information may be the same type of biological information as the first biological information, or may be a different type of biological information from the first biological information. The second biological information may be biological information measured by the same biological information measuring device 12 as the biological information measuring device 12 that measures the first biological information, or may be biological information measured by a biological information measuring device 12 different from the biological information measuring device 12 that measures the first biological information. For example, when brain waves are used as the first biological information, biological information other than brain waves is used as the second biological information. As another example, a brain wave in a first frequency band may be used as the first biological information, and a brain wave in a second frequency band different from the first frequency band may be used as the second biological information. The processor 24 acquires, as the second biological information, biological information measured by the biological information measuring device 12 defined as a device for measuring the second biological information. For example, the biological information described in the above-mentioned Examples 1 to 6 is used as the second biological information.

[0094] Furthermore, environmental information may be measured by the environmental information measuring device 14. The environmental information measured by the environmental information measuring device 14 is transmitted to the information processing device 10, and the processor 24 acquires the environmental information (S02). The environmental information is information for determining whether or not to adopt the first biometric information acquired in step S01 as biometric information for operating the device 16.

[0095] The processor 24 may acquire both the second biological information and the environmental information, or may acquire either the second biological information or the environmental information. The order of the processes of steps S01 and S02 may be the same, the process of step S02 may be executed after the process of step S01, or the process of step S01 may be executed after the process of step S02.

[0096] Next, the processor 24 determines whether or not to adopt the first biometric information acquired in step S01 as biometric information for operating the device 16 based on at least one of the second biometric information and the environmental information (S03). That is, if the first biometric information indicates the operation content of the device 16, the processor 24 determines whether or not the operation content is the operation content intended by the user. In other words, the processor 24 determines whether or not the operation instruction grasped from the first biometric information is the instruction intended by the user. For example, the processor 24 determines whether or not to adopt the first biometric information acquired in step S01 as biometric information for operating the device 16 by executing any one of the above-mentioned Examples 1 to 7.

[0097] When the user's situation is the specific situation described in the above embodiment, the processor 24 determines not to adopt the first biometric information acquired in step S01 as biometric information for operating the device 16 (S03, No). That is, the processor 24 determines that the operation instruction grasped from the first biometric information is not an instruction intended by the user. In this case, the processor 24 maintains the current situation of the device 16 (S04).

[0098] For example, if the second biometric information measured in step S02 includes specific biometric information described in the above embodiments, or if specific biometric information is measured as the second biometric information, the processor 24 determines not to adopt the first biometric information acquired in step S01 as biometric information for operating the device 16.

[0099] If the user's situation is not the specific situation described above, the processor 24 determines that the first biometric information acquired in step S01 is to be adopted as biometric information for operating the device 16 (S03, Yes). That is, the processor 24 determines that the operation instruction grasped from the first biometric information is an instruction intended by the user.

[0100] For example, if the second biometric information measured in step S02 does not include the above-mentioned specific biometric information, or if the above-mentioned specific biometric information is not measured as the second biometric information, the processor 24 determines to adopt the first biometric information acquired in step S01 as the biometric information for operating the device 16.

[0101] If the first biometric information acquired in step S01 is adopted as biometric information for operating the device 16 (S03, Yes), the processor 24 identifies the device 16 to be operated based on the first biometric information and identifies the operation content of the device 16 (S05).

[0102] For example, the processor 24 compares the first biometric information acquired in step S01 with each reference biometric information registered in the above-mentioned management information, and searches for reference biometric information whose difference from the first biometric information is within an allowable range. The processor 24 identifies the operation content of the device 16 linked to the searched reference biometric information. This identifies the device 16 to be operated, and the operation content of the device 16. The allowable range is determined in advance. The allowable range may be changed by the user. The allowable range may be determined for each user.

[0103] When a plurality of reference biometric information pieces whose difference from the first biometric information acquired in step S01 falls within an allowable range are found, the processor 24 identifies the reference biometric information piece whose difference from the first biometric information is the smallest among the plurality of reference biometric information pieces. The processor 24 identifies the operation content of the device 16 linked to the identified reference biometric information.

[0104] The reference biometric information may be information indicating a characteristic component of the biometric information. In this case, the processor 24 may extract a characteristic component from the first biometric information and search for reference biometric information having a component whose difference from the extracted component is within an allowable range. For example, when an electroencephalogram is used as the first biometric information, the processor 24 may extract a characteristic component from the measured electroencephalogram and analyze the component to infer the operation content represented by the electroencephalogram.

[0105] Next, the processor 24 transmits control information including information indicating the operation content grasped in step S05 to the device 16 (S06). The device 16 that receives the control information operates according to the control information. In this manner, the processor 24 operates the device 16.

[0106] In addition, when the user agrees to operate the device 16, the processor 24 may transmit control information to the device 16 to operate the device 16. For example, the processor 24 may display a message on the UI 20 to inquire of the user whether or not the user agrees to operate the device 16, or may output the message by voice. When biometric information indicating agreement is measured from the user and the processor 24 acquires the biometric information, the processor 24 transmits control information to the device 16 to operate the device 16. For example, when biometric information indicating agreement is measured from the user within a predetermined time from the time when the message is displayed or output as voice, and the processor 24 acquires the biometric information, the processor 24 transmits control information to the device 16 to operate the device 16. When biometric information indicating agreement is not measured from the user, the processor 24 does not transmit control information to the device 16.

[0107] In step S01, if no biometric information is acquired whose difference from the reference biometric information falls within the allowable range, the operation content is not specified and the current state of the device 16 is maintained.

[0108] For example, when first biometric information indicating the operation content for turning on a certain device 16 is measured from a user, the processor 24 transmits control information including information indicating the operation content for turning on the power of the device 16 to the device 16. This causes the device 16 to be powered on. Also, when first biometric information indicating the operation content for setting the function level of a certain device 16 to a certain value is measured from a user, the processor 24 transmits control information including information indicating the operation content for setting the function level of the device 16 to that value to the device 16. This causes the function level of the device 16 to be set to that value.

[0109] A specific example of the management information will be described below.

[0110] 4 shows an example of a management table, which is an example of management information. The data of the management table is stored in the storage device 22. The data of the management table may not be stored in the storage device 22, but may be stored in an external device such as a server.

[0111] In the management table, an ID, a reference electroencephalogram, and operation information indicating the operation of the device 16 are linked in advance. The reference electroencephalogram is an example of reference biological information. Here, the electroencephalogram is used as an example of the reference biological information, but biological information other than the electroencephalogram may be used as the reference biological information.

[0112] The ID is information for managing the information registered in the management table.

[0113] The reference electroencephalogram is, for example, an electroencephalogram determined by statistical processing, and is assumed to be generally generated by a user performing an operation content linked to the reference electroencephalogram, or an electroencephalogram assumed to be generally generated by a user requesting the execution of the operation content.

[0114] The reference electroencephalogram may be an electroencephalogram in a specific frequency band, or may be an electroencephalogram including electroencephalograms in a plurality of frequency bands.

[0115] The operation information includes device identification information for identifying the device 16 to be operated, and information indicating the operation content performed on the device 16. For example, the operation content may be an operation to turn on or off the power of the device 16, or an operation to set the function level of the device 16.

[0116] For example, the reference electroencephalogram associated with ID "1" is an electroencephalogram representing the operation of turning on the cooling function of an air conditioner. The reference electroencephalogram associated with ID "2" is an electroencephalogram representing the operation of turning off the cooling function of an air conditioner.

[0117] For example, when an electroencephalogram measured from a user includes a difference within an allowable range from a reference electroencephalogram associated with ID "1," processor 24 identifies the operation content "turn on cooling of the air conditioner" associated with the reference electroencephalogram. Then, processor 24 transmits control information including information indicating the operation content "turn on cooling of the air conditioner" to the air conditioner. The air conditioner operates according to the control information. This causes the cooling of the air conditioner to be turned on.

[0118] The processor 24 may also calculate a similarity between the electroencephalogram measured from the user and the reference electroencephalogram, and determine whether the similarity is equal to or greater than a threshold. The threshold is a value corresponding to an allowable range. If the similarity between the electroencephalogram measured from the user and the reference electroencephalogram is equal to or greater than the threshold, the processor 24 determines that the electroencephalogram measured from the user and the reference electroencephalogram are similar. In other words, the processor 24 determines that the difference between the electroencephalogram measured from the user and the reference electroencephalogram is within an allowable range. If an electroencephalogram is measured from the user such that the similarity between the electroencephalogram measured from the user and the reference electroencephalogram linked to ID "1" is equal to or greater than the threshold, the processor 24 identifies the operation content of "turn on the cooling function of the air conditioner."

[0119] For each user, reference biometric information and operation information may be associated with each other and registered in the management table. For example, biometric information measured from a user may be registered in the management table as reference biometric information of the user.

[0120] Fig. 5 shows an example of a management table in which specific reference biometric information of each user is registered. In the management table shown in Fig. 5, an ID, a reference electroencephalogram which is an example of reference biometric information, operation information, and user information are linked. The user information is information for identifying a user (e.g., a user name, a user ID, etc.).

[0121] The reference electroencephalogram linked to the user information is an electroencephalogram measured from the user indicated by the user information when the user performs the operation linked to the reference electroencephalogram, or an electroencephalogram measured from the user when the user requests the operation. Each reference electroencephalogram measured from each user is measured from each user in advance and registered in the management table.

[0122] For example, when user A manually turns on the cooling function of the “air conditioner,” user A's brain waves are measured by the bioinformation measuring device 12, and the measured brain waves are registered in the management table as reference brain waves representing the operation content of user A's “turning on the cooling function of the air conditioner.”

[0123] In this case, the measured reference electroencephalogram of user A, operation information indicating the operation content of "turning on the cooling of the air conditioner", and user information for identifying user A are associated and registered in the management table. The registration may be performed using the information processing device 10 or another device. In the example shown in Fig. 5, this information is registered as information of ID "1". The same applies to other operations and other users.

[0124] The registration operation may be performed multiple times, and the average of the multiple brain waves measured thereby may be registered as the reference brain wave. For example, the operation of having user A manually turn on the cooling function of the "air conditioner" and measuring the brain waves generated by user A at that time using the biological information measuring device 12 may be performed multiple times, and the average of the multiple brain waves measured thereby may be registered in the management table as the reference brain wave of user A.

[0125] For example, when a user A is logged in to the information processing device 10 and an electroencephalogram is measured from the user A, the processor 24 turns on the cooling of the "air conditioner" by transmitting control information including operation information linked to the ID "1" to the "air conditioner". More specifically, when the user A is logged in to the information processing device 10 and an electroencephalogram is measured by the biological information measuring device 12, the processor 24 searches for a reference electroencephalogram registered in the management table in association with user information for identifying the logged-in user A. In the example shown in FIG. 5, the reference electroencephalogram linked to the ID "1" and the reference electroencephalogram linked to the ID "3" are registered in the management table as the reference electroencephalogram of the user A, so these reference electroencephalograms are searched. When a difference between the measured electroencephalogram and the reference electroencephalogram linked to the ID "1" is within an acceptable range, the processor 24 turns on the cooling of the "air conditioner" by transmitting control information including operation information linked to the ID "1" to the "air conditioner". If the difference between the measured brain waves and the reference brain waves linked to ID "3" is within an acceptable range, the processor 24 turns off the cooling of the "air conditioner" by sending control information including operation information linked to ID "3" to the "air conditioner."

[0126] As another example, in a state where the user who operates the device 16 is set to be "user A" in the information processing device 10, when an electroencephalogram is measured from the user A, the difference between which is within an allowable range from a reference electroencephalogram linked to ID "1", the processor 24 may turn on the cooling of the "air conditioner" by transmitting control information including operation information linked to ID "1" to the "air conditioner". In more detail, when an electroencephalogram is measured by the biological information measuring device 12 in a state where the user who operates the device 16 is set to be "user A" in the information processing device 10, the processor 24 searches for a reference electroencephalogram registered in the management table in association with user information for identifying the user A who is the user who operates the device 16. When the difference between the measured electroencephalogram and the reference electroencephalogram linked to ID "1" is within an allowable range, the processor 24 turns on the cooling of the "air conditioner" by transmitting control information including operation information linked to ID "1" to the "air conditioner". The user who operates the device 16 is set in the information processing device 10 by the user, for example.

[0127] For users other than user A, each piece of information is registered in the management table in the same way as for user A. For example, each piece of information linked to ID "2" is information related to the operation when user B turned on the cooling function of the "air conditioner." Each piece of information linked to ID "3" is information related to the operation when user A turned off the cooling function of the "air conditioner."

[0128] In the management table shown in Figures 4 and 5, operation information indicating the operation of turning the power of the device 16 on or off is registered, but operation information indicating the function level of the device 16 may also be registered in the management table.

[0129] In the above embodiment, the processing by the processor 24 may be executed by a device other than the information processing device 10 (e.g., an external device such as a server), and information indicating the results of the processing may be displayed on the UI 20 or output as audio information.

[0130] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operation of the processor in each of the above embodiments may not only be performed by one processor, but may also be performed by multiple processors located at physically separate locations working together. Furthermore, the order of each operation of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0131] 10 information processing device, 12A, 12B vital information measuring device, 14 environmental information measuring device, 16 equipment, 24 processor.

Claims

1. The computer in a predetermined first time period including before and after a time point when the biometric information indicating the operation content of a specific device and associated with a specific process is measured from a user, in a predetermined second time period before the time point when the biometric information associated with the specific process is measured from the user, or in a predetermined third time period after the time point when the biometric information associated with the specific process is measured from the user, The present invention is characterized in that the present invention is configured to execute a process including a decision to execute a process of operating the specific device according to the biometric information indicating the operation content of the specific device when specific biometric information different from the biometric information associated with the specific process, i.e., information that is not expected to be measured under the circumstances in which the biometric information associated with the specific process is measured, is not measured from the user, or when the biometric information indicating the operation content of the specific device is measured for less than a predetermined time even if it is measured from the user, and a decision to execute a process of not operating the specific device when the above-mentioned cases are not met and when the biometric information is measured for more than the predetermined time. Information processing methods.

2. A computer comprising: a predetermined first time period including a time point when the biometric information indicating the operation content of a specific device and associated with a specific process is measured from a user, a predetermined second time period before the time point when the biometric information associated with the specific process is measured from the user, and a predetermined third time period after the time point when the biometric information associated with the specific process is measured from the user, The present invention is characterized in that the device is configured to execute a process including a decision to execute a process of operating the specific device when specific biometric information different from the biometric information associated with the specific process, i.e., information that is not expected to be measured under the circumstances in which the biometric information associated with the specific process is measured, is not measured from the user, and a decision to execute a process of not operating the specific device according to biometric information indicating the operation content of the specific device when the specific biometric information is measured from the user.

2. The information processing method according to claim 1.

Images