Information processing device and program

Abstract

To manipulate a manipulation target according to biological information of a plurality of users.SOLUTION: A processor manipulates a manipulation target according to biological information of each of a plurality of users. When the number of users changes, the processor manipulates the manipulation target according to biological information of users after the change.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] It is conceivable to operate an object to be operated using biological information such as electroencephalograms.

[0003] Patent Document 1 describes a BMI device that uses steady-state visually evoked potentials (SSVEP).

[0004] Patent Document 2 describes a BMI exercise support device that detects electroencephalogram biosignals from a user's electroencephalogram, detects surface myoelectric potential biosignals from the user's surface myoelectric potential, and calculates a control signal based on both biosignals.

[0005] Patent document 3 describes an imaging control device that pre-stores electroencephalogram conditions, which are the conditions for electroencephalograms that occur when a specific physical movement is imagined, acquires electroencephalogram information, and, if the electroencephalograms contained in the acquired electroencephalogram information satisfy the stored electroencephalogram conditions, outputs to the imaging device imaging conditions, which are the conditions under which the imaging device will image a subject.

[0006] Patent Document 4 describes a method for matching electroencephalogram signals with a person's intentions. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2014-071825 A [Patent Document 2] JP 2015-211705 A [Patent Document 3] JP 2015-102650 A [Patent Document 4] JP 2002-166050 A Summary of the Invention [Problem to be solved by the invention]

[0008] An object of the present invention is to operate an operation target based on biometric information of a plurality of users. [Means for solving the problem]

[0009] The invention according to claim 1 has a processor, and the processor includes potential signals obtained from the heads of multiple users within a specific range. , a plurality of users corresponding to each of the plurality of users; Biometric information and combining the plurality of pieces of biometric information by weighting the pieces of biometric information. and an information processing device that operates an operation target according to the 。 request request 2 The invention according to claim 1, wherein the weighting process is a weighted average, and a plurality of weighting coefficients are determined based on attributes of the plurality of users, or a plurality of weighting coefficients are determined based on distances between the operation target and each of the plurality of users. 1 The present invention relates to an information processing device.

[0010] Claim 3 The invention relates to a computer that stores potential signals obtained from the heads of multiple users within a specific range. , a plurality of users corresponding to each of the plurality of users; Biometric information and combining the plurality of pieces of biometric information by weighting the pieces of biometric information. This is a program for operating the target object in accordance with the received signal. Claim 4 The invention relates to a computer that stores potential signals obtained from the heads of multiple users within a specific range. , a plurality of users corresponding to each of the plurality of users; Biometric information and combining the plurality of pieces of biometric information by weighting the pieces of biometric information. The present invention is an information processing method for operating an operation target in accordance with a Effect of the Invention

[0011] According to the invention as set forth in claims 1 to 3, it is possible to operate an operation target based on biometric information of a plurality of users. [Brief description of the drawings]

[0012] [Figure 1]1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of a terminal device. [Figure 4] FIG. 13 illustrates a management table. [Diagram 5] FIG. 13 illustrates a management table. [Figure 6] FIG. 13 illustrates a management table. [Figure 7] FIG. 13 illustrates a management table. [Figure 8] FIG. 1 is a diagram showing a schematic diagram of people and devices present in a certain location. [Figure 9] FIG. 1 is a diagram showing a schematic diagram of people and devices present in a certain location. [Figure 10] FIG. [Figure 11] FIG. [Figure 12] FIG. [Figure 13] FIG. [Figure 14] FIG. [Figure 15] FIG. 1 is a diagram showing a schematic diagram of people and devices present in a certain location. [Figure 16] FIG. [Figure 17] FIG. [Figure 18] FIG. 1 is a schematic diagram showing a tomographic image of the brain. [Figure 19] FIG. [Figure 20] FIG. [Figure 21] FIG. 13 is a diagram showing a specific example of a graph. [Figure 22] FIG. 13 is a diagram showing a specific example of a graph. [Diagram 23] FIG. [Figure 24] FIG. [Diagram 25]FIG. [Figure 26] FIG. [Figure 27] 13 is a flowchart showing a process flow according to a twentieth embodiment. [Figure 28] 13 is a flowchart showing a process flow according to a twentieth embodiment. [Figure 29] FIG. 2 is a perspective view showing the entire hearable device. [Diagram 30] FIG. 2 is a diagram of the biological information measuring device as viewed from above. [Diagram 31] FIG. 2 is a view of the left earphone as seen from the left side. [Diagram 32] FIG. 2 is a perspective view showing the right earphone. [Diagram 33] FIG. 2 is a perspective view showing the left earphone. [Diagram 34] FIG. 13 is a top view of the right earphone. [Diagram 35] FIG. 2 is an exploded perspective view showing the right earphone. [Diagram 36] FIG. 2 is a cross-sectional view of the right earphone. [Figure 37] FIG. 2 is a perspective view showing the inside of the left earphone. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] An information processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 shows an example of the configuration of the information processing system according to this embodiment.

[0014] The information processing system according to the present embodiment includes an information processing device 10, one or more biological information measuring devices 12, one or more environmental information measuring devices 14, one or more devices 16, and one or more terminal devices 18. The number of each device shown in FIG. 1 is merely an example, and the number of each device is not limited to the number of each device shown in FIG. 1. In addition, the information processing system may include other devices (e.g., external devices such as a server) other than the devices shown in FIG. 1. If the device 16 is not used, the device 16 may not be included in the information processing system. If the terminal device 18 is not used, the terminal device 18 may not be included in the information processing system.

[0015] Each device included in the information processing system is configured to communicate with other devices. The communication may be wired communication using a cable, or wireless communication. That is, each device may be physically connected to other devices by a cable and transmit and receive information to each other, or may transmit and receive information to each other by wireless communication. For example, short-range wireless communication or Wi-Fi (registered trademark) is used as wireless communication. Wireless communication of other standards may also be used. For example, short-range wireless communication is Bluetooth (registered trademark), RFID (Radio Frequency Identifier), NFC, etc. Each device may communicate with other devices via a communication path such as a LAN (Local Area Network) or the Internet.

[0016] The information processing device 10 is, for example, a personal computer (hereinafter referred to as "PC"), a tablet PC, a smartphone, a mobile phone, or other devices (for example, a server, a robot, or other equipment). The information processing device 10 may be a terminal device that can be carried by a person (for example, a tablet PC, a smartphone, a mobile phone, etc.), a device that is installed on a table or the like and used, a mobile device (for example, a self-propelled device), a device that performs work, or a device that can interact with a person. For example, the information processing device 10 may be a smart speaker having a communication function, a microphone, and a speaker, or a robot that interacts with a person and works. The information processing device 10 may be a device equipped with artificial intelligence (AI).

[0017] The biological information measuring device 12 is a device configured to measure biological information of a living organism. The living organism may be a human, a non-human animal, or a plant.

[0018] For example, the biological information measuring device 12 has various sensors, electrodes, etc., and measures biological information of a living organism. When multiple biological information measuring devices 12 are used, each biological information measuring device 12 may measure different types of biological information. Some or all of the multiple biological information measuring devices 12 may measure the same type of biological information. Furthermore, the biological information measuring device 12 may measure one type of biological information, or may measure multiple types of biological information.

[0019] The biological information may include various information emitted from a living organism. The biological information may be, for example, information indicating brain activity (for example, brain waves, brain blood flow, brain magnetic field signals, etc.), information indicating pulse rate, electromyographic information such as electromyographic waveform, information on saliva (for example, information indicating saliva volume), information indicating pulse wave, information indicating blood pressure, information indicating blood flow, information indicating pulse, information indicating heart rate, information indicating electrocardiographic waveform, information indicating eye movement, information indicating body temperature, information indicating sweat amount, information indicating gaze, voice information, information indicating human movement, or information obtained from bodily fluids (for example, blood, etc.). Information specified by a biomarker may also be used as the biological information. The biological information may also be information resulting from an electric potential detected from a living organism. For example, the biological information may be an electroencephalogram which is a measurement result of a microcurrent generated with brain activity, an electrocardiogram which is a measurement result of a microcurrent generated with heartbeat, an electromyogram which is a measurement result of a microcurrent generated with muscle activity, or a skin potential which is a measurement result of a microcurrent generated on the skin. These are merely examples of biometric information, and other biometric information may be used.

[0020] By analyzing the biometric information, emotional information, mental information, or psychological information may be obtained as information on a person's condition. For example, by analyzing a person's biometric information, information indicating the person's emotion, information indicating the person's mental state, or information indicating the person's psychological state may be obtained. Of course, by analyzing the biometric information of non-human animals or plants, information indicating the state of the non-human animals or plants may be obtained. For example, information about a person's state may include the state of feeling the three basic needs of appetite, sleepiness, and sexual desire, as well as consciousness centered on feelings, but more specifically, information such as relief, anxiety, gratitude, surprise, excitement, sexual excitement, curiosity, sexual curiosity, calm, impatience, wonder, happiness, good fortune, relaxation, tension, honor, responsibility, respect, closeness, admiration, desire, fear, courage, pleasure, pleasure (regarding good deeds and virtues), regret, satisfaction, dissatisfaction, regret, disgust, shame, contempt, jealousy, guilt, murderous intent, Schadenfreude, saudade, expectation, feelings of superiority, feelings of inferiority, resentment, bitterness, suffering, sadness, pain, emotion, anger, trouble (agony, anguish, distress), resignation, despair, hope, hatred (love-hate), loveliness, and emptiness. Other examples include itching, pain, the need to urinate or defecate, hunger, and satiety. In addition, communication information transmitted from person to person includes information about a person's thoughts and intentions. For example, approval, agreement, opposition, rejection, indecision, confusion, etc. In particular, human brainwave information is the primary type of bioinformation, as it is highly likely that the above information can be detected in greater detail than other bioinformation.

[0021] The biological information measuring device 12 may be a device (for example, a wearable device) that is attached to a living subject to measure biological information, or may be a device that measures biological information without being attached to a living subject.

[0022] Examples of the wearable device include a hearable device attached to the ear of an animal, a device attached to the head of an animal, a device attached to the wrist, arm, or finger (hereinafter referred to as "wrist, etc.") of an animal (for example, a watch-type device such as a smart watch), a device attached to the neck of an animal, a device attached to the torso of an animal (for example, the chest, abdomen, etc.), a device attached to the lower limbs (for example, the thigh, lower leg, knee, foot, ankle, etc. of a human), a glasses-type device, or a contact lens-type device. The biological information measuring device 12 may be a wearable device attached to a part other than these. Furthermore, the biological information measuring device 12 may be attached to multiple parts.

[0023] The hearable device may be, for example, an earphone, a hearing aid, a piercing-type device, a clip-type device, or a device including a band or cable wrapped around the ear. The device worn on the head may be, for example, a headset including a band or cable wrapped around the head. The device worn on the wrist may be, for example, a device including a band or cable wrapped around the wrist. The device worn on other parts may also include a band or cable. In particular, when measuring brainwave information, a hearable device that can be measured with a natural appearance is preferable, and is excellent in that it has a highly versatile form like earphones that are commonly used as a device for measuring brainwaves. A natural appearance that does not make it obvious to a person that it is measuring brainwaves at first glance is an important element, and medical electroencephalographs that have electrodes on the head are obviously measuring brainwaves, which is a major obstacle when worn on a daily basis for measurement, and many people hesitate to wear them. In addition, hearable devices can be used to input voice into the human ear and output voice information from the mouth, so they are superior in that they can detect brainwave information, confirm the interpretation of that information by voice, and correct the interpretation if there is an error in the interpretation. Furthermore, BMI / BCI, which mainly uses brainwaves, is often used in situations where devices are operated or people communicate with each other, but the challenge is how to operate the device with high accuracy and how to correctly judge the content of the communication to be conveyed. Since BMI / BCI using hearable devices are worn in the ear, it is possible to improve accuracy by utilizing the above-mentioned voice input method and voice information from the mouth, or by utilizing bioinformation that a person can intentionally express other than brainwaves (for example, it is possible to check whether the information, which appears as myoelectric information when intentionally moving the jaw three times, or the information on the eye that appears when intentionally moving the eyeballs strongly, is correctly judged, and use it to decide whether to perform the operation or to transmit the communication).

[0024] The biological information measuring device 12 may be a contact type device that is a device that comes into contact with a living organism and measures the biological information of the living organism, or the biological information measuring device 12 may be a non-contact type device that measures the biological information of the living organism without coming into contact with the living organism. The biological information measuring device 12 may be a device that combines both a contact type device and a non-contact type device. In other words, the biological information measuring device 12 may come into contact with a living organism and measure the biological information of the living organism, and further, may measure the biological information of the living organism without coming into contact with the living organism. The biological information measured by coming into contact with a living organism and the biological information measured without coming into contact with the living organism may be the same type of biological information or different types of biological information.

[0025] The biological information measuring device 12 includes electrodes for measuring biological information and sensors other than electrodes. The electrodes may be in contact with a living body to detect an electric potential, which is an example of biological information, or may be in contact with the living body to detect an electric potential. The sensors other than the electrodes may be in contact with a living body to measure biological information, or may be in contact with the living body to measure biological information. For example, the electrodes may be in contact with an animal to detect an electric potential indicating the brain waves of the animal. Also, the sensors may be in contact with the animal to measure biological information indicating the body temperature of the animal. Of course, these are merely examples, and other biological information may be measured.

[0026] For example, the biological information measuring device 12 includes one or more electrodes. For example, a plurality of electrodes are provided on an organism, and an electric potential is detected by the plurality of electrodes. The detected electric potential includes a bioelectric potential, which is an example of the biological information of the organism, and the detected electric potential is processed to extract the bioelectric potential from the detected electric potential. For example, the detected electric potential may include noise other than that derived from the biological potential, and a noise cancellation process or other process is performed to obtain a bioelectric potential from which the noise has been removed. The noise may include, for example, an electric potential generated due to the movement of the organism, an electric potential transmitted from the outside of the organism, an electric potential generated due to the global environment, an electric potential representing biological information other than that of the measurement target, and the like. An electric potential generated by a device such as a PC or a smartphone may be included in the noise. The electrodes used for detecting the electric potential may be switched depending on the measurement conditions such as the detection sensitivity of the electric potential and the noise. In addition, when electric potentials derived from various biological potentials are measured in a compounded manner, the electric potentials derived from various biological potentials may be estimated and separated by performing a process of dividing the electric potentials measured for each biological potential by frequency, or by performing a process of dividing the electric potentials into a compounded portion and a weakly compounded portion during the measurement time. For example, in the above-mentioned hearable device, information indicating brain activity, information indicating pulse rate, electromyographic information derived from muscle movements, information resulting from blood flow such as pulse waves and heart rate may be measured in a composite manner, but these pieces of information often have different frequencies or absolute values ​​of output, and can be estimated by dividing them into frequency analysis of the measurement signal or differences in output value levels. For example, in the case of brain waves, they are divided into α, β, θ, δ, and γ wave bands depending on their frequencies. As an alternative method, each of these may be measured by a hearable device under the same state conditions, and a single measurement signal may be measured multiple times individually using a commercially available electroencephalograph, electromyographic measuring device, or blood flow measuring device, and a means may be introduced to analyze the composite measurement signal information of the hearable device using a frequency analysis method such as Fourier transform or wavelet transform to determine which biological information the information is derived from, and then to estimate the information by applying statistical processing.

[0027] The vital information measuring device 12 transmits the measured vital information to the information processing device 10. The vital information measuring device 12 may transmit the vital information to the information processing device 10 every time it measures the vital information, may store the vital information and transmit the vital information to the information processing device 10 at predetermined time intervals, or may transmit the vital information to the information processing device 10 at a timing specified by a person. In addition, the vital information measuring device 12 may receive vital information measured by another vital information measuring device 12 from the other vital information measuring device 12, and transmit the vital information measured by itself and the vital information measured by the other vital information measuring device 12 to the information processing device 10.

[0028] The biological information measuring device 12 may analyze biological information measured by its own device or another biological information measuring device, and transmit information indicating the result of the analysis to the information processing device 10. For example, the biological information measuring device 12 may include a processor, and the processor may analyze the biological information. Of course, the analysis may be performed by the information processing device 10 or another device.

[0029] The biological information measuring device 12 may include a battery and be driven by power supplied from the battery, or may be driven by power supplied from another device (such as the information processing device 10).

[0030] The information processing device 10 receives bioinformation from the bioinformation measuring device 12, and performs the following operations: analyzing the bioinformation, storing the bioinformation, outputting the bioinformation, storing information indicating the results of the analysis of the bioinformation, and outputting information indicating the results of the analysis of the bioinformation. Of course, the analysis of the bioinformation may be performed by the bioinformation measuring device 12 or another device. Outputting the bioinformation means, for example, displaying the bioinformation or outputting the bioinformation as audio information. Outputting information indicating the results of the analysis of the bioinformation means, for example, displaying information indicating the results of the analysis or outputting the results of the analysis as audio information. The information processing device 10 may transmit the bioinformation or information indicating the results of the analysis to another device.

[0031] The information processing device 10 may include one or more biological information measuring devices 12. That is, one or more biological information measuring devices 12 may be incorporated into the information processing device 10 to configure one device. The entire information processing device 10 including one or more biological information measuring devices 12 may be worn by a living being to measure biological information. That is, the information processing device 10 may be a wearable device. For example, the information processing device 10 may be a device worn on a person's head (e.g., forehead, etc.), a hearable device worn on a person's ear (e.g., earphones, headphones, etc.), a device worn on a person's arm, hand, wrist, finger, etc. (e.g., a watch-type device such as a smart watch), a device hung around a person's neck, a device worn on a person's torso (e.g., abdomen, chest, etc.), or a device worn on a person's lower limbs (e.g., thighs, lower legs, knees, feet, ankles, etc.). The information processing device 10 may be a health device worn on a person's arm, hand, torso, or lower limbs. The information processing device 10 may be attached to a part other than these.

[0032] The information processing device 10 and the biological information measuring device 12 may be separate devices. For example, the information processing device 10 may be a device such as a robot, a smart speaker, or a server, and the biological information measuring device 12 may be a wearable device attached to a living being.

[0033] The environmental information measuring device 14 is configured to measure environmental information related to living things and the environment surrounding the environmental information measuring device 14. For example, the environmental information measuring device 14 is a camera which is an imaging device, a microphone which collects sound, a temperature sensor which measures temperature, a humidity sensor which measures humidity, an odor sensor which measures odor, an illuminance sensor which measures brightness, an infrared sensor, a pressure sensor such as an air pressure sensor, a position sensor which detects the movement of an object, a human presence sensor, etc. One or more of these sensors may be included in the information processing system as the environmental information measuring device 14.

[0034] For example, the surroundings of the information processing device 10 or other places are photographed by a camera, and image data representing the surroundings or other places is generated. The image data may be moving image data or still image data. The image data photographed by the camera corresponds to an example of environmental information related to the environment included in the shooting range of the camera. Furthermore, image data representing an organism, which is generated by photographing the organism with a camera, may be used as biometric information of the organism. For example, the movement of the organism (e.g., a person) detected from the image data and the body shape of the organism correspond to an example of biometric information of the organism. In this sense, the camera corresponds to an example of a biometric information measuring device 12 that measures the biometric information of the organism.

[0035] Furthermore, sounds around the microphone (for example, human voice and other sounds) are input to the microphone, and sound data is generated by the microphone. The sound data representing the sounds input to the microphone corresponds to an example of environmental information relating to the environment around the microphone. Furthermore, the sound data representing the human voice input to the microphone may be used as bioinformation of the person. In this sense, the microphone corresponds to an example of a bioinformation measuring device 12 that measures the bioinformation of the person. Furthermore, the sound data input to the microphone representing the cries of animals other than humans may be used as bioinformation of the animal.

[0036] Data measured by a temperature sensor, a humidity sensor, an odor sensor, an illuminance sensor, an infrared sensor, a pressure sensor, a position sensor, a human presence sensor, and the like correspond to examples of environmental information. Data measured from a living organism by these sensors may be used as bioinformation of the living organism. In this sense, these sensors correspond to examples of a bioinformation measuring device 12 that measures the bioinformation of the living organism.

[0037] The environmental information may include information indicating the size of the room in which the living thing is located, information indicating the size of the room in which the device 16 is installed, information indicating the number of windows installed in the room, etc. Furthermore, the environmental information is not limited to information measured by the environmental information measuring device 14, and may be predetermined information or information measured in advance.

[0038] One or more environment information measuring devices 14 may be included in the information processing device 10.

[0039] The device 16 is an example of an operation target. The device 16 is, for example, a PC, a tablet PC, a smartphone, a mobile phone, a robot (for example, a humanoid robot, a non-human animal-type robot, a cleaning robot, and other robots), a display device such as a projector or a liquid crystal display, a recording device, a playback device, an imaging device such as a camera, a refrigerator, a rice cooker, a microwave oven, a coffee maker, a vacuum cleaner, a washing machine, an air conditioner, a lighting device, a clock, a surveillance camera, an automobile, a two-wheeled vehicle, an aircraft (for example, an unmanned aerial vehicle (for example, a drone)), a game machine, a gas range, a warm water washing toilet seat, a ventilation fan, a doorbell, a front door monitor, an elevator, a door, a window, or various sensing devices (for example, a temperature sensor, a humidity sensor, a voltage sensor, a current sensor, etc.). The category of the concept of the device 16 may include devices in general. For example, information devices, video devices, audio devices, and other devices may also be included in the category of the device 16 according to this embodiment.

[0040] The device 16 includes a communication device which is a communication interface, a memory which stores information, and a processor which controls the operation of the device 16. The device 16 may have a user interface which includes a display, an operation device, and the like. The device 16 may transmit device identification information for identifying the device 16 which is the device itself to the information processing device 10. The device identification information is, for example, the ID, name, model number, or address (e.g., MAC address, IP address, etc.) of the device 16.

[0041] In this embodiment, the operation target is operated based on the biometric information of one or more living beings (for example, one or more people). For example, the information processing device 10 acquires the biometric information of the living being from the biometric information measuring device 12, and operates the operation target based on the acquired biometric information. For example, the information processing device 10 acquires the biometric information of the living being from the biometric information measuring device 12, identifies the operation content of the operation target based on the acquired biometric information, and operates the operation target according to the identified operation content. The process of identifying the operation content may be performed by the biometric information measuring device 12 or another device (for example, a server, a device 16, a terminal device 18, etc.). A device other than the information processing device 10 (for example, a server, a terminal device 18, etc.) may operate the operation target. For example, the information processing device 10 may identify the operation content based on the biometric information, and a device other than the information processing device 10 may operate the operation target according to the operation content identified by the information processing device 10.

[0042] The object of operation may be hardware, software, or both hardware and software.

[0043] The hardware to be operated is not particularly limited, and may be, for example, the device 16 or a device other than the device 16 (for example, the information processing device 10, the vital information measuring device 12, the environmental information measuring device 14, or a server, etc.). A plurality of hardware may be the target of operation.

[0044] The software to be operated is not particularly limited, and may be, for example, an application program or an OS (operating system). The software to be operated may be software installed in the information processing device 10, may be software installed in the device 16, may be software installed in the terminal device 18, may be software installed in other devices (for example, the biological information measuring device 12, the environmental information measuring device 14, a server or other devices), or may be software provided via a communication path such as the Internet. A plurality of software may be the object of operation. When a plurality of software is the object of operation, each software may be installed in the same hardware or in different hardware.

[0045] The configuration of the information processing device 10 will be described in detail below with reference to Fig. 2. Fig. 2 shows an example of the hardware configuration of the information processing device 10.

[0046] The information processing device 10 includes, for example, a communication device 20, a UI 22, a memory 24, and a processor 26. The information processing device 10 may include other components.

[0047] The communication device 20 is a communication interface having a communication chip, a communication circuit, etc., and has a function of transmitting information to other devices and a function of receiving information transmitted from other devices. The communication device 20 may have a wireless communication function or a wired communication function. The communication device 20 may communicate with other devices by using, for example, short-range wireless communication, or may communicate with other devices via a communication path such as a LAN or the Internet.

[0048] The UI22 is a user interface and includes at least one of a display and an operation device. The display is a liquid crystal display, an EL display, or the like. The operation device is a keyboard, an input key, an operation panel, or the like. The UI20 may be a UI such as a touch panel that combines a display and an operation device. The UI20 may include a microphone and a speaker. Note that the information processing device 10 does not necessarily include the UI20.

[0049] The memory 24 is a device constituting one or more storage areas for storing various information. The memory 24 is, for example, a hard disk drive, various types of memory (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more memories 24 are included in the information processing device 10.

[0050] Management information is stored in the memory 24. The management information is information for identifying the operation content of the operation target based on the biometric information.

[0051] The processor 26 is configured to control the operation of each unit of the information processing device 10. The processor 26 may include a memory.

[0052] For example, predetermined reference biometric information and operation information indicating the operation content of the operation target are linked in advance and registered in the management information. The reference biometric information may be biometric information presumed to be generated from an animal (e.g., a human or other animal) performing the operation linked to the reference biometric information, or biometric information presumed to be generated from an animal requesting or wanting the execution of the operation, or biometric information presumed to be generated from a plant. The reference biometric information can also be said to be biometric information indicating the operation content of the operation target. The operation information indicating the operation content may include information indicating the operation target to be operated according to the operation content.

[0053] The reference biometric information and the operation information may be linked for each living thing and registered in the management information. For example, the reference biometric information and the operation information may be linked for each type of living thing and registered in the management information. For example, management information for humans, management information for animals other than humans, and management information for plants may be created, and each management information may be stored in the memory 24. In the management information for humans, the reference biometric information and the operation information that are assumed to be measured from humans are linked and registered. The reference biometric information and the operation information may be linked for each individual person (for example, each individual user) and registered in the management information. In the management information for animals other than humans, the reference biometric information and the operation information that are assumed to be measured from the animals are linked and registered. Management information may be created for each type of animals other than humans. In addition, in the management information for plants, the reference biometric information and the operation information that are assumed to be measured from the plants are linked and registered. Management information may be created for each type of plants.

[0054] The operation information may be information indicating the power operation content of device 16, information indicating the function level operation content of device 16, or information indicating both the power operation content and the function level operation content.

[0055] The power operation content of the device 16 is an operation to turn on or off the power of the device 16. The operation information regarding the power operation content is information indicating an operation to turn on or off the power of the device 16. It can be said that the biological information linked to the operation information regarding the power operation content is biological information corresponding to the on or off of the power of the device 16. For each type of living organism or each individual living organism, the operation information regarding the power operation content of the device 16 and the reference biological information may be linked and registered in the management information.

[0056] The operation content of the functional level of the device 16 is an operation to set the functional level of the device 16. The operation information regarding the operation content of the functional level is information indicating an operation to set the functional level of the device 16. It can be said that the biometric information linked to the operation information regarding the operation content of the functional level is biometric information corresponding to the functional level of the device 16. For each type of living organism or each individual living organism, the operation information regarding the operation content of the functional level of the device 16 and the reference biometric information may be linked and registered in the management information.

[0057] The function level is, for example, a level related to the performance or output of the device 16. To explain by giving specific examples, examples of the device level include the set temperature of an air conditioner, the air volume of the air conditioner, the air direction of the air conditioner, the presence or absence of a dehumidification function of the air conditioner, the brightness of a display device, the brightness of a lighting device, the volume of a speaker, the moving speed of a self-propelled device (e.g., a robot or a self-propelled vacuum cleaner, etc.), the setting values ​​of devices such as an imaging device, a recording device, and a playback device, the setting values ​​of devices such as a refrigerator, a rice cooker, and a microwave oven, and the setting values ​​of various sensing devices. These are merely examples, and other setting values, etc. may be the function level.

[0058] The management information may not be stored in the information processing device 10, or may be stored in the information processing device 10 and also in a device other than the information processing device 10.

[0059] The processor 26 is configured to obtain biometric information of a living organism, and to operate an operation target based on the biometric information.

[0060] For example, when the biometric information is measured from a living organism by the biometric information measuring device 12, the biometric information is transmitted from the biometric information measuring device 12 to the information processing device 10. The processor 26 receives the biometric information and identifies the operation content of the operation target based on the biometric information. The processor 26 operates the operation target according to the identified operation content. The processor 26 may identify the operation target and the operation content of the operation target based on the biometric information of the living organism. As another example, the operation target may be determined in advance. For example, the operation target may be specified by a person. In this case, the processor 26 identifies the operation content of the determined operation target based on the biometric information of the living organism.

[0061] When the operation target is the device 16, the processor 26 transmits control information including operation information indicating the specified operation content to the device 16, thereby operating the device 16. The device 16 that is the operation target operates according to the control information. The same applies when the operation target is hardware other than the device 16.

[0062] When the operation target is software, the processor 26 operates the software according to the identified operation content. For example, the processor 26 starts the software or performs processing using the software. When the software is installed in the information processing device 10, the processor 26 operates the software installed in the information processing device 10. When the software is installed in a device other than the information processing device 10, the processor 26 operates the software by transmitting control information including operation information indicating the operation content to the device in which the software is installed.

[0063] For example, the processor 26 compares the biometric information measured from the living organism with each reference biometric information registered in the management information, and searches for reference biometric information whose difference from the biometric information falls within an allowable range. The processor 26 identifies the operation content for the operation target linked to the searched reference biometric information. This identifies the operation target and the operation content for the operation target. The allowable range is determined in advance. The allowable range may be changed by the user. The allowable range may be determined for each type of organism or each individual organism.

[0064] When multiple reference biometric information are searched for, the difference between the reference biometric information measured from the living organism falls within an acceptable range, the processor 26 may identify, from the multiple reference biometric information, the reference biometric information that has the smallest difference from the biometric information in question, and may identify the operation content linked to the identified reference biometric information, or may identify the operation content linked to each of the multiple reference biometric information.

[0065] The reference bioinformation may be information indicating a characteristic component of the bioinformation. In this case, the processor 26 may extract a characteristic component from the bioinformation measured from the living organism, and search for reference bioinformation having a component whose difference from the extracted component is within an allowable range. For example, when an electroencephalogram is used as the bioinformation, the processor 26 may extract a characteristic component from the electroencephalogram of an animal, and infer the operation content represented by the electroencephalogram by analyzing the component.

[0066] The reference biometric information may be used as the threshold. For example, when the reference biometric information shows a waveform, the amplitude of the peak in the waveform is used as the threshold. In this case, the processor 26 searches for reference biometric information having a waveform similar to or similar to the waveform shown by the biometric information measured from the living organism, the measured biometric information being equal to or greater than the threshold, and identifies the operation content linked to the searched reference biometric information. In other words, when the waveform shown by the measured biometric information is similar to or similar to the waveform shown by a certain reference biometric information, and the amplitude of the peak of the waveform shown by the measured biometric information is equal to or greater than the threshold, which is the amplitude of the peak of the waveform shown by the reference biometric information, the processor 26 identifies the operation content linked to the reference biometric information.

[0067] When the operation target is operated using information indicating the activity of the animal's brain, a brain-machine interface (BMI) may be constructed by the bioinformation measuring device 12 that measures the activity of the brain and the information processing device 10. In general, the brain-machine interface is a general term for devices that interface the brain with a computer or the like by detecting brain waves or by stimulating the brain. When the connection destination is a computer, it is also called a brain-computer interface (BCI). In this embodiment, examples of using bioinformation including brain waves to operate devices or to communicate between people (mutual transmission of information about people's thoughts and intentions and status information, etc.) are shown, but these are considered to be part of the brain-machine interface and will be described hereinafter. The brain-machine interface may be invasive or non-invasive. In this case, the processor 26 operates the operation target based on the activity of the brain (for example, brain waves, etc.). In order to operate the operation target, the processor 26 may extract characteristic components from the electroencephalogram and operate the operation target based on the extracted components. In order to extract characteristic components from the electroencephalogram, a fast Fourier transform (FFT), a wavelet transform (WT), a time frequency distribution (TFD), an eigenvector method (EM), an auto regressive method (ARM), or the like may be used. In addition, as a method for linking the electroencephalogram and the operation content of the operation target using a feature vector obtained by extracting the features, or as a method for linking the transmission target and the transmission content in communication between people, for example, an independent component analysis (ICA), a k-means method, a support vector machine (SVM), a convolutional neural network, or the like may be used.

[0068] Furthermore, the processor 26 may receive identification information of the operation target transmitted from the operation target to identify the operation target. For example, the processor 26 transmits a request for acquiring the identification information to the operation target, and acquires the identification information transmitted from the operation target in response to the request. Furthermore, identification information of the hardware and identification information of the software installed in the hardware may be transmitted to the information processing device 10 from hardware that is connected to the information processing device 10 and is capable of communicating with the information processing device 10. The processor 26 receives the identification information transmitted in this manner.

[0069] The processing by the processor 26 may be executed by a device other than the information processing device 10 (for example, the biological information measuring device 12, the environmental information measuring device 14, the equipment 16, the terminal device 18, or other devices, etc.). Also, a part of the processing may be executed by the information processing device 10, and another part of the processing may be executed by a device other than the information processing device 10. For example, analysis of the biological information (for example, processing to identify the operation content) may be executed by a device other than the information processing device 10, and the operation of the operation target may be executed by the information processing device 10.

[0070] The configuration of the terminal device 18 will be described in detail below with reference to Fig. 3. Fig. 3 shows an example of the hardware configuration of the terminal device 18.

[0071] The terminal device 18 includes, for example, a communication device 28, a UI 30, a memory 32, and a processor 34.

[0072] The communication device 28 is a communication interface having a communication chip, a communication circuit, etc., and has a function of transmitting information to other devices and a function of receiving information transmitted from other devices. The communication device 28 may have a wireless communication function or a wired communication function. The communication device 28 may communicate with other devices by using, for example, short-range wireless communication, or may communicate with other devices via a communication path such as a LAN or the Internet.

[0073] The UI 30 is a user interface and includes at least one of a display and an operation device. The display is a liquid crystal display, an EL display, or the like. The operation device is a keyboard, an input key, an operation panel, or the like. The UI 30 may be a UI such as a touch panel that combines a display and an operation device. The UI 30 may include a microphone and a speaker.

[0074] The memory 32 is a device constituting one or more storage areas for storing various types of information. The memory 32 is, for example, a hard disk drive, various types of memory (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more memories 32 are included in the terminal device 18.

[0075] The processor 34 is configured to control the operation of each part of the terminal device 18. The processor 34 may include a memory.

[0076] A specific example of the management information will be described below.

[0077] 4 shows an example of a management table, which is an example of management information. The data of the management table is stored in the memory 24. The data of the management table may not be stored in the memory 24, but may be stored in a device other than the information processing device 10 (e.g., a device such as a server).

[0078] In the management table, an ID, a reference electroencephalogram, and operation information indicating the operation content of an operation target are linked in advance.

[0079] The ID is information for managing the information registered in the management table.

[0080] The reference electroencephalogram is an example of reference biological information. The reference electroencephalogram is an electroencephalogram that is estimated to be measured from a person. Here, the electroencephalogram is used as an example of the reference biological information, but biological information other than the electroencephalogram may be used as the reference biological information.

[0081] The reference brain waves are, for example, determined by statistical processing, and are brain waves that are estimated to be generated by a person performing an operation content linked to the reference brain waves, or brain waves that are estimated to be generated by a person requesting the execution of the operation content.

[0082] The reference electroencephalogram may be an electroencephalogram in a specific frequency band, or may be an electroencephalogram including electroencephalograms in a plurality of frequency bands.

[0083] 4, the operation target is an example of hardware, that is, device 16. Of course, the operation target may be hardware other than device 16, or may be software.

[0084] The operation information includes device identification information for identifying the device 16 to be operated, and information indicating the operation content performed on the device 16. For example, the operation content may be an operation to turn on or off the power of the device 16, an operation to set a function level of the device 16, or both an operation to turn on or off the power and an operation to set a function level.

[0085] For example, the reference electroencephalogram of ID "1" is an electroencephalogram representing the operation of turning on the cooling function of an air conditioner, and the reference electroencephalogram of ID "2" is an electroencephalogram representing the operation of turning off the cooling function of an air conditioner.

[0086] For example, when an electroencephalogram of a person is measured, the difference from the reference electroencephalogram of ID "1" falls within an allowable range, and the processor 26 identifies the operation content "turn on the cooling function of the air conditioner" that is linked to the reference electroencephalogram. The processor 26 then transmits control information including information indicating the operation content "turn on the cooling function of the air conditioner" to the air conditioner. The air conditioner operates according to the control information. This causes the cooling function of the air conditioner to be turned on.

[0087] Processor 26 may also calculate a similarity between the electroencephalogram measured from the person and the reference electroencephalogram, and determine whether the similarity is equal to or greater than a threshold. The threshold is a value corresponding to an acceptable range. If the similarity between the electroencephalogram measured from the person and the reference electroencephalogram is equal to or greater than the threshold, processor 26 determines that the electroencephalogram measured from the person and the reference electroencephalogram are similar. In other words, processor 26 determines that the difference between the electroencephalogram measured from the person and the reference electroencephalogram is within an acceptable range. If an electroencephalogram is measured from the person such that the similarity between the electroencephalogram measured from the person and the reference electroencephalogram is equal to or greater than the threshold, processor 26 identifies the operation content as "turn on the cooling function of the air conditioner."

[0088] 4, a reference electroencephalogram, which is an example of bioinformation, is registered in the management table, but emotional information, mental information, or psychological information obtained by analyzing the bioinformation may be registered in the management table instead of the reference electroencephalogram. For example, information indicating an emotion or mental state, such as hot or cold, may be registered in the management table as reference information instead of the reference electroencephalogram. Processor 26 analyzes the electroencephalogram measured from a person to identify the emotion or mental state of the person, identify reference information indicating the identified emotion or mental state, and identify an operation content linked to the identified reference information in the management table.

[0089] To explain this with a specific example, reference information indicating an emotion, mental state, or psychological state of "it's hot" is linked to operation information indicating an operation content of "turning on the cooling function of the air conditioner" and registered in the management table. Processor 26 identifies the emotion, mental state, or psychological state of a person by analyzing biometric information (e.g., brain waves) measured from the person. When the emotion, etc. is "it's hot," processor 26 identifies the operation content "turning on the cooling function of the air conditioner" that is linked to the reference information of "it's hot."

[0090] In this way, the processor 26 may identify the operation contents based on the biometric information itself, or may identify the operation contents based on various information (e.g., emotional information, mental information, psychological information, etc.) obtained from the biometric information. The same applies to each of the following embodiments.

[0091] For each living thing, the reference biometric information and the operation information may be associated and registered in the management table. For example, biometric information measured from an individual person may be registered in the management table as the reference biometric information of the person. For animals or living things other than humans, biometric information measured from each animal or living thing may be registered in the management table as the reference biometric information of the animal or living thing.

[0092] Fig. 5 shows an example of a management table in which specific reference biometric information of each person is registered. In the management table shown in Fig. 5, an ID, a reference electroencephalogram which is an example of reference biometric information, operation information, and user information are linked. The user information is information for identifying a user who is a person (for example, a user name, a user ID, etc.).

[0093] The reference electroencephalogram linked to the user information is an electroencephalogram measured from the user indicated by the user information when the user performs the operation linked to the reference electroencephalogram, or an electroencephalogram measured from the user when the user requests the operation. Each reference electroencephalogram measured from each user is measured from each user in advance and registered in the management table.

[0094] For example, when user A manually turns on the cooling function of the "air conditioner", the brain waves of user A are measured by the bioinformation measuring device 12, and the measured brain waves are registered in the management table as reference brain waves representing the operation content of "turning on the cooling function of the air conditioner" by user A. In this case, the measured reference brain waves of user A, operation information indicating the operation content of "turning on the cooling function of the air conditioner", and user information for identifying user A are associated and registered in the management table. The registration may be performed using the information processing device 10 or another device. In the example shown in FIG. 5, these pieces of information are registered as information of ID "1". The same applies to other operations and other users.

[0095] The registration operation may be performed multiple times, and the average of the multiple brain waves measured thereby may be registered as the reference brain wave. For example, the operation of having user A manually turn on the cooling function of the "air conditioner" and measuring the brain waves generated by user A at that time using the biological information measuring device 12 may be performed multiple times, and the average of the multiple brain waves measured thereby may be registered in the management table as the reference brain wave of user A.

[0096] For example, when a user A is logged in to the information processing device 10 and an electroencephalogram is measured from the user A and the difference between the electroencephalogram and the reference electroencephalogram of ID "1" is within the allowable range, the processor 26 turns on the cooling of the "air conditioner" by transmitting control information including operation information of ID "1" to the "air conditioner". More specifically, when the user A is logged in to the information processing device 10 and the electroencephalogram is measured by the biological information measuring device 12, the processor 26 searches for the reference electroencephalogram registered in the management table in association with the user information for identifying the logged-in user A. In the example shown in FIG. 5, the reference electroencephalogram of ID "1" and the reference electroencephalogram of ID "3" are registered in the management table as the reference electroencephalogram of user A, so these reference electroencephalograms are searched. When the difference between the measured electroencephalogram and the reference electroencephalogram of ID "1" is within the allowable range, the processor 26 turns on the cooling of the "air conditioner" by transmitting control information including operation information of ID "1" to the "air conditioner". If the difference between the measured brainwaves and the reference brainwaves of ID "3" is within an acceptable range, processor 26 turns off the cooling of the "air conditioner" by sending control information including operation information of ID "3" to the "air conditioner."

[0097] As another example, in a state where the user who operates the device 16 is set to be "user A" in the information processing device 10, when an electroencephalogram is measured from the user A, the difference of which from the reference electroencephalogram of ID "1" falls within an allowable range, the processor 26 may turn on the cooling of the "air conditioner" by transmitting control information including operation information of ID "1" to the "air conditioner". More specifically, when an electroencephalogram is measured by the biological information measuring device 12 in a state where the user who operates the device 16 is set to be "user A" in the information processing device 10, the processor 26 searches for a reference electroencephalogram registered in the management table in association with user information for identifying the user A who is the user who operates the device 16. When the difference between the measured electroencephalogram and the reference electroencephalogram of ID "1" falls within an allowable range, the processor 26 turns on the cooling of the "air conditioner" by transmitting control information including operation information of ID "1" to the "air conditioner". The user who operates the device 16 is set in the information processing device 10 by the user, for example.

[0098] For users other than user A, each piece of information is registered in the management table in the same way as for user A. For example, each piece of information linked to ID "2" is information related to the operation when user B turned on the cooling function of the "air conditioner." Each piece of information linked to ID "3" is information related to the operation when user A turned off the cooling function of the "air conditioner."

[0099] In the management table shown in Figures 4 and 5, operation information indicating the operation of turning the power of the device 16 on or off is registered, but operation information indicating the function level of the device 16 may also be registered in the management table.

[0100] Another example of the management table is shown in Fig. 6. The management table shown in Fig. 6 is a management table in which reference biometric information and operation information are associated and registered for each user. In the example shown in Fig. 6, as an example, an electroencephalogram and a body temperature are used as the reference biometric information. Therefore, in the management table, an ID, a reference electroencephalogram which is an example of reference biometric information, a reference body temperature which is an example of reference biometric information, operation information, and user information are associated with each other.

[0101] For example, when an electroencephalogram is measured from user A whose difference from the reference electroencephalogram of ID "1" falls within an acceptable range, and a body temperature equal to or greater than a threshold is measured from user A, processor 26 identifies the operation content indicated by the operation information of ID "1" as the operation content for device 16. Then, processor 26 transmits control information including the operation information indicating the operation content of ID "1" to an air conditioner, thereby turning on the cooling function of the air conditioner.

[0102] In the example shown in FIG. 6, the reference body temperature, the reference electroencephalogram, and the operation information are linked for each person, but the reference body temperature, the reference electroencephalogram, and the operation information may be linked, instead of for each person.

[0103] Another example of the management table is shown in Fig. 7. The management table shown in Fig. 7 is a management table in which reference electroencephalograms, reference environmental information, and operation information are associated with each other and registered for each user. Although the reference electroencephalograms are used as an example of the reference biological information, other biological information may be used as the reference biological information.

[0104] The reference environment information is information indicating an environment in which a reference electroencephalogram associated with the reference environment information is estimated to be measured. Although room temperature is used as the reference environment information, other environmental information may be used as the reference environment information.

[0105] For example, when an electroencephalogram is measured from user A and the difference from the reference electroencephalogram of ID "1" falls within an allowable range, and the room temperature when the electroencephalogram is measured is equal to or higher than a threshold value, processor 26 identifies the operation content indicated by the operation information of ID "1" as the operation content for device 16. In this manner, the operation content for device 16 is identified based on the electroencephalogram measured from the user and the room temperature when the electroencephalogram is measured.

[0106] In the example shown in FIG. 7, the reference biometric information, the reference environmental information, and the operation information are linked for each person. However, the reference biometric information, the reference environmental information, and the operation information may be linked, not for each person.

[0107] Each embodiment will be described below.

[0108] <Example 1> Hereinafter, a first embodiment will be described. In the first embodiment, biometric information measured from a plurality of living beings in a certain place in the real space is used. As an example, a case in which a device 16 is operated based on biometric information measured from a plurality of people gathered in a certain room will be described.

[0109] 8 shows a schematic diagram of the room 36. In the room 36, there are users A, B, and C, who are people. Furthermore, devices 16 are installed in the room 36. Note that the number of users and devices 16 is merely an example, and a plurality of devices 16 may be installed in the room 36.

[0110] The biometric information of each user is measured by the biometric information measuring device 12. For example, each of users A, B, and C wears a biometric information measuring device 12, which is a wearable device, and the biometric information of each of users A, B, and C is measured by the biometric information measuring device 12 worn by each user. The biometric information measuring device 12 may be worn by each user, or the biometric information measuring device 12 may not be worn by each user, and the biometric information of each user may be measured by the biometric information measuring device 12 that is not worn by the user. One or more types of biometric information may be measured from each user. Here, as an example, it is assumed that brain waves are measured from each of users A, B, and C. Of course, biometric information other than brain waves may be measured.

[0111] For example, bioinformation (e.g., electroencephalogram signals) of users in the room 36 is transmitted from the bioinformation measuring device 12 of each user to the information processing device 10. In the example shown in Fig. 8, bioinformation (e.g., electroencephalogram signals) of each of users A, B, and C is transmitted from the bioinformation measuring device 12 of each user to the information processing device 10.

[0112] The processor 26 of the information processing device 10 operates the device 16 based on the brain waves of each of the users A, B, and C. For example, the processor 26 calculates an average of the brain waves measured from each of the users A, B, and C, and specifies the operation content of the device 16 based on the average brain waves. For example, the processor 26 refers to the management table shown in FIG. 4, specifies a reference brain wave whose difference from the average brain wave falls within an allowable range, and specifies the operation content linked to the specified reference brain wave. Then, the processor 26 operates the device 16 according to the specified operation content. For example, the processor 26 calculates an average of the amplitude of the brain waves, specifies a reference brain wave whose difference from the brain wave having the average amplitude falls within an allowable range, and specifies the operation content.

[0113] As another example, when the operation content of the device 16 is the setting of the function level, the processor 26 may separately identify the function levels of the device 16 based on the brain waves of each of the users A, B, and C. Then, the processor 26 calculates the average of each function level and operates the device 16 according to the average function level. For example, when the device 16 is an air conditioner having a cooling function, if the operation content identified based on the brain waves of the user A is the operation content of "setting the cooling setting temperature of the air conditioner to 26°C", the operation content identified based on the brain waves of the user B is the operation content of "setting the cooling setting temperature of the air conditioner to 27°C", and the operation content identified based on the brain waves of the user C is the operation content of "setting the cooling setting temperature of the air conditioner to 28°C", the processor 26 calculates the average of these set temperatures. Then, the processor 26 sets the cooling setting temperature of the air conditioner to the average temperature.

[0114] When the operation content of the device 16 is an operation to turn the device 16 on or off, the processor 26 may separately determine whether the device 16 is on or off based on the brain waves of each of the users A, B, and C. Then, the processor 26 determines whether the device 16 is on or off according to a majority vote of the users A, B, and C, and turns the device 16 on or off according to the determination.

[0115] (Operation based on distance between device 16 and user) The processor 26 of the information processing device 10 may identify the operation content of the device 16 based on the biometric information (e.g., electroencephalogram signal) of a user whose distance from the device 16 to be operated is equal to or less than a threshold, and operate the device 16 according to the operation content. The positions of the device 16 and each user are identified, for example, by using a global positioning system (GPS) or other communication technology, and the position information of the device 16 and each user is transmitted to the information processing device 10. For example, when a user carries a biometric information measuring device 12 or a terminal device 18, the positions of the biometric information measuring device 12 and the terminal device 18 are identified by using a GPS or other communication technology, and the position information indicating the positions is transmitted to the information processing device 10 as the position information indicating the position of the user.

[0116] For example, when the distance between users A, B and the device 16 is equal to or less than a threshold and the distance between user C and the device 16 exceeds the threshold, the processor 26 of the information processing device 10 calculates the average of the brain waves of users A and B, identifies the operation content of the device 16 based on the average brain waves, and operates the device 16 according to the identified operation content. Even when the brain wave signal of user C is transmitted to the information processing device 10, the processor 26 identifies the operation content of the device 16 without using the brain waves of user C. In this way, the device 16 can be operated based on the bioinformation of a user whose distance from the device 16 is equal to or less than a threshold.

[0117] (Operations based on distance between information processing device 10 and user) The processor 26 of the information processing device 10 may identify the operation content of the device 16 based on the biometric information of a user whose distance to the information processing device 10 is equal to or less than a threshold, and operate the device 16 according to the operation content. For example, when the information processing device 10 is a device such as a smart speaker and is installed in a room 36, the processor 26 may operate the device 16 based on the biometric information of a user whose distance to the information processing device 10 is equal to or less than a threshold.

[0118] (Weighting process) The processor 26 of the information processing device 10 may perform weighting processing on the electroencephalograms of each user, calculate an average of the weighted electroencephalograms, and identify the operation content of the device 16 based on the average electroencephalogram.

[0119] For example, a weighting factor is determined for each user, and information indicating the weighting factor of each user is stored in the memory 24 of the information processing device 10. For example, each user may determine his / her own weighting factor, or a user selected from users A, B, and C may determine the weighting factor of each user. The information indicating the weighting factor of each user may be transmitted from the terminal device 18 of each user to the information processing device 10, or may be input to the information processing device 10 using the UI 22 of the information processing device 10.

[0120] The weighting coefficient of each user may be determined based on the attributes of the user. The attributes are, for example, age, sex, height, or weight. The weighting coefficient of each attribute is determined in advance, and information indicating the weighting coefficient of each attribute is stored in advance in the memory 24 of the information processing device 10. For example, information indicating the attributes of each user is transmitted from the terminal device 18 of each user to the information processing device 10, and the processor 26 of the information processing device 10 determines the weighting coefficient of each user based on the information indicating the attributes of each user.

[0121] The weighting factor may also be determined based on the distance between the device 16 and the user. For example, the closer the distance between the device 16 and the user, the larger the weighting factor may be, and the longer the distance, the smaller the weighting factor may be. Of course, the opposite may also be true. The distance is calculated based on the position information of the device 16 and the user, respectively.

[0122] (If biological information changes) When the brain waves of each of users A, B, and C change, the processor 26 of the information processing device 10 changes the operation content of the device 16 in response to the change. For example, the processor 26 calculates an average of the brain waves of users A, B, and C for each predetermined period, identifies the operation content for each period based on the average, and operates the device 16 for each period according to the identified operation content. In this case, the weighting process described above may also be performed.

[0123] In the first embodiment, the living things whose biological information is measured may be a combination of at least two of humans, non-human animals, and plants. For example, the device 16 may be operated based on the biological information of the humans and the living things. The same applies to the embodiments described below.

[0124] (What to do if an organism moves) Hereinafter, a process when a living being moves will be described with reference to Fig. 9. Fig. 9 shows a schematic diagram of a room 36. As in the example shown in Fig. 8, the room 36 contains users A, B, and C, who are people. Also, a device 16 is installed in the room 36.

[0125] For example, as indicated by the arrow in Fig. 9, when the user A moves and the distance between the user A and the device 16 exceeds a threshold value, and the distance between the users B and C and the device 16 is equal to or less than a threshold value, the processor 26 of the information processing device 10 identifies the operation content based on the brain waves of the users B and C, and operates the device 16 according to the identified operation content. As described above, the processor 26 may identify the operation content based on the average of the brain waves of the users B and C, and operate the device 16 according to the identified operation content, or may identify the functional level based on the brain waves of the users B and C, and operate the device 16 according to the average of the identified functional level, or may determine on or off based on the brain waves of the users B and C, and turn the device 16 on or off according to the determination.

[0126] The processor 26 may perform the weighting process described above. In this case, if the user A before moving is a user who influences the decision of the operation content (for example, if the weighting coefficient of the user A is larger than the weighting coefficients of other users), the operation content of the device 16 may change when the user A moves.

[0127] In addition, even when the biometric information of a non-human animal or living thing is measured, if the non-human animal moves or the living thing's location changes, the operation content of the device 16 may change accordingly.

[0128] <Example 2> A second embodiment will be described below. In the second embodiment, biometric information measured from a plurality of living organisms in different locations is utilized. As an example, a case where a device 16 is operated based on biometric information measured from people in different locations (e.g., locations not in the same room) is described here. The second embodiment is applied, for example, to a situation where a plurality of people have a conversation or a meeting online (e.g., a remote meeting or a Web meeting).

[0129] For example, assume that users A, B, C, and D are participating in an online conference. User A uses terminal device 18A, user B uses terminal device 18B, user C uses terminal device 18C, and user D uses terminal device 18D. Terminal devices 18A, 18B, 18C, and 18D have the same configuration as terminal device 18.

[0130] The terminal devices 18A, 18B, 18C, and 18D can receive services from a server that provides online conversation or conference services, and exchange information (e.g., images, sounds, character strings, etc.) with each other via the server. As such a service, for example, a publicly known service is used.

[0131] Hereinafter, the second embodiment will be described in detail with reference to Fig. 10. A screen 38A is shown in Fig. 10. The screen 38A is a screen displayed on the display of the UI 30 of the terminal device 18A of the user A.

[0132] Images of users A, B, C, and D participating in the online conference are displayed on screen 38A. For example, screen 38A includes areas 40A, 40B, 40C, and 40D. An image of user A is displayed in area 40A, an image of user B is displayed in area 40B, an image of user C is displayed in area 40C, and an image of user D is displayed in area 40D. Screens having a configuration similar to that of screen 38A are also displayed on the displays of each of terminal devices 18B, 18C, and 18D.

[0133] For example, user A is photographed by a camera provided in terminal device 18A of user A or a camera not provided in terminal device 18A, and processor 34 of terminal device 18A displays an image generated by the photograph in area 40A. Furthermore, data of the image of user A is transmitted from terminal device 18A to terminal devices 18B, 18C, and 18D via the server, and the image of user A is displayed on the display of each of terminal devices 18B, 18C, and 18D. The same applies to the images of users B, C, and D.

[0134] As in the first embodiment, the biometric information of each user is measured by the biometric information measuring device 12. As an example, it is assumed here that brain waves are measured from each of the users A, B, C, and D, as in the first embodiment. Of course, biometric information other than brain waves may be measured. The biometric information (e.g., brainwave signals) of each of the users A, B, C, and D is transmitted from the biometric information measuring device 12 of each user to the information processing device 10.

[0135] The processor 26 of the information processing device 10 operates the device 16 based on the brain waves of each of the users A, B, C, and D. The device 16 to be operated may be the terminal devices 18A, 18B, 18C, and 18D, or may be a device other than the terminal device 18.

[0136] Each terminal device 18 may have the functions of the information processing device 10, and the processor 34 of each terminal device 18 may operate the target device 16. For example, each terminal device 18 receives bioinformation from the bioinformation measuring device 12 of each user via a communication path and operates the device 16.

[0137] For example, the processor 34 of each terminal device 18 controls the volume of the speaker provided in the terminal device 18 based on the brain waves of each user. For example, when it is determined that user B is excited by analyzing the brain waves of user B, the processor 34 of each terminal device 18 controls the volume of the speaker of each terminal device 18 so that the volume of user B is reduced. For example, the processor 34 of the terminal device 18A controls the volume of the speaker of the terminal device 18A so that the volume of user B is reduced. The same applies to the other terminal devices 18. Note that when a speaker not provided in the terminal device 18 is used, the processor 34 controls the volume of the speaker.

[0138] In addition, authentication of users participating in the online conference may be performed. Users who have been successfully authenticated can participate in the online conference. For example, authentication techniques such as face authentication, voice authentication, retina authentication, or ear acoustic authentication are used. The authentication may be performed by the information processing device 10 or an authentication server.

[0139] In the second embodiment, the processor 26 of the information processing device 10 may operate the device 16 based on the brain waves of each user (for example, based on the average of the brain waves). The device 16 to be operated may be a device used by at least one of the users A, B, C, and D, or may be another device. In this case, weighting processing may be performed.

[0140] (What to do if an organism moves) Hereinafter, the process when a living thing moves will be described with reference to Fig. 11. Fig. 11 shows a screen 38A.

[0141] For example, if user B, who had been participating in an online conference, stops participating in the online conference or if the image of user B is no longer displayed in area 40B, processor 26 of information processing device 10 operates device 16 based on the brain waves of each of users A, C, and D.

[0142] <Example 3> Hereinafter, a third embodiment will be described with reference to Fig. 12. The third embodiment is a combination of the first and second embodiments. In Fig. 12, a screen 38A is shown.

[0143] For example, users A1 and A2 participate in an online conference using terminal device 18A, users B1 and B2 participate in an online conference using terminal device 18B, user C1 participates in an online conference using terminal device 18C, and users D1, D2, and D3 participate in an online conference using terminal device 18D. For example, users A1 and A2 belong to group A, users B1 and B2 belong to group B, user C1 belongs to group C, and users D1, D2, and D3 belong to group D. Information about the groups (for example, identification information of the group and identification information of each user belonging to the group, etc.) is stored in the information processing device 10, a server, etc.

[0144] Area 40A displays images of users A1 and A2, area 40B displays images of users B1 and B2, area 40C displays an image of user C1, and area 40D displays images of users D1, D2, and D3.

[0145] As in the first embodiment, the biometric information of each user is measured by the biometric information measuring device 12. Note that the biometric information of one or more users participating in the online conference does not have to be measured. As an example, it is assumed here that brain waves are measured from each user as in the first embodiment. Of course, biometric information other than brain waves may also be measured. The biometric information of each user (e.g., brain wave signals) is transmitted from the biometric information measuring device 12 of each user to the information processing device 10.

[0146] As in the second embodiment, the processor 26 of the information processing device 10 operates the device 16 based on the brainwaves of each user. The device 16 to be operated may be the terminal devices 18A, 18B, 18C, and 18D, or may be a device other than the terminal device 18. Each terminal device 18 may have the function of the information processing device 10, and the processor 34 of each terminal device 18 may operate the device 16 to be operated.

[0147] For example, similarly to the second embodiment, the processor 34 of each terminal device 18 controls the volume of the speaker provided in the terminal device 18 based on the brain waves of each user. For example, when it is determined that the user B1 is excited by analyzing the brain waves of the user B1, the processor 34 of each terminal device 18 controls the volume of the speaker of each terminal device 18 so as to reduce the volume of the user B1.

[0148] As another example, the processor 34 of each terminal device 18 may calculate an average of the brain waves of one or more users belonging to a group, and determine the state of the group based on the average. For example, when it is determined that users B1 and B2 belonging to group B are excited by analyzing the average of the brain waves of users B1 and B2, the processor 34 of each terminal device 18 controls the volume of the speaker of each terminal device 18 so that the volume of users B1 and B2 is reduced.

[0149] In the third embodiment, the processor 26 of the information processing device 10 may also operate the device 16 based on the brain waves of each user (for example, based on an average of the brain waves). The processor 26 may also operate the device 16 based on the brain waves of one or more users belonging to a group (for example, based on an average of the brain waves of one or more users belonging to a group).

[0150] (What to do if an organism moves) Hereinafter, the process when a living thing moves will be described with reference to Fig. 13. In Fig. 13, a screen 38A is shown.

[0151] For example, if user C1, who had been participating in an online conference, stops participating in the online conference, or if the image of user C is no longer displayed in area 40C, processor 26 of information processing device 10 operates device 16 based on the brain waves of users belonging to groups A, B, and D.

[0152] <Example 4> Hereinafter, a description will be given of the fourth embodiment. In the fourth embodiment, a process for encouraging people to move is performed.

[0153] For example, the processor 26 of the information processing device 10 may obtain emotional, mental, and psychological information of a person by analyzing the biometric information of the person. When multiple types of biometric information are measured, the processor 26 may obtain emotional, mental, and psychological information of the person by analyzing all or a part of the multiple types of biometric information. The processor 26 may perform weighting processing on each piece of biometric information based on the priority of each piece of biometric information, and obtain emotional, mental, and psychological information of the person based on the result of the weighting processing. Furthermore, a reference level may be set for each type of biometric information, and the processor 26 may determine that biometric information having the same reference level indicates the same tendency in terms of emotion, mental state, and psychological information.

[0154] The processor 26 prompts the person to move to an optimal location based on the result of the analysis of the biometric information. For example, when the processor 26 determines that the person feels uncomfortable based on the result of the analysis of the biometric information, the processor 26 prompts the person to move to a location where the person would feel comfortable. The location where the person would feel comfortable corresponds to an example of an optimal location.

[0155] The processor 26 identifies an optimal location (for example, a location where a person would feel comfortable) based on the biological information measured by the biological information measuring device 12 and the environmental information measured by the environmental information measuring device 14.

[0156] To explain with a specific example, when the processor 26 determines that the person feels hot based on the person's biological information, the processor 26 prompts the person to move to a cooler place. A cooler place corresponds to an example of an optimal place. For example, the processor 26 analyzes the person's sweat rate and brain waves, and determines that the person feels hot when the sweat rate is equal to or higher than a threshold and the brain waves indicate discomfort. For example, the environmental information measuring device 14 measures environmental information such as temperature, humidity, and air volume of each place, and the environmental information of each place is transmitted from the environmental information measuring device 14 to the information processing device 10. The processor 26 identifies a place where the person may feel cool based on the environmental information of each place. For example, the processor 26 identifies a place where the temperature is lower than the temperature of the current place where the person is, and prompts the person to move to the identified place.

[0157] Furthermore, when the processor 26 determines that the person feels cold based on the person's biological information, the processor 26 prompts the person to move to a warmer place. In this case, the warm place corresponds to an example of an optimal place.

[0158] For example, the processor 26 encourages each person to move to an optimal location. For example, as shown in FIG. 8, when users A, B, and C are present in a room 36, the processor 26 encourages each of the users A, B, and C to move to an optimal location for each user. The processor 26 may guide users whose biometric information has the same tendency to the same location. For example, when users A and B feel hot and user C feels cold, the processor 26 guides users A and B to a cool location and guides user C to a warm location.

[0159] As another example, if a person is looking at images or text displayed on a display, and processor 26 determines, based on the person's biometric information, that the person feels that the images or text are difficult to see, the processor 26 prompts the person to move closer to the display.

[0160] As yet another example, when sound is being emitted from a speaker, if processor 26 determines, based on the person's biometric information, that the person is finding it difficult to hear the sound, it prompts the person to move closer to the speaker.

[0161] For example, the processor 26 transmits information encouraging the person to move (for example, information showing guidance or guidance, etc.) to the terminal device 18 of the person. The processor 34 of the terminal device 18 displays the information on the display of the UI 30. FIG. 14 shows an example of the display. A screen 42 is a screen displayed on the display of the terminal device 18. For example, a message such as "Please move to XX (a comfortable place)" and a map 44 showing the place are displayed on the screen 42. On the map 44, for example, a mark 46 indicating the current location of the person, a mark 48 indicating a comfortable place, and an arrow 50 guiding the person to the comfortable place are displayed. The current location of the person and the comfortable place are identified by using a technology such as GPS. In the example shown in FIG. 8, information encouraging the person to move is displayed on the display of the terminal device 18 of each of the users A, B, and C.

[0162] FIG. 15 shows another display example. FIG. 15 shows a room 36. For example, the processor 26 of the information processing device 10 may display guidance by light or the like on the floor of the room 36, as indicated by the reference numeral 52. For example, the processor 26 displays guidance by controlling lighting installed in the room 36. If the location where the person is located is a road or the like, the processor 26 displays guidance on the ground or the like. Alternatively, the processor 26 may display guidance on a wall or may guide the person by voice. The voice may be emitted from the terminal device 18 or from another speaker. The processor 26 may also display guidance on a display installed around the person.

[0163] The processor 26 may guide multiple people whose biometric information shows the same tendency to the same place, and guide multiple people whose biometric information shows different tendencies to different places. For example, the processor 26 may guide multiple people who like each other to the same place, and guide multiple people who dislike each other to different places.

[0164] <Example 5> A fifth embodiment will be described below. In the fifth embodiment, biological information of a living organism is displayed. The fifth embodiment will be described in detail with reference to FIG. 16. FIG. 16 shows a screen 38A. As an example, it is assumed that an online conference is being held by a plurality of users, similar to the third embodiment.

[0165] For example, the biometric information of each user is transmitted to the terminal device 18 of each user, and the processor 34 of the terminal device 18 displays the biometric information of each user on a display. Fig. 16 shows a screen 38A of the terminal device 18A of the user A, and the processor 34 of the terminal device 18A displays the biometric information of each user on the screen 38A. Fig. 16 shows brain waves, which are an example of biometric information. For example, the waveform of the brain waves is displayed.

[0166] 16, an electroencephalogram 54 of a user A1, an electroencephalogram 56 of a user B1, an electroencephalogram 58 of a user C1, and an electroencephalogram 60 of a user D3 are displayed. For example, the processor 34 of the terminal device 18A links the electroencephalogram 54 of the user A1 to an image of the user A1 and displays it in the area 40A. The same is true for the electroencephalograms of the other users.

[0167] The user can select whether to display or not display the brainwaves. The processor of the terminal device 18A may, for example, display or not display the brainwaves of each user according to an instruction from the user of the terminal device 18A, or may display or not display the brainwaves of the user according to an instruction from the user whose brainwaves are measured.

[0168] For example, when the user of terminal device 18A instructs to display the brain waves of each of users A1, B1, C1, and D3, processor 34 of terminal device 18A displays the brain waves of each of users A1, B1, C1, and D3, as shown in FIG. 16.

[0169] As another example, the user A1 may set whether to display or hide the user A1's own brain waves. When the display is set, the user A1's brain waves are displayed, and when the non-display is set, the user A1's brain waves are not displayed. The same applies to other users.

[0170] Furthermore, the processor 34 of the terminal device 18 may display the waveform of the electroencephalogram and information indicating the emotion, mental state, or psychological state obtained by analyzing the electroencephalogram. For example, the electroencephalogram 58 indicates a state of "alertness (excitement)," and the processor 34 displays a character string indicating "alertness (excitement)" in the area 40C together with the electroencephalogram 58. The same applies to other electroencephalograms.

[0171] The processor 26 of the information processing device 10 may convert voice data during an online conference into text data, and store the text data in memory by linking it to biometric information (e.g., brain waves) of each user. The processor 26 may also store the voice data in memory by linking it to the biometric information of each user. The processor 34 of the terminal device 18 may also obtain the text data and display the text. For example, it may be possible to use biometric information as a search key to search for voice data or text data linked to the biometric information.

[0172] Furthermore, the processor 34 of the terminal device 18 may display electroencephalograms representing predetermined emotions, mental states, psychological states, and intention information. For example, when electroencephalograms inappropriate for a meeting (for example, electroencephalograms representing drowsiness, sleep, etc.) are measured, the processor 34 displays the electroencephalograms in association with an image of the user whose electroencephalograms were measured.

[0173] Images representing emotions, mental states, psychological states, and intention information may be displayed. An example of such a display is shown in Fig. 17. For example, an image 62 representing concentration, an image 64 representing sleepiness, an image 66 representing relaxation, and the like are displayed. For example, the processor 34 of the terminal device 18 displays each image in association with the user together with the brainwaves or instead of the brainwaves.

[0174] The processor 34 of the terminal device 18 may change the number, color, emphasis, and type of images depending on the degree of emotion, mental state, psychological state, and intention information.

[0175] The processor 34 may display a score according to the degree of the emotion, mental state, psychological state, or intention information, in association with the user.

[0176] The processor 34 may process and display the user's image itself (e.g., an image of the user's face) according to the degree of emotion, mental state, psychological state, and will information, or may change the user's background image. For example, the processor 34 may display the color of the user's face white or pale when the user is feeling negative, and display the skin color of the user's face bright when the user is feeling positive. The processor 34 may display a darker background image when the user is feeling negative, and a brighter background image when the user is feeling positive.

[0177] <Example 6> Example 6 will be described below with reference to Fig. 18. Fig. 18 shows a schematic cross-sectional image of a human brain.

[0178] For example, the bioinformation measuring device 12 measures a person's brain waves, and the processor 26 of the information processing device 10 identifies an activated part of the person's brain based on the measured brain waves. For example, by providing electrodes at three or more locations on the person's head to measure brain waves, the processor 26 identifies the part from which the measured brain waves are generated. The processor 26 may display a mark or the like indicating the identified part on a tomographic image of the person's brain, or may fill in the identified part.

[0179] The tomographic image may be an image taken by a computed tomography (CT) device or a magnetic resonance imaging (MRI) device, or may be an image that diagrammatically represents the brain (eg, an image created in advance).

[0180] <Example 7> The screen on which the biometric information is displayed will be described below. An example of the home screen is shown in FIG.

[0181] The home screen 100 is displayed, for example, on the display of the terminal device 18. The home screen 100 displays a "Scan" button, a "Record" button, a "Reset" button, a "File" button, and an "Edit" button. The screen 100 also includes display areas 102, 104, and 106. In the example shown in FIG. 19, the areas 102, 104, and 106 are arranged vertically in that order, but this arrangement is merely an example. The arrangement may be changed by the user.

[0182] When the "Scan" button is pressed, a list of the vital information measuring devices 12 that can communicate with the terminal device 18 (for example, can be paired with BLE or the like) is displayed on the home screen 100. When the "Record" button is pressed, vital information measured by the vital information measuring device 12 is stored in memory. For example, the vital information measuring device 12 and the terminal device 18 communicate with each other, and vital information measured by the vital information measuring device 12 is transmitted from the vital information measuring device 12 to the terminal device 18 and stored in the memory 32 of the terminal device 18. When the "Reset" button is pressed, the calibration settings are reset. When the "File" button is pressed, a screen for managing vital information files is displayed. When the "Edit" button is pressed, a setting screen (see FIG. 23) described later is displayed.

[0183] Area 102 displays the biological information measured by biological information measuring device 12, and area 104 displays information obtained by processing the biological information. Hereinafter, unprocessed biological information will be referred to as "Raw data." For example, area 102 displays a graph of the Raw data (hereinafter referred to as "Graph 1"), and area 104 displays a graph of processed biological information (hereinafter referred to as "Graph 2"). In area 106, the setting values ​​of the graph displayed in area 104 and the like are displayed.

[0184] The vertical axis (Y axis) of Graph 1 is the potential (e.g., [μV]) representing biological information, and the horizontal axis (X axis) is time. For example, the range of the vertical axis can be changed by a pinch operation, a zoom operation, or the like.

[0185] The vertical axis (Y axis) of Graph 2 before switching is the potential (e.g., [μV]) representing the biological information, and the horizontal axis (X axis) is time. For example, the range of the vertical axis can be changed by a pinch operation, a zoom operation, or the like.

[0186] The display contents of the areas 104 and 106 can be switched to other display contents. For example, when an operation of sliding the areas 104 and 106, such as a flick operation, is performed, the display contents are switched. Of course, the display contents may be switched by another operation.

[0187] FIG. 20 shows the display content after switching.

[0188] The graph 2 before switching is, for example, a graph (for example, a real-time graph) generated by performing processing set on a setting screen described later on raw data. In the area 106 before switching, setting values ​​related to the graph and the like are displayed. For example, an index indicating that biological information (for example, brain waves) is being acquired, an index indicating that a person whose biological information is being measured is concentrating, an index indicating that a person whose biological information is being measured is relaxed, and values ​​of a six-axis sensor mounted on the biological information measuring device 12 (for example, values ​​indicating the amount of movement in the axial directions of the X, Y, and Z axes, values ​​indicating the amount of rotation around the X, Y, and Z axes, etc.) are displayed in the area 106. Each value displayed in the area 106 is updated, for example, at predetermined time intervals (for example, every second). In addition, an image having a color corresponding to each value may be displayed.

[0189] The graph 2 after the switching is, for example, a graph generated by performing FFT (Fast Fourier Transform) processing on raw data (hereinafter referred to as an "FFT graph"). In the area 106 after the switching, setting values ​​related to the graph 1, graph 2, and FFT graph are displayed. For example, the items "Graph X", "Graph Y", "FFT Max X", and "FFT Max Y" are displayed in the area 106. The item "Graph X" is an item for setting the scale value of the X-axis common to the graph 1 and the FFT graph. The item "Graph Y" is an item for setting the scale value of the Y-axis separately for the graphs 1 and 2. The item "Graph 1" is an item for setting the scale value of the Y-axis of the graph 1, and the item "Graph 2" is an item for setting the scale value of the Y-axis of the graph 2. The item "FFT Max X" is an item for setting the maximum value of the X-axis of the FFT graph. The item "FFT Max Y" is an item for setting the maximum value of the Y-axis of the FFT graph.

[0190] Fig. 21 shows a specific example of raw data Graph 1 and pre-switching Graph 2. Fig. 22 shows a specific example of post-switching Graph 2 (graph after FFT processing).

[0191] An example of the setting screen is shown in Fig. 23. When the "Edit" button is pressed on the home screen 100, a setting screen 110 shown in Fig. 23 is displayed.

[0192] The item "Micro Volts" is used to set the Y-axis display of Graphs 1 and 2 to "μV."

[0193] The item "Cut Off Filter" is an item for setting the setting value of a filter (for example, a high pass filter (HPF) and a low pass filter (LPS)) to be applied to the biological information. According to the setting value set here, the filter processing is applied to the biological information. For example, the filter processing is applied to the graphs 1 and 2. The filter processing may be executed by the biological information measuring device 12, may be executed by the information processing device 10, or may be executed by the terminal device 18. By executing the filter processing, it is possible to extract information having a specific frequency from the measured biological information. For example, it is possible to extract only brain waves from biological information including brain waves and other myoelectric signals, or to extract other myoelectric signals (for example, signals caused by eye movement, face movement, jaw movement, etc.). The device 16 may be operated based on the extracted signal. For example, the processor 26 of the information processing device 10 may distinguish between brain waves and other myoelectric signals and extract them, and may operate the same device 16 or different devices 16 separately based on the brain waves and other myoelectric signals.

[0194] The item "Peak Filter" is an item for setting a value related to processing for peak values ​​of raw data. For example, the item "Peak Filter" includes an item for setting that measurement of biological information (e.g., electroencephalogram) is impossible when the peak value of raw data is equal to or greater than a set value, and an item for setting a value for measuring impulse noise.

[0195] <Example 8> An eighth embodiment will be described below. In the eighth embodiment, the processor 26 of the information processing device 10 infers the emotion, mental state, and psychological state of another person who has a specific relationship with the person based on the biological information of the person. The processor 26 may infer the emotion, mental state, and psychological state of another person based on the emotion information, mental information, and psychological information of the person.

[0196] The specific relationship is, for example, a parent-child relationship, a blood relationship, etc. For example, processor 26 may infer the emotion, mental state, or psychological state of a child based on the biological information, emotional information, mental information, or psychological state of a parent, or may infer the emotion, mental state, or psychological state of a parent based on the biological information, emotional information, mental information, or psychological state of a child, or may infer the emotion, mental state, or psychological state of a person's siblings based on the biological information, emotional information, mental information, or psychological information of the person.

[0197] For example, the biological information of the parent and the biological information of the child are measured by the biological information measuring device 12, and when the biological information of the child is similar to the biological information of the parent (for example, when the child's brain waves have a waveform similar to that of the parent's brain waves), the processor 26 of the information processing device 10 infers that the child's emotions, mental state, and psychological state are the same as those of the parent. For example, when the biological information of the parent expresses the emotion of "feeling good," and the biological information of the child is similar to the biological information of the parent, the processor 26 infers that the child also has the emotion of "feeling good." For example, the above inference is made when the child is an infant or a young child who has difficulty controlling his or her emotions, mental state, and psychological state.

[0198] As another example, if a parent has difficulty controlling their emotions, mental state or psychological state (e.g., they suffer from dementia, etc.), processor 26 may infer the emotions, mental state or psychological state of the parent based on the child's biometric information, emotional information, mental information or psychological information.

[0199] As yet another example, processor 26 may infer the emotions, mental state, or psychological state of people who have similar lifestyles. For example, people who have lived together for a long time may have similar lifestyles, and in that case, processor 26 infers the emotions, mental state, or psychological state of other people who have similar lifestyles to one person based on the biometric information, emotional information, mental information, or psychological information of the person.

[0200] <Example 9> Hereinafter, a ninth embodiment will be described. In the ninth embodiment, the processor 26 of the information processing device 10 informs a human subject of specific information that the subject does not know. For example, the processor 26 transmits the specific information to the terminal device 18 of the subject.

[0201] The specific information to be notified to the subject is, for example, visual information, environmental information, and information about the atmosphere of a specific location. For example, information that cannot be known unless the subject is in the specific location is the information to be notified to the subject. The location is specified, for example, by an administrator of the information processing device 10, the subject, etc. Also, information indicating a danger that has occurred in the specific location or a danger that may occur in the specific location may be transmitted to the terminal device 18 of the subject, as specified by the administrator, the subject, etc.

[0202] The visual information is, for example, an image taken by a fixed camera installed at a specific location, an image taken by a camera worn by a person at a specific location, etc. The visual information may include sound information measured by a microphone installed at a specific location or a microphone worn by a person. The environmental information is information measured by various sensors installed at a specific location (for example, temperature, humidity, air chamber, weather, etc.), information measured by a sensor worn by a person at a specific location, etc. The information regarding the atmosphere is, for example, information indicating that a specific location is a comfortable or uncomfortable place for a person, information indicating that a specific location is a place where a person can concentrate or is distracted, etc. For example, when a person at a specific location inputs information regarding the atmosphere of the specific location to his / her own terminal device 18, the input information is transmitted from the terminal device 18 to the information processing device 10.

[0203] For example, when a subject uses terminal device 18 to specify a specific location and request acquisition of specific information at the specific location, information indicating the request is transmitted from terminal device 18 to information processing device 10. In response to the request, processor 26 of information processing device 10 transmits specific information at the specified specific location to terminal device 18. Processor 34 of terminal device 18 causes the specific information to be displayed on a display or to be generated as sound from a speaker.

[0204] Furthermore, the processor 26 of the information processing device 10 may observe the situation at each location, and when the situation at a specific location designated by the subject meets a predetermined condition, may transmit specific information at the specific location to the subject's terminal device 18. The situation at the location may be, for example, an environmental situation (e.g., temperature, humidity, weather, etc.) obtained by analyzing environmental information, a situation obtained from visual information (e.g., congestion in a city, on a train, etc.), and a dangerous situation obtained from visual information, environmental information, etc. The situation at the location may be inferred or predicted from visual information, environmental information, etc.

[0205] For example, when the weather in a specific location specified by the subject falls under a predetermined weather condition or is predicted to fall under the predetermined weather condition, the processor 26 of the information processing device 10 transmits information related to the weather in the specific location to the terminal device 18 of the subject. Also, when the congestion state of the specific location falls under a predetermined congestion state or is predicted to fall under the predetermined congestion state, the processor 26 of the information processing device 10 transmits information related to the congestion state of the specific location to the terminal device 18 of the subject.

[0206] Furthermore, the processor 26 of the information processing device 10 may select specific information to be transmitted to the subject's terminal device 18 based on the subject's purpose (e.g., avoiding a crisis, etc.). The purpose is specified by the subject, for example.

[0207] For example, when a car is running out of control, if a person uses the terminal device 18 to give an instruction to transmit information about the running out of control car to the information processing device 10, information about the car (e.g., information including information indicating danger) is transmitted from the terminal device 18 to the information processing device 10. The information about the car includes location information of the terminal device 18. In addition, if the running out of control car is photographed by a camera worn by a person, the information about the car may include the photographed image. When the processor 26 of the information processing device 10 receives the information about the car, it acquires an image (e.g., an image photographed from a higher position) photographed by a fixed camera installed around the position of the terminal device 18 from the fixed camera, and transmits the image to the terminal device 18 of the subject. The image is displayed on the display of the terminal device 18 of the subject. The processor 26 may also transmit information indicating danger to the terminal device 18 of the subject. The information indicating the danger is displayed on the display of the terminal device 18 of the subject. Furthermore, in the case where multiple fixed cameras are installed, the processor 26 may transmit to the subject's terminal device 18 an image captured by the fixed camera that is most effective in detecting danger.

[0208] Furthermore, the processor 26 may select information included in the specific information or process the specific information according to the attributes of the subject (e.g., age, sex, medical history, etc.) and transmit the specific information to the terminal device 18 of the subject. For example, since there may be differences in visual function due to age (e.g., there may be differences in visual function due to cataracts, glaucoma, etc.), the processor 26 may adjust the clarity of the image included in the specific information according to the age of the subject. Furthermore, since there may be differences in crisis awareness due to gender and age, the processor 26 may select information included in the specific information according to the gender and age of the subject. For example, the processor 26 may include only an image in the specific information, may include a character string (e.g., a character string indicating a crisis) in the specific information, or may include both an image and a character string in the specific information, according to the gender and age of the subject.

[0209] <Example 10> The tenth embodiment will now be described.

[0210] For example, the biological information of the plant may be measured by the biological information measuring device 12, and the equipment 16 may be operated based on the measured biological information. The measured biological information may be, for example, the electric potential of the plant (e.g., electric potential including membrane potential, etc.) or water content. Of course, information other than these may be measured as the biological information.

[0211] For example, when bioinformation of a plant is being measured, if playing music near the plant improves the plant's growth compared to when music is not being played, the processor 26 of the information processing device 10 operates the device 16, such as a music player, to play music from the device 16 so that the bioinformation (e.g., electric potential) measured while the music is being played is continuously measured from the plant. The processor 26 may also control how water and fertilizer are applied to the plant. For example, the processor 26 may provide water and fertilizer to the plant so that the bioinformation is continuously measured from the plant.

[0212] Furthermore, the processor 26 may detect a change in the state of the plant based on the biological information measured from the plant, and display information corresponding to the change (e.g., an image, a character string, etc.) on a display of a device such as the terminal device 18. The processor 26 may anthropomorphize the plant and display information corresponding to the change on the display.

[0213] The processor 26 may determine whether or not the plant is diseased based on the biological information measured from the plant, and may display the result of the determination on the display of a device such as the terminal device 18.

[0214] Furthermore, the amount of water in an organism changes over time, and this change is reflected in the bioelectric potential. The bioelectric potential is measured by the biological information measuring device 12, and the processor 26 of the information processing device 10 may detect the behavior or abnormality of the organism based on the measured bioelectric potential. For example, when an abnormality is detected, the processor 26 may transmit warning information indicating the abnormality to a device such as the terminal device 18. For example, when the bioelectric potential of a person is measured and an abnormality of the person is detected based on the bioelectric potential, warning information is transmitted to the terminal device 18 of the person, and the warning information is displayed on the display of the terminal device 18.

[0215] Generally, moisture is distributed evenly throughout the body in the morning, and during the day, moisture gradually increases in the lower part of the body as the person is active. The change in moisture amount is reflected in the bioelectric potential. The processor 26 of the information processing device 10 may measure the bioelectric potential of each part of the person and measure time based on the change in the bioelectric potential. For example, the change in the bioelectric potential may be used as a clock or a timer.

[0216] Furthermore, the processor 26 of the information processing device 10 may measure biological information (eg, brain waves) of a non-human animal and infer the condition (eg, illness) of the animal and the environment in which the animal is placed.

[0217] Furthermore, the processor 26 of the information processing device 10 may detect the type of living thing wearing the biological information measuring device 12.

[0218] For example, the size of the skull is different between humans and non-human animals. When the bioinformation measuring device 12 has a band for attaching the bioinformation measuring device 12 to the head, and the bioinformation measuring device 12 is attached to the head by wrapping the band around the head, the processor 26 may detect the length of the band using a sensor or the like, and detect the type of animal to which the bioinformation measuring device 12 is attached based on the length of the band. To explain this by taking a specific example, since the size of the skull is different between humans and dogs, the length of the band also differs accordingly. The processor 26 detects the length of the band and detects whether the living thing to which the bioinformation measuring device 12 is attached is a human or a dog.

[0219] As another example, if the bioinformation measuring device 12 is a device that is worn on the ear of a living organism, the processor 26 may detect the length of the hair on the ear to which the bioinformation measuring device 12 is worn using a sensor or the like, and detect the type of animal to which the bioinformation measuring device 12 is worn based on the length of the hair.

[0220] As yet another example, since body temperature differs depending on the type of living thing, the processor 26 may measure the body temperature of the living thing wearing the vital information measuring device 12 and identify the type of the living thing based on the body temperature.

[0221] As yet another example, since different types of living things have different heights and weights, and therefore different resistance values, the processor 26 may identify the type of living thing wearing the vital information measuring device 12 based on the resistance value.

[0222] Biometric information measured from a living organism may be linked to the type of the living organism (e.g., human, dog, plant, etc.). The linking may be performed by the biological information measuring device 12 or the information processing device 10. In this way, it is possible to identify the type of organism from which certain biological information was measured. When the type of organism from which biological information is measured is identified as described above, information indicating the identified type may be linked to the biological information.

[0223] Animals may communicate with each other using biometric information such as brain waves. For example, communication may be performed between humans or between humans and non-human animals (e.g., dogs) using biometric information. When the type of living thing whose biometric information is measured is specified as described above, the processor 26 may change the method of converting the brain waves in the communication.

[0224] <Example 11> An eleventh embodiment will be described below. In the eleventh embodiment, a stimulus is given to a living organism, and the processor 26 of the information processing device 10 links stimulus information indicating the content of the stimulus with biometric information measured from the living organism. The linked stimulus information and biometric information are stored, for example, in the memory 24 of the information processing device 10. A plurality of pieces of biometric information may be linked. For example, brain waves, pulse, blood pressure, complexion, body temperature, etc. may be linked to the stimulus information.

[0225] The stimuli are, for example, images such as still images and moving images, environments, tastes, etc. For example, images expressing joy, anger, sorrow, and happiness, images and environments that cause fear, images and environments that cause healing, etc. are given as stimuli. The images are, for example, virtual images such as VR (Virtual Reality). Of course, images that show real objects may also be used. The stimuli may be electrical stimuli given by electrodes provided in the vital information measuring device 12.

[0226] For example, when a person is looking at an image, the biometric information measuring device 12 measures biometric information (e.g., brain waves) from the person, and the image and the biometric information are linked. The measured brain waves are brain waves emitted from the left brain, brain waves emitted from the right brain, or brain waves emitted from both the left and right brain.

[0227] In addition, the processor 26 may link the words uttered by the person when the stimulus is given (i.e., the voice information) to the stimulus information and the bio-information. Furthermore, the processor 26 may link information indicating the person's movement to the stimulus information and the bio-information.

[0228] A virtual image such as a VR image may be an image showing a background captured in a person's daily life, and surrounding sounds may be played along with the VR image.

[0229] Furthermore, as the VR image, an image corresponding to the things that the person whose biological information was measured has experienced may be used. The things that the person has experienced may be investigated in advance (for example, a questionnaire).

[0230] When the biometric information measured from the person who viewed the VR image changes significantly, the processor 26 may link the changed portion of the biometric information to the stimulation information as valid biometric information. For example, when the amount of change in the biometric information is equal to or greater than a threshold, the processor 26 links the changed portion of the biometric information to the stimulation information as valid biometric information.

[0231] The order in which the stimulation information is given to the person may be controlled. For example, the person may be given visual information that the person dislikes, and then may be given auditory information, olfactory information, or bodily sensation information that the person likes. The processor 26 may link each piece of stimulation information to the biological information measured at that time.

[0232] Processor 26 may vary the stimulus information provided to the person in response to changes in the measured physiological information.

[0233] Processor 26 may also change the playback speed of the VR image, change the viewpoint of the VR image, change the color of the VR image, or change the quality or focus of the VR image.

[0234] The linked stimulation information and biometric information may be used for digital marketing (e.g., video production, attractions, equipment, and other products) and digital healthcare (e.g., a service that records daily life and evaluates it based on the degree of stress and refreshment).

[0235] <Example 12> The biological information measuring device 12 may be shared by a plurality of people, etc. For example, after a person A uses the biological information measuring device 12, another person, a person B, may use the biological information measuring device 12 that was used by the person A.

[0236] For example, the contact-type biological information measuring device 12 (i.e., a device in which electrodes or sensors come into contact with a living organism such as a human to measure biological information) may output information indicating that a living organism (e.g., a person) wearing the biological information measuring device 12 has changed or information encouraging cleaning when the living organism (e.g., a person) wearing the biological information measuring device 12 has changed. Outputting information means, for example, displaying the information on a display, emitting the information as sound from a speaker, or both.

[0237] For example, if the vital information measuring device 12 is provided with a microphone, the vital information measuring device 12 recognizes the person using the vital information measuring device 12 based on the person's voice input from the microphone, thereby determining whether the person using the vital information measuring device 12 has changed. The vital information measuring device 12 may recognize the person using the vital information measuring device 12 based on an image generated by photographing the person using the vital information measuring device 12 using a camera provided in the terminal device 18 or the like.

[0238] As another example, if the account of a person using the biometric information measuring device 12 is managed by the information processing device 10, the biometric information measuring device 12, the terminal device 18, a server, etc., and that account is switched to another account, the biometric information measuring device 12 may determine that the person using the biometric information measuring device 12 has changed.

[0239] By outputting information indicating that the person using the vital information measuring device 12 has changed, the person using the vital information measuring device 12 can recognize that the vital information measuring device 12 has been used by another person. This allows, for example, a new person to use the vital information measuring device 12 to recognize that the vital information measuring device 12 needs to be cleaned. The same applies when information encouraging cleaning is output.

[0240] Also, a sensor or the like that detects that the vital information measuring device 12 has been cleaned may be used. For example, a sensor that detects that the electrodes or sensors that measure vital information have been cleaned with alcohol (e.g., a sensor that detects alcohol) is used, and when the vital information measuring device 12 is about to be used in a state in which the electrodes or sensors have not been cleaned with alcohol (e.g., when the power supply of the vital information measuring device 12 is turned on), the vital information measuring device 12 outputs information indicating that the vital information measuring device 12 has not been cleaned. Also, the vital information measuring device 12 does not need to turn on the power supply of the vital information measuring device 12. When the vital information measuring device 12 is about to be used in a state in which the electrodes or sensors have been cleaned with alcohol, the vital information measuring device 12 may output information indicating that the vital information measuring device 12 has been cleaned, or may not output such information.

[0241] <Example 13> Hereinafter, the thirteenth embodiment will be described with reference to Fig. 24. A screen 38A is shown in Fig. 24. The screen 38A is the same as the screen according to the third embodiment (see Fig. 12).

[0242] In the thirteenth embodiment, information (e.g., an image or a numerical value showing a bar) showing an emotion, will, mental state, psychological state, etc. is displayed. The emotion, will, mental state, psychological state, etc. are estimated by the processor 26 based on the biometric information of the user. For example, a bar showing the user's concentration level, level of relaxation, level of stress, etc., or a numerical value showing these may be displayed. Bars and numerical values ​​for all users may be displayed, or bars and numerical values ​​for users designated by an administrator, etc. may be displayed.

[0243] In the example shown in FIG. 24, bars and numerical values ​​representing the mental state, etc., of user C1 are displayed. Bar 120 is an image representing the degree of relaxation of user C1. Bar 122 is an image representing the degree of stress of user C1. The degree of relaxation is "100" and the degree of stress is "30". These numerical values ​​are calculated by processor 26 based on the biometric information of user C1.

[0244] Furthermore, the processor 26 may identify an event that contributed to the user's state (e.g., emotion, will, mental state, psychological state, etc.) and calculate the degree to which the event contributed to the user's state. For example, the processor 26 may identify an event that contributed to stress or recovery from the stress and display information indicating the degree of contribution on the screen 38A. This makes it easy for the user to understand what event influenced the user's state and to what extent. For example, the processor 26 detects the occurrence of an event using various sensors, etc., and estimates the change in the user's state when the event occurred and the user's state before and after the occurrence of the event based on the user's biometric information. The processor 26 estimates the degree to which the event contributed to the user's state based on the change in the user's state, for example.

[0245] The processor 26 may also store in the memory 24 history information indicating a history regarding the occurrence of an event, the user's biometric information before and after the occurrence of the event, changes in the user's biometric information, the user's state before and after the occurrence of the event, and changes in the user's state. The processor 26 may reproduce a past event based on the history indicated by the history information. For example, the processor 26 may record events before and after a point in time when the user relaxed, and reproduce the same situation at that time. For example, a target value for the relaxation is specified, and the processor 26 reproduces a situation in which the target value is obtained. The processor 26 may reproduce the situation by combining the playback of music, the display of an image, etc., and an instruction to a person to perform an action. The processor 26 may acquire the state of the user's brainwaves, classify the state of the brainwaves, and record the history by linking it to working hours (for example, hours excluding sleeping hours). The processor 26 may display a number or a bar indicating a state such as relaxation, sandwiched between an image of the meeting by a virtual camera.

[0246] <Example 14> Hereinafter, a fourteenth embodiment will be described with reference to Fig. 25 and Fig. 26. In Fig. 25, images representing users A, B, and C are shown, similar to Fig. 17. Also, information indicating the state of each user (e.g., brain waves, etc.) is displayed in association with each user. Here, as an example, it is assumed that users A, B, and C are participating in the same online conference.

[0247] For example, when all users participating in an online conference, or a number of users equal to or greater than a predetermined number, have the same state (e.g., the same emotion, will, mental state, or psychological state), the processor 26 may display the frame of the image of the user having the same state in a specific color (e.g., red, etc.) or apply a specific decoration. The state of the user is estimated based on the biometric information of the user.

[0248] For example, when users A, B, and C are in the same state (e.g., awake state), as shown in Fig. 26, the processor 26 displays the entire frame surrounding the images of users A, B, and C in a specific color or applies a specific decoration (e.g., a thick frame). In this way, it is possible to show that users A, B, and C are in the same state. For example, when all users participating in an online conference are in an awake state, applying such a display or decoration shows that the conference mode is a hot-blooded mode.

[0249] As in the fourteenth embodiment, information expressing the overall status of a plurality of users participating in a conference may be displayed.

[0250] <Example 15> Example 15 will now be described.

[0251] The data transmitted from the biological information measuring device 12 to another device (for example, the information processing device 10) may be data generated by applying processing such as FFT instead of raw data of the biological information. This prevents information that is included in the raw data but not included in the data after processing such as FFT from leaking.

[0252] Furthermore, information indicating the items of measured biological information (for example, information indicating a channel or a body part) may be encrypted by the biological information measuring device 12 and transmitted from the biological information measuring device 12 to another device (for example, the information processing device 10). This makes it difficult to know what items of biological information are transmitted, thereby improving the security of the biological information.

[0253] Furthermore, when the measured biometric information or information obtained from the biometric information is lent or sold to a user, the processor 26 may restrict the use of the biometric information. The contents of the restrictions may include, for example, deleting the information after a predetermined period of time has elapsed, returning the information to the provider, etc. Furthermore, the places where the information is permitted to be used, the purpose of the use, etc. may be specified.

[0254] <Example 16> Example 16 will now be described.

[0255] In the sixteenth embodiment, an AI assistant (i.e., an artificial intelligence assistant) that emits voice is used. For example, the AI ​​assistant executes processing or operates a device according to the user's biometric information or the user's instruction. The AI ​​assistant may be mounted, for example, on a user's terminal device, on a device other than the terminal device, or on an information processing device 10, a server, or the like.

[0256] For example, the processor 26 may change the volume of the voice emitted by the AI ​​assistant or the speed at which the AI ​​assistant speaks, depending on the user's biometric information.

[0257] Specifically, if the user's biometric information indicates that the user wants to relax, processor 26 changes the voice emitted by the AI ​​assistant to a voice that the user prefers (e.g., high-pitched, low-pitched, speaking speed, etc.) or to the voice of a family member of the user.

[0258] Also, if the user's biometric information indicates that the user is in a hurry, the processor 26 may increase the speed at which the AI ​​assistant speaks or cause the AI ​​assistant to output voice quickly.

[0259] In addition, when the user learns by using the voice uttered by the AI ​​assistant, the processor 26 may control the AI ​​assistant to utter the voice at a speaking speed and volume that is effective for learning for the user. The learning effect of the user is estimated based on the user's biometric information. For example, the processor 26 changes the speaking speed and volume of the AI ​​assistant based on the user's biometric information.

[0260] In addition, when the user is participating in an online conference or a telephone conference, the processor 26 may estimate the state of the other party of the user based on the biometric information of the other party, and based on the result of the estimation, change the voice uttered by the user in a manner that makes the voice uttered by the user easier to convey to the other party. For example, when the user is angry and makes a voice that frightens the other party, the processor 26 may change the voice of the user to a milder voice, or may express the contents of the user's voice by using figures, text, etc. without conveying the voice to the other party. Such processing may be executed by an AI assistant.

[0261] In addition, in an online conference or the like, a certain user (for example, a first user) may wear the biometric information measuring device 12, and the other user (for example, a second user) may not wear the biometric information measuring device 12. In this case, the biometric information of the first user is acquired, and the biometric information of the second user is not acquired. Even in such a case, if the first user can recognize the facial expression of the second user through the screen, the biometric information of the first user may change due to a change in the facial expression or voice of the second user. When a change occurs in the biometric information of the first user in this way, the processor 26 determines that a change has occurred in the biometric information of the first user due to the second user. The processor 26 may estimate the state (for example, emotions, mental state, or psychological state) of the second user based on the determination. For example, when the facial expression of the second user is a painful expression, the first user who sees the face may have an apologetic feeling, and therefore the processor 26 estimates the state of the second user based on the biometric information of the first user. The AI ​​assistant may also tell the first user how to interact with a second user who has such a condition.

[0262] Furthermore, when adding voice to an animated character or dubbing voice to a drama or movie, a user who has the authority to determine the voice may listen to the voice of the voice actor provisionally assigned to the character or actor, and processor 26 may determine the official voice actor based on the user's biometric information. For example, when biometric information indicating approval is obtained from the user, processor 26 may determine the voice actor who produced the voice as the official voice actor. When biometric information indicating a sense of incongruity or discomfort is obtained from the user, processor 26 may introduce a different voice actor to the user.

[0263] <Example 17> Example 17 will now be described.

[0264] This embodiment may be applied to a content service such as a game. For example, a game may be provided to a user via the user's terminal device, or via another device (e.g., a game device). The game may be an online game, or a game provided by executing a program installed in the terminal device or game device. Also, a recording medium storing a game program may be used, and the game may be provided by executing the program by the terminal device or game device.

[0265] For example, while the user is playing a game, the biometric information of the user is measured by the biometric information measuring device 12. The processor 26 may change the progress of the game, game scenes, game sounds, image presentation, and the like, according to the user's emotions obtained from the biometric information.

[0266] For example, for a user who does not like scary games or scary things, if bio-information indicating a feeling of fear above a predetermined level is measured, processor 26 may omit violent scenes, reduce the volume of the game's sound, simplify the presentation, or change the language to something easier to understand.

[0267] On the other hand, for a user who likes scarier scenes, if bio-information indicating an emotion of fear above a predetermined level is not measured, processor 26 may change the presentation of the game so that more violent scenes or scenes that emphasize scarier scenes appear.

[0268] The user's preferences are set before the game is started. Processor 26 may change the game presentation in conjunction with bio-information according to the user's chronic illness or age-related burden. Processor 26 may change the presentation for educational reasons.

[0269] Processor 26 may determine the state of the user based on the user's biological information at predetermined intervals. This allows the scene to be changed in real time, allowing the user to progress through the game without feeling any discomfort or strangeness. Note that scenes that cannot be changed due to the convenience of the game story may be set.

[0270] Furthermore, when a feeling that the game is difficult or easy is detected, the processor 26 may change the difficulty level of the game in response to the feeling. For example, the processor 26 may increase or decrease the action level of an AI that is an enemy of the user, or increase or decrease the number of enemy characters. Furthermore, when a mystery game is being played, the processor 26 may increase or decrease the number of hints leading to the answer. Furthermore, the processor 26 may increase or decrease a set value such as the attack power or defense power of a character operated by the user.

[0271] Processor 26 may notify the user of the change in difficulty level based on the biometric information. In this way, the user can understand, for example, at what difficulty level the user was able to clear the game. In this case, the game play time, the change in the user's emotions during the play time, the game difficulty level, etc. may be displayed in the form of a graph or other graphic.

[0272] In addition, when the user achieves a desired state by playing a game, information indicating the state may be stored as history information in the memory 24 of the information processing device 10. The processor 26 may change the scene or difficulty of the game so that the user's state becomes the desired state. For example, when a target state of relieving stress and feeling refreshed is set in the information processing device 10, the processor 26 may change the scene or difficulty of the game so that the user's state becomes the target state. For example, the processor 26 may change the game scene to a scene showing a refreshing blue sky and plains. As another example, the processor 26 may change an enemy character to a weak character so that the enemy character can be easily defeated by the character operated by the user.

[0273] <Example 18> Example 18 will now be described.

[0274] In the eighteenth embodiment, the biological information measuring device 12 is a hearable device and is worn on the ear of a user. The hearable device may have a voice assistant function. The voice assistant function is, for example, a function that assists a user based on the biological information of the user. The voice assistant function may be realized by AI.

[0275] The process according to the eighteenth embodiment will be described below.

[0276] First, a hearable device, which is an example of the bioinformation measuring device 12, measures bioinformation (e.g., electrocardiogram waveforms, electroencephalograms, skin potential, etc.) from the user's ear (e.g., ear canal), and further measures the user's voice. The hearable device links the bioinformation and voice information in chronological order and stores them. These pieces of information may be stored in the hearable device, in the user's terminal device, or in the information processing device 10. Here, as an example, it is assumed that these pieces of information are stored in the information processing device 10.

[0277] The processor 26 of the information processing device 10 judges whether or not a large fluctuation has occurred in the user's biological information. For example, if the biological information is an electroencephalogram, the large fluctuation is an epileptic abnormal wave, and if the biological information is an electrocardiogram, the large fluctuation is a waveform indicating an abnormality in the myocardium, such as arrhythmia.

[0278] When the processor 26 detects that a large fluctuation has occurred in the user's biometric information, the processor 26 records the time when the large fluctuation occurred, the location of the user at that time, and information indicating the user's state (hereinafter, such information is referred to as "context information"). The processor 26 may remove noise and determine whether the large fluctuation is caused by a mind-body stimulus response.

[0279] When the large fluctuation (i.e., change in mental and physical stimulus response) is a critical situation, the processor 26 controls the audio function of the hearable device. Critical situations include, for example, when a heart disease occurs (for example, when an arrhythmia is detected based on the pulse rate, which is bioinformation), when the user falls (for example, when the user's fall is detected based on the electromyogram, which is bioinformation, or data from an acceleration sensor), when the user swallows (for example, when an irregular waveform is included in the electromyogram, which is bioinformation, and when it is detected that swallowing may have occurred based on the irregular waveform), etc.

[0280] Processor 26 also transmits the above-mentioned context information, biometric information, and audio information to a device (e.g., a PC, etc.) installed in a predetermined location (e.g., a hospital, etc.). At this time, processor 26 transmits to the device the context information, biometric information, and audio information measured during a time period (e.g., a predetermined length of time) before and after the time point when large fluctuations occurred.

[0281] The processor 26 may also control the sound emitted from the hearable device based on the above-mentioned context information, biometric information, and audio information. For example, the processor 26 controls the playback speed of the content played on the hearable device. Specifically, the processor 26 changes the reading speed of a learning video or an e-book. The processor 26 may also stop the playback of the content. The processor 26 may also play content (e.g., a subject, a comic, music, etc.) that matches the user's preferences. The processor 26 may also execute an auto-call function. The auto-call function here is a function that automatically notifies a call or message to a pre-designated emergency contact (e.g., a family member, a family doctor, etc.). A call or message may be notified to multiple people at the same time, or a call or message may be notified to an outside person gradually. The processor 26 may also change the contact depending on the severity of the user's medical condition. Of course, the user himself / herself may also be notified.

[0282] The above voice assistant function may be switched for each hearable device or each user account. Also, a function for setting whether or not to execute the voice assistant function based on biometric information may be provided. The voice assistant function of the hearable device may be executed without being based on biometric information. Also, the situation in which the voice assistant function based on biometric information is applied may be limited to a situation in which a symptom of a disease is diagnosed, or may be limited to a reading function such as a content service. A function for setting such a limitation may be provided.

[0283] The processor 26 may train an AI that realizes a voice assistant function on the above-mentioned context information, biometric information, and voice information.

[0284] For example, the processor 26 may accumulate voice information when the user is speaking in a relaxed state and have the AI ​​learn the voice information. For example, by applying voice synthesis and voice adjustment technology, the AI ​​may synthesize a voice assistant voice optimized for biological reactions and read the voice.

[0285] As another example, the processor 26 may accumulate voice information when the user is speaking in a stressed state and have the AI ​​learn the voice information. For example, by applying voice synthesis and voice adjustment technology, the AI ​​may synthesize an optimized voice assistant voice to help the user regain their composure and read the voice.

[0286] As yet another example, the processor 26 may accumulate voice information when the user is nervous, and allow the AI ​​to learn when, where, and in what state the user is nervous. For example, by applying voice synthesis and voice adjustment technology, the AI ​​may synthesize an optimized voice assistant voice to help the user regain their composure, and read the voice.

[0287] As yet another example, the processor 26 may convert the user's voice into text, accumulate the converted data together with information indicating the user's emotions in chronological order, and allow the AI ​​to learn them. The data may be stored in a memory. The emotions may be selected by the user. The processor 26 may also change the size and color of the converted text according to the user's voice. By applying voice synthesis and voice adjustment technology, the AI ​​may synthesize an optimized voice assistant voice to help the user regain his or her composure, and read the voice.

[0288] For example, the processor 26 links and records the user's voice information obtained from the user's terminal device (e.g., a smartphone, a PC, etc.) and information such as icons, characters, pictures, etc. input by the user with the biometric information measured from the user. The processor 26 converts the user's biometric information into emotion index data such as joy, anger, sorrow, pleasure, stress, concentration, relaxation, burden, understanding, mastery, or interest.

[0289] For example, electrodermal activity may be used as a parameter indicating the mental activity state. Alternatively, skin potential level (SPL), which is a direct current component of skin potential activity (SPA), may be used. SPL generally indicates a high negative value when the level of arousal is high (i.e., when excited), and indicates a high positive value when drowsy or relaxed. Skin potential reflex (SPR), which is an alternating current component of skin potential activity (SPA), may be used. SPR frequently occurs due to stimuli caused by changes in the external environment for pain, touch, hearing, and vision, as well as deep breathing, body movements, thinking, etc.

[0290] The processor 26 visualizes and displays the degree of the state (e.g., joy, anger, sadness, happiness, stress, concentration, relaxation, burden, understanding, mastery, interest, etc.) represented by the emotion index data and the type of the state together with information indicating the state of the user acquired from an interface such as a camera or a keyboard. For example, the processor 26 changes the display color, the type of font, and the size of the characters according to the degree and type of the state. For example, in a reading function realized by existing software, the processor 26 may change the speaking style (e.g., joy, anger, sadness, fear, kindness, etc.) and the tone of voice (e.g., emphasis, speaking speed, voice pitch, intonation, volume, etc.) according to a stimulus response (e.g., relaxation, stress, tension, etc.). The processor 26 may change the volume and speed of the voice of the reading function according to the degree of the user's emotion.

[0291] The above examples may be applied to, for example, e-books, movie content (including content distributed via the Internet), electronic learning materials, online conference tools, and the like.

[0292] As another example, when creating a document or a material, if a function for changing the size or color of characters or figures according to the biometric information of the user is executed, the document or material may lose its consistency if the function is always executed. Therefore, the processor 26 may not change the size or color of characters or figures according to the biometric information in a specific area of ​​the document or material. In other words, the processor 26 may change the size or color of characters or figures in an area where the change is permitted according to the biometric information of the user.

[0293] In addition, when the reading function is executed to read out the characters written in a sentence or a document, the processor 26 may change the volume and speed of the voice during reading out in accordance with the biometric information of the user. Also, the reading out speed may be limited to a range that is neither too fast nor too slow.

[0294] The time required for the process of determining the emotion, mental state, or psychological state based on the biological information, and the time required for measuring the emotion, mental state, or psychological state may vary depending on the type of emotion, mental state, or psychological state. For example, the measurement interval for the concentration level and relaxation level is 1 second, the measurement interval for the interest level, comprehension level, proficiency level, activity load, emotional strength, and creativity is 1 to 5 seconds, and the measurement interval for the pleasantness and unpleasantness is 5 seconds or 10 seconds. Of course, these times are merely examples and may vary depending on the performance of the biological information measurement device 12, etc.

[0295] In the case where there is a time variance as described above, in a particular mode, the processor 26 may determine some of the emotions, mental states, and psychological states determined in advance, without determining all of the emotions, mental states, and psychological states determined in advance, or may limit the time and the situation in which the determination is performed. For example, when the user is learning, the processor 26 may determine only the learning effect based on the user's biological information, and may not determine emotions and the like other than the learning effect. However, when the user's condition is in a particular condition (for example, when the user suffers from an illness such as an underlying disease), the processor 26 may constantly determine all of the emotions, mental states, and psychological states determined in advance.

[0296] In addition, when the user stops wearing the hearable device but is wearing another biological information measurement device 12, the processor 26 may determine the emotion, mental state, or psychological state based on the biological information measured by the other biological information measurement device 12. For example, when the user is wearing a hearable device and a smart watch and stops wearing the hearable device, the processor 26 may determine the emotion, mental state, or psychological state based on the biological information measured by the smart watch.

[0297] <Example 19> Hereinafter, a nineteenth embodiment will be described. In each of the above-mentioned embodiments, when bioinformation of a specific frequency band (for example, brain waves of a specific frequency band) is measured and bioinformation of a frequency band other than the specific frequency band is not measured, the processor 26 may estimate bioinformation of the other frequency band. For example, the processor 26 estimates bioinformation of the same type as the measured bioinformation but of a frequency band different from the measured frequency band.

[0298] For example, when only the potential of the frequency band of α waves, which is an example of brain waves, is measured and the potential of frequency bands other than α waves is not measured, the processor 26 estimates brain waves of frequency bands that are not actually measured (for example, brain waves other than α waves such as β waves) or brain waves of frequency bands that are not clearly detected based on one or more pieces of information among information measured by various sensors (for example, a gyro sensor, an acceleration sensor, etc.) at the time of the measurement, environmental information (for example, information indicating temperature, humidity, weather, position, etc.), information on the user's current behavior (for example, information indicating the user is exercising, working at a desk, sleeping, etc.), information on the user's current illness, and information on the measurement history of the measured α waves (for example, information indicating the shape, period, amplitude, etc. of the waveform of α waves). For example, the processor 26 estimates brain waves of frequency bands that are not measured based on the correlation of the above various pieces of information. The processor 26 may output the brain waves of the estimated frequency band to a terminal device or the like. For example, the brain waves of the estimated frequency band are displayed on the display of the terminal device. At this time, the measured frequency band electroencephalogram (e.g., α waves) and the estimated frequency band electroencephalogram (e.g., β waves) may be displayed as a graph having the same scale (e.g., a graph in which the horizontal axis indicates time and the vertical axis indicates potential). Processor 26 may output the estimated frequency band electroencephalogram as a candidate for the frequency band electroencephalogram. In the case where α waves are not measured and only β waves are measured, α waves may be estimated in the same manner.

[0299] When bioinformation of a plurality of frequency bands is measured, the processor 26 may estimate bioinformation of a frequency band that is not measured based on the bioinformation of the plurality of frequency bands. For example, when alpha waves and beta waves are measured but no other brain waves are measured, the processor 26 may estimate brain waves of the other frequency bands based on the actually measured alpha waves and beta waves.

[0300] When multiple candidates are estimated, the processor 26 may display the multiple candidates on the display of the terminal device, or output a sound indicating that multiple candidates have been estimated. The processor 26 may also change the order in which the multiple candidates are displayed, or display a score according to the probability or possibility of occurrence of each of the multiple candidates.

[0301] For example, when measuring brain waves using a hearable device, it may be difficult to detect bioelectric potentials generated from a location on the head that is relatively far from the ears. In such cases, it may be possible to estimate brain waves in unmeasured frequency bands based on brain waves in the measured frequency bands and other information (e.g., environmental information and information about the user).

[0302] The function of estimating biological information of an unmeasured frequency band may be set to on or off by the user. For example, when the user turns on the function using a terminal device or the like, the function is executed. When the user turns off the function using a terminal device or the like, the function is not executed. Since unmeasured biological information (e.g., brain waves of an unmeasured frequency band) may not actually be generated from the user's brain, it is possible to turn off the function in such a case.

[0303] <Example 20> The following describes Example 20.

[0304] When using brain waves or other such BMI for communication between people, the information to be communicated is identified and a means for transmitting that information is provided.

[0305] For example, the information sender extracts a target of communication and specifies the content of information to be communicated, and transmits it to the receiver. The target of communication may be one receiver for one sender, or multiple receivers (e.g., multiple receivers) for one sender. The content of information to be communicated is information or status information about a person's thoughts or intentions, such as approval, agreement, opposition, rejection, indecision, confusion, etc. When information is provided to a person, the information sender conveys to the receiver whether the person approves of the information. The method of communication may be a method of stimulating the five senses of a person, such as information transmission by sound, information transmission by light emission, and stimulation transmission by vibration. To give a more specific example, for example, a hearable device can simultaneously measure biological information such as brain waves (measurement of the content of information to be communicated) and transmit information by sound by utilizing the ear, and is therefore excellent in that BMI can be realized without preparing various devices. Here, sound may be, for example, music or a chime, other than linguistic information, as long as the meaning of the sound is mutually determined in advance. There are also ways to communicate the degree or level of one's intention by chiming twice to indicate agreement or by changing the pitch. A low-pitched chime indicates a low level of agreement, while a high-pitched chime indicates a high level of agreement, resulting in a more diverse range of ways of communicating. With the advent of BMI, communication between people will be able to acquire a variety of communication methods that utilize biological information such as brain waves, in addition to the traditional method of communication that is centered on language through voice and GUI (Graphical User Interface).

[0306] A specific example of the process according to the 20th embodiment will be described below with reference to Fig. 27 and Fig. 28. Fig. 27 and Fig. 28 show flow charts showing the flow of the specific example. Fig. 27 shows the flow of the process when operating a device. Fig. 28 shows the flow of the process when people communicate with each other.

[0307] As shown in FIG. 27, first, an operation mode (OP mode) is selected (S01), and information on the user's operation intention is extracted and detected (S02). As a result, the device to be operated and the operation contents are extracted, and they are linked. In addition, a method for determining the accuracy of the operation intention is presented to the user. If the accuracy of the operation intention is high (S03, high accuracy), the operation on the device is immediately executed (S04). After that, the correctness of the operation is verified (S05). For example, the verification is performed by the user. If the operation is correct (S06, Yes), the device is operated in the same way next time (S07). For example, the operation contents are set in the device, and the same operation is performed next time. If the operation is incorrect (S06, No), the process proceeds to step S11. If the accuracy of the operation intention is low (S03, low accuracy), the operation intention is confirmed using another method (S08). Note that even if the accuracy of the operation intention is high, the operation intention may be confirmed using another method. For example, information indicating another method is displayed on a display or output as a sound from an earphone of the hearable device. This presents the user with another method. If the confirmed operation intention is correct (S09, Yes), that is, if the confirmed operation intention matches the operation intention that the user had in mind, an operation on the device is executed (S10). If the confirmed operation intention is incorrect (S09, No), the process proceeds to step S11. In step S11, the reason why the operation intention was incorrect is analyzed. For example, the analysis is performed by the user. Based on the analysis result, the next determination method (i.e., the operation determination method) is selected (S12). In addition, the determination criteria for the operation intention are changed (S13), and the changed content is reflected in the process from step S03 onwards. For example, the threshold value registered in the management table is changed, and the operation intention is confirmed, the accuracy is determined, and the like based on the changed threshold value. Note that each of the above processes may be selected by the user.

[0308] Also, when people communicate with each other, as shown in FIG. 28, a communication mode (hereinafter referred to as a "CM mode") is selected (S20). For example, either an automatic selection mode or a manual selection mode is set by the user, and the CM mode is selected according to the setting. Next, a target of communication (hereinafter referred to as a "CM target") is selected (S21). For example, in a mode in which a CM target is automatically selected, a participant of a meeting or the like is selected as the CM target. In a mode in which a CM target is selected by a manual operation of a user, candidates for the CM target are presented to the user, for example, by the voice of a hearable device or the display on a display, and the user selects the CM target. Next, information that may be conveyed to the CM target is selected by the user (S22). For example, information that may be conveyed to the CM target is selected from a group of intention information, a group of status information, etc. In addition, information that is not desired to be conveyed to the CM target may be selected. For example, information indicating an emotion is selected as information that is not desired to be conveyed to the CM target. Next, biological information is measured or observed, and intention information, status information, etc. are extracted based on the biological information (S23). Next, information to be conveyed to the commercial target is detected from the extracted intention information, state information, etc. (S24). For example, a detection test is performed in advance, and intention information, state information, etc. similar to information detected by the test (for example, intention information or state information whose similarity is equal to or greater than a threshold) is detected from the extracted intention information, state information, etc. The detection method may be presented to the user. Then, the detected intention information, state information, etc. are conveyed to the commercial target. For example, the conveyance method may be presented to the user, and the user may select the conveyance method. The conveyance method may be, for example, conveyance by vibration (for example, conveyance by voice), conveyance by light, etc. Information may be conveyed to the commercial target by language, music, light emission signals, etc. When information has been conveyed to the commercial target, the user who is the sender may be notified that the conveyance has been completed. The notification may be omitted. Furthermore, after the process of step S24 is executed, the process of step S03 and subsequent steps shown in FIG. 27 may be executed.

[0309] The process according to the 20th embodiment may be executed using a hearable device. Since the hearable device is attached to a person's ear and the biometric information is measured, each piece of information can be conveyed to the user by voice while the biometric information can be measured. Since the hearable device is attached to the ear, it has an advantage that it looks more natural than an electroencephalograph attached to the head to measure brain waves, etc. Furthermore, the accuracy of the above-mentioned operation mode and CM mode can be improved by the BCI using the hearable device.

[0310] In each of the above embodiments, all or part of the processing executed by the processor 26 of the information processing device 10 may be executed by the processor 34 of the terminal device 18, or may be executed by the vital information measuring device 12. In addition, the processing may be executed by one or more devices selected from the information processing device 10, the terminal device 18, and the vital information measuring device 12.

[0311] <Hearable device> 29 to 37, a hearable device, which is an example of the biological information measuring device 12 according to this embodiment, will be described in detail below. In the following, an earphone that is attached to a person's ear will be described as an example of the hearable device, but the hearable device described here may be a hearable device other than an earphone.

[0312] For example, one or more electrodes are used as biopotential detection electrodes (hereinafter referred to as "sensor electrodes") for detecting electrical potentials including biopotentials, another one or more electrodes are used as electrodes for grounding (hereinafter referred to as "ground electrodes"), and yet another one or more electrodes are used as electrodes (hereinafter referred to as "reference electrodes") for detecting electrical potentials to be compared with the electrical potentials detected by the sensor electrodes.

[0313] Hereinafter, the potential detected by the sensor electrode will be referred to as the "sensor potential," the potential detected by the reference electrode will be referred to as the "reference potential," and the potential detected by the ground electrode will be referred to as the "ground potential."

[0314] Fig. 29 is a perspective view showing the entire hearable device 200. Fig. 30 is a view of the hearable device 200 as viewed from above. Fig. 31 is a view of the left earphone as viewed from the left. Fig. 32 is a perspective view of the right earphone. Fig. 33 is a perspective view of the left earphone. Fig. 34 is a view of the right earphone as viewed from above. Fig. 35 is an exploded perspective view of the right earphone. Fig. 36 is a cross-sectional view of the right earphone. Fig. 37 is a perspective view showing the inside of the left earphone.

[0315] For ease of explanation, the front-back direction, the up-down direction, and the left-right direction are defined as shown in Fig. 29. The front direction is the direction in which a person's face faces, and the back direction is the opposite direction to the front direction. The up direction is the direction in which the top of a person's head faces, and the down direction is the opposite direction to the up direction. The right direction is the direction toward a person's right hand side, and the left direction is the direction toward a person's left hand side. The front-back direction, the up-down direction, and the left-right direction are perpendicular to each other.

[0316] Also, as an example, the hearable device 200 measures biological information including brain waves. Of course, the hearable device 200 may measure biological information other than brain waves while measuring brain waves, or without measuring only brain waves. For example, when the right earphone 216R or the left earphone 216L described later is worn on a person's ear, an electric potential representing biological information is detected. The electric potential signal indicating the detected electric potential usually includes an electric potential signal representing brain waves, but in addition to the electric potential signal representing brain waves, an electric potential signal indicating an electric potential generated due to the movement of the person's head may include an electric potential signal indicating an electric potential generated due to the movement of the head of the person, such as an electric potential signal indicating an electric potential generated due to the movement of the head of the person, such as an electric potential signal indicating an electric potential generated due to the movement of the head of the person. In addition, as biological information caused by a change in blood flow accompanying the movement of the head of the person, cerebral blood flow, pulse waves related to the cardiac blood flow, heart rate, etc. may be significantly measured. In this way, the hearable device 200 may measure biological information other than brain waves together with brain waves. The biological information other than brain waves may be processed as noise (for example, may be removed), or may be separated from the brain waves and used for some processing.

[0317] 29 and 30, the hearable device 200 mainly includes a right earphone 216R to be worn on a person's right ear, a left earphone 216L to be worn on a person's left ear, and a cable 218 that electrically connects the right earphone 216R and the left earphone 216L. The right earphone 216R and the left earphone 216L may transmit and receive signals and data via the cable 218. A remote control for operating the hearable device 200 may be provided on the cable 218.

[0318] Either the right earphone 216R or the left earphone 216L may function as a biological information measuring device that measures biological information, or both the right earphone 216R and the left earphone 216L may function as a biological information measuring device.

[0319] For example, one earphone may have a sensor electrode, a reference electrode and a ground electrode, while the other earphone has no electrodes.

[0320] As another example, both the right earphone 216R and the left earphone 216L may have a sensor electrode, a reference electrode, and a ground electrode.

[0321] As yet another example, one earphone may have one or two of a sensor electrode, a reference electrode, and a ground electrode, and the other earphone may have electrodes that the other earphone does not have.

[0322] As yet another example, the right earphone 216R and the left earphone 216L may each be provided with one or more electrodes, and the sensor electrode, the reference electrode, and the ground electrode may each be assigned to one of the electrodes.

[0323] As yet another example, one of the earphones may be provided with multiple sensor electrodes, multiple reference electrodes, and multiple ground electrodes, such as the right earphone 216R being provided with multiple sensor electrodes and the left earphone 216L being provided with multiple reference electrodes.

[0324] The right earphone 216R is, for example, a canal-type earphone, and includes a right housing 220R, a right earpiece 222R, a right support 224R, a right earhook 226R, and an electrode member 228R. The earpiece is sometimes called an ear pad.

[0325] The right housing 220R is, for example, a housing having a thin rectangular parallelepiped shape and housing electronic circuits and the like. In the right housing 220R, a right earpiece 222R and a right support 224R are provided on a surface facing a person's right ear when a user wears the hearable device 200. A control device, a speaker, a communication device (for example, a communication chip), an electronic circuit for analyzing and processing biological information, a six-axis sensor (for example, a sensor having a three-axis sensor for detecting acceleration and a three-axis sensor for detecting angular velocity), a memory, a GPS (Global Positioning System) sensor, and the like are stored inside the right housing 220R. The communication device is a device for realizing wireless communication functions such as Bluetooth and BLE (Bluetooth Low Energy). This communication device is equipped with a wireless LAN (for example, a network using WiFi, etc.) and a cellular (3G, 4G, 5G, LPWA, etc.) module to widen the communication area, and can directly transmit data to a device in a local area with a longer communication distance than Bluetooth or to a distant device via the Internet. The six-axis sensor detects the direction of movement, orientation, and rotation of the right housing 220R. The biological information may be stored in a memory. Note that the electronic circuit for analyzing the biological information does not have to be provided in the biological information measurement device 12.

[0326] The right support 224R has a columnar or other cylindrical shape, protrudes from a side surface of the right housing 220R (for example, a surface facing the right ear when the right earphone 216R is worn on the right ear), and is provided between the right housing 220R and the right earpiece 222R. For example, the outer diameter of the right support 224R is larger than the outer diameter of the right earpiece 222R. An electrode member 228R is provided on a part or all of the side surface of the right support 224R.

[0327] For example, the electrode member 228R has a ring shape and is supported by a columnar right support 224R. The whole or a part of the electrode member 228R functions as an electrode. That is, an electrode may be provided on the whole surface of the electrode member 228R, or an electrode may be provided on a part of the surface of the electrode member 228R, and no electrode may be provided on that part of the surface. The electrode member 228R is made of, for example, conductive rubber made of carbon. Even in this case, the whole of the electrode member 228R may be made of conductive rubber, and the whole of the electrode member 228R (for example, the whole surface) may function as an electrode, or a part of the electrode member 228R may be made of conductive rubber, and a part of the electrode member 228R (for example, a part of the surface) may function as an electrode.

[0328] The right earpiece 222R is provided at the tip of the right support 224R (i.e., the end opposite to the end connected to the right housing 220R). For example, the right earpiece 222R has a cylindrical shape that narrows toward the tip. Of course, this shape is merely an example, and the right earpiece 222R may have a different shape.

[0329] A sound guide tube or the like (for example, a conductive tube member described later) is stored inside the right earpiece 222R, and sound emitted by a speaker is transmitted through the sound guide tube or the like inside the right earpiece 222R and emitted to the outside from the right earpiece 222R. An electrode is provided on a part or all of the outer surface (for example, a side surface, etc.) of the right earpiece 222R. The electrode is made of, for example, conductive rubber made of carbon. The right earpiece 222R itself may be made of conductive rubber. For example, the whole or part of the right earpiece 222R may be made of conductive rubber. In other words, the whole or part of the surface of the right earpiece 222R may function as an electrode.

[0330] The right earpiece 222R and the electrode member 228R may be made of, for example, an elastic material. For example, a resin such as rubber is used as the elastic material. Specifically, Si-based rubber (for example, S1734 manufactured by NOK) or urethane-based rubber may be used for the right earpiece 222R and the electrode member 228R. Furthermore, the hardness (for example, hardness according to the standard of durometer type A (instantaneous)) of each of the right earpiece 222R and the electrode member 228R is, for example, 40 to 75. As an example, a resin with a hardness of 70 may be used for the right earpiece 222R and the electrode member 228R.

[0331] As described below, the right earpiece 222R is positioned on the right ear so as to be inserted into and contact the ear canal of the right ear, and the electrode member 228R supported by the right support 224R is positioned on the right ear so as to contact the concha cavity of the right ear.

[0332] The right ear hook 226R has a curved shape as a whole, and is a member that is hung on the right ear of the user when the user wears the right earphone 216R. For example, the right ear hook 226R is hung on the helix of the right ear. To explain in more detail, the right ear hook 226R is provided in contact with the helix on the back side of the helix. One end of the right ear hook 226R is connected to a front part of the right housing 220R, and the right ear hook 226R has a curved shape from the connection part to the rear side of the right housing 220R, and this part forms a curved part. The curved part is provided in contact with the helix on the back side of the helix of the right ear. For example, the curved part has a shape that follows the shape of the back side of the helix. The other part of the right ear hook 226R is connected to one end of the cable 218.

[0333] The right earpiece 222R and the right support 224R are replaceable members. For example, multiple types (e.g., 3 to 5 types) of right earpieces 222R and right support bodies 224R with different shapes and sizes are prepared in advance, and the right earpiece 222R and right support body 224R can be replaced to suit the shape of the user's ear (e.g., the ear canal, the concha cavity, or other parts).

[0334] The left earphone 216L is, for example, a canal-type earphone, and includes a left housing 220L, a left earpiece 222L, a left support 224L, a left earhook 226L, and an electrode member 228L.

[0335] The left housing 220L is, for example, a housing having a thin rectangular parallelepiped shape, and stores a battery and the like. In the left housing 220L, the left earpiece 222L and the left support 224L are provided on a surface facing the left ear of the user when the user wears the hearable device 200. Power from the battery is supplied to the right earphone 216R and the left earphone 216L, thereby driving the right earphone 216R and the left earphone 216L. For example, power from the battery is supplied to a speaker and each circuit. Note that the battery may be provided in both the right housing 220R and the left housing 220L, or may be provided in either the right housing 220R or the left housing 220L.

[0336] The left support 224L has a columnar or other cylindrical shape, protrudes from a side surface of the left housing 220L (for example, a surface facing the left ear when the left earphone 216L is worn on the left ear), and is provided between the left housing 220L and the left earpiece 222L. For example, the outer diameter of the left support 224L is larger than the outer diameter of the left earpiece 222L. An electrode member 228L is provided on a part or all of the side surface of the left support 224L.

[0337] For example, the electrode member 228L has a ring shape and is supported by the columnar left support 224L. The whole or a part of the electrode member 228L functions as an electrode. That is, an electrode may be provided on the whole surface of the electrode member 228L, or an electrode may be provided on a part of the surface of the electrode member 228L, and no electrode may be provided on that part of the surface. The electrode member 228L is made of, for example, conductive rubber made of carbon. Even in this case, the whole of the electrode member 228L may be made of conductive rubber, and the whole of the electrode member 228L (for example, the whole surface) may function as an electrode, or a part of the electrode member 228L may be made of conductive rubber, and a part of the electrode member 228L (for example, a part of the surface) may function as an electrode.

[0338] The left earpiece 222L is provided at the tip of the left support 224L (i.e., the end opposite the end connected to the left housing 220L). For example, the left earpiece 222L has a cylindrical shape that narrows toward the tip. Of course, this shape is merely an example, and the left earpiece 222L may have a different shape.

[0339] A sound guide tube and the like are stored inside the left earpiece 222L, and sound emitted by a speaker is transmitted through the sound guide tube and the like inside the left earpiece 222L and emitted to the outside from the left earpiece 222L. An electrode is provided on a part or the whole of the outer surface (e.g., side surface, etc.) of the left earpiece 222L. The electrode is made of, for example, conductive rubber made of carbon. The left earpiece 222L itself may be made of conductive rubber. For example, the whole or part of the left earpiece 222L may be made of conductive rubber. In other words, the whole or part of the surface of the left earpiece 222L may function as an electrode.

[0340] The left earpiece 222L and the electrode member 228L may be made of, for example, an elastic material. For example, a resin such as rubber is used as the elastic material. Specifically, Si-based rubber (for example, S1734 manufactured by NOK) or urethane-based rubber may be used for the left earpiece 222L and the electrode member 228L. Furthermore, the hardness (for example, hardness according to the standard of durometer type A (instantaneous)) of each of the left earpiece 222L and the electrode member 228L is, for example, 40 to 75. As an example, a resin with a hardness of 70 may be used for the left earpiece 222L and the electrode member 228L.

[0341] As described below, the left earpiece 222L is positioned on the right ear so as to be inserted into and contact the ear canal of the right ear, and the electrode member 228L supported by the left support 224L is positioned on the left ear so as to contact the concha cavity of the left ear.

[0342] The left ear hook 226L has a curved shape as a whole, and is a member that is hung on the left ear of the user when the user wears the left earphone 216L. For example, the left ear hook 226L is hung on the helix of the left ear. To explain in more detail, the left ear hook 226L is provided in contact with the helix on the back side of the helix. One end of the left ear hook 226L is connected to the front part of the left housing 220L, and the left ear hook 226L has a curved shape from the connection part to the rear side of the left housing 220L, and this part forms a curved part. The curved part is provided in contact with the helix on the back side of the helix of the left ear. For example, the curved part has a shape that follows the shape of the back side of the helix. The other part of the left ear hook 226L is connected to one end of the cable 218.

[0343] The left earpiece 222L and the left support 224L are replaceable members. For example, multiple types (e.g., 3 to 5 types) of left earpieces 222L and left support bodies 224L with different shapes and sizes are prepared in advance, and the left earpiece 222L and the left support body 224L can be replaced to suit the shape of the user's ear (e.g., the ear canal, the concha cavity, or other parts).

[0344] The control device, communication device, electronic circuitry, six-axis sensor, memory, etc. may be stored in either the right housing 220R or the left housing 220L, or in both the right housing 220R and the left housing 220L.

[0345] Further, the right housing 220R or the left housing 220L is provided with a power button, a switch for adjusting the volume, etc. The power button, the switch, etc. may be provided on both the right housing 220R and the left housing 220L.

[0346] For example, either the electrode provided on the right earpiece 222R or the electrode provided on the left earpiece 222L is used as a sensor electrode, and the other electrode is used as a reference electrode. Also, the electrode members 228R and 228L are used as ground electrodes. Of course, either the electrode member 228R or the electrode member 228L may be used as a sensor electrode, the other used as a reference electrode, and the electrodes provided on the right earpiece 222R and the left earpiece 222L may be used as ground electrodes.

[0347] As another example, the right earpiece 222R may be provided with a plurality of electrodes separated from each other, and at least one of the plurality of electrodes may be used as a sensor electrode, a reference electrode, or a ground electrode. For example, the plurality of electrodes may be used as sensor electrodes, and a potential detected by an electrode having the highest detection sensitivity, the least noise, or the most stable noise level among the plurality of electrodes may be used as a sensor potential. The same applies when the plurality of electrodes are used as reference electrodes or ground electrodes. The same applies to the left earpiece 222L, the electrode member 228R, and the electrode member 228L.

[0348] As shown in FIG. 34, the electrode member 228R supported by the right support 224R is disposed at a height h from the right housing 220R. That is, the distance between the electrode member 228R and the right housing 220R is set to a height h. The height h is set to a value that prevents interference between the right housing 220R and the helix and prevents poor contact, for example. Specifically, the height h is set so that the right housing 220R does not contact the helix of the right ear when the right earpiece 222R is inserted into the ear canal of the right ear and the right earphone 216R is worn on the right ear. That is, the height h is set so that the right housing 220R is separated from the helix to a position where it does not contact the helix. In this way, for example, the upper surface of the right housing 220R does not contact the helix. If the right housing 220R comes into contact with the helix and interferes with the right housing 220R, the right earpiece 222R may not be inserted to a stable position in the ear canal, and poor contact may occur between the electrode of the right earpiece 222R and the ear canal. Also, the electrode member 228R may not be positioned in a position that makes good contact with the concha of the right ear, and poor contact may occur between the electrode member 228R and the concha of the right ear. If such poor contact occurs, the detection sensitivity of the sensor potential, the reference potential, and the ground potential may decrease, and the measurement accuracy of the biological information may decrease. By setting the height h so that the right housing 220R does not come into contact with the helix, the right housing 220R does not interfere with the helix, and the right earpiece 222R can be inserted to a stable position in the ear canal, and good contact may be achieved between the electrode of the right earpiece 222R and the ear canal. In addition, the electrode member 228R can be placed in a position that allows good contact with the cavity of the concha of the right ear, and good contact between the electrode member 228R and the cavity of the concha can be achieved. This improves the detection sensitivity of the bioelectric potential. The same applies to the left earphone 216L.

[0349] Also, as shown in FIG. 30, the right ear hook 226R is arranged offset from the position of the right housing 220R in the direction where the right earpiece 222R is installed when the right earphone 216R is viewed from above. In other words, when the right earphone 216R is viewed from above, a gap is formed between the right housing 220R and the right ear hook 226R. It can be said that the right ear hook 226R is arranged offset from the position of the right housing 220R in the direction where the right earpiece 222R is installed. The gap is an ear space, and the right ear is inserted and arranged in the gap. By providing such an ear space and inserting the right ear into the ear space, the right ear can be arranged in the ear space regardless of the size of the right ear, and the right earphone 216R can be stably worn on the right ear. As a result, the contact between the electrode of the right earpiece 222R and the ear canal and the contact between the electrode member 228R and the concha cavity can be improved. The same applies to the left earphone 216L.

[0350] As shown in FIG. 35, a conductive pipe member 230R made of metal is provided on the right support 224R so as to protrude in a direction away from the right housing 220R. The rear end of the conductive pipe member 230R is installed on the right support 224R, and the right earpiece 222R is provided on the front end of the conductive pipe member 230R. For example, the rear end of the conductive pipe member 230R is attached to the front end of the right support 224R by screwing or the like. A conductive pipe member 232R made of metal is provided on a side surface of the right support 224R in a direction protruding from the side surface. In addition, a ring-shaped electrode member 228R is provided on the side surface so as to cover the conductive pipe member 232R. To explain in more detail, a groove is formed on the side surface of the right support 224R along the circumferential direction of the right support 224R, and a hole 224R1 into which one end of the conductive pipe member 232R is inserted is formed in the groove. One end of the conductive tube member 232R is inserted into the hole 224R1, so that the conductive tube member 232R is attached to the side surface of the right support body 224R. For example, the conductive tube member 232R is attached to the side surface of the right support body 224R by screwing or the like. With the conductive tube member 232R attached to the side surface of the right support body 224R, the electrode member 228R is fitted into a groove in the side surface of the right support body 224R, so that the electrode member 228R is attached to the side surface of the right support body 224R in a state in which it is in contact with the conductive tube member 232R. The conductive tube member 232R is connected to an electric wire within the right support body 224R. As a result, a potential signal indicating the potential detected by the electrode member 228R is sent to the electric wire via the conductive tube member 232R, and is output to the main board 234, which will be described later, via the electric wire. The left earphone 216L has a configuration similar to that of the right earphone 216R.

[0351] 36, a main board 234 and a sub-board 236 are housed in the right side housing 220R. Boards other than the main board 234 and the sub-board 236 may be housed in the left side housing 220L.

[0352] 37, left housing 220L contains battery 238 and relay board 240. Power is supplied from battery 238 via relay board 240 to right earphone 216R and left earphone 216L.

[0353] The cable 218 has a hardness that allows the cable 218 to maintain its overall shape. For example, the cable 218 has a shape that extends backward from the right housing 220R and the left housing 220L so that the cable 218 does not come into contact with the back of the head or the hair at the back of the head when the hearable device 200 is worn on the ears of a person. In this way, the cable 218 does not come into contact with the back of the head or the hair at the back of the head, so that noise caused by such contact can be prevented from occurring in the biometric information.

[0354] Even if the cable 218 comes into contact with the back of the head, the load caused by the contact is transmitted to the cable 218, and the left earphone 216L as a whole rotates backward as indicated by the arrow Y in FIG. 31. This allows the left earpiece 222L and the electrode member 228L to be more firmly fixed to the left ear. That is, the left earphone 216L as a whole rotates in a direction that more firmly fixes the left earpiece 222L and the electrode member 228L. As a result, the contact between the electrode of the left earpiece 222L and the ear canal and the contact between the electrode member 228L and the cavity of the concha can be improved, thereby improving the detection sensitivity. The same applies to the right earphone 216R.

[0355] The hearable device 200 may be a device capable of outputting bone conduction. This allows information to be transmitted to a person by bone conduction instead of sound. For example, even a user who is deaf can use the hearable device 200 to measure biological information such as brain waves and transmit information to the user by bone conduction.

[0356] Each of the above-described embodiments may be implemented using the hearable device 200. That is, the hearable device 200 may be used to measure biometric information of a user, and each embodiment may be implemented using the measured biometric information. Of course, by using the hearable device 200 and another biometric information measuring device 12, multiple types of biometric information may be measured, and a person's state (e.g., emotional information, mental information, and psychological information, etc.) may be obtained based on the multiple types of measured biometric information.

[0357] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operation of the processor in each of the above embodiments may not only be performed by one processor, but may also be performed by multiple processors located at physically separate locations working together. Furthermore, the order of each operation of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0358] 10 information processing device, 12 vital information measuring device, 14 environmental information measuring device, 16 equipment, 18 terminal device.

Claims

1. A processor is included. The processor, acquiring a plurality of pieces of biometric information corresponding to each of a plurality of users, the biometric information including potential signals obtained from the heads of the respective users within a specific range, and operating an operation target according to a combination of the plurality of pieces of biometric information obtained by performing weighting processing on the plurality of pieces of biometric information; Information processing device.

2. The information processing device described in claim 1, wherein the weighting process is a weighted average, and multiple weighting coefficients are determined based on the attributes of the multiple users, or multiple weighting coefficients are determined based on the distance between the operation target and each of the multiple users.

3. The computer acquiring a plurality of pieces of biometric information corresponding to each of a plurality of users, the biometric information including potential signals obtained from the heads of the respective users within a specific range, and operating an operation target according to a combination of the plurality of pieces of biometric information obtained by performing weighting processing on the plurality of pieces of biometric information; A program to make it work like this.

4. The computer acquiring a plurality of pieces of biometric information corresponding to each of a plurality of users, the biometric information including potential signals obtained from the heads of the respective users within a specific range, and operating an operation target according to a combination of the plurality of pieces of biometric information obtained by performing weighting processing on the plurality of pieces of biometric information; Information processing methods.

Images