Information processing device, information processing method, and program

Abstract

The present disclosure relates to an information processing device, an information processing method, and a program that make it possible to estimate cognitive function. This information processing device comprises: a cognitive function estimation processing unit that estimates the current cognitive function of a user on the basis of brain information and peripheral information obtained by performing sensing on the user; and a recommended stimulation determination unit that uses a cognitive function transition model, which expresses the relationships between different stimulations to be provided in order to transition between various states of cognitive function and a relaxed state, to determine a recommended stimulation to be provided to the user on the basis of the current cognitive function of the user and a desired cognitive function according to a selection by the user. The present technology can be applied in, for example, a cognitive function intervention system.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and in particular to an information processing device, an information processing method, and a program that are capable of estimating cognitive function.

[0002] In recent years, the effectiveness of taVNS (Transcutaneous auricular vagus nerve stimulation), a treatment method that directly electrically stimulates the area of ​​the vagus nerve that is distributed in the auricle, has been reported.

[0003] For example, Non-Patent Document 1 discloses that a meta-analysis has confirmed that taVNS is effective in reducing symptoms of depression, and that multiple interventions over a certain period of time are required. Non-Patent Document 2 also discloses the effects of taVNS on cognition in healthy individuals.

[0004] Wu, C., Liu, P., Fu, H., Chen, W., Cui, S., Lu, L., & Tang, C. (2018). Transcutaneous auricular vagus nerve stimulation in treating major depressive disorder: a systematic review and meta-analysis. Medicine, 97, doi: 10.1097 / MD.0000000000013845.Ridgewell, C., Heaton, KJ, Hildebrandt, A., Couse, J., Leeder, T., & Neumeier, WH (2021). The effects of transcutaneous auricular vagal nerve stimulation on cognition in healthy individuals: A meta-analysis. Neuropsychology, 35, 352-365.

[0005] However, when cognitive load is increased through cognitive tasks, people enter a state of high arousal, but a state of high arousal does not necessarily mean that cognitive function is high. Therefore, it is difficult to estimate cognitive function based on the state of arousal, and a method for estimating cognitive function has been sought.

[0006] The present disclosure has been made in light of such circumstances, and makes it possible to estimate cognitive function.

[0007] An information processing device according to one aspect of the present disclosure includes a cognitive function estimation processing unit that estimates a current cognitive function of a user based on brain information and peripheral information obtained by sensing the user.

[0008] An information processing method or program according to one aspect of the present disclosure includes estimating a current cognitive function of a user based on brain information and peripheral information obtained by sensing the user.

[0009] In one aspect of the present disclosure, a user's current cognitive function is estimated based on brain information and peripheral information obtained by sensing the user.

[0010] 1 is a block diagram showing a configuration example of a first embodiment of a cognitive function intervention system to which the present technology is applied; FIG. 2 is a diagram showing an example of a correspondence relationship between brain information and peripheral information and cognitive function; FIG. 3 is a diagram explaining an example of a cognitive function transition model; FIG. 4 is a diagram explaining a first display example of a user interface screen; FIG. 5 is a diagram explaining a second display example of a user interface screen; FIG. 6 is a diagram explaining a third display example of a user interface screen; FIG. 7 is a flowchart explaining cognitive function intervention processing; FIG. 8 is a block diagram showing a configuration example of a second embodiment of a cognitive function intervention system; FIG. 9 is a block diagram showing a configuration example of a third embodiment of a cognitive function intervention system; FIG. 10 is a diagram explaining conductive materials; FIG. 11 is a diagram showing an example of a head-mounted display equipped with a sensor; FIG. 12 is a diagram showing an example of a headband equipped with a sensor; FIG. 13 is a diagram showing an example of headphones equipped with a sensor; FIG. 14 is a diagram showing an example of earphones equipped with a sensor; FIG. 15 is a diagram showing an example of a watch equipped with a sensor; FIG. 16 is a diagram showing an example of glasses equipped with a sensor; FIG. 17 is a block diagram showing a configuration example of an embodiment of a computer to which the present technology is applied.

[0011] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0012] <First Configuration Example of Cognitive Function Intervention System> FIG. 1 is a block diagram showing a configuration example of a first embodiment of a cognitive function intervention system to which the present technology is applied.

[0013] As shown in FIG. 1, the cognitive function intervention system 11 includes an electroencephalogram sensor 21 , a peripheral sensor 22 , an information processing device 23 , an information presentation device 24 , and a stimulus provision device 25 .

[0014] The brain wave sensor 21 is mounted on a device worn by the user (for example, various devices such as headphones (ears), earphones (ears), caps (head), headsets (head), and glasses (eyes)) and supplies brain wave data obtained by sensing the user's brain waves to the information processing device 23.

[0015] The peripheral sensor 22 is mounted on a device worn by the user (e.g., various devices such as headphones (ears), earphones (ears), caps (head), rings (finger), headsets (head), wristbands (wrists), and eyeglasses (eyes)) and supplies peripheral data obtained by sensing the user's peripheral activities (e.g., pulse waves, sweating, eye movement, pupil diameter, etc.) to the information processing device 23. Examples of peripheral data that can be used include laser Doppler flow meter (LDF), electrocardiogram (ECG), electrodermal activity (EDA), pupil diameter, and eye movement. In addition to a device worn by the user, the user's peripheral activities may also be sensed using a remote camera or the like.

[0016] The information processing device 23 executes cognitive function intervention processing to intervene in the user's cognitive functions (e.g., processing ability, thinking ability, performance ability, etc.) based on the brain wave data supplied from the brain wave sensor 21 and the peripheral data supplied from the peripheral sensor 22. The information processing device 23 is configured to include a brain information processing unit 31, a peripheral information processing unit 32, a cognitive function estimation processing unit 33, a recommended stimulus determination unit 34, and a stimulus readjustment unit 35.

[0017] The information presentation device 24 displays a user interface screen that presents information indicating the user's current cognitive function and displays a GUI (Graphical User Interface) for inputting user operations. For example, cognitive function information indicating the user's current cognitive function is supplied from the cognitive function estimation processing unit 33 to the information presentation device 24, and the information presentation device 24 supplies desired cognitive function information indicating a desired cognitive function selected in response to user operations to the recommended stimulus determination unit 34 and the stimulus readjustment unit 35.

[0018] The stimulus providing device 25 provides various stimuli to the user in accordance with recommended stimulus information indicating recommended stimuli to be provided to the user, which is supplied from the recommended stimulus determination unit 34. For example, the stimulus providing device 25 can provide electrical stimulation using taVNS, provide stressful cognitive tasks such as work, calculations, mental arithmetic, or games, provide stimuli such as relaxing music, mindfulness, or fragrance, or provide stimuli to induce paced breathing, which involves breathing at a predetermined frequency. Note that the stimulus providing device 25 can use a device (e.g., earphones or headphones) equipped with the EEG sensor 21 and peripheral sensor 22 as a device for providing electrical stimulation using taVNS.

[0019] The brain information processing unit 31 estimates the user's level of alertness and calculates feature quantities such as alpha waves, beta waves, and theta waves according to the electroencephalogram data supplied from the electroencephalogram sensor 21, thereby performing a brain state estimation process to acquire the user's level of alertness and the brain state represented by the feature quantities of alpha waves, beta waves, and theta waves, and then acquires brain information indicating the user's brain state and supplies it to the cognitive function estimation processing unit 33. Note that the estimation method used by the brain information processing unit 31 to estimate the user's level of alertness is described in detail, for example, in International Publication No. 2022 / 209499, which has already been filed.

[0020] The peripheral information processing unit 32 calculates the user's heart rate (e.g., heart rate variability such as RMSSD (Root Mean Square of Successive Differences) and HR (Heart Rate)), the degree of sweating (e.g., SCR (Skin Conductance Response) representing short-term changes in conductivity due to sweating, and SCL (Skin Conductance Level) representing long-term changes in the reference value of conductivity), eye information (pupil diameter, eye movement speed, etc.) according to the peripheral data supplied from the peripheral sensor 22, and obtains peripheral information indicating the user's peripheral state represented by this information and supplies it to the cognitive function estimation processing unit 33.

[0021] The cognitive function estimation processing unit 33 performs cognitive function estimation processing to estimate the user's current cognitive function based on the brain information supplied from the brain information processing unit 31 and the peripheral information supplied from the peripheral information processing unit 32, in accordance with the correspondence relationship shown in Figure 2.

[0022] For example, as shown in FIG. 2 , if the user's brain state (e.g., alertness) is at a medium level and the user's peripheral state (e.g., heart rate) is at a low level, the cognitive function estimation processing unit 33 estimates that the user's current cognitive function is at a high level (a state in which cognitive function is enhanced). Furthermore, if the user's brain state is at a high level and the user's peripheral state information is at a low level, the cognitive function estimation processing unit 33 estimates that the user's current cognitive function is at a medium level (a state in which cognitive function is at a medium level). Furthermore, if the user's brain state is at a medium level and the user's peripheral state is at a high level, the cognitive function estimation processing unit 33 estimates that the user's current cognitive function is at a low level (a state in which cognitive function is reduced).

[0023] The cognitive function estimation processing unit 33 then supplies cognitive function information indicating the user's current cognitive function to the information presentation device 24 and the recommended stimulus determination unit 34, and the user's current cognitive function is presented on the information presentation device 24. Thereafter, when the user operates the GUI displayed on the information presentation device 24 to select a desired cognitive function, desired cognitive function information indicating the desired cognitive function is supplied from the information presentation device 24 to the recommended stimulus determination unit 34 and the stimulus readjustment unit 35. Furthermore, the cognitive function estimation processing unit 33 updates the user's cognitive function by estimating it based on the brain information and peripheral information supplied after the recommended stimulus is provided by the stimulus provision device 25, and supplies updated cognitive function information indicating the updated cognitive function to the stimulus readjustment unit 35.

[0024] The recommended stimulus determination unit 34 uses the cognitive function information supplied from the cognitive function estimation processing unit 33 and the desired cognitive function information supplied from the information presentation device 24 to determine recommended stimuli to be provided to the user in accordance with a cognitive function transition model such as that shown in Figure 3, and supplies recommended stimulus information indicating the recommended stimuli to the stimulus provision device 25.

[0025] FIG. 3 shows an example of a cognitive function transition model that illustrates the relationship between stimuli to be provided to transition between each cognitive function state and a relaxed state. When the user's current cognitive function is in a relaxed state and a high state is selected as the desired cognitive function, a decision is made to provide taVNS and a cognitive task as the recommended stimuli. When the user's current cognitive function is in a relaxed state and a medium state is selected as the desired cognitive function, a decision is made to provide a cognitive task as the recommended stimuli. When the user's current cognitive function is in a relaxed state and a low state is selected as the desired cognitive function, a decision is made to provide paced breathing and a cognitive task as the recommended stimuli. When the user's current cognitive function is in a medium state and a high state is selected as the desired cognitive function, a decision is made to provide taVNS as the recommended stimuli. When the user's current cognitive function is in a high, medium, or low state and a relaxed state is selected as the desired cognitive function, a decision is made to provide paced breathing as the recommended stimuli. Note that the intensity of the taVNS electrical stimulation may be adjusted to the maximum current that cannot be perceived by the user within a range confirmed to be safe.

[0026] The stimulation readjustment unit 35 determines whether the user has achieved the desired cognitive function as a result of the provision of the recommended stimuli by the stimulation provision device 25, based on the desired cognitive function information supplied from the information presentation device 24 and the updated cognitive function information supplied from the cognitive function estimation processing unit 33. If the stimulation readjustment unit 35 determines that the user has not achieved the desired cognitive function, that is, if the updated cognitive function of the user deviates from the desired cognitive function, the stimulation readjustment unit 35 adjusts parameters of the stimulation to be provided to the user in accordance with the deviation and supplies the adjusted parameters to the stimulation provision device 25, so that the adjusted stimulation can be provided to the user again. For example, the stimulation readjustment unit 35 can adjust parameters indicating the intensity of the electrical stimulation provided by taVNS or parameters indicating the cycle of paced breathing.

[0027] The cognitive function intervention system 11 is configured as described above and can estimate the current cognitive function based on the user's brain information and peripheral information. The cognitive function intervention system 11 then determines appropriate recommended stimuli to be provided to the user to transition from the current cognitive function to a desired cognitive function, thereby more reliably transitioning the user to the desired cognitive function.

[0028] In addition, multiple types of biosensors can be used as the EEG sensor 21 and the peripheral sensor 22, and the EEG data and peripheral data may be acquired by the same biosensor, or the EEG data and peripheral data may be acquired by different biosensors.

[0029] <Display Examples of User Interface Screens> Display examples of user interface screens displayed on the information presentation device 24 will be described with reference to FIGS. 4 to 6. FIG.

[0030] FIG. 4 is a diagram illustrating a first display example of the user interface screen.

[0031] The user interface screen 41 a is an example of a screen that presents the user's current cognitive function status (high, medium, low). The user interface screen 41 a displays a display GUI 51 indicating that the user's current cognitive function is high, a display GUI 52 indicating that the user's current cognitive function is medium, and a display GUI 53 indicating that the user's current cognitive function is low. In the illustrated example, the GUI 51 is hatched, indicating that the user's current cognitive function is high.

[0032] The user interface screen 41b is an example of a screen that presents a message indicating a recommended intervention method depending on the user's current cognitive function state. In the illustrated example, since the user's current cognitive function is in a medium state, the user interface screen 41b displays a message indicating a recommended intervention method, "If you want to improve your cognitive function, try taVNS."

[0033] The user interface screen 41c is an example of a screen that displays multiple options to allow the user to select a desired cognitive function. The user interface screen 41c displays a selection GUI 54 for selecting a high state as the desired cognitive function and a selection GUI 55 for selecting a relaxed state as the desired cognitive function. For example, when the user taps (or presses and holds) the selection GUI 54, the high state is selected as the desired cognitive function, and desired cognitive function information indicating the high state is supplied from the information presentation device 24 to the information processing device 23.

[0034] As a result, a user interface screen 41d presenting a message indicating a recommended stimulus determined in accordance with the cognitive function transition model ( FIG. 3 ) in response to the desired cognitive function information is displayed on the information presentation device 24. For example, if the user's current cognitive function is in a medium state and a high state is selected as the desired cognitive function, taVNS is determined as the recommended stimulus in accordance with the cognitive function transition model, and the message "taVNS" indicating the recommended stimulus is displayed on the user interface screen 41b as shown in the figure.

[0035] In addition, in the user interface screen 41, in addition to using the expression "cognitive function," expressions such as "processing ability," "thinking ability," and "performance" may be used, and in addition to using the expression "desired state," expressions such as "desired state" may be used.

[0036] FIG. 5 is a diagram illustrating a second display example of the user interface screen.

[0037] For example, the information presentation device 24 may allow the user to select a desired cognitive function via a user interface screen 41e that displays the selection GUI 54 and the selection GUI 55, without displaying the current state of the user's cognitive function. Then, the information presentation device 24 displays a user interface screen 41f that presents a message indicating a recommended stimulus determined in accordance with the cognitive function transition model in response to the user's selection on the user interface screen 41e.

[0038] FIG. 6 is a diagram illustrating a third display example of the user interface screen.

[0039] For example, the information presentation device 24 may present only the current state of the user's cognitive function on the user interface screen 41 g on which the display GUI 51 , the display GUI 52 , and the display GUI 53 are displayed.

[0040] Here, since the cognitive function intervention system 11 can estimate the current state of the user's cognitive function with high accuracy by utilizing a large amount of data, it is desirable to collect a large amount of data from more users or to collect data from a single user over a long period of time.

[0041] Furthermore, the cognitive function intervention system 11 can transition the user's cognitive function to a high state when the user wants to improve their cognitive function in various situations, such as work, study, exams, sports, games, racing, e-sports, shogi, and go, not limited to general users and professional users. Conversely, the cognitive function intervention system 11 can transition the user's cognitive function to a relaxed state when the user is excited and unable to relax in such situations. The cognitive function intervention system 11 is effective when actually playing sports such as soccer, baseball, basketball, swimming, judo, archery, and kyudo, and when performing mind training for these sports.

[0042] Furthermore, by collecting a larger history of intervention effects, the cognitive function intervention system 11 can evaluate, for example, when and what the same intervention is most effective, and can provide recommended stimuli that are more appropriate for the situation.

[0043] <Processing Example of Cognitive Function Intervention Processing> An example of cognitive function intervention processing executed in the cognitive function intervention system 11 will be described with reference to the flowchart shown in FIG.

[0044] For example, when a user wears a device equipped with the EEG sensor 21 and the peripheral sensor 22 and instructs the start of the processing, the cognitive function intervention processing is started, and in step S11, the EEG sensor 21 starts sensing EEGs, and the peripheral sensor 22 starts sensing peripheral signals. Then, the EEG sensor 21 supplies EEG data indicating the user's EEGs to the brain information processing unit 31, and the peripheral sensor 22 supplies peripheral data indicating the user's peripheral state to the peripheral information processing unit 32.

[0045] In step S12, the brain information processing unit 31 performs a brain state estimation process to estimate the state of the user's brain according to the brain wave data supplied from the brain wave sensor 21 in step S11, acquires brain information indicating the state of the user's brain, and supplies it to the cognitive function estimation processing unit 33.

[0046] In step S13, the peripheral information processing unit 32 calculates the user's peripheral state according to the peripheral data supplied from the peripheral sensor 22 in step S11, obtains peripheral information indicating the user's peripheral state, and supplies it to the cognitive function estimation processing unit 33.

[0047] In step S14, the cognitive function estimation processing unit 33 performs cognitive function estimation processing to estimate the user's current cognitive function in accordance with the correspondence relationship shown in Fig. 2 based on the brain information supplied from the brain information processing unit 31 in step S12 and the peripheral information supplied from the peripheral information processing unit 32 in step S13. Then, the cognitive function estimation processing unit 33 supplies cognitive function information indicating the user's current cognitive function to the information presentation device 24 and the recommended stimulus determination unit 34.

[0048] Then, the information presentation device 24 presents the user's current cognitive function, and when the user operates the information presentation device 24 to select a desired cognitive function, the process proceeds to step S15.

[0049] In step S15, the recommended stimulus determination unit 34 acquires the desired cognitive function information supplied from the information presentation device 24. Then, using the cognitive function information supplied from the cognitive function estimation processing unit 33 in step S14 and the desired cognitive function information supplied from the information presentation device 24, the recommended stimulus determination unit 34 determines a recommended stimulus to be provided to the user in accordance with the cognitive function transition model shown in FIG.

[0050] In step S16, the stimulus providing device 25 provides various stimuli (taVNS, cognitive tasks, or paced breathing) to the user in accordance with the recommended stimulus information supplied from the recommended stimulus determining unit 34 in step S15.

[0051] In step S17, the cognitive function estimation processing unit 33 estimates and updates the user's cognitive function based on the brain information and peripheral information supplied after the recommended stimulus is provided by the stimulus providing device 25 in step S16, and supplies updated cognitive function information indicating the updated cognitive function to the stimulus readjustment unit 35.

[0052] In step S18, the stimulus readjustment unit 35 determines whether the user has achieved the desired cognitive function as a result of the recommended stimulus provided by the stimulus provision device 25, based on the desired cognitive function information supplied from the information presentation device 24 and the updated cognitive function information supplied from the cognitive function estimation processing unit 33 in step S17.

[0053] If the stimulus readjustment unit 35 determines in step S18 that the user has not achieved the desired cognitive function, the process proceeds to step S19. In step S19, the stimulus readjustment unit 35 adjusts parameters of the stimulus to be provided to the user and supplies the adjusted parameters to the stimulus provision device 25. Thereafter, the process returns to step S16, where the stimulus provision device 25 provides the user with a stimulus that has been readjusted by the stimulus readjustment unit 35, and the same process is repeated thereafter.

[0054] On the other hand, if the stimulation readjustment unit 35 determines in step S18 that the user has achieved the desired cognitive function, the processing ends.

[0055] By executing the cognitive function intervention process described above, the cognitive function intervention system 11 can estimate the user's current cognitive function based on the user's brain information and peripheral information and determine an appropriate recommended stimulus to be provided to the user to transition from the current cognitive function to a desired cognitive function. This allows the cognitive function intervention system 11 to more reliably transition the user to the desired cognitive function.

[0056] <Second Configuration Example of Cognitive Function Intervention System> Fig. 8 is a block diagram showing a configuration example of a second embodiment of a cognitive function intervention system to which the present technology is applied. In the cognitive function intervention system 11A shown in Fig. 8, components common to those in the cognitive function intervention system 11 in Fig. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0057] That is, the cognitive function intervention system 11A is configured with an EEG sensor 21, a peripheral sensor 22, an information processing device 23A, an information presentation device 24, and a stimulus providing device 25, and has a configuration in common with the cognitive function intervention system 11 in Figure 1 in that the stimulus providing device 25A is equipped with a brain information processing unit 31, a peripheral information processing unit 32, a cognitive function estimation processing unit 33A, a recommended stimulus determination unit 34A, and a stimulus readjustment unit 35.

[0058] On the other hand, the cognitive function intervention system 11A has a different configuration from the cognitive function intervention system 11 in FIG. 1 in that the cloud storage unit 26 is connected to the information processing device 23A via a network, and the information processing device 23A is equipped with a history recording unit 36.

[0059] For example, every time the cognitive function estimation processing unit 33A estimates the user's cognitive function, it supplies and records cognitive function information indicating that cognitive function to the history recording unit 36. Similarly, every time the recommended stimulus determination unit 34A determines a recommended stimulus to be provided to the user, it supplies and records recommended stimulus information indicating the recommended stimulus to the history recording unit 36. As a result, the history recording unit 36 ​​records a history of intervention effects that accumulates changes in the user's cognitive function and the distribution of recommended stimuli provided to the user.

[0060] The recommended stimulus determination unit 34A can then update the cognitive function transition model to improve the accuracy of transitioning the user's cognitive function by referring to the history of intervention effects recorded in the history recording unit 36. This allows the cognitive function intervention system 11A to more accurately determine an appropriate recommended stimulus to be provided to the user to transition from the current cognitive function to a desired cognitive function.

[0061] Furthermore, the cognitive function intervention system 11A can upload the history of intervention effects recorded in the history recording unit 36 ​​to the cloud storage unit 26 for storage, and can also download the intervention effect histories of many users who use various cognitive function intervention systems 11 from the cloud storage unit 26. This allows the recommended stimulus determination unit 34A to update the cognitive function transition model by referring to the intervention effect histories of many users, and the cognitive function intervention system 11A can further improve the accuracy of transitioning the user's cognitive function.

[0062] Therefore, by appropriately updating the cognitive function transition model, the cognitive function intervention system 11A can transition the user's cognitive function with high accuracy in accordance with the cognitive function transition model.

[0063] <Third Configuration Example of Cognitive Function Intervention System> Fig. 9 is a block diagram showing a configuration example of a third embodiment of a cognitive function intervention system to which the present technology is applied. In the cognitive function intervention system 11B shown in Fig. 9, components common to those in the cognitive function intervention system 11A of Fig. 8 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0064] That is, the cognitive function intervention system 11B is configured to include an EEG sensor 21, a peripheral sensor 22, an information processing device 23B, an information presentation device 24, and a stimulus provision device 25, and has a cloud storage unit 26 connected to the information processing device 23B, and the information processing device 23B includes a brain information processing unit 31, a peripheral information processing unit 32, a cognitive function estimation processing unit 33B, a recommended stimulus determination unit 34B, a stimulus readjustment unit 35, and a history recording unit 36B, in that it has a configuration in common with the cognitive function intervention system 11A of FIG. 8.

[0065] On the other hand, the cognitive function intervention system 11B is configured to include an acceleration sensor 27, and the information processing device 23B is configured to include a behavioral state estimation processing unit 37, which differs from the cognitive function intervention system 11A in FIG.

[0066] For example, the acceleration sensor 27 senses the acceleration accompanying the user's behavior and supplies the obtained acceleration data (acceleration and time) to the behavioral state estimation processing unit 37 .

[0067] The behavioral state estimation processing unit 37 performs behavioral content estimation processing to estimate the user's behavioral content (e.g., context such as walking state, sleeping state, sitting state, etc.) based on the acceleration data supplied from the acceleration sensor 27, and supplies behavioral content information indicating the behavioral content to the recommended stimulus determination unit 34B and the history recording unit 36B.

[0068] As a result, the recommended stimulus determination unit 34B can determine recommended stimuli to be provided to the user using, for example, a cognitive function transition model classified by the user's behavioral content. Furthermore, the history recording unit 36B records the user's behavioral content as a history of intervention effects, in addition to changes in the user's cognitive function and the distribution of recommended stimuli provided to the user, and the recommended stimulus determination unit 34B can update the cognitive function transition model classified by the user's behavioral content.

[0069] Therefore, by also using the user's behavioral content, the cognitive function intervention system 11B can accurately transition the user's cognitive function for each behavioral content.

[0070] <Regarding conductive materials> For example, a wireless earphone 300 as shown in Fig. 10 can be equipped with a biological sensor such as an electroencephalogram sensor 21 or a peripheral sensor 22. In the wireless earphone 300, earpiece portions 302L and 302R attached to main body 301L and main body 301R, respectively, are made of a conductive material, and earpiece portions 302L and 302R that come into contact with the outer ear can serve as electrical contact points with the living body.

[0071] Furthermore, the earpiece portion 302L and the earpiece portion 302R have multiple conductor portions, and by contacting a first surface of a first conductor portion with a portion of the inner surface of the external auditory canal that is closer to the top of the head, where the EEG intensity is relatively strong, and by contacting a second surface of a second conductor portion with a portion of the inner surface of the external auditory canal that is closer to the neck, where the EEG intensity is relatively weak, it is possible to easily measure the potential difference between the portion closer to the top of the head and the portion closer to the neck, thereby enabling more accurate detection of human brain waves.

[0072] For example, the conductive material constituting earpiece portion 302L and earpiece portion 302R includes at least one selected from the group consisting of PEDOT-PSS, polypyrrole, polyacetylene, polyphenylene vinylene, polythiophene, polythiol, polyaniline, and analogs thereof. Also, the first conductor portion and the second conductor portion are at least one thermoplastic resin selected from the group consisting of polyvinyl chloride resin, polypropylene resin, polyethylene resin, polyurethane resin, polyacetal resin, polyamide resin, polycarbonate resin, and copolymers thereof.

[0073] 11 shows an example of a head-mounted display 400 equipped with an EEG sensor 21 and a peripheral sensor 22 used in the cognitive function intervention system 11. In the head-mounted display 400, detection electrodes 403 for the EEG sensor 21 and the peripheral sensor 22 can be provided on the inner surfaces of the pad part 401 and the band part 402, for example.

[0074] 12 shows an example of a headband 500 equipped with the EEG sensor 21 and peripheral sensor 22 used in the cognitive function intervention system 11. In the headband 500, detection electrodes 503 for the EEG sensor 21 and peripheral sensor 22 can be provided, for example, on the inner surfaces of band portions 501 and 502 that come into contact with the head.

[0075] 13 shows an example of headphones 600 equipped with the EEG sensor 21 and peripheral sensor 22 used in the cognitive function intervention system 11. In the headphones 600, detection electrodes 603 for the EEG sensor 21 and peripheral sensor 22 can be provided on, for example, the inner surface of a band portion 601 that comes into contact with the head, ear pads 602, or the like.

[0076] 14 shows an example of an earphone 700 equipped with the EEG sensor 21 and peripheral sensor 22 used in the cognitive function intervention system 11. In the earphone 700, for example, detection electrodes 702 for the EEG sensor 21 and peripheral sensor 22 can be provided on an earpiece 701 that is inserted into the ear.

[0077] 15 shows an example of a watch 800 equipped with an EEG sensor 21 and a peripheral sensor 22 used in the cognitive function intervention system 11. In the watch 800, detection electrodes 804 for the EEG sensor 21 and the peripheral sensor 22 can be provided, for example, on the inner surface of a display unit 801 that displays the time, or on the inner surface of a band unit 802 (e.g., the inner surface of a buckle unit 803).

[0078] 16 shows an example of eyeglasses 900 equipped with an EEG sensor 21 and a peripheral sensor 22 used in the cognitive function intervention system 11. In the eyeglasses 900, detection electrodes 902 for the EEG sensor 21 and the peripheral sensor 22 can be provided on the inner surface of temples 901, for example.

[0079] Alternatively, the brain wave sensor 21 and peripheral sensor 22 may be mounted on, for example, gloves, rings, pencils, pens, glasses, a VR (Virtual Reality) headset, a game console controller, or the like.

[0080] <Regarding the Brain State Estimation Process> Here, the brain information processing unit 31 will be described as an example of the alertness level, which is one of the brain states of the user estimated by executing the brain state estimation process based on the user's electroencephalogram data.

[0081] It is known that alpha waves included in brain waves increase when relaxed, such as when at rest, and beta waves included in brain waves increase when active thinking or concentrating. Therefore, the brain information processing unit 31 can estimate that the user's level of alertness is high when any of the following conditions is satisfied: a first condition that the power spectrum area of ​​the frequency band of alpha waves included in the brain waves is smaller than a predetermined threshold th1; a second condition that the power spectrum area of ​​the frequency band of beta waves included in the brain waves is larger than a predetermined threshold th2; or a third condition that the power spectrum area of ​​the frequency band of theta waves included in the user's brain waves is larger than a predetermined threshold th3. In this way, in addition to independently determining the first condition, the second condition, and the third condition, the brain information processing unit 31 may also estimate that the user's level of alertness is high when a fourth condition defined by a combination of the first to third conditions (e.g., the first condition and the second condition, the first condition and the third condition, the second condition and the third condition, or the first condition, the second condition and the third condition) is satisfied.

[0082] Furthermore, when estimating the user's level of alertness using electroencephalograms, the brain information processing unit 31 can use an estimation model such as machine learning instead of the thresholds th1, th2, and th3. This estimation model is, for example, a model trained using the power spectrum of electroencephalograms when the user's level of alertness is clearly high as training data. When the power spectrum of electroencephalograms is input, for example, this estimation model estimates the user's level of alertness based on the input power spectrum of electroencephalograms. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0083] The brain information processing unit 31 may also divide the electroencephalogram into a plurality of segments on the time axis, derive a power spectrum for each segment, and derive the power spectrum area of ​​the α wave frequency band for each derived power spectrum. In this case, for example, when the derived power spectrum area is smaller than a predetermined threshold tha, it can be estimated that the user's level of arousal is high.

[0084] The brain information processing unit 31 can also estimate the user's level of arousal using, for example, an estimation model that estimates the user's level of arousal based on the derived power spectrum area. This estimation model is, for example, a model that is trained using, as training data, a power spectrum area corresponding to an obviously high level of arousal. When, for example, a power spectrum area is input, this estimation model estimates the user's level of arousal based on the input power spectrum area. This estimation model includes, for example, a neural network. This learning model may include, for example, a deep neural network such as a convolutional neural network (CNN).

[0085] In addition to using electroencephalograms, the brain information processing unit 31 may estimate the user's level of alertness using fNIRS (functional near-infrared spectroscopy), MEG (Magneto-Encephalo-Graphy), fMRI (functional magnetic resonance imaging), or the like.

[0086] <Example of Computer Configuration> Next, the above-described series of processes (information processing method) can be performed by hardware or software. When the series of processes is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

[0087] FIG. 17 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0088] The program can be recorded in advance on the hard disk 1005 or ROM 1003 as a recording medium built into the computer.

[0089] Alternatively, the program can be stored (recorded) on a removable recording medium 1011 driven by the drive 1009. Such a removable recording medium 1011 can be provided as a so-called package software. Here, examples of the removable recording medium 1011 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0090] The program can be installed into the computer from the removable recording medium 1011 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 1005. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer via a wired network such as a LAN (Local Area Network) or the Internet.

[0091] The computer includes a CPU (Central Processing Unit) 1002 , to which an input / output interface 1010 is connected via a bus 1001 .

[0092] When a user inputs a command via an input / output interface 1010 by operating an input unit 1007, the CPU 1002 executes a program stored in a read-only memory (ROM) 1003 in accordance with the command. Alternatively, the CPU 1002 loads a program stored on a hard disk 1005 into a random access memory (RAM) 1004 and executes the program.

[0093] As a result, the CPU 1002 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 1002 outputs the processing results from the output unit 1006 via the input / output interface 1010, or transmits them from the communication unit 1008, or further records them on the hard disk 1005, as necessary.

[0094] The input unit 1007 is made up of a keyboard, a mouse, a microphone, etc. The output unit 1006 is made up of an LCD (Liquid Crystal Display), a speaker, etc.

[0095] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0096] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0097] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0098] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0099] Furthermore, for example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0100] Furthermore, for example, the above-described program can be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0101] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0102] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0103] It should be noted that the present technologies described in this specification can be implemented independently and singly, unless a contradiction arises. Of course, any two or more of the present technologies can also be implemented in combination. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. Furthermore, part or all of any of the present technologies described above can also be implemented in combination with other technologies not described above.

[0104] <Examples of Combinations of Configurations> The present technology can also be configured as follows. (1) An information processing device comprising: a cognitive function estimation processing unit that estimates a current cognitive function of the user based on brain information and peripheral information obtained by sensing the user. (2) The information processing device according to (1), further comprising: a recommended stimulus determination unit that determines a recommended stimulus to be provided to the user based on the user's current cognitive function and a desired cognitive function selected by the user. (3) The information processing device according to (2), wherein the user selects a desired cognitive function in response to the current cognitive function of the user estimated by the cognitive function estimation processing unit being presented to the user, and the selected desired cognitive function is supplied to the recommended stimulus determination unit. (4) The information processing device according to (3), wherein a user interface screen that presents the user's current cognitive function displays options for selecting multiple desired cognitive functions. (5) The information processing device according to any one of (2) to (4), wherein the recommended stimulus determination unit determines, as the recommended stimulus to be provided to the user, electrical stimulation using taVNS (Transcutaneous auricular vagus nerve stimulation), a cognitive task that imparts stress to the user, or paced breathing that guides the user's breathing. (6) The information processing device according to (5), wherein the recommended stimulus determination unit determines the recommended stimulus using a cognitive function transition model that represents the relationship between stimuli to be provided to transition between each state of cognitive function and a relaxed state. (7) The information processing device according to (6), wherein the recommended stimulus determination unit determines, using the cognitive function transition model, to provide the taVNS and the cognitive task as the recommended stimuli when the user's current cognitive function is in a relaxed state and a high state is selected as the desired cognitive function. (8) The information processing device according to (6), wherein the recommended stimulus determination unit determines, using the cognitive function transition model, to provide the cognitive task as the recommended stimulus when the user's current cognitive function is in a relaxed state and a medium state is selected as the desired cognitive function.(9) The information processing device according to (6), wherein the recommended stimulus determination unit uses the cognitive function transition model to determine to provide the paced breathing and the cognitive task as the recommended stimuli when the user's current cognitive function is in a relaxed state and a low state is selected as the desired cognitive function. (10) The information processing device according to (6), wherein the recommended stimulus determination unit uses the cognitive function transition model to determine to provide the taVNS as the recommended stimulus when the user's current cognitive function is in a medium state and a high state is selected as the desired cognitive function. (11) The information processing device according to (6), wherein the recommended stimulus determination unit uses the cognitive function transition model to determine to provide the paced breathing as the recommended stimulus when the user's current cognitive function is in a high state, a medium state, or a low state and a relaxed state is selected as the desired cognitive function. (12) The information processing device according to any of (1) to (11), further comprising a brain information processing unit that estimates the brain state of the user according to electroencephalogram data obtained by sensing the user's electroencephalogram. (13) The information processing device according to (12), wherein the brain information processing unit estimates that the level of alertness, which is one of the brain states of the user, is high when any of a first condition that a power spectrum area of ​​a frequency band of alpha waves included in the user's electroencephalograms is smaller than a first threshold, a second condition that a power spectrum area of ​​a frequency band of beta waves included in the user's electroencephalograms is greater than a second threshold, or a third condition that a power spectrum area of ​​a frequency band of theta waves included in the user's electroencephalograms is greater than a third threshold, or when a fourth condition defined by a combination of the first to third conditions is satisfied. (14) The information processing device according to (12), wherein the brain information processing unit estimates the level of alertness of the user using an estimation model trained with training data that includes a power spectrum of electroencephalograms when the level of alertness, which is one of the brain states of the user, is clearly high.(15) The information processing device according to any of (2) to (14), further comprising: a stimulus readjustment unit that, when the user does not have a desired cognitive function as a result of providing the recommended stimulus, adjusts parameters of the stimulus to be provided to the user according to the deviation from the desired cognitive function and provides the adjusted stimulus to the user. (16) The information processing device according to (6), wherein the recommended stimulus determination unit updates the cognitive function transition model by referring to a history of intervention effects in which changes in the user's cognitive function and a distribution of the recommended stimuli provided to the user are accumulated. (17) The information processing device according to (6), wherein the recommended stimulus determination unit determines the recommended stimulus also using behavioral content of the user. (18) An information processing method, including: an information processing device estimating the user's current cognitive function based on brain information and peripheral information obtained by sensing the user. (19) A program for causing a computer of the information processing device to execute information processing, including estimating the user's current cognitive function based on brain information and peripheral information obtained by sensing the user.

[0105] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0106] REFERENCE SIGNS LIST 11 Cognitive function intervention system, 21 EEG sensor, 22 Peripheral sensor, 23 Information processing device, 24 Information presentation device, 25 Stimulus providing device, 26 Cloud storage unit, 27 Acceleration sensor, 31 Brain information processing unit, 32 Peripheral information processing unit, 33 Cognitive function estimation processing, 34 Recommended stimulus determination unit, 35 Stimulus readjustment unit, 36 History recording unit, 37 Behavioral state estimation processing unit

Claims

1. An information processing device comprising a cognitive function estimation processing unit that estimates a current cognitive function of a user based on brain information and peripheral information obtained by sensing the user.

2. The information processing device according to claim 1, further comprising a recommended stimulus determination unit that determines a recommended stimulus to be provided to the user based on the user's current cognitive function and a desired cognitive function selected by the user.

3. An information processing device as described in claim 2, wherein a desired cognitive function is selected by the user in response to the current cognitive function of the user estimated by the cognitive function estimation processing unit being presented to the user, and the selected desired cognitive function is supplied to the recommended stimulus determination unit.

4. The information processing device according to claim 3, wherein a user interface screen presenting the user's current cognitive function displays options for selecting a plurality of desired cognitive functions.

5. The information processing device according to claim 2, wherein the recommended stimulus determination unit determines, as the recommended stimulus to be provided to the user, electrical stimulation by taVNS (Transcutaneous auricular vagus nerve stimulation) for the user, a cognitive task that imposes stress on the user, or paced breathing that guides the user's breathing.

6. The information processing device according to claim 5, wherein the recommended stimulus determination unit determines the recommended stimulus using a cognitive function transition model that represents the relationship between stimuli to be provided to transition between each cognitive function state and a relaxed state.

7. The information processing device described in claim 6, wherein the recommended stimulus determination unit uses the cognitive function transition model to determine that when the user's current cognitive function is in a relaxed state and a high state is selected as the desired cognitive function, the taVNS and the cognitive task are to be provided as the recommended stimuli.

8. The information processing device described in claim 6, wherein the recommended stimulus determination unit uses the cognitive function transition model to determine that when the user's current cognitive function is in a relaxed state and an intermediate state is selected as the desired cognitive function, the cognitive task is to be provided as the recommended stimulus.

9. The information processing device of claim 6, wherein the recommended stimulus determination unit uses the cognitive function transition model to determine that when the user's current cognitive function is in a relaxed state and a low state is selected as the desired cognitive function, to provide the paced breathing and the cognitive task as the recommended stimuli.

10. The information processing device described in claim 6, wherein the recommended stimulus determination unit uses the cognitive function transition model to determine that when the user's current cognitive function is in a medium state and a high state is selected as the desired cognitive function, the recommended stimulus is to be provided by the taVNS.

11. The information processing device described in claim 6, wherein the recommended stimulus determination unit uses the cognitive function transition model to determine that when the user's current cognitive function is in a high state, a medium state, or a low state and a relaxed state is selected as the desired cognitive function, paced breathing is to be provided as the recommended stimulus in all cases.

12. The information processing device according to claim 1, further comprising a brain information processing unit that estimates the state of the user's brain according to electroencephalogram data obtained by sensing the user's electroencephalogram.

13. The information processing device of claim 12, wherein the brain information processing unit estimates that the user's brain state is in a high level of alertness when a first condition is satisfied that a power spectral area of ​​the frequency band of alpha waves contained in the user's brain waves is smaller than a first threshold, a second condition is satisfied that a power spectral area of ​​the frequency band of beta waves contained in the user's brain waves is larger than a second threshold, or a third condition is satisfied that a power spectral area of ​​the frequency band of theta waves contained in the user's brain waves is larger than a third threshold, or when a fourth condition defined as a combination of the first condition to the third condition is satisfied.

14. The information processing device according to claim 12, wherein the brain information processing unit estimates the level of alertness of the user using an estimation model trained on the power spectrum of electroencephalograms when the level of alertness, which is one of the brain states of the user, is clearly high, as training data.

15. An information processing device as described in claim 2, further comprising a stimulus readjustment unit which, when the user does not have the desired cognitive function as a result of the provision of the recommended stimulus, adjusts parameters of the stimulus to be provided to the user in accordance with the deviation from the desired cognitive function, and provides the adjusted stimulus to the user.

16. The information processing device according to claim 6, wherein the recommended stimulus determination unit refers to a history of intervention effects accumulated from changes in the user's cognitive function and the distribution of the recommended stimuli provided to the user, and updates the cognitive function transition model.

17. The information processing device according to claim 6, wherein the recommended stimulus determination unit determines the recommended stimulus based on the user's behavior as well.

18. An information processing method comprising: an information processing device estimating a current cognitive function of a user based on brain information and peripheral information obtained by sensing the user.

19. A program for causing a computer of an information processing device to execute information processing including estimating the current cognitive function of a user based on brain information and peripheral information obtained by sensing the user.

Images