Information processing device and information processing system

Abstract

An information processing device according to an embodiment of the present disclosure comprises: a measurement unit having an electrode that can be attached to a living body and capable of outputting a first signal based on the potential of the electrode; a first processing unit capable of generating a second signal related to a period in which the myoelectric potential of the living body changes; and a determination unit capable of determining the attachment state of the electrode on the basis of the first signal and the second signal.

Description

Information processing device and information processing system

[0001] The present disclosure relates to an information processing device and an information processing system.

[0002] An electroencephalogram (EEG) measurement system has been proposed that has electrodes for measuring electroencephalograms and determines whether the electrodes are properly attached or not by utilizing the amount of AC noise mixed in (Patent Document 1).

[0003] International Publication No. 2011 / 158481

[0004] It is desirable that a device for measuring electric potential be able to determine the attachment state of the electrodes.

[0005] It is desirable to provide an information processing device that can suitably determine the wearing state.

[0006] An information processing device according to an embodiment of the present disclosure includes an electrode that can be attached to a living body, a measurement unit that can output a first signal based on the potential of the electrode, a first processing unit that can generate a second signal related to a period during which the myoelectric potential of the living body changes, and a determination unit that can determine the attachment state of the electrode based on the first signal and the second signal.An information processing system according to an embodiment of the present disclosure includes an information processing device that includes an electrode that can be attached to a living body, a measurement unit that can output a first signal based on the potential of the electrode, a first processing unit that can generate a second signal related to a period during which the myoelectric potential of the living body changes, and a determination unit that can determine the attachment state of the electrode based on the first signal and the second signal.

[0007] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing device according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of the configuration of a measurement unit according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment of the present disclosure. FIG. 4A is a diagram illustrating an example of a measurement signal obtained by an information processing device according to an embodiment of the present disclosure. FIG. 4B is a diagram illustrating an example of a measurement signal obtained by an information processing device according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating an example of a determination process performed by an information processing device according to an embodiment of the present disclosure. FIG. 6A is a diagram illustrating an example of a calibration task performed by an information processing device according to an embodiment of the present disclosure. FIG. 6B is a diagram illustrating an example of a calibration task performed by an information processing device according to an embodiment of the present disclosure. FIG. 6C is a diagram illustrating an example of a calibration task performed by an information processing device according to an embodiment of the present disclosure. FIG. 7A is a diagram illustrating an example of a method for presenting a calibration task performed by an information processing device according to an embodiment of the present disclosure. FIG. 7B is a diagram illustrating an example of a method for presenting a calibration task performed by an information processing device according to an embodiment of the present disclosure. FIG. 8 is a flowchart illustrating an example of an operation of an information processing device according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment of the present disclosure. Fig. 10 is a diagram for explaining a configuration example of an information processing device according to an embodiment of the present disclosure. Fig. 11 is a diagram for explaining a configuration example of an information processing device according to an embodiment of the present disclosure. Fig. 12 is a diagram for explaining a configuration example of an information processing device according to a modified example of the present disclosure. Fig. 13 is a flowchart showing an operation example of an information processing device according to a modified example of the present disclosure.

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order: 0. Background 1. Embodiment 2. Modification

[0009] <0. Background> When measuring biometric information, such as electroencephalograms, electrodes are placed on the skin surface of the head or ears to measure potential changes due to neural activity in the brain. To accurately measure potential changes due to neural activity in the brain, it is important that the electrodes are securely attached. If the electrodes are not properly attached to the skin, potential measurement itself may not be possible, or the measurement signal may be contaminated by various electromagnetic noises present in our daily lives. This may result in an inability to accurately measure potential changes due to neural activity in the brain. Therefore, it is desirable to be able to appropriately determine the contact state of the electrodes with the skin.

[0010] 1. Embodiment Fig. 1 is a diagram illustrating an example of a schematic configuration of an information processing device according to an embodiment of the present disclosure. The information processing device 1 is a device capable of acquiring signals related to a living body (hereinafter referred to as biosignals). The information processing device 1 includes a measuring unit 100 having a plurality of electrodes that can come into contact with a living body, and is configured to detect the biosignals. The biosignals can also be referred to as signals based on a bioelectric potential, i.e., biopotential signals.

[0011] The information processing device 1 is configured to be capable of performing biomeasurements such as electroencephalography (EEG) and electrocardiogram (ECG). The information processing device 1 is, for example, a device capable of measuring biopotentials and can also be called a biopotential measuring device (or a biopotential measuring circuit). The information processing device 1 is also configured to be capable of acquiring biosignals and can also be called a biosignal acquiring device.

[0012] The biological signal is, for example, a signal related to an electric potential (voltage) generated in association with the activity of a living body. Examples include electroencephalograms (EEGs) which are signals associated with brain activity, electromyograms (EMGs) which are signals associated with muscle activity, and electrocardiograms which are signals associated with cardiac activity. The biological signal is a signal related to electroencephalograms, electromyograms, electrocardiograms, pulse waves, etc.

[0013] The information processing device 1 detects biosignals such as signals related to brain waves (EEG signals). The detected EEG signals are, for example, electrical signals corresponding to the activity state of the user's brain. The information processing device 1 measures electrical potentials (biopotential measurement) using electrodes attached to the surface of the user's body, and can measure brain waves, electromyography, and the like. The information processing device 1 acquires biosignals based on biopotentials, making it possible to check the state of the living body.

[0014] The information processing device 1 can be used, for example, in electronic devices that are worn by a user. The information processing device 1 can be applied to electronic devices that can be worn on the body, such as the ears, head, face, neck, hands, wrists, arms, legs, or chest. The information processing device 1 may also be realized as a device installed in a wearable device, such as earphones, headphones, or eyewear.

[0015] 1 , the information processing device 1 has a measurement unit 100 and a signal processing unit 110. The information processing device 1 may also have an estimation unit 120. The estimation unit 120 may be provided within the signal processing unit 110 or may be provided separately from the signal processing unit 110. The information processing device 1 (or the measurement unit 100) may be configured as, for example, a sensor (biosensor) capable of acquiring a biosignal.

[0016] The measurement unit 100 is a measurement circuit and is configured to be able to generate a biosignal. As will be described later, the measurement unit 100 has a plurality of electrodes for detecting potential. The measurement unit 100 (measurement circuit) is configured to be able to measure, for example, electroencephalograms, electromyograms, etc. The measurement unit 100 can acquire a biosignal of the user and output the biosignal to the signal processing unit 110.

[0017] The signal processing unit 110 is a signal processing circuit configured to be able to perform signal processing (information processing). The signal processing unit 110 has, for example, a processor and memory (ROM, RAM, etc.) and is configured to perform various types of signal processing. The signal processing unit 110 can read and execute a program embedded therein to perform signal processing (information processing).

[0018] The signal processing unit 110 is configured to be able to perform signal processing on the input signal. The signal processing unit 110 (signal processing circuit) is configured, for example, by a circuit that performs various signal processing on the biosignal. The signal processing unit 110 can perform various signal processing on the biosignal output from the measurement unit 100, such as noise reduction processing, frequency analysis processing, and normalization processing.

[0019] The estimation unit 120 is configured to be able to execute a process of estimating the state of a living organism using a biological signal. The estimation unit 120 is configured, for example, with a processor, a memory, etc. The estimation unit 120 can perform a process of calculating a feature amount using, for example, a biological signal after noise reduction processing. The estimation unit 120 extracts the feature amount by performing, for example, fast Fourier transform (FFT) processing or wavelet analysis processing on the biological signal. Examples of the feature amount include alpha wave components, beta wave components, gamma wave components, etc. contained in the biological signal.

[0020] The estimation unit 120 performs processing to estimate the state of the living organism based on the calculation results of the feature amounts. The estimation unit 120 estimates a state such as whether the living organism is relaxed or concentrated based on, for example, alpha wave components and beta wave components. The estimation unit 120 can be configured to be able to determine the state of the living organism.

[0021] As another example, the estimation unit 120 may determine whether a living organism is asleep using a biosignal related to an electroencephalogram. Furthermore, for example, the estimation unit 120 may estimate a heart rate using a biosignal related to an electrocardiogram. The estimation unit 120 may also analyze the biosignal to estimate the emotion of the living organism. In this way, analyzing the biosignal makes it possible to confirm the state of the living organism.

[0022] The information processing device 1 may be configured to acquire information (context information) related to the state or situation of the user, and execute a process of estimating the state of the living body, a process of suggesting (presenting) an action to the user, etc. The context information includes, for example, information related to the user's actions (movements), information related to the user's position, information related to the time, information related to sound, etc.

[0023] The information processing device 1 (or the measurement unit 100) may have, as a sensor unit, various sensors capable of acquiring context information, such as an acceleration sensor, a gyro sensor (angular velocity sensor), a GPS sensor, a sound sensor, etc. The sensor unit may be provided within the information processing device 1 or may be provided separately from the information processing device 1.

[0024] The estimation unit 120 is configured to be able to generate information indicating the state of a living organism (hereinafter referred to as state information). The estimation unit 120 can generate and output the state information as an estimation result. The state information includes, for example, information indicating whether the living organism is in a relaxed state, information indicating whether the living organism is in a sleeping state, information indicating the psychological state of the living organism, such as emotions, information indicating the heart rate, etc.

[0025] The status information may be used, for example, to display an image indicating the status of the living body, or to output a sound indicating the status of the living body. As an example, the information processing device 1 may have a display unit capable of displaying an image based on the status information. The display unit may be, for example, an organic EL display, a liquid crystal display, or the like. The display unit may include a touch panel.

[0026] The information processing device 1 may have a sound output unit, such as a speaker, that can output sounds (music (BGM), sound effects, etc.) based on the status information. The information processing device 1 may be configured to be able to provide vibrations or the like to the user based on the status information.

[0027] The information processing device 1 is configured to include a presentation unit (display unit, sound output unit, etc.) that can present (provide) images, sounds, vibrations, etc. to the user. The information processing device 1 can be configured to determine (decide) an action recommended to the user using biometric information and context information. The information processing device 1 may display an image encouraging the recommended action on the display unit, or may output an audio message encouraging the recommended action by the sound output unit.

[0028] At least one or both of the signal processing unit 110 and the estimation unit 120 may be provided in a device external to the information processing device 1. Examples of the external device include an electronic device that is a terminal device (terminal) used by a user, a server (e.g., a cloud server), etc. Examples of the electronic device include a smartphone, a tablet terminal, a wearable terminal, a computer, or other information processing device.

[0029] The information processing device 1 and the external device can be collectively referred to as an information processing device. The information processing device 1 and the external device can transmit and receive signals (information) via wireless communication or wired communication. For example, the information processing device 1 is connectable to a network and configured to be able to communicate with external devices. The information processing device 1 and the external device connected via a network can be collectively referred to as an information processing device or an information processing system.

[0030] 2 is a diagram illustrating an example of the configuration of a measurement unit according to an embodiment. The measurement unit 100 has electrodes used for detecting potential and is configured to be able to measure biopotentials. The measurement unit 100 has, for example, multiple electrodes 10 (a signal electrode 10a and a reference electrode 10b in FIG. 2), an amplifier circuit 30, and an AD conversion circuit 40.

[0031] The signal electrode 10a and the reference electrode 10b are configured to be attachable to a living body. The signal electrode 10a and the reference electrode 10b are provided at a distance from each other and contact different positions. The signal electrode 10a can be placed at any position from which a biological signal is to be acquired. The reference electrode 10b can be placed at any position, including near the signal electrode 10a.

[0032] The measuring unit 100 detects the potential (voltage) of the surface of the living body using the signal electrode 10 a and the reference electrode 10 b. For example, a potential difference may occur between the signal electrode 10 a and the reference electrode 10 b that are in contact with the skin of the living body due to electricity generated inside the living body.

[0033] The signal electrode 10a and the reference electrode 10b are each made of a conductive material. For example, the signal electrode 10a and the reference electrode 10b are made of silver-silver chloride (Ag / AgCl), aluminum (Al), copper (Cu), gold (Au), or the like. Each of the signal electrode 10a and the reference electrode 10b may be made of, for example, a resin material that is conductive and elastic. The shape of each of the signal electrode 10a and the reference electrode 10b is not particularly limited and may be circular, elliptical, polygonal, or another shape.

[0034] As shown in the example diagrammatically in Fig. 2, the signal electrode 10a is brought into contact with a measurement site (a location to be measured) during actual use, and the potential of the contacted site is applied to the signal electrode 10a. For example, the signal electrode 10a is placed directly above the active area of ​​the living body from which a biosignal is to be measured. For example, the signal electrode 10a is electrically connected to an amplifier circuit 30, and supplies the amplifier circuit 30 with a signal Sig1, which is a signal corresponding to the potential (biopotential) of the contacted living body site. The signal Sig1 is a biosignal obtained by the signal electrode 10a.

[0035] 2, the reference electrode 10b contacts a location on the living body different from the location of the signal electrode 10a during actual use, and is given the potential of the contacted portion. The reference electrode 10b may be disposed, for example, at any location around the signal electrode 10a. The reference electrode 10b is electrically connected, for example, to the amplifier circuit 30, and supplies the amplifier circuit 30 with a signal Sig2 corresponding to the potential of the contacted portion of the living body. The signal Sig2 is a biological signal obtained by the reference electrode 10b.

[0036] The amplifier circuit 30 is configured to be able to amplify an input signal. The amplifier circuit 30 is configured, for example, using a differential amplifier circuit (differential amplifier) ​​capable of amplifying a signal. The amplifier circuit 30 (differential amplifier unit) generates a measurement signal S1 based on, for example, a signal obtained by the signal electrode 10a and a signal obtained by the reference electrode 10b. The measurement signal S1 is a biosignal based on the potential of the signal electrode 10a and the potential of the reference electrode 10b.

[0037] 2 , a signal Sig1 is input to the amplifier circuit 30 via the signal electrode 10a. A signal Sig2 is also input to the amplifier circuit 30 via the reference electrode 10b. The amplifier circuit 30 generates a measurement signal S1 based on, for example, the difference between the signal Sig1 and the signal Sig2. The amplifier circuit 30 generates the measurement signal S1 according to the difference between the potential of the signal Sig1 and the potential of the signal Sig2, and outputs the measurement signal S1 to the AD conversion circuit 40.

[0038] The AD conversion circuit 40 is configured to convert an input analog signal into a digital signal. The AD conversion circuit 40 is an ADC (Analog to Digital Converter). For example, a measurement signal S1 is input to the AD conversion circuit 40 from the amplifier circuit 30. The AD conversion circuit 40 performs AD conversion processing on the measurement signal S1, which is an analog signal input from the amplifier circuit 30.

[0039] The AD conversion circuit 40 (AD conversion unit) converts the measurement signal S1 output by the amplifier circuit 30 into a digital signal. The AD conversion circuit 40 can output the digitally converted measurement signal S1 to the signal processing unit 110 (see FIG. 1 ). The signal processing unit 110 acquires the measurement signal S1 as a biosignal from the measurement unit 100 and performs various signal processing on the measurement signal S1. The estimation unit 120 can perform processing such as generating state information using the measurement signal S1.

[0040] 3 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment. As shown in the example of FIG. 3, the information processing device 1 includes a signal processing unit 110. The information processing device 1 may also include the measurement unit 100, the detection unit 130, and the presentation unit 140.

[0041] As described above, the measurement unit 100 has electrodes 10 (e.g., a signal electrode 10a and a reference electrode 10b) that can be attached to a living body, and is configured to acquire a measurement signal S1 based on the potential of the electrodes 10. The measurement unit 100 generates the measurement signal S1 and outputs the measurement signal S1 to the signal processing unit 110.

[0042] The detection unit 130 is configured to be able to detect attachment of the electrode 10 to a living body. The detection unit 130 is configured to detect attachment of the electrode 10 to a living body in response to a signal (operation signal) based on the operation of an operation member (not shown) by a user, for example.

[0043] The detection unit 130 may be configured to detect the attachment of the electrode 10 to a living body based on the output signal of a sensor (e.g., an acceleration sensor, an optical sensor, an ultrasonic sensor, a temperature sensor, a sound sensor (microphone), etc.) that can detect the contact (or proximity) of the electrode 10 to a living body.

[0044] The detection unit 130 may be configured to include, for example, an operating member (an operation button (switch), a touch panel, etc.) and a sensor (an acceleration sensor, an optical sensor, etc.) that detects contact (or proximity) of the electrode 10. The signal processing unit 110 may also include the detection unit 130. The detection unit 130 may generate a signal (detection signal) indicating that the electrode 10 has been attached to a living body, and output the signal to the processing unit 60a of the signal processing unit 110.

[0045] The presentation unit 140 is configured to be able to present images and sounds to the user. In the example shown in Fig. 3, the presentation unit 140 has a display unit 141 that can display images and a sound output unit 142 that can output sounds. The display unit 141 is configured to display images based on image data. The display unit 141 is an organic EL display, a liquid crystal display, or the like. The display unit 141 may include a touch panel.

[0046] The sound output unit 142 is configured to output sound based on the audio data. The sound output unit 142 is a speaker or the like, and can also be considered a converter that converts the audio data, which is an electrical signal, into sound (sound waves). The sound output unit 142 can output sound corresponding to the audio data. Note that the display unit 141 and part or all of the sound output unit 142 may be configured integrally.

[0047] The signal processing unit 110 has, for example, a determination unit 50, a processing unit 60a, and a processing unit 60b. The determination unit 50 is configured to be able to determine the attachment state of the electrode 10 based on the signal output from the measurement unit 100. The determination unit 50 can determine whether the attachment state (contact state) of the electrode 10 to the living body is good or not by using the signal component of the myoelectric potential included in the measurement signal S1 measured by the measurement unit 100.

[0048] Fig. 4A is a diagram showing an example of a measurement signal when the electrode 10 is well attached, and Fig. 4B is a diagram showing an example of a measurement signal when the electrode 10 is poorly attached. In Fig. 4A and Fig. 4B, the vertical axis represents the signal level of the measurement signal S1, and the horizontal axis represents time.

[0049] When the electrode 10 is properly attached, changes in myoelectric potentials generated by the muscles of the living body are observed, as in the example shown in FIG. 4A . The signal value (amplitude) of the measurement signal S1 increases or decreases depending on tension and relaxation. For example, when the electrode 10 is attached to the ear or near the ear, the amplitude (signal level) of the measurement signal S1 increases when the facial muscles are tense, and decreases when the facial muscles are relaxed. When the electrode 10 is not properly attached, various noises (e.g., electromagnetic noise) tend to be mixed into the measurement signal S1, as in the example shown in FIG. 4B .

[0050] The determination unit 50 is configured to determine the attachment state of the electrode 10 based on, for example, the amplitude of the measurement signal S1 during a period in which the myoelectric potential of the living body changes. The determination unit 50 may also determine the attachment state of the electrode 10 based on the distribution of values ​​of the measurement signal S1 during a period in which the myoelectric potential of the living body changes. The determination unit 50 can grasp the myoelectric potential component mixed into the measurement signal S1 and determine (estimate) whether the contact state of the electrode 10 is good or bad.

[0051] The processing unit 60a is configured to be able to generate a signal (referred to as a timing signal) relating to a period during which the myoelectric potential of the living body changes. The processing unit 60a is configured to generate, for example, a timing signal S2 relating to a period during which the myoelectric potential included in the measurement signal S1 changes (transitions), and output the timing signal S2 to the determination unit 50.

[0052] The processing unit 60a generates, for example, a timing signal S2 indicating a period during which the user performs an action accompanied by a change in myoelectric potential. The period during which the user performs an action accompanied by a change in myoelectric potential may be set automatically by the information processing device 1 or may be set by the user. The processing unit 60a may generate, for example, a signal relating to the start timing and end timing of the action accompanied by a change in myoelectric potential as the timing signal S2 and output it to the determination unit 50.

[0053] The determination unit 50 is configured to be able to determine the attachment state of the electrode 10 to the living body based on the measurement signal S1 input from the measurement unit 100 and the timing signal S2 input from the processing unit 60a. The determination unit 50 is configured to determine the attachment state of the electrode 10 based on the measurement signal S1 during a period in which the myoelectric potential of the living body changes, as indicated by the timing signal S2. The determination unit 50 can determine the attachment state using, for example, the value of the measurement signal S1 during the period in which the myoelectric potential changes, the distribution (shape) of the value of the measurement signal S1, etc.

[0054] The determination unit 50 is configured to generate a signal (referred to as a determination signal) relating to the determination result of the attachment state. The determination unit 50 can generate a determination signal relating to the attachment state of the electrode 10 and output it to the processing unit 60b. The determination signal is a signal indicating the attachment state of the electrode 10, such as a signal indicating that the attachment state (contact state) of the electrode 10 is good or a signal indicating that the attachment state of the electrode 10 is bad.

[0055] The processing unit 60b is configured to be able to execute a process of presenting at least one of an image and sound based on the determination result by the determination unit 50. The processing unit 60b is configured to control the presentation unit 140, for example, in response to a determination signal output from the determination unit 50. As an example, the processing unit 60b controls the presentation unit 140 to present an image or sound indicating the attachment state of the electrode 10 to the user.

[0056] Furthermore, the processing unit 60b can output an image, a sound, or the like that prompts the user to reattach the electrode 10 (or the information processing device 1) by controlling the presentation unit 140. Note that part or all of the above-described processing units 60a and 60b may be configured integrally.

[0057] The processing unit 60a is configured to be capable of executing a process of prompting the user to perform an action accompanied by a change in myoelectric potential. The processing unit 60a is configured to be capable of executing a process of presenting at least one of an image and a sound that prompts the user to perform an action accompanied by a change in myoelectric potential (referred to as a calibration task). The processing unit 60a may be configured to execute a process of prompting the user to perform the calibration task when the detection unit 130 detects that the electrode 10 is attached.

[0058] 3, the processing unit 60a may control the presentation unit 140 to present the user with an image, sound, or the like prompting the user to perform a calibration task. The calibration task is, for example, an action (processing) for changing the myoelectric potential of a living body, and is an action of sequentially tensing and relaxing muscles in the face or neck. The processing unit 60a is configured to, for example, control the presentation unit 140 to prompt the user to perform a calibration task that includes tensing and relaxing muscles at least once.

[0059] The processing unit 60a may perform processing to display an image representing the calibration task on the display unit 141. For example, the processing unit 60a controls the display unit 141 to display an image representing the timing to tense muscles and the timing to relax muscles on the display unit 141. The processing unit 60a may also perform processing to output, from the sound output unit 142, a voice message, a sound effect (beep sound), or the like instructing (requesting) muscle tension and relaxation.

[0060] The processing unit 60a is configured to be able to generate a signal related to the duration of the calibration task as the timing signal S2. The processing unit 60a generates, for example, the timing signal S2 related to the start and end timings of the calibration task, and outputs it to the determination unit 50. The timing signal S2 can also be said to be a signal indicating the time when the user performs a movement that changes the myoelectric potential.

[0061] The determination unit 50 is configured to be able to determine the attachment state of the electrode 10 based on the measurement signal S1 during the calibration task period indicated by the timing signal S2. The determination unit 50 can determine the attachment state using, for example, the value of the measurement signal S1 during the execution period of the calibration task, the distribution of the values ​​of the measurement signal S1, etc. The information processing device 1 generates a change in the myoelectric potential that is mixed into the measurement signal S1 by the calibration task, and can accurately determine the attachment state using the measurement signal S1.

[0062] 5A and 5B are diagrams illustrating an example of a determination process performed by an information processing device according to an embodiment. Fig. 5A shows an example of a measurement signal S1 during a calibration task when the electrode 10 is properly attached. Fig. 5B shows an example of a measurement signal S1 during a calibration task when the electrode 10 is poorly attached.

[0063] The determination unit 50 is configured to be able to execute a process of calculating a feature quantity using, for example, the measurement signal S1. As an example, the determination unit 50 can calculate amplitude intensity, which is a feature quantity that changes (increases or decreases) according to myoelectric potential. The determination unit 50 is configured to calculate, for example, root mean square (RMS) as the amplitude intensity. The RMS value at a certain time can be expressed, for example, by the following equation (1).

[0064] In equation (1), x is the value of the measurement signal S1 at a certain time. N is the number of samples in a certain interval. When the device is worn properly, the feature value (RMS value) changes over time depending on muscle tension and relaxation, as shown in the example of FIG. 5(c). On the other hand, when the device is worn poorly, the feature value remains approximately constant, as shown in the example of FIG. 5(d).

[0065] ​Figures 5(e) and (f) each show the distribution of feature quantities, with the horizontal axis representing the feature quantity (RMS value) and the vertical axis representing the frequency. When the device is worn well, the feature quantity changes over time, resulting in a relatively flat distribution of feature quantities, as shown in the example of Figure 5(e). On the other hand, when the device is worn poorly, the feature quantity remains roughly constant, resulting in a distribution with a peak at a certain value, as shown in the example of Figure 5(f).

[0066] The determination unit 50 calculates, for example, kurtosis, which is an index representing the sharpness of a frequency distribution, as shown in Figures 5(e) and 5(f). For example, in the example shown in Figure 5(e), the kurtosis is -0.2, and in the example shown in Figure 5(f), the kurtosis is 1.9. The determination unit 50 can determine the attachment state of the electrode 10 by comparing the calculated kurtosis with a threshold value. Note that the threshold value used to determine whether the attachment state is good or bad can be adjusted (changed) depending on the placement of the electrode 10, how tension and relaxation are applied in the calibration task, etc.

[0067] 6A to 6C are diagrams illustrating an example of a calibration task performed by an information processing device according to an embodiment. For example, the calibration task includes at least one muscle tension and one relaxation, as shown schematically in FIG. 6A . The calibration task may also be an action in which muscle tension and relaxation are performed multiple times, as shown in the example of FIG. 6B . Increasing the number of tensions and relaxations can improve the accuracy of the process of determining the wearing state.

[0068] Furthermore, the calibration task may be set so that the time spent tensing and relaxing are approximately equal. For example, in the example shown in FIG. 6C , the proportions of time spent tensing and time spent relaxing are approximately equal. When the proportions of time spent tensing and relaxing during calibration are equal, if the wearing condition is good, the distribution of the feature amount (e.g., RMS value) will be flatter and the kurtosis will be lower. This allows for accurate determination of the wearing condition.

[0069] 6A to 6C, the processing unit 60a may generate a timing signal S2 relating to, for example, the start timing (time t1) and end timing (time t2) of the calibration task, and output the timing signal S2 to the determination unit 50. Based on the timing signal S2 output by the processing unit 60a, the determination unit 50 can analyze the measurement signal S1 during the period from time t1, which is the start timing of the calibration task, to time t2, which is the end timing, thereby making it possible to appropriately determine the attachment state of the electrode 10.

[0070] 7A and 7B are diagrams illustrating an example of a method for presenting a calibration task by an information processing device according to an embodiment. For example, the information processing device 1 may present a prompt to perform an action at a timing when the user is tense or relaxed, as in the example shown in FIG. 7A. The processing unit 60a of the information processing device 1 may display the image shown in FIG. 7A using the presentation unit 140, or may display the image shown in FIG. 7A on a display of an electronic device external to the information processing device 1.

[0071] The processing unit 60a of the information processing device 1 may present a calibration task including tension and relaxation actions using an image showing a time-series pattern and the movement of a dot, as shown in Fig. 7B. As in the example shown in Fig. 7B, by using dots (circles) or other marks that move over time, it is possible to present to the user in an easy-to-understand manner which actions to perform.

[0072] The information processing device 1 may present the calibration task using audio. For example, when relaxed, audio messages such as "Please keep your mouth half open" or "Please relax" may be output by the sound output unit 142. Furthermore, when tense, audio messages such as "Please clench your molars" or "Please bite your molars" may be output by the sound output unit 142.

[0073] The information processing device 1 may use a beep sound to prompt the user to repeatedly tense and relax at short intervals (for example, one second or less). For example, the information processing device 1 may present a message saying "Please clench your molars while a beep is playing" before the start of the calibration task, and then present a beep sound at the timing to tense up.

[0074] The processing unit 60b may cause the presentation unit 140 or an external electronic device of the information processing device 1 to output an image or sound according to the determination result by the determination unit 50. For example, the processing unit 60b may control the presentation unit 140 to present a message such as "Attachment is complete" or "The device is not properly attached, please reattach it."

[0075] Fig. 8 is a flowchart showing an example of the operation of the information processing device 1 according to the embodiment. The example of the operation of the information processing device 1 will be described with reference to the flowchart of Fig. 8. For example, after an application of the information processing device 1 is launched, the flowchart of Fig. 8 starts.

[0076] In step S11, the detection unit 130 of the information processing device 1 detects that the electrode 10 has been attached to, for example, the ear, by the user operating an operation member, etc. Based on the detection signal output from the detection unit 130, the processing unit 60a determines that the electrode 10 has been attached to the ear.

[0077] In step S12, the processing unit 60a causes the presentation unit 140 to present an image, sound, or the like indicating the calibration task. The processing unit 60a also generates a timing signal S2 indicating the period of the calibration task and outputs it to the determination unit 50.

[0078] In step S13, the determination unit 50 uses the measurement signal S1 and the timing signal S2 to determine the attachment state of the electrode 10 (for example, to calculate the above-mentioned feature amount, etc.). In step S14, the determination unit 50 determines whether the electrode 10 is attached to the ear properly.

[0079] If the determination result in step S14 is negative ("No" in step S14), the processing unit 60b and the presentation unit 140 present a prompt to the user to reattach the electrode 10, and then the process returns to step S12.

[0080] If the determination result in step S14 is positive ("Yes" in step S14), that is, if it is determined that the contact state between the ear and the electrode 10 is good, the information processing device 1 notifies the user that the device (or the electrode 10) has been attached, via the processing unit 60b and the presentation unit 140. Thereafter, the process shown in the flowchart of FIG. 8 ends.

[0081] As described above, the information processing device 1 according to this embodiment has a processing unit 60a capable of generating a timing signal S2 relating to a period during which the myoelectric potential changes. By using the measurement signal S1 and the timing signal S2, the information processing device 1 can determine the attachment state of the electrodes 10. Compared to a measurement method in which an AC current is supplied to the electrodes 10 to confirm the contact state of the electrodes 10, it is possible to suppress increases in device costs, power consumption, and the like.

[0082] The information processing device 1 according to this embodiment can quantitatively and relatively easily determine the attachment state of the electrodes 10. It is possible to perform electroencephalogram measurement while ensuring a good attachment state between the living body and the electrodes 10. It is possible to enjoy services based on highly accurate electroencephalogram data, and it is possible to improve the availability of the services.

[0083] When the information processing device 1 is applied to an ear electroencephalograph, it is possible to relatively easily determine the attachment state of the electrodes 10 by utilizing the myoelectric potential (myoelectric information) mixed into the measurement signal S1 via the electrodes 10 attached to the ears, thereby realizing a wearable sensor that can suitably determine the attachment state.

[0084] The technology disclosed herein can be applied to a variety of products. For example, the information processing device and measurement unit disclosed herein can be used in various electronic devices with biometric functions. The information processing device 1 and measurement unit 100 disclosed herein can be applied to wearable devices such as earphone devices and headphone devices.

[0085] For example, the information processing device 1 may be implemented in an earphone device such as a TWS (True Wireless Stereo) device, as shown in Fig. 9. One of the signal electrode 10a and the reference electrode 10b, for example, the signal electrode 10a, may be formed as an earpiece-type dry electrode and positioned so as to contact the ear canal during actual use. The other of the signal electrode 10a and the reference electrode 10b, for example, the reference electrode 10b, may be positioned so as to contact another part of the ear during actual use.

[0086] The information processing device 1 may be applied to a headphone device, as shown in the example of Fig. 10. For example, in the case of overhead headphones, the signal electrode 10a and the reference electrode 10b may be mounted on the surface of the earbuds. Of the two ear pads, the signal electrode 10a may be provided on one ear pad, and the reference electrode 10b may be provided on the other ear pad.

[0087] The information processing device 1 may be applied to a head-mounted display (HMD) as shown in the example of Fig. 11. Electrodes 10 (such as a signal electrode 10a and a reference electrode 10b) may be provided on a pad portion 201, a band portion 202, etc. of the head-mounted display.

[0088] The signal processing unit 110 and the estimation unit 120 (see also FIG. 1 ) may be implemented in a wearable device (such as an earphone device or a headphone device), or the signal processing unit 110 and the estimation unit 120 may be provided in an electronic device 200, as in the example shown in FIG. 9 . The electronic device 200 is, for example, a terminal device used by a user. The electronic device 200 may be a smartphone, a tablet terminal, a wearable terminal, a computer, or any other information processing device.

[0089] 9 , the information processing device 1 may be configured as an information processing system 300 including a measuring unit 100, an electronic device 200 having a signal processing unit 110, and the like. The information processing system 300 may be configured to include, for example, the above-mentioned estimation unit 120, a display unit 141, a sound output unit 142, and the like. Note that the signal processing unit 110 and the estimation unit 120 may be configured integrally. The signal processing unit 110 may be configured to include the estimation unit 120.

[0090] [Actions and Effects] The information processing device (information processing device 1) according to this embodiment has an electrode (electrode 10) that can be attached to a living body, and is equipped with a measurement unit (measurement unit 100) that can output a first signal (measurement signal S1) based on the potential of the electrode, a first processing unit (processing unit 60a) that can generate a second signal (timing signal S2) related to a period during which the myoelectric potential of the living body changes, and a determination unit (determination unit 50) that can determine the attachment state of the electrode based on the first signal and the second signal.

[0091] The information processing device 1 according to this embodiment includes a processing unit 60a capable of generating a timing signal S2, and a determination unit 50 capable of determining the attachment state of the electrodes 10 based on the measurement signal S1 and the timing signal S2. This makes it possible to realize an information processing device that can preferably determine the attachment state.

[0092] Next, a modified example of the present disclosure will be described. In the following, the same components as those in the above embodiment will be denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0093] 2. Modifications In the above-described embodiment, an example configuration of the information processing device 1 has been described, but this is merely an example, and the configuration of the information processing device 1 is not limited to the above-described example. Also, in the above, an example method for determining the wearing state has been described, but the method for determining the wearing state is not limited to the above-described example.

[0094] Fig. 12 is a diagram illustrating a configuration example of an information processing device according to a modified example of the present disclosure. As shown in the example of Fig. 12, the information processing device 1 may have an extraction unit 70 and a quality estimation unit 80. For example, the signal processing unit 110 is configured to include the extraction unit 70 and the quality estimation unit 80. The extraction unit 70 and the quality estimation unit 80 may be configured by a logic circuit (digital circuit) capable of various types of digital signal processing.

[0095] The extraction unit 70 is configured to be able to extract myoelectric components. The extraction unit 70 may extract the myoelectric components contained in the measurement signal S1 using, for example, a signal separation technique, and output the measurement signal S1 representing the myoelectric components to the determination unit 50. The determination unit 50 may perform a process of determining the wearing state using the measurement signal S1 from which unnecessary signal components have been removed, i.e., the extracted myoelectric components.

[0096] The extraction unit 70 may use the technology described in non-patent document (Chen, Xun, et al. "A novel EEMD-CCA approach to removing muscle artifacts for pervasive EEG." IEEE Sensors Journal 19.19 (2018): 8420-8431) as the signal separation technology. Removing potential signals other than myoelectric potentials, such as potential changes caused by brain activity, through signal separation processing enables the wearing state to be determined with high accuracy.

[0097] The determination unit 50 may use the integrated intensity of the measurement signal S1 in a high frequency band (e.g., 20 Hz or higher) in which the frequency characteristics of myoelectric potential are strongly reflected, as a feature used in the process of determining the attachment state. The determination unit 50 can determine whether the attachment state of the electrode 10 is good or bad, for example, by comparing the integrated intensity of the measurement signal S1 with a predetermined threshold value.

[0098] Depending on the type of calibration task, myoelectric potentials may be generated in only a small number of muscle fibers, resulting in a small number of frequencies at which signal strength increases during tension. Therefore, the determination unit 50 may use, as a feature, the integrated strength in a frequency band in which signal strengths equal to or greater than a predetermined value are generated.

[0099] The determination unit 50 may use the timing of tension and relaxation presented in the calibration task to calculate the feature amount during tension and the feature amount during relaxation separately and use these as the feature amounts. Alternatively, the determination unit 50 may determine the wearing state by comparing the feature amount distributions during tension and relaxation.

[0100] The information processing device 1 may include a quality estimation unit 80, as shown in the example in Fig. 12. The quality estimation unit 80 is configured to estimate the signal quality of the measurement signal S1. The quality estimation unit 80 may, for example, estimate the signal quality of the measurement signal S1 and output an output signal indicating the signal quality of the measurement signal S1 to the determination unit 50. The quality estimation unit 80 may, for example, output an output signal indicating a period during which the signal quality of the measurement signal S1 is estimated to be relatively good, among periods of a movement accompanied by a change in myoelectric potential, to the determination unit 50.

[0101] The determination unit 50 may determine the wearing state using the measurement signal S1 during a period when the signal quality is relatively good, based on the output signal of the quality estimation unit 80. The wearing state determination process can be performed taking into account the quality of the measurement signal S1, making it possible to perform the wearing state determination process with high accuracy.

[0102] 12 , the information processing device 1 may be configured to include a sensor unit 150. The sensor unit 150 includes sensors (such as an acceleration sensor, an optical sensor, an ultrasonic sensor, a temperature sensor, and a microphone) that can detect contact (or proximity) of the electrode 10 with a living body. The sensor unit 150 may also include a sensor that can measure the contact impedance between the living body and the electrode 10.

[0103] The determination unit 50 may determine whether the contact state of the electrode 10 is good or bad based on the measurement signal S1 and the output signal of the sensor unit 150. In this case, the determination process of the wearing state can be performed taking into account the detection result by the sensor unit 150, and the determination process of the wearing state can be performed with high accuracy.

[0104] The determination unit 50 may classify the kurtosis of the feature distribution using machine learning (learning model) and perform the process of determining the wearing state. Note that the machine learning method is not particularly limited, and various methods can be used. The kurtosis of the feature distribution, the feature, the measurement signal S1, etc. may be used as the input signal when performing machine learning.

[0105] The determination unit 50 of the information processing device 1 may calculate the feature distribution during the calibration task and determine whether or not a sufficient change in myoelectricity has been measured to determine the wearing state. If a sufficient change in myoelectricity has not been measured, a dynamic presentation of muscle tension and relaxation may be continued until the wearing state is determined to be good.

[0106] When the quality estimation unit 80 estimates that the signal quality of the measurement signal S1 during the calibration task is poor, the information processing device 1 may prompt the user to reattach the electrode 10. Furthermore, the information processing device 1 may determine whether the calibration task is being performed appropriately by the user using the output signal of the sensor unit 150. The information processing device 1 may also determine whether the calibration task is being performed appropriately based on an image captured by an external camera.

[0107] The calibration task may be started in response to a user's operation of an operating member (e.g., a button operation), or in response to attachment or detachment of the device from a charging case. Alternatively, the calibration task may be started when the sensor unit 150 detects that the device is approaching a living body.

[0108] The information processing device 1 may decide to start a calibration task when it detects that the wearing state has deteriorated based on the output signal of the quality estimation unit 80, the output signal of the sensor unit 150, or the like. Note that the calibration task is not limited to the above-described example. The action of tensing the muscles may be, for example, a movement of pursing the lips, a movement of tilting the head, or the like. Furthermore, the action of relaxing the muscles may be, for example, a movement of resting, relaxing, a movement of half-opening the mouth, or the like.

[0109] The information processing device 1 may change the content of the calibration task based on the determination result by the determination unit 50. For example, as shown in Fig. 13 , if the determination result in step S14 is negative ("No" in step S14), the information processing device 1 may change the calibration task in step S15 and present the changed calibration task to the user in step S12.

[0110] Although the present disclosure has been described above with reference to embodiments and modifications, the present technology is not limited to the above embodiments and various modifications are possible. For example, although the above modifications have been described as modifications of the above embodiments, the configurations of the modifications can be appropriately combined. Furthermore, the present disclosure is applicable not only to the human body but also to living organisms other than the human body, such as animals, such as pets and livestock.

[0111] According to an embodiment of the present disclosure, an information processing device includes an electrode that can be attached to a living body, a measurement unit that can output a first signal based on the potential of the electrode, a first processing unit that can generate a second signal related to a period during which the myoelectric potential of the living body changes, and a determination unit that can determine the attachment state of the electrode based on the first signal and the second signal. Thus, it is possible to realize an information processing device that can preferably determine the attachment state.

[0112] According to an embodiment of the present disclosure, there is provided an information processing system including an information processing device having an electrode that can be attached to a living body, a measurement unit that can output a first signal based on the potential of the electrode, a first processing unit that can generate a second signal related to a period during which the myoelectric potential of the living body changes, and a determination unit that can determine the attachment state of the electrode based on the first signal and the second signal. Thus, it is possible to realize an information processing system that can preferably determine the attachment state.

[0113] Note that the effects described in this specification are merely examples and are not limited to those described, and other effects may be present. The present disclosure may also be configured as follows: (1) An information processing device including: a measurement unit having an electrode attachable to a living body and capable of outputting a first signal based on the potential of the electrode; a first processing unit capable of generating a second signal related to a period during which the myoelectric potential of the living body changes; and a determination unit capable of determining the attachment state of the electrode based on the first signal and the second signal. (2) The information processing device described in (1), wherein the first processing unit is capable of executing a process to prompt a first movement accompanied by a change in myoelectric potential. (3) The information processing device described in (2), wherein the first processing unit is capable of generating, as the second signal, a signal related to the period of the first movement. (4) The information processing device described in (2) or (3), wherein the first processing unit is capable of generating, as the second signal, signals related to the start timing and end timing of the first movement. (5) The information processing device described in any one of (2) to (4), wherein the determination unit is capable of determining the attachment state of the electrode based on the first signal during the period of the first movement. (6) The information processing device according to any one of (2) to (5), wherein the first processing unit is capable of executing a process to prompt the first movement of tensing and relaxing facial or neck muscles. (7) The information processing device according to any one of (2) to (6), wherein the first processing unit is capable of executing a process to prompt the first movement of repeatedly tensing and relaxing facial or neck muscles. (8) The information processing device according to any one of (2) to (7), wherein the first processing unit is capable of executing a process to present at least one of an image and a sound that prompts the first movement. (9) The information processing device according to any one of (1) to (8), further including a second processing unit that is capable of executing a process to present at least one of an image and a sound based on a determination result by the determination unit. (10) The information processing device according to any one of (1) to (9), wherein the determination unit is capable of determining the attachment state of the electrodes based on the amplitude of the first signal during a period in which the myoelectric potential of the living body changes.(11) The information processing device according to any one of (1) to (10), wherein the determination unit is capable of determining the attachment state of the electrode based on a distribution of values ​​of the first signal during a period in which the myoelectric potential of the living organism changes. (12) The information processing device according to any one of (1) to (11), further comprising a detection unit capable of detecting attachment of the electrode to the living organism. (13) The information processing device according to (12), wherein the first processing unit is capable of executing processing to prompt a first action accompanied by a change in myoelectric potential when the detection unit detects that the electrode is attached. (14) The information processing device according to any one of (1) to (13), wherein the electrode is capable of coming into contact with an ear or the vicinity of the ear, and the measurement unit is capable of outputting the first signal based on the myoelectric potential. (15) An information processing system comprising: an information processing device having an electrode attachable to a living body, a measurement unit capable of outputting a first signal based on the potential of the electrode, a first processing unit capable of generating a second signal related to a period during which the myoelectric potential of the living body changes, and a determination unit capable of determining an attachment state of the electrode based on the first signal and the second signal. (16) The information processing system described in (15), wherein the first processing unit is capable of executing a process to prompt a first movement accompanied by a change in myoelectric potential. (17) The information processing system described in (16), wherein the first processing unit is capable of generating, as the second signal, a signal related to the period of the first movement.

[0114] This application claims priority based on Japanese Patent Application No. 2023-194584, filed on November 15, 2023, in the Japan Patent Office, the entire contents of which are incorporated herein by reference.

[0115] Those skilled in the art will recognize that various modifications, combinations, subcombinations, and variations may occur depending on design requirements and other factors, and are intended to be within the scope of the appended claims and their equivalents.

Claims

1. An information processing device having electrodes attachable to a living body, comprising: a measuring unit capable of outputting a first signal based on the potential of the electrodes; a first processing unit capable of generating a second signal related to a period during which the myoelectric potential of the living body changes; and a judgment unit capable of judging the attachment state of the electrodes based on the first signal and the second signal.

2. The information processing device according to claim 1, wherein the first processing unit is capable of executing a process for encouraging a first movement accompanied by a change in myoelectric potential.

3. The information processing device according to claim 2, wherein the first processing unit is capable of generating, as the second signal, a signal related to a period of the first operation.

4. The information processing device according to claim 2, wherein the first processing unit is capable of generating, as the second signal, a signal related to a start timing and an end timing of the first operation.

5. The information processing device according to claim 2, wherein the determination unit is capable of determining the attachment state of the electrodes based on the first signal during the period of the first action.

6. The information processing device according to claim 2, wherein the first processing unit is capable of executing a process for encouraging the first movement of tensing and relaxing muscles in the face or neck.

7. The information processing device according to claim 2, wherein the first processing unit is capable of executing a process for encouraging the first movement of repeatedly tensing and relaxing muscles in the face or neck.

8. The information processing device according to claim 2, wherein the first processing unit is capable of executing a process of presenting at least one of an image and a sound that prompts the user to perform the first action.

9. The information processing device according to claim 1, further comprising a second processing unit capable of executing a process of presenting at least one of an image and a sound based on a result of the determination by the determination unit.

10. The information processing device according to claim 1, wherein the determination unit is capable of determining the attachment state of the electrodes based on the amplitude of the first signal during a period in which the myoelectric potential of the living body changes.

11. The information processing device according to claim 1, wherein the determination unit is capable of determining the attachment state of the electrodes based on a distribution of values ​​of the first signal during a period in which the myoelectric potential of the living body changes.

12. The information processing device according to claim 1, further comprising a detection unit capable of detecting attachment of the electrode to the living body.

13. The information processing device according to claim 12, wherein the first processing unit is capable of executing a process for prompting a first movement accompanied by a change in myoelectric potential when the attachment of the electrode is detected by the detection unit.

14. The information processing device according to claim 1, wherein the electrode is capable of contacting the ear or the vicinity of the ear, and the measuring unit is capable of outputting the first signal based on a myoelectric potential.

15. An information processing system comprising: an information processing device having an electrode attachable to a living body, a measuring unit capable of outputting a first signal based on the potential of the electrode; a first processing unit capable of generating a second signal related to a period during which the myoelectric potential of the living body changes; and a judgment unit capable of judging the attachment state of the electrode based on the first signal and the second signal.

16. The information processing system according to claim 15, wherein the first processing unit is capable of executing a process for encouraging a first movement accompanied by a change in myoelectric potential.

17. The information processing system according to claim 16, wherein the first processing unit is capable of generating, as the second signal, a signal related to a period of the first operation.

Images