Earphone, information processing device, and information processing method

Abstract

To provide an earphone which facilitates adhesion of a brain wave electrode during wearing.SOLUTION: The earphone includes: a housing which has elasticity on an outer layer of at least one end side thereof; a speaker which is housed in the housing; and an ear tip which is fixed on the end side having the elasticity of the housing, and has a sound guiding part through which a sound from the speaker passes and an elastic electrode for sensing brain waves of a wearer.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an earphone, an information processing device, and an information processing method. [Background technology]

[0002] Earphones that acquire electroencephalogram signals are known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-159908 A Summary of the Invention [Problem to be solved by the invention]

[0004] When acquiring an EEG signal, it is important to bring the EEG electrodes into close contact with the user in order to acquire the EEG signal with high accuracy. However, in conventional earphones, the positions of the electrodes are not determined according to the shapes of the ears and ear canals that differ from user to user, so that the EEG electrodes are not necessarily brought into close contact with the sensing positions for users with ears and ear canals of various shapes. In addition, when evaluating a state from an EEG signal, it is common to use a generalized model in which the EEG signals and states of an unspecified number of users are learned in advance, and it cannot be said that the EEG signals that differ from user to user appropriately estimate the state.

[0005] Therefore, one aspect of the present invention aims to provide an earphone that allows the EEG electrodes to be more easily attached when worn. Another aspect of the present invention aims to appropriately estimate the state of the wearer from the EEG signal using a model that is customized for the wearer. [Means for solving the problem]

[0006] An earphone in one aspect of the present invention comprises an earphone having a housing with elasticity on the outer layer at least on one end side, a speaker contained inside the housing, and an ear tip fixed to the elastic end side of the housing, the ear tip having a sound conducting portion through which sound from the speaker passes and an elastic electrode for sensing the wearer's brain waves.

[0007] In another aspect, an information processing device includes an acquisition unit that acquires an EEG signal, an estimation unit that estimates a state of the wearer of the earphones from the acquired EEG signal using a model that has learned a specified EEG signal of the wearer and the state of the wearer at the time the specified EEG signal was acquired, and a processing unit that performs processing based on the estimated state of the wearer. Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to provide earphones that allow the EEG electrodes to be more closely attached when worn. According to another aspect of the present invention, it is possible to appropriately estimate the state of the wearer from the EEG signal using a model generated for the wearer. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an entire earphone set according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an enlarged example of an earphone portion in the first embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of the right part of the earphone according to the first embodiment. [Figure 4] 3 is a diagram showing an example of a configuration of a gripping part in the first embodiment. FIG. [Diagram 5] 3 is a diagram showing an example of a configuration of a gripping part in the first embodiment. FIG. [Figure 6] FIG. 13 is a diagram showing an example of each configuration of an electroencephalogram signal processing system in a second embodiment. [Figure 7] FIG. 11 is a block diagram showing an example of a user terminal according to the second embodiment. [Figure 8] FIG. 11 is a block diagram showing an example of a server according to the second embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a good state according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a training band based on IAP in the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a training screen according to the second embodiment. [Figure 12] 13 is a flowchart showing an example of a guidance process according to the second embodiment. [Figure 13] 13 is a flowchart showing an example of a training process according to the second embodiment. [Figure 14] FIG. 13 is a diagram showing an enlarged example of an earphone according to a modified example. [Figure 15] FIG. 11 is a diagram showing an example of an earphone set in Modification Example 2. [Figure 16] FIG. 11 is a diagram showing an example of a schematic cross section of an earphone in Modification Example 2. [Figure 17] 13 is a diagram showing an example of a structure around the electrodes of a neck hanging part in Modification 2. FIG. [Figure 18] 13 is a diagram showing an example of a schematic exploded view of an earphone set according to Modification 2. FIG. [Figure 19A] FIG. 13 is a diagram showing an example of a display screen during concentration training. [Figure 19B] FIG. 13 is a diagram showing an example of a screen displaying measurement results relating to relaxation, concentration, and meditation on a daily basis. [Figure 20] FIG. 1 is a diagram showing an example of a frequency waveform during meditation and non-meditation. [Figure 21] FIG. 13 is a diagram showing an example of a frequency waveform of an electroencephalogram signal in the morning and after lunch. [Figure 22] FIG. 13 is a diagram showing an example of the positions of the electrodes on the neck hanging part in Modification 3. [Figure 23] 13 is a diagram showing the positional relationship between the carotid artery and the left electrode 162. FIG. [Figure 24] FIG. 13 is a diagram showing an example of a reference signal mixed with the pulsation of the carotid artery. [Figure 25A] FIG. 13 is a diagram showing an example of a signal during mouth breathing. [Figure 25B] FIG. 13 is a diagram showing an example of a signal during nasal breathing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and application of techniques not specified below. In other words, the present invention can be implemented with various modifications within the scope of the gist of the present invention. In addition, in the following description of the drawings, the same or similar parts are represented by the same or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. The drawings may also include parts with different dimensional relationships and ratios.

[0011] [First embodiment] An example of the earphone according to the first embodiment will be described below with reference to the drawings.

[0012] <Earphone Overview> First, an overview of the earphones according to the first embodiment will be described with reference to Figs. 1 to 3. Fig. 1 is a diagram showing an example of an entire earphone set 10 according to the first embodiment. Fig. 2 is a diagram showing an enlarged example of an earphone 100 part according to the first embodiment. Fig. 3 is a diagram showing an example of the right part of the earphone 100 according to the first embodiment. The earphone set may simply be called an earphone.

[0013] 1 to 3, the earphone set 10 includes a pair of earphones 100R, 100L, a cable 102 connected to each of the pair of earphones 100R, 100L, a first housing case 104 and a second housing case 106 provided at any position of the cable 102. The first housing case 104 and the second housing case 106 may include, for example, a communication circuit (communication interface) that communicates sound signals with other devices, an operation unit that has a function of operating the earphones 10, a power source (battery), a microphone, and the like. The cable 102 may include, for example, a plurality of signal lines that connect the first housing case 104, the second housing case 106, and each circuit in the earphones 100R(L).

[0014] The first housing case 104 and the second housing case 106 may be integrated into one. As described below, the earphones 10 may be configured as a wireless type in which the circuits and the like housed inside the first housing case 104 and the second housing case 106 are housed in the housing 112 of the earphones 10, eliminating the need for the cable 102. When there is no particular distinction between right (R) and left (L) for each configuration, the reference symbol RL is omitted.

[0015] As shown in FIGS. 2 and 3, the earphone 100 has an ear tip 110, a housing 112, a speaker 113 accommodated inside the housing 112, a joint mechanism 114, a connection part 116, a cable 118, and a grip part 120.

[0016] The ear tip 110 is attached to one end of the housing 112. At this time, one end of the housing 112 is formed of a flexible material, such as an elastic material or a rubber material. This is to prevent vibrations of the housing 112 from being transmitted to the ear tip 110. The ear tip 110 also includes a sound guide section 115 through which sound from the speaker 113 housed inside the housing 112 passes, and an elastic electrode for sensing the brain waves of the wearer. The elastic electrode is, for example, composed of the whole or part of the ear tip 110, and a rubber electrode capable of acquiring a biosignal can be used. As a result, the ear tip 110 including the elastic electrode can acquire the brain wave signal of the wearer by being in close contact with the inner wall of the ear canal.

[0017] The ear tip 110 is detachably attached to a nozzle protruding from one end of the housing 112. The sound guide 115 functions as a passage through which sound from the speaker 113 passes. The nozzle has a sound guide inside it that passes the sound output from the speaker 113, and the sound passes through the sound guide 115 of the ear tip 110 that partially overlaps the sound guide of the nozzle, and reaches the wearer's eardrum. The elastic electrode included in the ear tip 110 and the copper wire (first signal wire described later) in the housing 112 are provided as far away from the sound guide 115 as possible. For example, an elastic electrode is provided on the outer edge of the ear tip 110, and a copper wire is provided on the outer edge of the housing 112. This makes the elastic electrode and the copper wire that transmits the brainwave signal less susceptible to the vibration of the sound.

[0018] The housing 112 has elasticity on at least one outer layer on the end side. A nozzle protrudes from the elastic end side, and the ear tip 110 is attached to this nozzle. As a result, when the earphone 10 is inserted into the ear canal, the ear tip 110 and the elastic parts of the housing 112 are elastically deformed to fit the shape of the wearer's ear canal and are attached, so that the elastic parts of the ear tip 110 and the housing 112 fit the inner wall of the ear canal. As a result, the elastic electrode of the ear tip 110 that fits the inner wall of the ear canal can accurately acquire brainwave signals.

[0019] At least one end of the housing 112 has elasticity and flexibility as described above. As long as the material has such characteristics, the material forming the housing 112 is not limited, but an example is a material that is soft, has low resilience, and is durable, such as silicone rubber. The housing 112 can be elastically deformed by human force. Due to the elastic deformation of the housing 112, the nozzle is configured to be able to adjust the direction of its extension along the ear canal.

[0020] For example, when the earphone 10 is worn, the elastic end of the housing 112 first comes into contact with the outer ear and is deformed so as to be dented by the contact pressure. When the earphone 10 is worn, the ear tip 110 including the elastic electrode is positioned in the ear canal and fits closely around the entire circumference of the ear canal.

[0021] Moreover, the housing 112 has a storage space in the opposite direction to the nozzle, and the storage space has a board including a sound processing circuit and a speaker 113. The speaker 113 may be placed in the housing 112, for example, with the position of the speaker 113 taken into consideration so that the directionality of the output sound is directly directed to the eardrum in the ear canal. As an example, the speaker 113 is placed so that sound is output from the center of the housing 112. The speaker 113 has a peripheral portion covered with a cushioning material such as a foam material, and this cushioning material prevents the housing 112 and the speaker 113 from coming into direct contact with each other. This makes it difficult for vibrations to be transmitted to the housing 112 when sound is output from the speaker 113, and makes it difficult for vibrations to be transmitted to the sensor (elastic electrode) of the ear tip 110 via the housing 112. That is, it becomes possible to reduce the influence of vibrations accompanying the output of sound when sensing an electroencephalogram signal.

[0022] The joint mechanism 114 is a mechanism that connects the end of the housing 112 opposite to the nozzle to the connection part 116. For example, the joint mechanism 114 is a ball joint mechanism that can at least rotate and adjust the housing 112 in the horizontal direction (the direction of the XY plane). The joint mechanism 114 may also enable the housing 112 to rotate 360 ​​degrees, making the position of the housing 112 adjustable.

[0023] This allows the position of the housing 112 to be adjusted by the joint mechanism 114 to suit the different ear shapes of each user, and the ear tip 110 having an elastic electrode can be brought into close contact with the inner wall of the ear canal, making it possible to properly acquire brainwave signals.

[0024] The connecting portion 116 is a mechanism that connects the housing 112 and the grip portion 120. For example, the connecting portion 116 has rigidity and is made of a resin or the like. The connecting portion 116 extends from the position where it is fixed to the housing 112 in a predetermined direction, for example, in the vertically downward direction. The extending portion may have a curved shape so as to approach the ear tip 110 side.

[0025] Cable 118 includes therein a first signal line that transmits a signal sensed from the elastic electrode of ear tip 110 to processing circuit 144 (see FIGS. 4 and 5) inside gripping portion 120, and a second signal line that connects a substrate inside housing 112 to communication circuit 150 (see FIGS. 4 and 5) and transmits sound. Note that the second signal line may be multiple signal lines.

[0026] The gripping portion 120 grips the wearer's earlobe and has electrodes in the peripheral area of ​​each end. For example, the gripping portion 120 has a second electrode 140 at one end and a third electrode 142 at the other end. The second electrode 140 is an electrode that is connected to earth, and the third electrode 142 is an electrode that functions as a reference electrode. In this case, by calculating the difference between the signal sensed by the elastic electrode (first electrode) of the ear tip 110 and the signal sensed by the third electrode, which is the reference electrode, it becomes possible to acquire an electroencephalogram signal with high accuracy. This is because the signal acquired from the earlobe contains almost no electroencephalogram signal.

[0027] The gripping portion 120 has a structure for pinching the earlobe, for example, a structure like a clip. In order to fit gently to the earlobe, the gripping portion 120 is preferably made of a material having elasticity such as rubber. The gripping portion 120 does not necessarily have to be configured to pinch the earlobe, but may have a plate-like structure that can adequately contact the earlobe.

[0028] The gripping unit 120 may also include a converter that converts the electroencephalogram signal acquired based on the elastic electrode (first electrode) into a digital signal. The converter processes the electroencephalogram signal sensed at a predetermined sampling rate, for example, and converts it into a digital signal. This shortens the signal line that transmits the analog electroencephalogram signal, making it less susceptible to noise and enabling the electroencephalogram signal to be digitized quickly, thereby improving noise resistance.

[0029] In addition, each end of the gripping part 120 may have a magnet. For example, a positive magnet is provided at one end, and a negative magnet is provided at the other end. This allows the second electrode and the third electrode to be appropriately pressed against the earlobe when they are brought into contact with the earlobe, reducing the strain on the ear.

[0030] Furthermore, the connection part 116 may have an adjustment mechanism that makes it possible to adjust the position of the grip part 120. For example, the adjustment mechanism is a slide mechanism that has a mechanism for moving the position of the grip part 120 up and down in the Z direction (the vertical direction, or the direction connecting the chin and the top of the head). The adjustment mechanism may also have a mechanism that can fix the grip part 120 along a predetermined direction in which the connection part 116 extends.

[0031] This makes it possible to adjust the position of the gripping portion 120 to suit the different shapes and sizes of the earlobes of each user, and by contacting the reference electrode at an appropriate position, it becomes possible to more appropriately acquire a signal from the reference electrode.

[0032] The adjustment mechanism is not limited to a slide mechanism, and any mechanism may be adopted as long as it can adjust the position of the gripping portion 120. For example, the connection portion 116 may have a plurality of holes in a predetermined direction, and the projections of the gripping portion 120 may be fitted into the holes to fix the gripping portion 120. The connection portion 116 may also have a fixing member 122 for fixing the gripping portion 120 to the connection portion 116 after the gripping portion 120 has been moved. For example, the fixing member 122 is rotated to press the gripping portion 120 against the connection portion 116, thereby fixing the gripping portion 120. As a specific example, the fixing member 122 has a threaded portion (bolt) extending toward the gripping portion 120, and the gripping portion 120 has a nut that receives the threaded portion of the fixing member 122.

[0033] <Grip configuration> Figures 4 and 5 are diagrams showing an example of the configuration of the grip part 120 in embodiment 1. Figure 4 shows the configuration of the grip part 120 in the case of a wireless type earphone 10, and Figure 5 shows the configuration of the grip part 120 of the wireless type earphone 10.

[0034] As shown in FIG. 4, the gripping portion 120R has a second electrode 140R, a third electrode 142R, and a processing circuit 144R. As described above, the second electrode 140R is an electrode for earth connection, and the third electrode 142R is an electrode that functions as a reference electrode. The second electrode 140R and the third electrode 142R may be elastic electrodes. For example, the gripping portion 120R is a plate-shaped member that is bent near the center in the longitudinal direction of the plate-shaped member, and at least the part that contacts the ear, such as the earlobe, may have elasticity. The gripping portion 120 bends near the center to contact the earlobe in a manner that pinches the earlobe.

[0035] The processing circuit 144R includes a signal processing circuit that converts into a digital signal an electroencephalogram signal sensed by the elastic electrodes of the ear tip 110. Moreover, the grip portion 120L has a similar configuration to the grip portion 120R.

[0036] The first housing case 104 includes a communication circuit 150 and an operation unit 152. The communication circuit 150 includes an antenna for wireless communication. The antenna is compatible with wireless communication standards such as Bluetooth (registered trademark). Therefore, the earphones 10 are wirelessly connected to devices such as mobile terminals and laptops, and exchange sound data with these devices. The operation unit 152 has an operation function for controlling the volume and playback of the sound processing circuit inside the housing 112.

[0037] The second housing case 106 has a power source 154, and the power source 154 supplies power to each circuit, etc. Also, the second housing case 106 charges the power source 154 through, for example, a charging opening. The cable 102 has a signal line and transmits a signal to each circuit. Note that the configuration shown in Fig. 4 is an example, and each part may be configured to be housed in any housing case.

[0038] Fig. 5 shows a wireless type earphone 10, and illustrates an example in which the components housed in the housing case shown in Fig. 4 are housed in the grip part 120. As shown in Fig. 5, the grip part 120R has a second electrode 140R, a third electrode 142R, a processing circuit 144R, a communication circuit 150, and a power source 154. The functions of each part are the same as those shown in Fig. 4.

[0039] This makes it possible to realize a wireless type, eliminating the need for cable 102 and each housing case, and reducing the number of parts required when manufacturing earphone 10, thereby lowering manufacturing costs.

[0040] 5 is just one example, and each unit may be provided inside housing 112. For example, communication circuit 150 and processing circuit 144 may be provided in housing 112. The internal configuration of housing 112 and grip portion 120 may be determined taking into account the load on the ear.

[0041] According to the earphone 10 according to the first embodiment described above, when worn by a wearer, the elastic housing 112 can deform to fit the size and shape of the concha, so that the ear tip 110 having the elastic electrode fits along the ear canal and fits closely around the entire circumference of the ear canal. As a result, it becomes possible to appropriately sense brainwave signals from the elastic electrode.

[0042] Furthermore, the earphone 10 may be formed of not only the ear tip 110 but also all exterior parts (housing 112) that come into contact with the skin using a material that has elasticity and flexibility. This allows the earphone 10 to fit the ear, even if there are individual differences in the shape of the ear, and makes it possible to obtain a high wearing comfort, high sound insulation, and low likelihood of falling off, as well as to properly acquire electroencephalogram signals.

[0043] [Second embodiment] An example of the electroencephalogram signal processing system according to the second embodiment will be described below with reference to the drawings. <System application examples> Fig. 6 is a diagram showing an example of each configuration of the electroencephalogram signal processing system 1 in the second embodiment. In the example shown in Fig. 6, earphones 10A, 10B, ..., information processing devices 20A, 20B, ..., and a server 30 that processes electroencephalogram signals used by each user are connected via a network N. In addition, when there is no need to distinguish between the individual components, alphabetical symbols such as A and B are omitted.

[0044] The earphone 10 is the earphone 10 described in the first embodiment, but is not necessarily limited to this. The earphone 10 acquires at least one brainwave signal from each of the left and right earphones 10R and 10L, acquiring a total of two brainwave signals. Note that the number of brainwave signals does not have to be two. In addition, the earphone 10 is not limited to earphones, and may be any device that can sense brainwaves.

[0045] The information processing device 20 is, for example, a smartphone, a mobile phone (feature phone), a computer, a tablet terminal, a PDA (Personal Digital Assistant), etc. The information processing device 20 is also referred to as a user terminal 20.

[0046] The information processing device 30 is, for example, a server, and may be composed of one or more devices. The information processing device 30 processes the electroencephalogram signal and analyzes the user's state from the electroencephalogram signal using, for example, a learning function of artificial intelligence (AI). The information processing device 30 is also referred to as the server 30.

[0047] In the example shown in FIG. 6, a user (user A) using a user terminal 20A wears earphones 10A, and the earphones 10A acquire an electroencephalogram signal of user A. The earphones 10A transmit the electroencephalogram signal of user A to the user terminal 20, and an application of the user terminal 20 processes the electroencephalogram signal. At this time, the application may analyze the electroencephalogram signal using an edge AI, or transmit the electroencephalogram signal to the server 30 to acquire the analysis result by the server 30. The application provides user A with the state of user A estimated using the electroencephalogram signal, training for transitioning from a current state to a predetermined state based on the electroencephalogram signal, and the like.

[0048] <Example of configuration> 7 is a block diagram showing an example of a user terminal 20 according to the second embodiment. The user terminal 20 includes one or more processing devices (controller: CPU) 210, one or more network communication interfaces 220, a memory 230, a user interface 250, and one or more communication buses 270 for interconnecting these components.

[0049] The user interface 250 includes, for example, a display device 251 and an input device (such as a keyboard and / or a mouse or any other pointing device) 252. The user interface 250 may also be a touch panel.

[0050] Memory 230 may be, for example, a high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state storage device, or may be a non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0051] As another example, memory 230 may be one or more storage devices located remotely from CPU 210. In one embodiment, memory 230 stores the following programs, modules and data structures, or a subset thereof:

[0052] The one or more processing units (CPU) 210 read and execute a program from the memory 230 as necessary. For example, the one or more processing units (CPU) 210 may configure an acquisition unit 212, an estimation unit 214, and a processing unit 216 by executing a program stored in the memory 230.

[0053] The acquisition section 212 acquires the electroencephalogram signal output from the earphones 10 via the network communication interface 220.

[0054] The estimation unit 214 estimates the state of the wearer from the acquired electroencephalogram signal by using a model that has learned a predetermined electroencephalogram signal of the wearer of the earphones 10 and the state of the wearer at the time of acquiring the predetermined electroencephalogram signal. The model used here is a trained model that is customized to the characteristics of the wearer's individual brain by using the wearer's own electroencephalogram signal. For example, this trained model is a model that includes an inference program that trains the electroencephalogram signal using teacher data in which the state (e.g., a state defined by emotional valence (Postivie / Negative) and arousal level) designated by the wearer himself at the time of acquiring the electroencephalogram signal is used as a correct answer label, and infers the state of the wearer from the electroencephalogram signal. The trained model may also be a model trained by the server 30.

[0055] The processing unit 216 performs processing based on the state of the wearer estimated by the estimation unit 214. This makes it possible to perform processing according to the state of the wearer estimated from the electroencephalogram signal of the wearer using a model trained for the wearer, and to output processing results customized for an individual.

[0056] The trained model may be a customized model in which the wearer's EEG signal and the wearer's state at the time of acquiring the EEG signal are additionally trained on a predetermined model in which the EEG signal of another person and the state of the other person at the time of acquiring the EEG signal are trained. This allows the model to have a certain degree of estimation performance even at an early stage, and allows for personal customization as the model is used (trained).

[0057] The processing unit 216 may perform a guidance process for inducing a transition from a state of the wearer estimated based on the current electroencephalogram signal to a predetermined state of the wearer indicated by the first electroencephalogram signal, and may perform a guidance process for providing feedback to the wearer based on the current electroencephalogram signal. For example, the processing unit 216 may show or play various contents to the wearer when using the earphones 10, and have the wearer indicate a better state. For example, the processing unit 216 may have the wearer indicate a good state, such as favorable, concentrating, relaxing, or inducing sleepiness. The contents may be, for example, music, videos, or games. As an example of a method of indication, the processing unit 216 may display a UI component (icon, button, etc.) indicating a good state on the screen of the user terminal 20, and the user may operate the UI component when in a good state. Note that the contents are given labels representing characteristics such as genres. For example, when the content is a video, the labels may include kids, family, documentary, comedy, suspense, romance, action, etc., and when the content is music, the labels may include rock, pop, ballad, classical, etc. Labels can also be the type of work or the mood of the work. Type of work can include cerebral work, violent work, enjoyable for everyone, controversial work, dark work, happy work, family-friendly work, witty work, etc. Mood of work can include gentle work, intense work, etc.

[0058] Processing unit 216 associates the waveform and characteristics of the electroencephalogram signal (first electroencephalogram signal) when the wearer is in a good state with the characteristics of the content, and causes learning unit 311 (see FIG. 8) of server 30 to learn these as teacher data. Processing unit 216 causes learning unit 311 of server 30 to learn what kind of video to watch, music to listen to, or game to play to transition to a good state, by using the electroencephalogram signals before and after the transition to the first electroencephalogram signal when the wearer is in a good state, the state at the time of each electroencephalogram signal, and the characteristics of the content as teacher data, thereby making it possible to generate a learned model including an inference algorithm.

[0059] By learning the electroencephalogram signals, states, and contents before and after the transition to the good state, the processing unit 216 can grasp, from the wearer's current electroencephalogram signals, what contents should be shown to the wearer in order to transition to a good state for the wearer. The processing unit 216 informs the wearer of the analysis results using the trained model as feedback results.

[0060] 8 is a block diagram showing an example of a server 30 according to the second embodiment. The server 30 includes one or more processing units (CPUs) 310, one or more network communication interfaces 320, a memory 330, and one or more communication buses 370 for interconnecting these components.

[0061] Server 30 may optionally include a user interface 350, which may include a display device (not shown) and a keyboard and / or mouse (or some other input device, such as a pointing device, not shown).

[0062] Memory 330 may be, for example, a high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state storage device, or may be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0063] Additionally, another example of memory 330 may include one or more storage devices located remotely from CPU 310. In one embodiment, memory 330 stores the following programs, modules and data structures, or a subset thereof:

[0064] One or more processing units (CPUs) 310 read and execute a program from the memory 330 as necessary. For example, the one or more processing units (CPUs) 310 may configure a learning unit 311 by executing a program stored in the memory 330.

[0065] The learning unit 311 analyzes the state of the wearer using the electroencephalogram signal of the wearer and generates a model (first model). For example, a model combining any one of a plurality of extraction methods for extracting learning data from an electroencephalogram signal and any one of a plurality of classifiers may be used. The extraction method includes a wavelet transform, a Fourier transform, and the like, and the classifier includes, for example, a random forest, a support vector machine (SVM), a neural network, a decision tree, and the like.

[0066] <Feedback Training> It is known that the frequency band and potential tendency of electroencephalograms differ from person to person. Therefore, an individual user who is undergoing electroencephalogram measurement is asked to annotate his / her own arbitrary state (for example, a first state) using symbols, emoticons, or the like displayed on the user terminal 20. In this way, the learning unit 311 learns the electroencephalogram signal and the annotation, and it becomes possible to predict and visualize the first electroencephalogram signal corresponding to the first state and the transition from the arbitrary electroencephalogram signal to the first electroencephalogram signal. In addition, it is aimed that the user can reproduce an arbitrary state by performing a predetermined training so as to transition from an arbitrary electroencephalogram signal to the first electroencephalogram signal.

[0067] The processing unit 216 calculates the frequency band for performing electroencephalogram training for each wearer. For example, the processing unit 216 detects an Individual Alpha frequency Peak (IAP) from the electroencephalogram signal. Next, the processing unit 216 sets each of the frequency bands, delta (0.5-3 Hz), theta (4-7 Hz), alpha (8-13 Hz), beta (14-30 Hz), and gamma (30 Hz or more), for each wearer, using the IAP as an axis. The processing unit 216 obtains the ratio between the total potential (e.g., the average value of all frequency bands) of the electroencephalogram signal and the potential of each frequency band (e.g., the average value of each frequency band). Alpha waves are said to appear when the wearer is at rest, awake, and has his / her eyes closed, and theta waves, delta waves, etc. are set based on the alpha waves.

[0068] Moreover, the processing unit 216 calculates a predetermined ratio (hereinafter, also referred to as the "golden ratio") for each individual wearer. The method of calculating the golden ratio is as follows: based on the analysis result of the artificial intelligence (AI) by the server 30, an electroencephalogram signal in any good state of the individual wearer is calculated, and divided into frequency bands using a predetermined Hz in the low frequency direction and a predetermined Hz in the high frequency direction with the IAP as the axis, and the ratio between the average value of all frequency bands and the average value or representative value of each frequency band is calculated. The representative value of each frequency band may be, for example, the frequency at the peak with the highest value in each frequency band. The golden ratio may also be the ratio between the average value or representative value of each frequency band.

[0069] Fig. 9 is a diagram showing an example of a good state according to the second embodiment. The example shown in Fig. 9 shows a good state of one wearer, and may be in a different position for another user. The good state may be specified by an instruction from the wearer. An instruction from the wearer may be, for example, pressing a UI component indicating a good state on the screen of the user terminal 20 while the electroencephalogram signal is being sensed.

[0070] The processing unit 216 converts the calculated average or representative value ratio of the electroencephalogram signal in each frequency band in a good state. For example, the processing unit 216 may calculate the standard deviation of the ratio of electroencephalogram signals categorized by gender, age, and time using the electroencephalogram signals of various wearers, and may convert the ratio into the golden ratio of each band suitable for the individual, taking into account the calculated standard deviation and the golden ratio of the individual.

[0071] <Training Method> 1: The learning unit 311 analyzes the frequency characteristics of an individual based on the calculated IAP, and records the tendency of the potential. 2: The learning unit 311 learns the feature quantities of the electroencephalogram signal of an individual in any state (not only a good state, but also various other states such as concentrating, relaxing, etc.) based on the annotations of the wearer. 3: When the AI ​​in the learning unit 311 has sufficiently learned the features of the EEG signal, it becomes possible to infer an individual's state from the EEG signal. 4: The processing unit 216 converts the feature amount of the electroencephalogram signal that can now be estimated into a frequency or potential. 5: The processing unit 216 performs training to approximate a certain arbitrary state of the learned individual, for example, to approximate the value of the transformed frequency or potential. 6: In this case, the desired state may be selected at the wearer's discretion, or the system may select it automatically. 7: The music and audiovisual materials used during training will be automatically selected or recommended by the system based on past training history. 8: After the training, the processing unit 216 asks the wearer to annotate the effect again to verify it.

[0072] Fig. 10 is a diagram showing an example of a training band based on IAP in the second embodiment. The example shown in Fig. 10 shows the IAP of one wearer, and the IAP of another wearer may be located at a different frequency. Therefore, in training using an electroencephalogram signal for each individual, it is preferable to set a training band for each individual as shown in Fig. 10.

[0073] <Feedback> The processing unit 216 selects content and stimuli while providing feedback. The processing unit 216 selects optimal music, videos, etc. that lead to an ideal potential state. For example, the processing unit 216 may generate recommendations from music and video viewing histories (individual and overall).

[0074] Furthermore, the processing unit 216 selects audio / video stimuli that lead to an optimal potential state. For example, the processing unit 216 may generate recommendations from training histories (individual / overall). FIG. 11 is a diagram showing an example of a training screen according to the second embodiment. As shown in FIG. 11, the wearer can learn the state of the wearer by pressing a face mark that indicates the mood at that time while listening to music.

[0075] <Operation> Next, a description will be given of the operation of the electroencephalogram signal processing system 1. Fig. 12 is a flowchart showing an example of a guidance process according to the second embodiment. In the example shown in Fig. 12, a process of guiding a subject to a predetermined state is performed using an acquired electroencephalogram signal.

[0076] In step S102, the acquisition unit 212 acquires an electroencephalogram signal output from a sensor (for example, an elastic electrode of the earphone 10).

[0077] In step S104, the estimation unit 214 estimates the state of the wearer from the acquired electroencephalogram signal using a model that has learned a specified electroencephalogram signal of the wearer wearing the sensor and the state of the wearer at the time this specified electroencephalogram signal was acquired.

[0078] In step S106, the processing unit 216 executes a predetermined process based on the estimated state of the wearer. The predetermined process is, for example, a process of notifying the wearer of the state.

[0079] Fig. 13 is a flowchart showing an example of a training process according to the second embodiment. The example shown in Fig. 13 shows a process of performing training to transition to a predetermined state while sensing an electroencephalogram signal.

[0080] In step S202, the acquisition unit 212 acquires an electroencephalogram signal output from a sensor (for example, an elastic electrode of the earphone 10).

[0081] In step S204, the processing unit 216 calculates an individual alpha frequency peak (IAP) from the electroencephalogram signal.

[0082] In step S206, the processing unit 216 performs feedback on whether to move the IAP to a higher potential or a lower potential based on the calculated IAP.

[0083] In step S208, the processing unit 216 determines whether or not the process is to be ended based on the operation of the wearer. If the user operates to end the process (step S216-YES), the process ends, and if the user does not operate to end the process (step S216-NO), the process returns to step S202.

[0084] [Variations] Although the embodiments of the technology disclosed in the present application have been described above, the technology disclosed in the present application is not limited to the above. In addition, the programs of the information processing devices 20 and 30 of the present invention can be installed or loaded into a computer through various non-transitory recording media such as optical disks such as CD-ROMs, magnetic disks, and semiconductor memories, or by downloading via a communication network or the like.

[0085] FIG. 14 is a diagram showing an enlarged example of an earphone according to the first modification. The example of the earphone shown in FIG. 14 differs from the earphone shown in FIG. 2 in the installation direction of the gripping part 120, but the other configurations are the same as those of the earphone shown in FIG. 2. As shown in FIG. 14, the gripping parts 120RA and 120LA are provided with their longitudinal direction along the vertical Z direction. In addition, the longitudinal direction of the gripping part 120 is provided along the movement direction of the adjustment mechanism provided in the connection part 116. The part that pinches the earlobe faces the upward direction of the Z axis (toward the housing). This allows most of the gripping part 120 to be hidden on the face side of the connection part 116, thereby improving the design. In addition, by adopting the configuration of the gripping part 120 shown in FIG. 14, it is possible to reduce the size of the gripping part 120. This is because the earlobe generally has a width in the horizontal direction, so by gripping the earlobe from below, as shown in Figure 14, the longitudinal length of the folded portion of gripping portion 120 can be made shorter than when the earlobe is gripped horizontally, as shown in Figure 1.

[0086] Fig. 15 is a diagram showing an example of an earphone set 50 in Modification 2. The earphone set 50 shown in Fig. 15 does not have the earphone grip part 120 shown in Fig. 1 and Fig. 14, and instead has a neck hanging part 160 and a pair of earphones 170R, 170L. Each of the earphones 170R, 170L is connected to the neck hanging part 160 using a cable capable of signal communication, but may also be connected using wireless communication.

[0087] The neck hanging part 160 has a central member that runs along the back of the neck, and rod-shaped members (arms) 182R, 182L that have a curved shape along both sides of the neck. Electrodes 162, 164 for sensing brainwave signals are provided on the surface of the central member that contacts the back of the neck. The electrodes 162, 164 are an electrode that is connected to earth and a reference electrode, respectively. This allows the electrodes 162, 164 to be spaced apart from the elastic electrodes provided on the ear tips of the earphones, making it possible to obtain brainwave signals with high accuracy. In addition, the tip sides of the rod-shaped members 182R, 182L on both sides of the neck hanging part 160 are heavier than the base sides (the central member sides), so that the electrodes 162, 164 are appropriately pressed against the neck of the wearer. For example, weights are provided on the tip sides of the rod-shaped members 182R, 182L.

[0088] Fig. 16 is a diagram showing an example of a schematic cross section of earphone 170 in Modification 2. Earphone 170 shown in Fig. 16 is basically the same as earphone 100, but has an elastic member (e.g., urethane) 174 provided between speaker 113 and nozzle 172. Providing this elastic member 174 makes it difficult for vibrations from speaker 113 to be transmitted to the elastic electrode of ear tip 176, preventing sound interference between the elastic electrode of ear tip 176 and speaker 113.

[0089] Furthermore, ear tip 176 including an elastic electrode is located at the sound guide port, but the elasticity of the elastic electrode itself makes it possible to prevent interference from sound vibrations. Also, since an elastic member is used for the housing, this elastic member makes it difficult for sound vibrations to be transmitted to the elastic electrode of ear tip 176, making it possible to prevent interference from sound vibrations.

[0090] The earphone 170 includes an audio sound processor, and uses the audio sound processor to cut sound signals below a predetermined frequency (for example, 50 Hz) corresponding to an electroencephalogram signal. In particular, the audio sound processor cuts sound signals below 30 Hz, which is a frequency band that is likely to be characteristic of an electroencephalogram signal, but amplifies sound signals with frequencies around 70 Hz in order not to impair the base sound. This can prevent interference between sound signals and electroencephalogram signals. Also, the audio sound processor may cut a predetermined frequency only when sensing of an electroencephalogram signal is being performed. The above-mentioned audio sound processor can also be applied to the earphone 10 in the embodiment.

[0091] Fig. 17 is a diagram showing an example of the structure around the electrodes of the neck hanging part 160 in Modification 2. In the example shown in Fig. 17, an elastic member 166 is provided on the neck hanging part 160 side of each of the electrodes 162, 164. This elastic member 166 is, for example, urethane, and the elasticity of this elastic member 166 allows the electrodes 162, 164 to change position.

[0092] As a specific example, the electrodes are hemispherical in shape, and elastic member 166 is attached so as to cover the central part of the hemisphere, allowing the electrodes to change angle by 360 degrees using the elasticity of elastic member 166. Also, elastic member 166 deforms to fit the shape of the wearer's neck, making it easier for electrodes 162, 164 to be pressed against the wearer's neck.

[0093] As described above, the tip sides of the rod-shaped members 182R, 182L on both sides of the neck hanging portion 160 are heavy, so that when the neck hanging portion 160 is worn around the neck, the weight of the tip sides of the rod-shaped members 182R, 182L makes it easier for each of the electrodes 162, 164 to be pressed against the neck of the wearer.

[0094] Fig. 18 is a diagram showing an example of a schematic exploded view of earphone set 50 in Modification 2. In the example shown in Fig. 18, rod-shaped member 182R has a bellows structure on the inside of an aluminum plate located on the outside, and rod-shaped member 182R is deformable according to the shape, thickness, etc. of the wearer's neck. Also, elastic member 184R such as rubber is provided on the inside of rod-shaped member 182R around the neck, and elastic member 184R comes into contact with the neck to reduce the burden of contact on the neck and improve the fit.

[0095] In addition, the rod-shaped members 182 located on both sides of the neck hanging part 160 can be folded toward the electrode side. For example, the rod-shaped member 182R can be folded toward the electrode side or the central member side by the folding structure 186R. The rod-shaped member 182L also has a folding structure (e.g., a hinge) and can be folded toward the electrode side. This allows the neck hanging part 160 to be stored compactly.

[0096] Moreover, it is preferable that the rod-shaped members 182R, 182L are slightly curved vertically downward so that they can easily fit along the collarbone of the wearer.

[0097] <Specific example of EEG feedback training> Next, a specific example of the above-mentioned individual brainwave feedback training will be described. In this specific example, it is an application that can improve productivity by managing concentration time and break time while learning brainwave signals. In this application, for example, training is performed by dividing time into sections and changing the use case for each section. For example, the first 10 minutes are meditation (meditation use case), the next 25 minutes are a task (concentration use case), and the next 5 minutes are a break (relaxation use case), with four sets of this task and break. The final break may be 25 minutes. This time management technique is based on the time management technique of the Pomodoro Technique, which is said to improve productivity.

[0098] During time management in the specific example, the acquisition unit 212 acquires an EEG signal using the above-mentioned earphones or the like, the estimation unit 214 estimates the current state of the user in each use case based on the EEG signal using a trained model customized for the individual by the learning unit 311, and the processing unit 216 performs guidance processing to transition from the current state to a good state for the user.

[0099] As an example of feedback, the processing unit 216 controls a predetermined area of ​​the display screen of the user terminal 20, for example, the background, to be displayed in gradation using colors previously assigned to each frequency band of the electroencephalogram signal. For example, when the electroencephalogram signal is transitioning to a good state (golden ratio state), the color of the predetermined area is displayed to become gradually brighter, and when the electroencephalogram signal is not transitioning to a good state, the color of the processing area is displayed to become gradually darker. Note that, since the good state of the electroencephalogram signal differs depending on the user, the gradation is determined based on the frequency band of the electroencephalogram signal in the good state of the user. This makes it possible to display the predetermined area using gradation in accordance with the change in the electroencephalogram signal, and to visually express the current state of the electroencephalogram signal to the user.

[0100] Fig. 19A is a diagram showing an example of a display screen during concentration training. The screen shown in Fig. 19A is a screen displayed during a task in time management, showing that the current score value is "42", and "25:00" indicates the timer for the task time. In this way, the processing unit 216 uses a timer to perform time management for each use case. The processing unit 216 may also display the waveform of the frequency band of the electroencephalogram signal related to each use case.

[0101] Furthermore, the processing unit 216 may control the speaker to output a sound preset in the frequency band of the electroencephalogram signal. The preset sound may be, for example, a natural sound, such as the sound of wind, rain, waves, or forest, or may be an appropriate combination of these sounds. For example, when the electroencephalogram signal is transitioning to a good state (golden ratio state), a sound is output from the speaker so as to gradually become gentler, and when the electroencephalogram signal is not transitioning to a good state, a sound is output from the speaker so as to gradually become rougher. The transition of the electroencephalogram signal to a good state refers to a decrease in the difference between the average value or the representative value ratio in each frequency band of the current electroencephalogram signal and the golden ratio, and each frequency band is the same as each frequency band for which the golden ratio is calculated. The preset sound may be a sound. When the electroencephalogram signal acquired by the acquisition unit 212 approaches the above-mentioned golden ratio, the processing unit 216 may display an emoticon on the screen, output a sound, or display the screen in a color.

[0102] Furthermore, the application in the specific example is capable of scoring and visualizing concentration, relaxation, tension, fatigue, and the like determined from the EEG signal for a predetermined period, for example, on a monthly, weekly, daily, or session basis. Regarding the score, the estimation unit 214 scores based on the distance between the golden ratio learned using the EEG signal of each individual user and the ratio of the representative values ​​of each frequency band when the current EEG signal is divided into each frequency band, in accordance with each use case, and the processing unit 216 displays the scored value on the screen.

[0103] Fig. 19B is a diagram showing an example of a screen displaying the measurement results for Relax, Focus, and Meditation on a daily basis. The screen shown in Fig. 19B is a so-called dashboard that tabulates and visualizes data, and shows that the time determined to be relaxed was "33 minutes," the time determined to be focused was "23 minutes," and the time determined to be calm (able to meditate) was "42 minutes" based on the EEG signal measured on the 15th (Wed).

[0104] The estimation unit 214 may obtain each score value in each use case by estimation processing using a trained model. In the example shown in FIG. 19B, the score value of relaxation is "68" and the score value of concentration is "72". When the score value of each use case is equal to or greater than a corresponding predetermined value, the processing unit 216 may determine that the subject is relaxed, concentrated, or calm. Each use case that is time-managed will be described below.

[0105] (meditation) Meditation (calmness) is performed, for example, in the first time slot in the time management technique of this application. First, the processing unit 216 performs meditation guidance using audio and video. The acquisition unit 212 acquires the EEG signal after the meditation guide, and the estimation unit 214 calculates a score value based on the acquired EEG signal and the golden ratio. When the EEG signal calms down and the score value increases (when the frequency approaches the golden ratio), the processing unit 216 changes the color of the background and controls so that the wind in the background image stops and / or birds start crying.

[0106] On the other hand, when the EEG signal becomes rough and the score value becomes low (when the frequency is far from the golden ratio), the processing unit 216 controls so that the color of the background changes, the wind blows in the background image, and / or the scenery of the background image becomes stormy. The processing unit 216 may also change the background (natural environment) and the contents of the guide depending on the time (morning, afternoon, etc.) when this application is executed and the purpose.

[0107] FIG. 20 is a diagram showing an example of frequency waveforms during meditation and non-meditation. In the example shown in FIG. 20, low frequencies of delta waves and theta waves appear during meditation (Focus) compared to non-meditation (Tired). The golden ratio and the like of the individual user are obtained using the characteristics of these individual waveforms. In addition, in order to make delta waves and theta waves appear from the current frequency value, the meditation guide is performed using the above-mentioned audio and video.

[0108] (task) In the time management technique of this application, the task (concentration, fatigue) is performed after meditation, and is alternated with rest thereafter. The estimation unit 214 estimates the concentration and fatigue level of the user based on the frequency of the acquired electroencephalogram signal. For example, the estimation unit 214 estimates that fatigue is increasing when theta waves increase during the task. This is because it is generally said that delta waves and theta waves do not appear in an awake state.

[0109] Fig. 21 is a diagram showing an example of the frequency waveform of an electroencephalogram signal in the morning and after lunch. In the example shown in Fig. 21, the value of theta waves is not large in the morning (Focus), but the value of theta waves increases after lunch (Tired2) when the user feels tired or sleepy. When the value of theta waves increases, the processing unit 216 plays a sound or the like to encourage the user to concentrate in order to lower the value of theta waves. In a state of concentration, the amplitude in each frequency band is constant and there is little fluctuation, and the golden ratio appears appropriately.

[0110] (break) In the time management technique of this application, a break (rest, concentration) is performed after a task, and is alternated thereafter with the task. Since alpha waves are said to appear when the eyes are closed, the estimation unit 214 can estimate that the eyes are closed when the value of alpha waves increases. In addition, since the low frequency values ​​of delta waves and theta waves increase when the user enters a meditative state, the estimation unit 214 estimates that the user is becoming calm when the values ​​of these frequency bands increase. Furthermore, even during a break, if the user is awake, the processing unit 216 provides content that is calming for the user, which is presented by the trained model.

[0111] The frequency of the electroencephalogram signal in each use case is obtained by taking an arithmetic average of the frequency signal obtained by converting the electroencephalogram signal into a time-divided frequency signal within a predetermined time. The predetermined time may be different for each use case.

[0112] As a result, the application in the specific example can support the optimization of how time is used by using each time period divided for each use case. In addition, this application can create a playlist that helps the user concentrate during the concentration time in time management, tailored to the individual's tastes and characteristics. For example, by using a trained model that has learned the user's annotation results, it is possible to select songs that will transition to a good state (golden ratio state), and a playlist is generated in which the songs are ordered according to a predetermined standard. In addition, this application can use the trained model to recommend guides such as simple meditation and stretching, games and videos that will tone the brain, etc., according to the characteristics of the user's brainwave signal during break times in time management.

[0113] <Heart rate or respiration analysis> In the third modification, the positions of the reference electrodes in the neck hanging part 160 in the second modification are changed to measure an electrocardiogram (ECG) signal based on the pulse of the carotid artery. FIG. 22 is a diagram showing an example of the positions of the parts of the neck hanging part 160 in the third modification. The example shown in FIG. 22 shows a cross-sectional view of the neck hanging part 160 in the XY plane, and the electrodes 162, 164 are arranged so as to be located on both sides of the wearer's neck and are composed of elastic electrodes. For example, the electrodes may be arranged on both ends of a rod-shaped member or a central member located on both sides of the neck of the neck hanging part 160, or the electrodes 162, 164 may be arranged at a position a predetermined distance (for example, 3 to 7 cm) to the left and right from the center of the neck hanging part 160. A battery BA and a board ST on which a processing unit is mounted are provided at a central position inside the neck hanging part 160. By arranging the substrate ST closer to the neck and arranging the battery BA, which is shorter than the substrate ST in the X direction, on the outside of the substrate ST, the neck hanging part 160 can be made into a curved shape that fits more closely to the neck. Also, an elastic member that can be attached to the inside of the neck hanging part 160 may be provided, and the electrodes 162, 164 may be provided on this elastic member. This makes it possible to obtain a reference signal with high accuracy even when there is movement, such as during exercise, by having the elastic member adhere to the wearer's neck.

[0114] Fig. 23 is a diagram showing the positional relationship between the carotid artery and the left electrode 162. In the example shown in Fig. 23, the electrode 162 is positioned around the carotid artery B10, so that the signal sensed by the electrode 162 includes a signal based on the pulse of the carotid artery.

[0115] FIG. 24 is a diagram showing an example of a reference signal mixed with the pulsation of the carotid artery. The example shown in FIG. 24 is an example of raw data sensed from the electrode 162 or 164. In the graph shown in FIG. 24, the vertical axis represents the value of the sensed signal, and the horizontal axis represents time. As shown in FIG. 24, since the reference signal is a signal subtracted from the signal sensed by the elastic electrode of the ear tip 176, the electrocardiogram signal (ECG) mixed into the reference signal is measured in antiphase. For example, the downwardly convex peak circled in FIG. 24 is a signal indicating an electrocardiogram signal. A signal indicating the normal pulse of the wearer is synonymous with an electrocardiogram signal.

[0116] By utilizing the inclusion of a carotid artery pulse, for example, CPU 210 acquires an electroencephalogram signal based on a signal output from an elastic electrode of ear tip 176, using as a reference signal the signals output from electrodes 162, 164 provided on the neck hanging part described in modification 3. Next, CPU 210 detects a pulse based on a peak that appears in the opposite phase to the peak of the acquired electroencephalogram signal.

[0117] This allows the electrodes for measuring the reference signal to be positioned on both sides of the wearer's neck, making it possible to measure the electroencephalogram signal and the electrocardiogram signal using a common signal. In other words, the CPU 210 can analyze the heart rate by measuring the electrocardiogram signal based on the pulse while analyzing the electroencephalogram signal.

[0118] Next, an example of measuring respiration using the neck hanging section 160 in the third modification will be described. In order to measure respiration, a predetermined filtering process is performed on the electroencephalogram signal. The predetermined filtering process is performed in the following procedure. 1: Low-pass filter 2: Median filter 3: Then calculate the power spectrum value using FFT (Fast Fourier Transform) 4: Time derivative (P t -P t-1 Calculate the energy difference value (P t is the power spectrum value at time t)

[0119] Fig. 25A is a diagram showing an example of a signal during mouth breathing. Fig. 25B is a diagram showing an example of a signal during nose breathing. In the graphs shown in Figs. 25A and B, the vertical axis represents the energy difference value after filtering, and the horizontal axis represents time. In the signals shown in Figs. 25A and B, downward convex parts appear when breathing in (inhalation), and upward convex parts appear when breathing out (exhalation). One of the reasons for these peaks and valleys is thought to be that changes in the myoelectric potential of the face during breathing are mixed into each electrode of the ear tip 176.

[0120] Using the above-mentioned changes in myoelectric potential, for example, CPU 210 uses signals output from electrodes 162, 164 provided on the neck hanging part described in Modification 3 as reference signals and acquires an electroencephalogram signal based on a signal output from an elastic electrode of ear tip 176. Next, CPU 210 processes the acquired electroencephalogram signal using a low-pass filter and a median filter, frequency-converts the filtered signal, time-differentiates the time-series power spectrum value after frequency conversion, and detects respiration based on the periodicity of the difference value of this time derivative.

[0121] This makes it possible to measure a signal representing respiration by performing a predetermined filter process on the EEG signal. That is, the CPU 210 can measure a signal representing respiration while analyzing the EEG signal, and analyze the pace, magnitude, etc. of respiration.

[0122] In addition, the predetermined filter processing used for respiration detection can be applied to detect the heartbeat to remove noise and detect the heartbeat more appropriately. That is, the CPU 210 detects the peak value of the electrocardiogram that appears in the opposite phase to the peak of the electroencephalogram signal, processes the detected signal with a low-pass filter and a median filter, frequency-converts the filtered signal, time-differentiates the power spectrum value of the time series after the frequency conversion, and detects the pulse based on the periodicity of the time-differentiated difference value. This makes it possible to detect the heartbeat using the pulse from which noise has been removed.

[0123] In addition, the above-mentioned heartbeat and respiration analysis is performed by the CPU 210 detecting the heartbeat and respiration by using an autocorrelation function to find periodicity from the data values ​​shown in Fig. 24 or 25. In addition, the above-mentioned heartbeat and respiration detection process may be implemented as a program including instructions for each process and installed in an information processing device for execution. [Explanation of symbols]

[0124] 10. Earphones 20, 30 Information processing device 110 Ear tip (elastic electrode) 112 Housing 113 Speaker 114 Joint Mechanism 116 Connection 118 Cable 120 Gripping part 212 Acquisition Department 214 Estimation Department 216 Processing section 311 Learning Department

Claims

1. An earphone, a housing having elasticity on at least one outer layer of an end portion; a speaker accommodated within the housing; an ear tip attached to the elastic end of the housing, the ear tip having a sound guide portion through which sound from the speaker passes and an elastic electrode for sensing the brain waves of the wearer; An earphone equipped with:

2. The earphone of claim 1 , comprising a gripping portion for gripping an earlobe, the gripping portion having electrodes in a peripheral region at each end.

3. The earphone according to claim 1 or 2, wherein the grip portion includes a converter that converts an electroencephalogram signal acquired based on the elastic electrodes into a digital signal.

4. The earphone of claim 1 , wherein each end of the gripping portion has a magnet.

5. The earphone of claim 4 , wherein the grip portion includes a power source and a communication unit.

6. a connecting portion that connects the grip portion and the housing, The earphone according to claim 1 , wherein the connection portion has an adjustment mechanism that enables a position of the grip portion to be adjusted.

7. The connection portion extends in a predetermined direction from a position fixed to the housing, The earphone according to claim 6 , wherein the adjustment mechanism includes a mechanism that allows the grip portion to move along the predetermined direction.

8. A neck strap is provided which is connected to the earphones so as to be capable of signal communication, The neck hanging portion has a central member and rod-shaped members provided on both sides of the central member, The earphone according to claim 1 , wherein the central member or the rod-shaped member has an electrode on a surface on the side facing the neck of the wearer.

9. The earphone according to claim 1 , further comprising a processor that controls the sound signal output by the speaker, the processor cutting out a predetermined frequency or lower corresponding to the brainwave signal from the sound signal.

10. An acquisition unit that acquires an electroencephalogram signal output from the earphone according to any one of claims 1 to 9; an estimation unit that estimates a state of the wearer from the acquired electroencephalogram signal using a model that has learned a predetermined electroencephalogram signal of the wearer of the earphones and the state of the wearer at the time of acquiring the predetermined electroencephalogram signal; A processing unit that performs processing based on the estimated state of the wearer; An information processing device comprising:

11. The information processing device according to claim 10, wherein the model is a customized model that is learned by adding the electroencephalogram signal of the wearer and the state of the wearer at the time the electroencephalogram signal is acquired to a predetermined model in which the electroencephalogram signals of other people and the state of the other people at the time the electroencephalogram signals are acquired.

12. The processing unit includes: The information processing device according to claim 10 or 11, which performs an induction process to induce a transition from a state of the wearer estimated based on a current electroencephalogram signal to a predetermined state of the wearer indicated by a first electroencephalogram signal, the induction process providing feedback to the wearer based on the current electroencephalogram signal.

13. One or more processors included in the information processing device Acquiring an electroencephalogram signal output from the earphone according to any one of claims 1 to 9; estimating a state of the wearer from the acquired electroencephalogram signal using a model that has learned a predetermined electroencephalogram signal of the wearer of the earphones and the state of the wearer at the time of acquiring the predetermined electroencephalogram signal; performing processing based on the estimated state of the wearer; An information processing method for performing the above.

14. One or more processors included in the information processing device A signal output from the electrode provided on the neck hanging part according to claim 8 is used as a reference signal, and an electroencephalogram signal is obtained based on a signal output from the elastic electrode of the ear tip. detecting a pulse based on a peak that appears in an opposite phase to the peak of the electroencephalogram signal; An information processing method for performing the above.

15. One or more processors included in the information processing device A signal output from the electrode provided on the neck hanging part according to claim 8 is used as a reference signal, and an electroencephalogram signal is obtained based on a signal output from the elastic electrode of the ear tip. performing low-pass filtering and median filtering on the electroencephalogram signal, frequency-converting the filtered signal, time-differentiating the time-series power spectrum value after the frequency conversion, and detecting respiration based on the periodicity of the time-differentiated difference value; An information processing method for performing the above.

Images