Earphones, information processing device, and information processing method

Abstract

To provide an earphone which facilitates adhesion of a brain wave electrode during wearing.SOLUTION: The earphone includes: a housing which has elasticity on an outer layer of at least one end side thereof; a speaker which is housed in the housing; and an ear tip which is fixed on the end side having the elasticity of the housing, and has a sound guiding part through which a sound from the speaker passes and an elastic electrode for sensing brain waves of a wearer.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to earphones, information processing devices, and information processing methods.

Background Art

[0002] Conventionally, earphones for acquiring electroencephalogram signals have been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When acquiring an electroencephalogram signal, it is important to closely attach an electrode for electroencephalogram to the user in order to accurately acquire the electroencephalogram signal. However, in conventional earphones, the position of the electrode is not determined according to the shapes of different ears and ear canals of users. Therefore, for users having ears and ear canals of various shapes, the electrode for electroencephalogram is not always in close contact with the sensing position. In addition, when evaluating a state from an electroencephalogram signal, it is usually evaluated using a model in which electroencephalogram signals and states of an unspecified number of users have been learned and generalized in advance, and it cannot be said that electroencephalogram signals different for each user appropriately estimate the state.

[0005] Therefore, one aspect of the present invention aims to provide earphones that are more likely to closely attach an electrode for electroencephalogram during wearing. Another aspect of the present invention aims to appropriately estimate the state of the wearer from an electroencephalogram signal using a model customized and generated for the wearer.

Means for Solving the Problems

[0006] An earphone according to one aspect of the present invention is an earphone comprising: a housing having elasticity on at least one end outer layer; a speaker housed inside the housing; and an ear tip fixed to the elastic end of the housing, the ear tip having a sound conductor through which sound from the speaker passes and an elastic electrode for sensing the wearer's brain waves.

[0007] An information processing device in another embodiment includes an acquisition unit that acquires brainwave signals, an estimation unit that estimates the wearer's state from the acquired brainwave signals using a model that has learned a predetermined brainwave signal of the wearer of the earphones and the wearer's state at the time of acquisition of the predetermined brainwave signal, and a processing unit that performs processing based on the estimated wearer's state. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to provide earphones that allow electroencephalogram (EEG) electrodes to adhere more closely when worn. Furthermore, according to another aspect of the present invention, it becomes possible to appropriately estimate the wearer's state from the EEG signal using a model generated for the wearer. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the overall earphone set in the first embodiment. [Figure 2] This figure shows a magnified example of the earphone portion in the first embodiment. [Figure 3] This figure shows an example of the right portion of the earphone in the first embodiment. [Figure 4] This figure shows an example of the configuration of the gripping part in Embodiment 1. [Figure 5] This figure shows an example of the configuration of the gripping part in Embodiment 1. [Figure 6] This figure shows examples of the configurations of the electroencephalogram signal processing system in the second embodiment. [Figure 7] This block diagram shows an example of a user terminal according to the second embodiment. [Figure 8] It is a block diagram showing an example of a server according to the second embodiment. [Figure 9] It is a diagram showing an example of a good state according to the second embodiment. [Figure 10] It is a diagram showing an example of a training band based on IAP in the second embodiment. [Figure 11] It is a diagram showing an example of a training screen according to the second embodiment. [Figure 12] It is a flowchart showing an example of a guidance process according to the second embodiment. [Figure 13] It is a flowchart showing an example of a training process according to the second embodiment. [Figure 14] It is a diagram showing an enlarged example of an earphone according to a modification example. [Figure 15] It is a diagram showing an example of an earphone set in modification example 2. [Figure 16] It is a diagram showing an example of a schematic cross-section of an earphone in modification example 2. [Figure 17] It is a diagram showing an example of the structure around an electrode of a neck-hanging part in modification example 2. [Figure 18] It is a diagram showing an example of a schematic disassembly of an earphone set in modification example 2. [Figure 19A] It is a diagram showing an example of a display screen during intensive training. [Figure 19B] It is a diagram showing an example of a screen for displaying measurement results regarding relaxation, concentration, and meditation on a daily basis. [Figure 20] It is a diagram showing an example of frequency waveforms during meditation and when not in meditation. [Figure 21] It is a diagram showing an example of frequency waveforms of electroencephalogram signals in the morning and after lunch. [Figure 22] It is a diagram showing an example of the positions of each electrode of a neck-hanging part in modification example 3. [Figure 23] It is a diagram showing the positional relationship between the carotid artery and the left electrode 162. [Figure 24] It is a diagram showing an example of a reference signal contaminated with the pulsation of the carotid artery. [Figure 25A] This figure shows an example of signals during mouth breathing. [Figure 25B] This figure shows an example of signals during nasal breathing. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. However, the embodiments described below are merely illustrative, and there is no intention to exclude various modifications or applications of techniques not explicitly stated below. That is, the present invention can be implemented in various modifications without departing from its spirit. In addition, in the following drawings, identical or similar parts are denoted by the same or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions or proportions. There may be parts in the drawings where the relationships between dimensions or proportions differ from those in the drawings.

[0011] [First Embodiment] An example of an earphone in the first embodiment will be described below with reference to the drawings.

[0012] <Overview of earphones> First, an overview of the earphones in the first embodiment will be described using Figures 1 to 3. Figure 1 is a diagram showing an example of the entire earphone set 10 in the first embodiment. Figure 2 is a diagram showing a magnified example of the earphone 100 portion in the first embodiment. Figure 3 is a diagram showing an example of the right portion of the earphone 100 in the first embodiment. The earphone set may also be simply called earphones.

[0013] In Figures 1-3, the earphone set 10 includes a pair of earphones 100R and 100L, a cable 102 connected to each of the earphones 100R and 100L, a first housing case 104 and a second housing case 106 provided at any position on the cable 102. The first housing case 104 and the second housing case 106 may include, for example, a communication circuit (communication interface) for communicating sound signals with other devices, an operating unit having a function for operating the earphones 10, a power supply (battery), a microphone, etc. The cable 102 may include, for example, multiple signal lines connecting the first housing case 104, the second housing case 106, and the various circuits within the earphones 100R(L).

[0014] The first housing case 104 and the second housing case 106 may be combined into one. Furthermore, as described later, the earphone 10 may be configured as a wireless type, with the circuits and other components housed inside the first housing case 104 and the second housing case 106 housed in the housing 112 of the earphone 10, thus eliminating the need for a cable 102. Note that when there is no particular distinction between right (R) and left (L) in each configuration, the RL designations are omitted.

[0015] As shown in Figures 2-3, the earphone 100 includes an ear tip 110, a housing 112, a speaker 113 housed inside the housing 112, a joint mechanism 114, a connecting part 116, a cable 118, and a gripping part 120.

[0016] The ear tip 110 is attached to one end of the housing 112. At this time, the one end of the housing 112 is made of a flexible material, such as an elastic material or rubber material. This is to prevent vibrations of the housing 112 from being transmitted to the ear tip 110. The ear tip 110 also includes a sound conductor 115 through which sound from the speaker 113 housed inside the housing 112 passes, and an elastic electrode that senses the wearer's brain waves. The elastic electrode can be made up of, for example, the entire ear tip 110 or a part of it, and can be a rubber electrode capable of acquiring biosignals. As a result, the ear tip 110, including the elastic electrode, can acquire the wearer's brain wave signals by making close contact with the inner wall of the ear canal.

[0017] Furthermore, the ear tip 110 is detachably attached to a nozzle protruding from one end of the housing 112. The sound conductor 115 functions as a passage through which sound from the speaker 113 passes. The nozzle also has a sound conductor inside that allows sound output from the speaker 113 to pass through, and the sound passes through the sound conductor 115 of the ear tip 110, which partially overlaps with the sound conductor of the nozzle, and reaches the wearer's eardrum. In addition, the elastic electrode included in the ear tip 110 and the copper wire (first signal wire described later) inside the housing 112 are positioned as far away from the sound conductor 115 as possible. For example, the elastic electrode is provided on the outer edge of the ear tip 110, and the copper wire is provided on the outer edge inside the housing 112. This makes the elastic electrode and the copper wire that transmits the brainwave signal less susceptible to the effects of sound vibrations.

[0018] The housing 112 has elasticity in at least one end of its outer layer. A nozzle protrudes from the elastic end, and the ear tip 110 is attached to this nozzle. As a result, when the earphone 10 is inserted into the ear canal, the elastic portion of the ear tip 110 and the housing 112 elastically deforms to conform to the shape of the wearer's ear canal, and the elastic portion of the ear tip 110 and the housing 112 fits snugly against the inner wall of the ear canal. Consequently, the elastic electrode of the ear tip 110, which fits snugly against the inner wall of the ear canal, can acquire electroencephalogram signals with high accuracy.

[0019] As described above, at least one end of the housing 112 is elastic and flexible. The material forming the housing 112 is not limited as long as it possesses these properties, but one example is a soft, low-rebound, and durable material, such as silicone rubber. The housing 112 can be elastically deformed by human force. This elastic deformation of the housing 112 allows the nozzle to be configured to adjust its orientation along the ear canal.

[0020] For example, the elastic end of the housing 112 first comes into contact with the outer ear when the earphone 10 is worn, and deforms by indenting under the contact pressure. When the earphone 10 is worn, the ear tip 110, which includes the elastic electrode, is positioned in the ear canal and makes close contact with the entire circumference of the ear canal.

[0021] Furthermore, the housing 112 has a storage space in the direction opposite to the nozzle, and this storage space contains a circuit board including the sound processing circuit and a speaker 113. The speaker 113 should be positioned within the housing 112 such that the directivity of the output sound is directed directly toward the eardrum in the ear canal. For example, the speaker 113 is positioned so that sound is output from the center of the housing 112. In addition, the periphery of the speaker 113 is covered with a cushioning material such as foam material, and this cushioning material prevents the housing 112 and the speaker 113 from coming into direct contact. As a result, vibrations when the speaker 113 outputs sound are less likely to be transmitted to the housing 112, and vibrations are less likely to be transmitted to the sensor (elastic electrode) of the ear tip 110 via the housing 112. In other words, it becomes possible to reduce the influence of vibrations associated with sound output when sensing brainwave signals.

[0022] The joint mechanism 114 is a mechanism that connects the end of the housing 112 opposite to the nozzle to the connecting part 116. For example, the joint mechanism 114 is at least a ball joint mechanism that allows the housing 112 to be rotated and adjusted horizontally (in the direction of the XY plane). Alternatively, the joint mechanism 114 may allow the housing 112 to rotate 360 ​​degrees and adjust the position of the housing 112.

[0023] This allows the joint mechanism 114 to adjust the position of the housing 112 to match the different ear shapes of each user, ensuring that the ear tip 110 with elastic electrodes adheres tightly to the inner wall of the ear canal, and enabling the proper acquisition of electroencephalogram (EEG) signals.

[0024] The connecting portion 116 is a mechanism that connects the housing 112 and the gripping portion 120. For example, the connecting portion 116 is rigid and made of resin or the like. The connecting portion 116 extends from a position fixed to the housing 112 in a predetermined direction, for example, vertically downward. The extended portion may have a curved shape that moves closer to the ear tip 110.

[0025] The cable 118 contains a first signal line that transmits signals sensed from the elastic electrodes of the ear tip 110 to the processing circuit 144 (see Figures 4 and 5) inside the gripping part 120, and a second signal line that connects the circuit board inside the housing 112 to the communication circuit 150 (see Figures 4 and 5) and transmits sound. The second signal line may consist of multiple signal lines.

[0026] The gripping portion 120 grips the wearer's earlobe and has electrodes in the peripheral area of ​​each end. For example, the gripping portion 120 has a second electrode 140 at one end and a third electrode 142 at the other end. The second electrode 140 is an electrode connected to ground, and the third electrode 142 is an electrode that functions as a reference electrode. In this case, by calculating the difference between the signal sensed by the elastic electrode (first electrode) of the ear tip 110 and the signal sensed by the third electrode of the reference electrode, it becomes possible to acquire the electroencephalogram (EEG) signal with high accuracy. This is because the signal acquired from the earlobe contains almost no EEG signal.

[0027] Furthermore, the gripping portion 120 has a structure that clamps onto the earlobe, for example, a clip-like structure. In addition, to ensure a gentle fit to the earlobe, the gripping portion 120 is preferably made of an elastic material such as rubber. Note that the gripping portion 120 does not necessarily have to be structured to clamp onto the earlobe; it is sufficient if it has a plate-like structure that can make appropriate contact with the earlobe.

[0028] Furthermore, the gripping portion 120 may include a converter that converts the electroencephalogram (EEG) signal acquired based on the elastic electrode (first electrode) into a digital signal. The converter, for example, processes the EEG signal sensed at a predetermined sampling rate and converts it into a digital signal. This shortens the signal line that transmits the analog EEG signal, making it less susceptible to noise, allowing for rapid digitization of the EEG signal and improving noise immunity.

[0029] Furthermore, each end of the gripping portion 120 may have a magnet. For example, a positive-pole magnet may be provided at one end and a negative-pole magnet at the other end. This allows for appropriate pressure to be applied to the earlobe when the second and third electrodes are brought into contact with the earlobe, thereby reducing the burden on the ear.

[0030] Furthermore, the connecting portion 116 may have an adjustment mechanism that allows the position of the gripping portion 120 to be adjusted. For example, the adjustment mechanism may be a sliding mechanism that moves the position of the gripping portion 120 up and down in the Z direction (vertical direction, or the direction connecting the jaw and the top of the head). The adjustment mechanism may also have a mechanism that can fix the gripping portion 120 along a predetermined direction in which the connecting portion 116 extends.

[0031] This makes it possible to adjust the position of the gripping part 120 to suit the different earlobe shapes and sizes of each user, and by making contact with the reference electrode at the appropriate position, it becomes possible to acquire signals from the reference electrode more accurately.

[0032] The adjustment mechanism is not limited to a sliding mechanism; any mechanism that allows the position of the gripping portion 120 to be adjusted may be used. For example, the connecting portion 116 may have a plurality of holes in a predetermined direction, and the gripping portion 120 may be fixed by fitting the projection of the gripping portion 120 into these holes. Alternatively, the connecting portion 116 may have a fixing member 122 for fixing the gripping portion 120 to the connecting portion 116 after it has been moved. For example, the fixing member 122 presses the gripping portion 120 against the connecting portion 116 by rotating it, thereby fixing the gripping portion 120. Specifically, the fixing member 122 may have a threaded portion (bolt) extending in the direction of the gripping portion 120, and the gripping portion 120 may have a nut that receives the threaded portion of the fixing member 122.

[0033] <Structure of the gripping part> Figures 4 and 5 show an example of the configuration of the gripping portion 120 in Embodiment 1. Figure 4 shows the configuration of the gripping portion 120 in the case of a wireless type earphone 10, and Figure 5 shows the configuration of the gripping portion 120 of a wireless type earphone 10.

[0034] As shown in Figure 4, the gripping portion 120R has a second electrode 140R, a third electrode 142R, and a processing circuit 144R. As described above, the second electrode 140R is an electrode for ground connection, and the third electrode 142R is an electrode that functions as a reference electrode. The second electrode 140R and the third electrode 142R may be elastic electrodes. For example, the gripping portion 120R is a plate-shaped member and has a structure that bends near the center in the longitudinal direction of the plate-shaped member, and at least the part that contacts the ear, such as the earlobe, should be elastic. The gripping portion 120 bends near the center, so as to grip the earlobe and make contact with it.

[0035] The processing circuit 144R includes a signal processing circuit that converts the electroencephalogram (EEG) signals sensed by the elastic electrodes of the ear tip 110 into digital signals. The gripping section 120L has the same configuration as the gripping section 120R.

[0036] The first housing case 104 includes a communication circuit 150 and an operating unit 152. The communication circuit 150 includes an antenna for wireless communication. The antenna is compatible with wireless communication standards such as Bluetooth®. Therefore, the earphone 10 is wirelessly connected to devices such as mobile terminals and laptops, and communicates sound data with these devices. The operating unit 152 has operating functions for controlling the volume and playback of the sound processing circuit inside the housing 112.

[0037] The second housing case 106 has a power supply 154, which supplies power to each circuit, etc. The second housing case 106 is also charged, for example, through a charging opening for the power supply 154. The cable 102 has signal lines and transmits signals to each circuit. Note that the configuration shown in Figure 4 is an example, and each part may be configured to be housed in any housing case.

[0038] Figure 5 shows a wireless earphone 10, illustrating an example where the components housed in the case shown in Figure 4 are housed in the gripping section 120. As shown in Figure 5, the gripping section 120R includes a second electrode 140R, a third electrode 142R, a processing circuit 144R, a communication circuit 150, and a power supply 154. The function of each part is the same as shown in Figure 4.

[0039] This makes it possible to create a wireless type, eliminating the need for cables 102 and each storage case, and reducing the number of parts used when manufacturing the earphones 10, thereby lowering manufacturing costs.

[0040] Furthermore, the example shown in Figure 5 is just one example, and each part may be provided inside the housing 112. For example, the communication circuit 150 and the processing circuit 144 may be provided in the housing 112. Also, the internal configuration of the housing 112 and the internal configuration of the gripping part 120 should be determined taking into consideration the load on the ear.

[0041] According to the earphone 10 of the first embodiment described above, when worn by the wearer, the elastic housing 112 can deform to match the size and shape of the concha, so that the ear tip 110 with elastic electrodes fits along the ear canal and makes close contact with the entire circumference of the ear canal. As a result, it becomes possible to appropriately sense electroencephalogram signals from the elastic electrodes.

[0042] Furthermore, the earphone 10 may be made of an elastic and flexible material not only for the ear tip 110, but also for all external parts (housing 112) that come into contact with the skin. This allows the earphone 10 to fit the ear well, even if there are individual differences in ear shape, resulting in a high level of comfort, high sound insulation, and resistance to falling out, while also enabling the proper acquisition of brainwave signals.

[0043] [Second Embodiment] Below, an example of an electroencephalogram (EEG) signal processing system in the second embodiment will be described with reference to the drawings. <Examples of system application> Figure 6 shows examples of the configurations of the electroencephalogram (EEG) signal processing system 1 in the second embodiment. In the example shown in Figure 6, each earphone 10A, 10B, ... used by each user, and the information processing devices 20A, 20B, ..., and the server 30 that processes EEG signals are connected via a network N. Note that when individual configurations are not distinguished, letters such as A and B are omitted.

[0044] The earphone 10 is the earphone 10 described in the first embodiment, but is not necessarily limited to it. The earphone 10 acquires at least one brainwave signal from each of the left and right earphones 10R and 10L, acquiring a total of two brainwave signals. Note that there do not necessarily have to be two brainwave signals. Also, the earphone 10 is not limited to earphones, but can be any device capable of sensing brainwaves.

[0045] The information processing device 20 can be, for example, a smartphone, a mobile phone (feature phone), a computer, a tablet device, or a PDA (Personal Digital Assistant). The information processing device 20 is also referred to as the user terminal 20.

[0046] The information processing device 30 is, for example, a server and may consist of one or more devices. The information processing device 30 also processes electroencephalogram (EEG) signals and analyzes the user's state from the EEG signals using, for example, the learning function of artificial intelligence (AI). The information processing device 30 is also referred to as the server 30.

[0047] In the example shown in Figure 6, a user (User A) using user terminal 20A wears earphones 10A, and earphones 10A acquire User A's brainwave signals. Earphones 10A transmit User A's brainwave signals to user terminal 20, and the application on user terminal 20 processes the brainwave signals. At this time, the application may analyze the brainwave signals using edge AI, or transmit the brainwave signals to server 30 to obtain analysis results from server 30. The application provides User A with information such as User A's state estimated using the brainwave signals, and training to transition from the current state to a predetermined state based on the brainwave signals.

[0048] <Example of configuration> Figure 7 is a block diagram showing an example of a user terminal 20 according to the second embodiment. The user terminal 20 includes one or more processing units (control units: CPU) 210, one or more network communication interfaces 220, memory 230, user interface 250, and one or more communication buses 270 for interconnecting these components.

[0049] The user interface 250 includes, for example, a display device 251 and an input device (such as a keyboard and / or mouse or some other pointing device) 252. Alternatively, the user interface 250 may be a touch panel.

[0050] The memory 230 is, for example, a high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory, and may also be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices.

[0051] Another example of memory 230 may be one or more storage devices located remotely from the CPU 210. In one embodiment, memory 230 stores the following programs, modules, and data structures, or subsets thereof:

[0052] One or more processing units (CPUs) 210 read and execute a program from the memory 230 as needed. For example, one or more processing units (CPUs) 210 may configure the acquisition unit 212, estimation unit 214, and processing unit 216 by executing a program stored in the memory 230.

[0053] The acquisition unit 212 acquires the brainwave signals output from the earphone 10 via the network communication interface 220.

[0054] The estimation unit 214 estimates the wearer's state from the acquired brainwave signals using a model that has learned a predetermined brainwave signal of the wearer of the earphones 10 and the wearer's state at the time of acquisition of this predetermined brainwave signal. The model used here is a pre-trained model customized to the wearer's individual brain characteristics using the wearer's own brainwave signals. For example, this pre-trained model is a model that learns brainwave signals using training data in which the wearer's own indication of their state (e.g., a state defined by positive / negative valence and arousal level) at the time of brainwave signal acquisition is used as the correct label, and includes an inference program that infers the wearer's state from the brainwave signals. Alternatively, the pre-trained model may be a model learned by the server 30.

[0055] The processing unit 216 performs processing based on the wearer's state estimated by the estimation unit 214. This enables processing according to the wearer's state estimated from the wearer's electroencephalogram (EEG) signals, using a model learned for the wearer, and allows for the output of personalized processing results.

[0056] The trained model may be a customized model in which the wearer's brainwave signals and the wearer's state at the time of acquisition are added to a predetermined model that has been trained with another person's brainwave signals and the other person's state at the time of acquisition. This allows the model to have a certain degree of estimation performance even in the initial stages, and can be customized for the individual as it is used (trained).

[0057] The processing unit 216 performs induction processing to guide the wearer to a predetermined state indicated by the first brainwave signal, based on the wearer's state estimated from the current brainwave signal. It may also perform induction processing that provides feedback to the wearer based on the current brainwave signal. For example, when the earphones 10 are used, the processing unit 216 may show or let the wearer listen to various content and ask the wearer to indicate a better state. For example, the processing unit 216 may ask the wearer to indicate a good state such as pleasant, focused, relaxed, or sleepy. The content may be music, videos, games, etc. As an example of how instructions are given, the processing unit 216 may display UI components (icons, buttons, etc.) indicating a good state on the screen of the user terminal 20, and the user may operate the UI component when in a good state. The content is assigned labels that represent characteristics such as genre. For example, if the content is a video, the labels may include kids, family, documentary, comedy, suspense, romance, action, etc. If the content is music, the labels may include rock, pop, ballad, classical, etc. The label can also describe the type or atmosphere of the work. Types of works include intellectual works, violent works, works for everyone, controversial works, dark works, happy works, family-friendly works, witty works, etc. Atmospheres of works can include gentle works, intense works, etc.

[0058] The processing unit 216 associates the waveform and characteristics of the brainwave signal (first brainwave signal) when the wearer is instructed to be in a good state with the characteristics of the content, and uses these as training data to train the learning unit 311 of the server 30 (see Figure 8). By training the learning unit 311 of the server 30 with the brainwave signals before and after the transition to the first brainwave signal when the wearer is in a good state, the state at each brainwave signal, and the characteristics of the content as training data, the processing unit 216 can learn what kind of videos to watch, music to listen to, or games to play to transition to a good state, and generate a trained model including an inference algorithm.

[0059] The processing unit 216 learns the brainwave signals, states, and content before and after the transition to this favorable state, and based on the wearer's current brainwave signals, it can determine what content should be shown to the wearer in order to transition them to a favorable state. The processing unit 216 then informs the wearer of the analysis results using the learned model as feedback.

[0060] Figure 8 is a block diagram showing an example of a server 30 according to the second embodiment. The server 30 includes one or more processing units (CPUs) 310, one or more network communication interfaces 320, memory 330, and one or more communication buses 370 for interconnecting these components.

[0061] The server 30 may optionally include a user interface 350, which may include a display device (not shown) and a keyboard and / or mouse (or some other pointing device or other input device; not shown).

[0062] The memory 330 is, for example, a high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state memory, and may also be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices.

[0063] Another example of memory 330 is one or more storage devices located remotely from the CPU 310. In one embodiment, memory 330 stores the following programs, modules, and data structures, or subsets thereof:

[0064] One or more processing units (CPUs) 310 read and execute a program from the memory 330 as needed. For example, one or more processing units (CPUs) 310 may constitute the learning unit 311 by executing a program stored in the memory 330.

[0065] The learning unit 311 analyzes the wearer's state using the wearer's electroencephalogram (EEG) signals and generates a model (first model). For example, a model may be used that combines any extraction method from multiple methods for extracting learning data from EEG signals with any classifier from multiple classifiers. The extraction method includes wavelet transform, Fourier transform, etc., and the classifier includes, for example, Random Forest, Support Vector Machine (SVM), neural network, decision tree, etc.

[0066] <Feedback Training> It is known that the frequency bands and potential trends of brainwaves differ from person to person. Therefore, users whose brainwaves are being measured are asked to annotate their own arbitrary state (for example, the first state) using symbols or emoticons displayed on the user terminal 20. By learning from the brainwave signals and annotations, the learning unit 311 can learn and predict the first brainwave signal corresponding to the first state, and the transition from an arbitrary brainwave signal to the first brainwave signal. Furthermore, by performing predetermined training to enable the transition from an arbitrary brainwave signal to the first brainwave signal, the goal is to enable users to reproduce any arbitrary state.

[0067] The processing unit 216 calculates the frequency bands for EEG training for each individual wearer. For example, the processing unit 216 detects the Individual Alpha frequency Peak (IAP) from the EEG signal. Next, using the IAP as the axis, the processing unit 216 sets each frequency band for each individual: delta (0.5~3Hz), theta (4~7Hz), alpha (8~13Hz), beta (14~30Hz), and gamma (30Hz and above). The processing unit 216 calculates the ratio between the total potential of the EEG signal (e.g., the average value of all frequency bands) and the potential of each frequency band (e.g., the average value of each frequency band). Alpha waves are said to appear when at rest, awake, and with eyes closed, and theta waves, delta waves, etc. are set based on these alpha waves.

[0068] Furthermore, the processing unit 216 calculates a predetermined ratio (hereinafter also referred to as the "golden ratio") for each individual wearer. The method for calculating the golden ratio involves calculating an ideal state of brainwave signals for each individual wearer based on the analysis results of artificial intelligence (AI) by the server 30, dividing the brainwaves into frequency bands using predetermined Hz in the low-frequency direction and predetermined Hz in the high-frequency direction, etc., with IAP as the axis, and calculating the ratio of the average value of all frequency bands to the average value or representative value of each frequency band. The representative value of each frequency band may be, for example, the peak frequency with the highest value in each frequency band. The golden ratio may also be the ratio of the average value or representative value of each frequency band.

[0069] Figure 9 shows an example of a good state according to the second embodiment. The example shown in Figure 9 shows a good state for one wearer, and for other users, it may be in a different position. The good state may also be identified by instructions from the wearer. Instructions from the wearer include, for example, pressing a UI component on the screen of the user terminal 20 that indicates a good state while sensing electroencephalogram signals.

[0070] The processing unit 216 converts the calculated optimal state of brainwave signals into a ratio of average or representative values ​​in each frequency band. For example, the processing unit 216 may use brainwave signals from various wearers to calculate the standard deviation of the ratio of brainwave signals categorized by gender, age, and time, and then, considering the calculated standard deviation and the individual's golden ratio, convert it into a golden ratio for each band that is suitable for that individual.

[0071] <Training Methods> 1: The learning unit 311 analyzes the individual's frequency characteristics and records the potential trend based on the calculated IAP. 2: The learning unit 311 learns the characteristic features of the wearer's brainwave signals in various states (not only good states, but also states such as being focused or relaxed) based on annotations by the wearer. 3. By allowing the AI ​​in the learning unit 311 to sufficiently learn the features of brainwave signals, it becomes possible to infer an individual's state from the brainwave signals. 4. The processing unit 216 uses the predictable feature quantities of the electroencephalogram signal to convert them into frequency or potential. 5. The processing unit 216 performs training to approach any given state of the learned individual, for example, to approach a converted frequency or potential value. 6. In this case, the selection of the desired state may be at the user's discretion, or it may be selected automatically by the system. 7. The music and audio-visual materials used for training will be automatically selected or recommended by the system based on your past training history. 8: After training, the processing unit 216 will have the wearer annotate the results again to verify the effectiveness.

[0072] Figure 10 shows an example of a training band based on IAP in the second embodiment. The example shown in Figure 10 shows the IAP of one wearer, and for other wearers, the IAP may be located at a different frequency. Therefore, in training using electroencephalogram signals on an individual basis, it is preferable to set a training band for each individual as shown in Figure 10.

[0073] <Feedback> The processing unit 216 selects content and stimuli while providing feedback. The processing unit 216 selects the optimal music, video, etc., to lead to the ideal potential state. For example, the processing unit 216 may generate recommendations from the viewing history of music and videos (individual and overall).

[0074] Furthermore, the processing unit 216 selects audio and video stimuli to guide the user to the optimal potential state. For example, the processing unit 216 may generate recommendations from the training history (individual and overall). Figure 11 shows an example of a training screen according to the second embodiment. As shown in Figure 11, the user can listen to music and press face icons that indicate their mood at the time, thereby allowing the system to learn the user's state.

[0075] <Operation> Next, the operation of the electroencephalogram (EEG) signal processing system 1 will be described. Figure 12 is a flowchart showing an example of induction processing according to the second embodiment. In the example shown in Figure 12, the acquired EEG signals are used to induce a predetermined state.

[0076] In step S102, the acquisition unit 212 acquires the electroencephalogram (EEG) signal output from the sensor (for example, the elastic electrode of the earphone 10).

[0077] In step S104, the estimation unit 214 uses a model that has learned a predetermined brainwave signal of the wearer wearing the sensor and the wearer's state at the time the predetermined brainwave signal was acquired to estimate the wearer's state from the acquired brainwave signal.

[0078] In step S106, the processing unit 216 performs a predetermined process based on the estimated wearer's condition. This predetermined process may include, for example, informing the wearer of their condition.

[0079] Figure 13 is a flowchart showing an example of a training process according to the second embodiment. The example shown in Figure 13 illustrates a process in which training is performed to transition to a predetermined state while sensing electroencephalogram (EEG) signals.

[0080] In step S202, the acquisition unit 212 acquires the electroencephalogram (EEG) signal output from the sensor (for example, the elastic electrode of the earphone 10).

[0081] In step S204, the processing unit 216 calculates the IAP (Individual Alpha frequency Peak) from the electroencephalogram signal.

[0082] In step S206, the processing unit 216 provides feedback based on the calculated IAP, deciding whether to move the IAP to a high potential or a low potential.

[0083] In step S208, the processing unit 216 determines whether or not the process will end based on the user's operation. If the user initiates an operation to end the process (step S216-YES), the process ends; if the user does not initiate an operation to end the process (step S216-NO), the process returns to step S202.

[0084] [Differentiation] Although several embodiments of the technology disclosed in this application have been described above, the technology disclosed in this application is not limited to those described above. Furthermore, the programs of the information processing devices 20 and 30 of the present invention can be installed or loaded onto a computer by downloading them via various non-temporary recording media such as optical discs such as CD-ROMs, magnetic discs, and semiconductor memory, or via a communication network.

[0085] Figure 14 shows an enlarged example of an earphone according to Modification 1. The example of the earphone shown in Figure 14 differs from the earphone shown in Figure 2 in the orientation of the gripping part 120, but the other configurations are the same as those of the earphone shown in Figure 2. As shown in Figure 14, the gripping parts 120RA and 120LA are provided with their longitudinal direction aligned with the vertical Z direction. The longitudinal direction of the gripping part 120 is also aligned with the direction of movement by the adjustment mechanism provided on the connecting part 116. The part that grips the earlobe faces upward along the Z axis (towards the housing). As a result, most of the gripping part 120 is hidden on the face side of the connecting part 116, thereby improving the design. Furthermore, by adopting the configuration of the gripping part 120 shown in Figure 14, it becomes possible to reduce the size of the gripping part 120. This is because, since earlobes generally have width in the horizontal direction, grasping the earlobe from below, as shown in Figure 14, allows for a shorter longitudinal length of the folded portion of the gripping part 120 compared to grasping the earlobe from the horizontal direction, as shown in Figure 1.

[0086] Figure 15 shows an example of the earphone set 50 in Modification 2. The earphone set 50 shown in Figure 15 lacks the earphone gripping portion 120 shown in Figures 1 and 14, and instead has a neck strap portion 160 and a pair of earphones 170R and 170L. Each earphone 170R and 170L is connected to the neck strap portion 160 using a signal-communication cable, but they may also be connected using wireless communication.

[0087] The neck strap 160 has a central member that follows the back of the neck and rod-shaped members (arms) 182R and 182L that curve along both sides of the neck. Electrodes 162 and 164 for sensing electroencephalogram (EEG) signals are provided on the back surface of the central member that contacts the neck. Electrodes 162 and 164 are a grounded electrode and a reference electrode, respectively. This allows for a distance between the EEG signals and the elastic electrodes provided on the earphone eartips, enabling accurate acquisition of EEG signals. Furthermore, the rod-shaped members 182R and 182L on both sides of the neck strap 160 are heavier at their tips than at their bases (on the central member side), ensuring that the electrodes 162 and 164 are properly pressed against the wearer's neck. For example, weights are provided at the tips of the rod-shaped members 182R and 182L.

[0088] Figure 16 shows a schematic example of a cross-section of the earphone 170 in modified example 2. The earphone 170 shown in Figure 16 is basically the same as the earphone 100, but an elastic member (e.g., urethane) 174 is provided between the speaker 113 and the nozzle 172. By providing this elastic member 174, vibrations from the speaker 113 are less likely to be transmitted to the elastic electrode of the ear tip 176, and interference between the elastic electrode of the ear tip 176 and the speaker 113 in terms of sound can be prevented.

[0089] Furthermore, although the ear tip 176, which includes elastic electrodes, is located at the sound duct, the elasticity of the elastic electrodes themselves prevents interference from sound vibrations. In addition, since the housing uses an elastic material, this elastic material makes it difficult for sound vibrations to be transmitted to the elastic electrodes of the ear tip 176, thus preventing interference from sound vibrations.

[0090] The earphone 170 includes an audio sound processor, which is used to cut sound signals below a predetermined frequency (e.g., 50Hz) that corresponds to brainwave signals. In particular, the audio sound processor cuts sound signals below 30Hz, which is a frequency band in which brainwave signals tend to exhibit distinctive characteristics, but amplifies sound signals around 70Hz to avoid compromising the bass tone. This prevents interference between sound signals and brainwave signals. Furthermore, the audio sound processor only needs to cut predetermined frequencies when brainwave signals are being sensed. The audio sound processor described above is also applicable to the earphone 10 in the embodiment.

[0091] Figure 17 shows an example of the structure around the electrodes of the neck strap portion 160 in Modification 2. In the example shown in Figure 17, an elastic member 166 is provided on the neck strap portion 160 side of each electrode 162 and 164. This elastic member 166 is made of, for example, urethane, and the elasticity of this elastic member 166 allows the electrodes 162 and 164 to change position.

[0092] As a specific example, by using a hemispherical electrode and bonding an elastic member 166 to cover the central part of the hemisphere, the electrode can change its angle 360 ​​degrees using the elasticity of the elastic member 166. In addition, the elastic member 166 deforms to conform to the shape of the wearer's neck, making it easier to press the electrodes 162 and 164 against the wearer's neck.

[0093] As described above, the ends of the rod-shaped members 182R and 182L on both sides of the neck strap 160 are heavier. Therefore, when the neck strap 160 is worn around the neck, the weight of the ends of the rod-shaped members 182R and 182L makes it easier for the electrodes 162 and 164 to press against the wearer's neck.

[0094] Figure 18 shows an example of a roughly disassembled earphone set 50 in modified example 2. In the example shown in Figure 18, the rod-shaped member 182R has, for example, a bellows structure inside the aluminum plate located on the outside, and the rod-shaped member 182R can be deformed according to the shape and thickness of the wearer's neck. In addition, an elastic member 184R, such as rubber, is provided on the inside of the neck of the rod-shaped member 182R, and the contact of the elastic member 184R with the neck reduces the burden of contact with the neck and improves the wearing comfort.

[0095] Furthermore, the rod-shaped members 182 located on both sides of the neck strap 160 can be folded toward the electrode side. For example, the rod-shaped member 182R can be folded toward the electrode side or the central member side by the folding structure 186R. Similarly, the rod-shaped member 182L also has a folding structure (e.g., a hinge) and can be folded toward the electrode side. This allows the neck strap 160 to be stored compactly.

[0096] Furthermore, the rod-shaped members 182R and 182L should be slightly curved downwards in the vertical direction so that they conform easily to the wearer's clavicle.

[0097] <Specific examples in electroencephalography (EEG) feedback training> Next, I will explain a specific example of the individualized EEG feedback training described above. This example is an application that allows users to improve productivity by managing their concentration and rest times while learning about their brainwave signals. In this application, for example, the training is conducted by dividing time into segments and changing the use case for each segment. For example, the first 10 minutes are for meditation (use case for meditation), the next 25 minutes are for a task (use case for concentration), and the next 5 minutes are for a break (use case for relaxation). This task-and-rest cycle is repeated for four sets. The final break can also be 25 minutes. This time management technique is based on the Pomodoro Technique, which is said to improve productivity.

[0098] During time management in the specific example, the acquisition unit 212 acquires brainwave signals using the aforementioned earphones, the estimation unit 214 estimates the user's current state based on the brainwave signals in each use case using a pre-trained model customized for the individual by the learning unit 311, and the processing unit 216 performs guidance processing to transition the user from their current state to a better state.

[0099] As an example of feedback, the processing unit 216 controls a predetermined area of ​​the user terminal 20's display screen, such as the background, to display a gradient using colors pre-assigned to each frequency band of the brainwave signal. For example, when the brainwave signal is transitioning to a good state (golden ratio state), the color of the predetermined area is displayed so that it gradually becomes brighter, and when the brainwave signal is not transitioning to a good state, the color of the processing area is displayed so that it gradually becomes darker. Since the good state of the brainwave signal differs depending on the user, the gradient is determined based on the frequency band of the brainwave signal in that user's good state. This makes it possible to display a gradient in a predetermined area in accordance with changes in the brainwave signal, and to visually represent the current state of the brainwave signal to the user.

[0100] Figure 19A shows an example of a display screen during concentration training. The screen shown in Figure 19A is displayed during a time management task, showing the current score value as "42" and "25:00" indicating the task time timer. In this way, the processing unit 216 uses a timer for time management in each use case. The processing unit 216 may also display the waveform of the frequency band of the electroencephalogram (EEG) signal related to each use case.

[0101] Furthermore, the processing unit 216 may control the speaker to output a sound pre-set to the frequency band of the brainwave signal. The pre-set sound may be, for example, a natural sound such as wind, rain, waves, or a forest, or a combination of these as appropriate. For example, when the brainwave signal is transitioning to a good state (golden ratio state), the speaker will output a sound that gradually becomes gentler, and when the brainwave signal is not transitioning to a good state, the speaker will output a sound that gradually becomes rougher. Transitioning the brainwave signal to a good state means that the difference between the ratio of the average or representative value in each frequency band of the current brainwave signal and the golden ratio becomes smaller, and each frequency band is the same as the frequency band from which the golden ratio is obtained. The pre-set sound may also be a voice. In addition, when the brainwave signal acquired by the acquisition unit 212 approaches the golden ratio described above, the processing unit 216 may display an emoji on the screen, output a sound, or indicate this with the screen color.

[0102] Furthermore, the application in the specific example can score and visualize concentration, relaxation, tension, fatigue, etc., determined from brainwave signals over a predetermined period, such as monthly, weekly, daily, or session-based. Regarding the score, the estimation unit 214 scores it based on the distance between the golden ratio learned using the individual user's brainwave signals and the ratio of representative values ​​for each frequency band when the current brainwave signal is divided into each frequency band, according to each use case. The processing unit 216 then displays the scored value on the screen.

[0103] Figure 19B shows an example of a screen displaying measurement results for relaxation, focus, and meditation on a daily basis. The screen shown in Figure 19B is a so-called dashboard that aggregates and visualizes data, and it shows that based on the electroencephalogram signals measured on the 15th (Wed), the time judged to be relaxed was "33 minutes", the time judged to be focused was "23 minutes", and the time judged to be calm (meditated) was "42" minutes.

[0104] The estimation unit 214 may obtain the respective score values ​​for each use case through estimation processing using a trained model. In the example shown in Figure 19B, the score value for relaxation is "68" and the score value for concentration is "72". If the score value for each use case is equal to or greater than the corresponding predetermined value, the processing unit 216 may determine that the user is relaxed, concentrated, or calm. The following describes each use case that is managed by time.

[0105] (meditation) Meditation (calming down) is performed, for example, during the initial time period as part of the time management techniques in this application. First, the processing unit 216 provides meditation guidance using audio and video. The acquisition unit 212 acquires brainwave signals after the meditation guidance, and the estimation unit 214 calculates a score value based on the acquired brainwave signals and the golden ratio. When the brainwave signals calm down and the score value increases (when the frequency approaches the golden ratio), the processing unit 216 changes the background color, stops the wind in the background image, and / or plays sounds such as bird songs.

[0106] On the other hand, if the brainwave signals become erratic and the score value decreases (the frequency deviates from the golden ratio), the processing unit 216 controls the background color, the wind to blow in the background image, and / or the landscape in the background image to become stormy. The processing unit 216 may also change the background (natural environment) and guide content depending on the time of day (morning, afternoon, etc.) and purpose for which the application is run.

[0107] Figure 20 shows an example of frequency waveforms during meditation and when not meditating. In the example shown in Figure 20, during meditation (Focus), low-frequency delta and theta waves appear compared to when not meditating (Tired). These individual waveform characteristics can be used to determine the user's personal golden ratio, etc. Furthermore, the audio and video described above are used to guide meditation in order to generate delta and theta waves from the current frequency values.

[0108] (task) Tasks (concentration, fatigue) are performed after meditation as part of the time management techniques in this application, and are then alternated with breaks. The estimation unit 214 estimates the user's level of concentration and fatigue based on the frequency of the acquired brainwave signals. For example, the estimation unit 214 estimates that fatigue is increasing when theta waves increase during a task. This is because delta waves and theta waves are generally said not to appear in a waking state.

[0109] Figure 21 shows an example of the frequency waveforms of brainwave signals in the morning and after lunch. In the example shown in Figure 21, the theta wave values ​​are not high in the morning (Focus), but they increase after lunch (Tired2), when the user is feeling tired or sleepy. When the theta wave values ​​increase, the processing unit 216 plays sounds or other means to encourage concentration in order to lower the theta wave values. In a state of concentration, the amplitude in each frequency band becomes constant and fluctuations decrease, and the golden ratio appears appropriately.

[0110] (break) Breaks (rest, concentration) are performed after tasks as part of the time management techniques in this application, and thereafter are performed alternately with tasks. The estimation unit 214 estimates that the eyes are closed because alpha waves are said to appear when the eyes are closed, and therefore when the alpha wave value increases, it is possible to estimate that the eyes are closed. Also, the estimation unit 214 estimates that the user is calming down because the values ​​of the lower frequencies of delta and theta waves increase when the user enters a meditative state, and therefore when the values ​​of these frequency bands increase. Furthermore, even during breaks, if the user is awake, the processing unit 216 provides calming content presented by the trained model to the user.

[0111] The frequency of the electroencephalogram (EEG) signal in each use case is determined by taking the average of the frequency signals obtained by time-division multiplexing and conversion of the EEG signal within a predetermined time period. The predetermined time period may differ for each use case.

[0112] This allows the application in the example to help optimize time usage by using time-divided time slots for each use case. Furthermore, during focused periods within time management, the application can create playlists tailored to individual preferences and characteristics to facilitate concentration. For example, by using a trained model that has learned from user annotation results, it can select music to transition to a good state (the golden ratio state), and generate a playlist with these songs ordered according to predetermined criteria. Additionally, during breaks within time management, the application can use the trained model to recommend simple meditation and stretching exercises, as well as brain-stimulating games and videos, based on the characteristics of the user's brainwave signals.

[0113] <Analysis of heart rate or respiration> In Modification 3, an electrocardiogram (ECG) signal based on the carotid artery pulse is measured by changing the position of each reference electrode in the neck strap 160 in Modification 2. Figure 22 is a diagram showing an example of the position of each part of the neck strap 160 in Modification 3. In the example shown in Figure 22, a cross-sectional view of the neck strap 160 in the XY plane is shown, and each electrode 162, 164 is positioned on both sides of the wearer's neck and is made of elastic electrodes. For example, each electrode may be positioned at both ends of the rod-shaped member or central member located on both sides of the neck of the neck strap 160, or each electrode 162, 164 may be positioned at a predetermined distance (e.g., 3-7 cm) to the left and right from the center of the neck strap 160. In addition, a battery BA and a substrate ST on which a processing unit is mounted are provided at the center of the inside of the neck strap 160. By positioning the circuit board ST closer to the neck and the battery BA, which is shorter in the X direction than the circuit board ST, outside of the circuit board ST, the neck strap portion 160 can be made into a curved shape that conforms more closely to the neck. Alternatively, an elastic member that can be attached to the inside of the neck strap portion 160 may be provided, and electrodes 162 and 164 may be provided on this elastic member. This makes it possible to acquire a reference signal with high accuracy even when there is movement such as during exercise, as the elastic member adheres closely to the wearer's neck.

[0114] Figure 23 shows the positional relationship between the carotid artery and the left electrode 162. In the example shown in Figure 23, the positioning of electrode 162 around carotid artery B10 ensures that the signal sensed by electrode 162 includes a signal based on the carotid artery pulse.

[0115] Figure 24 shows an example of a reference signal mixed with carotid artery pulse. The example shown in Figure 24 is an example of raw data sensed from electrode 162 or 164. In the graph shown in Figure 24, the vertical axis represents the value of the sensed signal, and the horizontal axis represents time. As shown in Figure 24, since the reference signal is the signal subtracted from the signal sensed by the elastic electrode of the ear tip 176, the electrocardiogram (ECG) signal mixed into the reference signal is measured in reverse phase. For example, the downward-convex peak circled in Figure 24 is the signal indicating the electrocardiogram signal. The signal indicating the wearer's normal pulse is synonymous with the electrocardiogram signal.

[0116] By utilizing the inclusion of the carotid artery pulse, for example, the CPU 210 uses the signals output from each electrode 162, 164 provided in the neck strap as described in Modified Example 3 as reference signals to acquire an electroencephalogram (EEG) signal based on the signals output from the elastic electrodes of the ear tip 176. Next, the CPU 210 detects the pulse based on a peak that appears in opposite phase to the peak of the acquired EEG signal.

[0117] This allows the electrodes for measuring the reference signal to be positioned on both sides of the wearer's neck, enabling the measurement of both electroencephalogram (EEG) and electrocardiogram (ECG) signals using a common signal. In other words, the CPU210 can analyze the EEG signal while simultaneously measuring the ECG signal based on the pulse rate and analyzing the heart rate as well.

[0118] Next, an example of measuring respiration using the neck strap 160 in Modification 3 will be described. In order to measure respiration, a predetermined filtering process is performed on the electroencephalogram (EEG) signal. The predetermined filtering process is performed in the following procedure. 1: Low-pass filter 2: Median filter 3. Then, calculate the power spectrum values ​​from the FFT (Fast Fourier Transform). 4: Time derivative from time series power spectrum (P t -P t-1 Calculate the energy difference value that indicates (P t (This is the power spectral value at time t)

[0119] Figure 25A shows an example of a signal during mouth breathing. Figure 25B shows an example of a signal during nasal breathing. In the graphs shown in Figures 25A and B, the vertical axis represents the energy difference after filtering, and the horizontal axis represents time. In the signals shown in Figures 25A and B, the downward peaks appear when inhaling, and the upward peaks appear when exhaling. One reason for these peaks and valleys is thought to be that changes in facial muscle potential during breathing are mixed into each electrode of the ear tip 176.

[0120] Using the changes in electromyography described above, for example, the CPU 210 uses the signals output from electrodes 162 and 164, which are provided in the neckband as described in Modified Example 3, as reference signals to acquire an electroencephalogram (EEG) signal based on the signals output from the elastic electrodes of the ear tips 176. Next, the CPU 210 processes the acquired EEG signal with a low-pass filter and a median filter, converts the frequency of the filtered signal, differentiates the time-series power spectrum value after frequency conversion, and detects respiration based on the periodicity of the difference in this time derivative.

[0121] This makes it possible to measure signals representing respiration by applying a predetermined filtering process to the electroencephalogram (EEG) signals. In other words, the CPU210 can analyze EEG signals while simultaneously measuring signals representing respiration, enabling analysis of respiration pace, volume, and other related parameters.

[0122] Furthermore, by applying the same filtering process used for respiration detection to heart rate detection, noise can be removed, enabling more accurate heart rate detection. Specifically, the CPU 210 detects the peak value of the electrocardiogram that appears in opposite phase to the peak of the electroencephalogram signal, processes the detected signal with a low-pass filter and a median filter, frequency-converts the filtered signal, differentiates the time-series power spectrum value after frequency conversion, and detects the pulse based on the periodicity of the time-differentiated difference value. This makes it possible to detect the heart rate using a pulse with noise removed.

[0123] Furthermore, the heart rate and respiration analysis described above is performed by the CPU 210, which detects heart rate and respiration by determining the periodicity from the data values ​​shown in Figure 24 or Figure 25 using an autocorrelation function. Alternatively, the process for detecting heart rate and respiration described above may be implemented as a program containing the instructions for each process, installed in the information processing device, and executed. [Explanation of symbols]

[0124] 10 earphones 20, 30 Information Processing Equipment 110 Ear Tips (Elastic Electrodes) 112 Housing 113 speakers 114 Joint Mechanism 116 Connection part 118 Cable 120 Gripping part 212 Acquisition Department 214 Estimation Department 216 Processing Unit 311 Learning Department

Claims

1. An acquisition unit that acquires brainwave signals output from earphones, which are brainwave measurement devices worn by the user, An estimation unit that estimates the user's state from the acquired brainwave signals using a model that has learned the user's predetermined brainwave signals and the user's state at the time the predetermined brainwave signals were acquired, A processing unit that performs brainwave training by providing the user with predetermined music via the earphones to induce a transition of the user to a predetermined state indicated by a first brainwave signal selected by the user, based on the user's state estimated based on the current brainwave signal acquired from the earphones, the processing unit that performs the brainwave training while providing feedback to the user on whether or not they are approaching the predetermined state, based on the current brainwave signal stimulated by music selected based on the history of performing the brainwave training, An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, wherein the model is a customized model that has been learned by adding the user's brainwave signal and the user's state at the time the brainwave signal was acquired to a predetermined model that has been learned of another person's brainwave signal and the other person's state at the time the brainwave signal was acquired.

3. One or more processors included in the information processing device, To acquire brainwave signals output from earphones, which are brainwave measurement devices worn by the user. Using a model that has learned a predetermined brainwave signal of the user and the user's state at the time the predetermined brainwave signal was acquired, the user's state is estimated from the acquired brainwave signal. Performing electroencephalography (EEG) training by providing the user with predetermined music via the earphones to induce a transition of the user to a predetermined state indicated by a first EEG signal selected by the user, based on the user's state estimated from the current EEG signal acquired from the earphones, and performing the EEG training while providing feedback to the user on whether or not they are approaching the predetermined state, based on the current EEG signal stimulated by the music selected based on the history of performing the EEG training. An information processing method that performs the following.

4. One or more processors included in the information processing device To acquire brainwave signals output from earphones, which are brainwave measurement devices worn by the user. Using a model that has learned a predetermined brainwave signal of the user and the user's state at the time the predetermined brainwave signal was acquired, the user's state is estimated from the acquired brainwave signal. Performing electroencephalography (EEG) training by providing the user with predetermined music via the earphones to induce a transition of the user to a predetermined state indicated by a first EEG signal selected by the user, based on the user's state estimated from the current EEG signal acquired from the earphones, and performing the EEG training while providing feedback to the user on whether or not they are approaching the predetermined state, based on the current EEG signal stimulated by the music selected based on the history of performing the EEG training. A program that executes the command.

Images