Information processing method, program, and information processing apparatus

Abstract

To improve usability in authentication using a brain wave signal.SOLUTION: The information processing method includes, by a processor included in an information processing apparatus, outputting stimulated music data for stimulating a brain of a user with different frequencies, acquiring a brain wave signal of the user from a brain wave measurement device worn by the user, and generating an authentication model for authenticating the user from the brain wave signal of the user based on feature data of the brain wave signal of the user during stimulation by the stimulated music data.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an information processing method, a program, and an information processing device. [Background technology]

[0002] In recent years, personal authentication technologies using a user's electroencephalogram (EEG) signals have been researched. For example, there are known technologies for performing authentication based on a user's EEG signals (see, for example, Patent Document 1), technologies for performing authentication based on EEG signals when an authentication task is performed (see, for example, Patent Document 2), and technologies for performing authentication based on EEG signals in response to visual stimuli (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2008 / 144174 [Patent Document 2] Patent No. 6373402 specification [Patent Document 3] EP 3647976 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in both technologies, active tasks are required for authentication, and there is room for improvement in usability.

[0005] Therefore, one aspect of the disclosed technology aims to provide an information processing method, a program, and an information processing device that enable usability to be improved in authentication using electroencephalogram signals. [Means for solving the problem]

[0006] An information processing method in one aspect of the disclosed technology includes a processor included in an information processing device outputting stimulation music data that stimulates a user's brain with different frequencies, acquiring the user's electroencephalogram (EEG) signals from an electroencephalogram (EEG) measuring device worn by the user, and generating an authentication model that authenticates the user from the user's EEG signals based on feature data of the user's EEG signals during stimulation by the stimulation music data. [Effects of the Invention]

[0007] According to one aspect of the disclosed technology, it is possible to improve usability in authentication using electroencephalogram signals. [Brief explanation of the drawings]

[0008] [Figure 1A] FIG. 1 is a diagram showing the experimental procedure of the first experiment. [Figure 1B] FIG. 1 is a diagram illustrating an example of a cognitive task used in this experiment. [Figure 1C] FIG. 10 shows trial average frequency data (peak data every 1 Hz) for each subject for each electrode site. [Figure 1D] FIG. 10 is a diagram showing trial average frequency data (sweep data) of each subject for each electrode site. [Figure 1E] FIG. 10 shows trial average frequency data (Sweep StimPeak data) for each subject for each electrode site. [Figure 1F] FIG. 10 is a diagram illustrating an example of authentication accuracy according to all analysis methods. [Figure 1G] FIG. 10 is a diagram showing the electrode sites and frequencies that contributed to authentication by the analysis method (peaks every 1 Hz). [Figure 1H] FIG. 10 is a diagram showing the electrode sites and frequencies that contributed to authentication by the analysis method (Sweep). [Figure 1I] FIG. 10 is a diagram showing the electrode sites and frequencies that contributed to authentication using the analysis method (Sweep StimPeak). [Figure 1J] FIG. 10 is a diagram showing the accuracy when only electroencephalogram signals from ear canal electrodes at rest are used. [Figure 1K] FIG. 10 is a diagram showing the accuracy when only the electroencephalogram signal from the ear canal electrode is used during cognitive load. [Figure 1L] FIG. 10 is a diagram showing an example of authentication accuracy using ear canal electrodes according to the full analysis method. [Figure 2] 1 is a block diagram illustrating an example of an information processing device according to a first embodiment. [Figure 3] 5A and 5B are diagrams illustrating an example of generating a sine wave according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of processing process A according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing an example of processing B according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of processing C according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of remix processing according to the first embodiment. [Figure 8] 5 is a flowchart showing an example of processing by the information processing device according to the first embodiment. [Figure 9] 6 is a flowchart showing an example of a process related to authentication according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of an earphone set according to a second embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a schematic cross section of an earphone according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or applications of techniques not explicitly described below. In other words, the present invention can be implemented with various modifications within the scope of its spirit. Furthermore, in the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. Parts in the drawings may have different dimensional relationships or ratios.

[0010] Before describing an overview of an embodiment of the disclosed technology, an experiment will be described below to investigate the possibility of seamless authentication while listening to music. Note that the music used in this experiment is not ordinary music, but music that can stimulate specific frequencies in the user's brain. This music will be described later.

[0011] <Experiment details> In this experiment, we verified whether it is possible to authenticate a person by simply listening to brainwave-eliciting auditory stimuli that have musical qualities that would not be unnatural even if they were listened to as normal background music (BGM).In addition, because the brainwave peaks in response to external stimuli differ from person to person, we verified whether it is possible to obtain information useful for personal authentication by observing the differences in the user's brainwave responses even when the music used to stimulate the user is the same.

[0012] <Experimental conditions> The number of subjects was 23. The analysis frequency is 4-45Hz. The reason for limiting it to 45Hz is to prevent power supply noise (around 50Hz) from being mixed in.

[0013] (Music used) The music used was an original piece of music created specifically for this experiment, combining instrumental sounds that evoke different frequencies (hereinafter referred to as "stimulus music data"). An example of the stimulus music data is shown below. 36Hz bass sound that stimulates the user's brain 40Hz drum sounds that stimulate the user's brain A 44Hz synthesizer sound that stimulates the user's brain The above three instrument sounds were synthesized to generate music that stimulates 36Hz, 40Hz, and 44Hz. The method for generating instrument sounds that stimulate specific frequencies will be described later. The above music is composed of a 16-second phrase repeated to form a 5-minute piece. The instruments and stimulation frequencies can be changed as desired, making it possible to compose complex music depending on the security level. For example, a low security level may stimulate a user's brain with fewer frequencies, while a high security level may stimulate a user's brain with more frequencies. As an example, a low security level may stimulate a user's brain with one frequency, a medium security level may stimulate a user's brain with three frequencies, and a high security level may stimulate a user's brain with five frequencies, but the number of frequencies is not limited to these.

[0014] (Feature data) As an analysis method, the following three feature data were used for the frequency data: (1) 1Hz bin Peak Frequency data was extracted from the frequency data in 1 Hz increments: 4, 5, 6, ..., 45 Hz. (2) Sweep Rather than pinpointing a 1 Hz bin, data around the frequency ±0.5 Hz was averaged. (3) Sweep StimPeak Of the sweep data, 1 Hz bin peak data was used for stimulation frequencies of 36, 40, and 44 Hz. To avoid increased authentication accuracy due to subject-specific noise (such as the degree of contact between the electrodes and hair) that is not an EEG component, analysis using (2) Sweep and (3) Sweep StimPeak is recommended.

[0015] (measuring equipment) The measuring device was an international 10-20 electroencephalograph, Polymate Mini AP108 (Miyuki Giken Co., Ltd.). The five electrodes measured were Cz, T3, T4, left external auditory canal, and right external auditory canal, with the mastoid process (right) used as the reference and the mastoid process (left) used as the ground.

[0016] (procedure) Figure 1A shows the experimental procedure for this experiment. As shown in Figure 1A, first, in the learning stage, learning data is acquired. In the learning stage, stimulus music data is output to the subject to stimulate specific frequencies in the user's brain. Music stimulation was performed for 5 sets of 5 minutes each (4 sets of 5 minutes for one subject). The measured EEG signals were extracted every 16 seconds, and this data was Fourier transformed and then converted into decibels. The 16-second data sampled at 500 Hz was Fourier transformed. The frequency resolution was 0.0625 Hz. An inference model was generated for each frequency band from the frequency data acquired here.

[0017] In the test phase, authentication data is acquired with or without a task. The task used in this experiment is the task shown in Figure 1B. Figure 1B is a diagram showing an example of the cognitive task used in this experiment. In the cognitive task shown in Figure 1B, the following steps are performed continuously for 5 minutes. (1) Numbers are presented on the screen in succession. (2) Remember the numbers in order and answer (3) Feedback (4) Return to (1) -Difficulty changes If you get the answer right, the number of digits you need to remember increases by one. If you get the answer wrong, the number of digits you have to remember will decrease by one. Start with 5 digits The EEG signals of the users were acquired by musical stimulation while performing the cognitive task, and by musical stimulation while at rest. The 23 subjects were divided into two groups: one group was measured by musical stimulation while at rest, followed by measurement by musical stimulation during the cognitive task, and the other group was measured by musical stimulation while at rest, followed by measurement by musical stimulation during the cognitive task.

[0018] Returning to Figure 1A, in this experiment, an inference model was generated for each frequency band for the training data, and the authentication accuracy was verified using the authentication data as test data. The experimental results for each feature data are explained below.

[0019] Figure 1C shows the trial-average frequency data (peak data every 1 Hz) for each subject at each electrode site. As shown in Figure 1C, the musical stimulation evoked frequencies of 36, 40, and 44 Hz. The 1 Hz bin peak data may contain frequency bands that are affected by noise and data fluctuations.

[0020] Figure 1D shows the trial-averaged frequency data (sweep data) for each subject at each electrode site. As shown in Figure 1D, by averaging the surrounding frequencies, the elicitation of the musical stimulation frequencies of 36, 40, and 44 Hz is weakened, but it can be said that the noise has been removed and the characteristics of each individual's EEG have been extracted.

[0021] Figure 1E shows the trial average frequency data (Sweep StimPeak data) for each subject at each electrode site. As shown in Figure 1E, by using only the pre-averaging values ​​for the musical stimulation frequencies of 36, 40, and 44 Hz, it is possible to extract the characteristics of the stimulated frequency while suppressing noise fluctuations.

[0022] Figure 1F shows an example of the authentication accuracy for all analysis methods. As shown in Figure 1F, among the resting EEG data, SweepStimPeak data had the highest accuracy, and individual authentication was performed correctly with a probability of 94%. As will be explained later, the stimulated frequency characteristics and low-frequency frequency characteristics data contribute to individual authentication.

[0023] Furthermore, recognition accuracy was low under cognitive load, indicating that authentication accuracy declines under external load, including threats. Even with EEG data acquired at different times, authentication accuracy was approximately 80%, eliminating the possibility that high accuracy was due to noise other than EEG signals, such as differences in the measurement environment. The recall indicator shown in Figure 1F indicates the proportion of correct predictions made by the model for the positive class, and indicates the number of times the actual class was correctly predicted for all observations in positive trials.

[0024] Figure 1G shows the electrode sites and frequencies that contributed to the recognition by the analysis method (peaks every 1 Hz). As shown in Figure 1G, the frequencies of 36, 40, and 44 Hz, where music stimulation was performed, contributed most to the recognition.

[0025] Figure 1H shows the electrode locations and frequencies that contributed to authentication using the analysis method (Sweep). As shown in Figure 1H, the frequencies of 36, 40, and 44 Hz, which were used for musical stimulation, did not contribute to authentication, and low frequencies contributed. This can be said to be due to the averaging process, which weakened the characteristics of the frequencies used for musical stimulation.

[0026] Figure 1I shows the electrode locations and frequencies that contributed to authentication using the analysis method (Sweep StimPeak). As shown in Figure 1I, the frequencies of 36, 40, and 44 Hz, where music stimulation was performed, contributed most to authentication, and theta waves (4-8 Hz) and alpha waves (8-12 Hz) also contributed to authentication.

[0027] <Authentication accuracy using only ear canal EEG signals> Here, we assumed an earphone-type EEG monitor and investigated the accuracy of the authentication model when training was performed using only ear canal EEG signals. The earphone-type EEG monitor will be described later using Figures 10 and 11. The following indicators were used for authentication accuracy. Accuracy: The proportion of correctly predicted observations, calculated as the number of correctly predicted observations divided by the total number of observations. Precision: The proportion of correctly predicted positive observations, calculated as the number of correctly predicted observations divided by all positive predictions. Recall: The proportion of correct predictions made by the model for the positive class. The actual class was correctly predicted for all observations in the positive trials. False Positive Rate: The proportion of false positives. The proportion of results that are incorrectly reported as positive compared to the total number of actual negatives. F1 Score: A combination of the precision and recall of a model, defined as their harmonic mean. The closer to 1, the better the accuracy, and the closer to 0, the less effective it is.

[0028] Figure 1J shows the accuracy of each test when only the EEG signal from the ear canal electrodes is used during rest. As shown in Figure 1J, the central Recall (recall rate) was over 90% with the analysis method (Sweep StimPeak), which is not significantly different from the 94% accuracy achieved when all electrodes were used.

[0029] Figure 1K shows the accuracy of each test when only the EEG signal from the ear canal electrode is used during cognitive load. As shown in Figure 1K, the recall (center) shows that even with the analysis method (Sweep StimPeak), the authentication accuracy is lower than the 62% when all electrodes are used.

[0030] Figure 1L shows an example of authentication accuracy using ear canal electrodes with the full analysis method. As shown in Figure 1L, the SweepStimPeak data was the most accurate among resting EEG data, and personal authentication was performed correctly with a probability similar to that when all electrodes were used. In this case, too, the stimulated frequency characteristics and low-frequency frequency characteristics data contributed to personal authentication. Furthermore, when ear canal electrodes were used only, the contribution rate increased because common noise could be removed from the differential signal between the left and right electrodes.

[0031] <Reduction in study time> When using all electrodes, the authentication accuracy was examined when the learning time for EEG data was limited to just 5 minutes, rather than 5 sets of 5 minutes each. Even when the learning time was limited to 5 minutes (using only one set in the above experiment), the same authentication accuracy was obtained as when using 5 sets of 5 minutes each. In other words, for resting EEG data, SweepStimPeak data was the most accurate, with an authentication accuracy of just under 90%.

[0032] <Experimental Considerations> (Consideration 1) It was suggested that in addition to the frequency band elicited by musical stimuli, low frequencies (6-10 Hz) are useful for personal authentication technology. It is known that each individual has a different response to stimulated frequencies, and even when using the 36, 40, and 44 Hz stimulation frequencies used in this study, we were able to extract and utilize brain wave components characteristic of each individual. Low frequencies are frequently present among EEG components, making them prone to individual differences. It was suggested that combining low frequencies with high frequencies of 36, 40, and 44 Hz may enable more accurate personal authentication. When the study time was short, such as 5 minutes, the stimulus frequency and the alpha band of approximately 10-13 Hz contributed to the prediction, suggesting that the frequency components that contributed differ depending on the study time.

[0033] (Consideration 2) We were able to eliminate the possibility that the authentication accuracy was improved due to noise other than brain waves, such as differences in the measurement environment. A certain level of discrimination accuracy (approximately 80%) was also achieved for data measured on the same subject at a later date. This ruled out the possibility that the accuracy was high due to noise other than brain waves, such as differences in the measurement environment.

[0034] (Consideration 3) It was suggested that the system may not be able to perform authentication in the first place when cognitive load is applied from the outside. It was suggested that the accuracy of personal authentication is lower under cognitive load compared to when at rest, and that brain activity differs from when at rest when performing complex calculations, and that the differences between individuals in personal characteristics at that time are smaller than when at rest. If it can be verified that the differences between individuals are small even under situations other than cognitive load, such as threats, it may lead to the creation of a system that cannot authenticate under extreme circumstances such as threats.

[0035] (Consideration 4) It was suggested that personal authentication may be possible using a simple electroencephalograph. Even when using signals from only two electrodes in the ear canal, authentication accuracy remained at the same level as when five electrodes on the head were used, suggesting the feasibility of personal authentication using simple electroencephalographs such as earphone-type devices.

[0036] Based on the above considerations, the present disclosure describes a technology for personal authentication using electroencephalogram signals evoked by musical stimuli. This enables users to seamlessly and unconsciously be authenticated by listening to music, without having to perform the active task of "authentication."

[0037] <Summary of the Disclosed Technology> The disclosed technology develops a system for personal authentication based on individual differences in brain wave responsiveness to music. By having users listen to stimulating music that stimulates the user's brain with specific frequencies, which makes it easier for personal characteristics to emerge during personal authentication, it becomes possible to improve the accuracy of personal authentication. Furthermore, since personal authentication can be performed while listening to music, usability is improved.

[0038] The stimulating music data also includes instrument sound data (referred to as "stimulating instrument sound data") that stimulates the user's brain with specific frequencies in a predetermined frequency band, and music can be generated using this stimulating instrument sound data. Furthermore, by having the user listen to the generated music, it is possible to provide the user with a stimulus of the specific frequency. This allows the user to achieve personal authentication using electroencephalogram signals while listening to more natural music.

[0039] [First embodiment] <Configuration example of information processing device 10> 2 is a block diagram showing an example of an information processing device 10 according to the first embodiment. The information processing device 10 is, for example, a personal computer, and may be composed of one or more devices. The information processing device 10 processes sound data and generates, for example, instrument sound data that stimulates a user in a predetermined frequency band. The information processing device 10 does not necessarily have to be a personal computer, and may also be a server, a smartphone, a tablet terminal, or the like that has information processing capabilities.

[0040] The information processing device 10 includes one or more processors (CPUs: Central Processing Units) 110, one or more network communication interfaces 120, a storage device 130, a user interface 150, and one or more communication buses 170 for interconnecting these components.

[0041] The user interface 150 includes a display device (not shown) and an input device (not shown), such as a keyboard and / or a mouse or some other pointing device.

[0042] The storage device 130 may be, for example, a high-speed random access memory such as a DRAM, an SRAM, a DDR RAM, or other random access solid-state storage device, or may be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The storage device 130 may be, for example, a computer-readable non-transitory recording medium that records a program that causes a processor to execute the processes described below.

[0043] Another example of storage device 130 may be one or more storage devices located remotely from processor 110. In one embodiment, storage device 130 stores the following programs, modules, and data structures, or a subset thereof:

[0044] The one or more processors 110 read and execute a program from the storage device 130 as needed. For example, the one or more processors 110 execute a program stored in the storage device 130 to configure a control unit 111 that executes the processing of the disclosed technology. The control unit 111 may also configure an output unit 112, an acquisition unit 113, a generation unit 114, and an authentication unit 115.

[0045] The output unit 112 outputs the stimulus music data used in the disclosed technology. For example, when a user performs an operation to receive a predetermined service, the output unit 112 may use this operation as a trigger to control the output of the stimulus music data from an output device such as a speaker or earphones.

[0046] Here, the generation process of the stimulating music data will be described. In the following, the stimulating music data will be described as being generated by the control unit 111. However, the control unit 111 may acquire stimulating music data generated by another device and store it in the storage device 130.

[0047] (Generation of stimulus music data) The control unit 111 acquires sound data. For example, the control unit 111 may acquire predetermined sound data stored in the storage device 130, may acquire predetermined sound data from an external device via the network communication interface 120, or may acquire predetermined sound data generated by a user operating the user interface 150.

[0048] The control unit 111 generates stimulation sound data that stimulates the user's brain with frequencies in a predetermined frequency band based on the acquired sound data. For example, the control unit 111 may use the user interface 150 (see FIG. 3, etc., described later) to generate stimulation instrument sound data based on sound data that stimulates gamma waves, for example. The control unit 111 may also process the acquired sound data into stimulation sound data. For example, the control unit 111 performs a predetermined processing process on the sound data to generate stimulation sound data that includes, as main components, predetermined frequencies in a predetermined frequency band.

[0049] The predetermined frequency band includes, for example, at least one of the following frequency bands related to brain waves generated by brain activity: Δ (delta) waves (0.5 to 4 Hz), θ (theta) waves (4 to 8 Hz), α (alpha) waves (8 to 12 Hz), β (beta) waves (12 to 30 Hz), and γ (gamma) waves (30 to 100 Hz). Furthermore, the control unit 111 may generate stimulating instrument sound data that includes a specific predetermined frequency as a main component within the predetermined frequency band. For example, the control unit 111 may generate stimulating instrument sound data that includes a 40 Hz frequency component as a main component of gamma waves, with 40 Hz being the predetermined frequency. This makes it possible to stimulate the user with a specific frequency, thereby stimulating the user with a specific frequency band that includes the specific frequency.

[0050] The predetermined processing performed by the control unit 111 includes processing the predetermined sound data so that it has the characteristics of each instrument sound, such as a drum, bass, guitar, keyboard (piano), etc. For example, the processing includes processing the envelope of the predetermined sound data or synthesizing sound data having the characteristics of the instrument sound. The control unit 111 may select a processing method corresponding to an instrument sound designated by a user or the like based on processing information in which processing methods are set for each instrument sound, and process the predetermined sound data using the selected processing method. The control unit 111 may also combine multiple stimulus instrument sound data. For example, the control unit 111 may combine first stimulus instrument sound data with second stimulus instrument sound data different from the first stimulus instrument sound data.

[0051] The output unit 112 outputs the stimulating instrument sound data generated by the control unit 111. For example, the output unit 112 outputs stimulating music data including the generated stimulating instrument sound data to a speaker and controls the speaker to output the stimulating music. The output unit 112 may also combine the stimulating music data with other sound data and output the combined data. The speaker may be provided in the information processing device 10, or may be connected to the information processing device 10 via a network.

[0052] The above processing makes it possible to reduce the sense of incongruity felt as an instrument sound, even for an instrument sound that stimulates the user with frequencies in a predetermined frequency band. For example, processing by the control unit 111 makes it possible to generate stimulating instrument sound data that has characteristics similar to those of an instrument sound, while having frequencies in a predetermined frequency band. This allows the listener to listen to stimulating music that includes stimulating instrument sound data without feeling much incongruity as music. Below, the processing will be explained using three examples.

[0053] (Processing A) The predetermined sound data may include stimulation sound data including a pure tone that stimulates a predetermined frequency band to the user. For example, if the predetermined frequency band is a gamma wave and the specific frequency is 40 Hz, the pure tone may be a 40 Hz sine wave or a 10,000 Hz sine wave that oscillates at a period of 40 Hz. A pure tone is also called a single tone, a master tone, or a carrier, and represents a sound generated by an oscillator or a vibration that becomes a master tone.

[0054] When the stimulation sound data includes the above-mentioned pure tones, the control unit 111 may perform envelope processing on the stimulation sound data, including processing related to attack and / or processing related to decay.

[0055] For example, the control unit 111 may process the stimulation sound data into a sound similar to a bass drum (bass drum). In this case, the control unit 111 may calculate an envelope for the stimulation sound data, and process the envelope in terms of attack, adjusting the timing at which the volume reaches a maximum, and in terms of decay, adjusting how the sound attenuates.

[0056] By performing the above processing, the stimulus sound data is envelope-processed so that it has the characteristics of a bass drum, and the stimulus sound data becomes similar to the sound of a bass drum.

[0057] Furthermore, the control unit 111 may process the stimulation sound data into a sound similar to a snare drum or a hi-hat. In this case, the control unit 111 may also synthesize sound data associated with a predetermined instrument sound with the stimulation sound data. For example, the control unit 111 may store preset sound data for a snare and / or sound data for a hi-hat, and synthesize the snare sound data or the hi-hat sound data with the stimulation sound data according to the application or user designation, thereby generating stimulation instrument sound data related to the snare and / or the hi-hat.

[0058] By performing the above process, the stimulus sound data is synthesized to have the characteristics of a snare or hi-hat, so that the stimulus sound data resembles the sounds of a snare or hi-hat. Furthermore, by synthesizing the bass drum, snare, and hi-hat, it becomes possible to produce a more natural drum sound.

[0059] (Processing B) The predetermined sound data may also include instrument sound data that does not include a specific frequency (e.g., 40 Hz) in a predetermined frequency band as a main component. For example, the instrument sound data may include sound data of drums, bass, keyboard, guitar, keyboard, etc., generated by normal playing or generation methods.

[0060] In the case of the above-mentioned instrument sound data, the control unit 111 may process the instrument sound data using a low frequency oscillator (LFO). For example, if it is desired to stimulate the user with a specific frequency of 40 Hz, the control unit 111 causes the LFO to oscillate at 40 Hz, thereby modulating the instrument sound data so that a 40 Hz vibration is generated.

[0061] Through the above process, by oscillating a specific frequency with an LFO, it is possible to modulate normal instrument sound data and stimulate the user with a specific frequency band or a specific frequency.

[0062] (Processing C) In addition, if the specified sound data includes instrument sound data that does not include a specific frequency (e.g., 40 Hz) in a specified frequency band as a main component, the control unit 111 may include synthesizing the differential frequency between the peak frequency of the instrument sound data and the specific frequency in the specified frequency band into the instrument sound data.

[0063] Here, we will explain why processing C, which synthesizes the differential frequency between the peak frequency of the instrument sound data and a specific frequency in a predetermined frequency band into the instrument sound data, can stimulate the user to a specific frequency.

[0064] First, we will explain why the LFO can stimulate a specific frequency. Hereinafter, θ1 is defined as the peak frequency of the original sound data, and θ2 is defined as the frequency at which the LFO oscillates. At this time, the frequency of the sound data modulated based on the frequency θ2 oscillated by the LFO is expressed by (Equation 1).

number

[0065] Here, in order to examine the time waveform of the modulated sound data, the modulated sound data is converted into an analytic signal by a Hilbert transform or the like, and the envelope is found by finding the absolute value (Equation 2).

number

[0066] When (Equation 2) is expanded using (Equation 3), it finally becomes √(3 / 2+2cosθ2+1 / 2cos(2θ2)).

number

[0067] As described above, the envelope of the modulated sound loses the original peak frequency θ1 and is affected only by the frequency θ2 generated by the LFO. This led the inventors to consider whether it might be possible to stimulate a specific frequency in a user listening to a synthesized sound by simply synthesizing a specific frequency with the peak frequency of the original sound data. In the following, the peak frequency θ1 is set to the specific frequency θ2. Equation 4 represents the frequency of the synthesized sound data obtained by synthesizing the peak frequency θ1 with the specific frequency θ2.

number

[0068] Here, in order to examine the time waveform of the synthetic sound data, the synthetic sound data is converted into an analytic signal by a Hilbert transform or the like, and the envelope is found by finding the absolute value (Equation 5).

number

[0069] When (Equation 5) is expanded using (Equation 6), it finally becomes √(2+2cos(θ1-θ2)).

number

[0070] Therefore, if you want to stimulate the user with a specific frequency of 40 Hz, and the peak frequency θ1 of the original sound data is, for example, 70 Hz, by setting the predetermined frequency (differential frequency) θ2 to 30 Hz, the frequency of 40 Hz can be stimulated to the user, since θ1-θ2=70 Hz-30 Hz. In other words, once the specific frequency to stimulate the user is determined, the differential frequency between the peak frequency of the original sound data and the specific frequency is calculated, and this differential frequency is synthesized into the sound data, thereby stimulating the user with the specific frequency. Therefore, even a simple process such as processing process C can stimulate the user with the specific frequency.

[0071] The control unit 111 also acquires predetermined music data. The predetermined music data may be music data of existing music, or music data of music newly generated by a generation AI (Artificial Intelligence) or the like. The control unit 111 may acquire predetermined music data selected by a user from a database in which music data is stored, or may acquire predetermined music data distributed by a music distribution service.

[0072] When predetermined music data is acquired, the control unit 111 analyzes the predetermined music data and extracts one or more instrument sound data. For example, the control unit 111 separates sound source data from existing music data using a publicly known AI technology for performing sound source separation. The sound source separation AI can input music data and output each sound source data by learning the characteristics of each sound source. Each sound source data may be, for example, vocals, drums, bass, guitar, keyboard, etc. The control unit 111 may be provided in a processing device capable of data communication with the information processing device 10 via a network, such as a server on a cloud.

[0073] Furthermore, the control unit 111 may select at least one piece of instrument sound data from one or more pieces of instrument sound data as the predetermined sound data, whereby the selected instrument sound data is acquired by the control unit 111 as the predetermined sound data.

[0074] The control unit 111 may also process at least one of the sound source-separated instrument sound data into stimulating instrument sound data. For example, the control unit 111 performs the above-described processing B or C on one piece of instrument sound data.

[0075] By the above processing, it becomes possible to separate existing musical instrument data into sound sources, extract each piece of sound source data, and generate stimulating musical instrument sound data in which a specific frequency is stimulated from the extracted sound source data.

[0076] Furthermore, the control unit 111 may synthesize the stimulating instrument sound data with predetermined music data. For example, the control unit 111 may replace sound source data separated from existing music with the stimulating instrument sound data or superimpose the sound source data on the original sound source data to generate stimulating music data.

[0077] The output unit 112 may also output synthesized stimulating music data including stimulating instrument sound data. This makes it possible to extract sound source data of a predetermined instrument sound from existing music data, replace this sound source data with stimulating instrument sound data, or superimpose the stimulating instrument sound data on the original sound source data, thereby generating and outputting stimulating music data that stimulates a predetermined frequency band or a specific frequency to the user.

[0078] By using the above-mentioned processing, it is possible to realize remixes that create derivative music by reconstructing the structure of the sound source using the sound source separation technology, which is a well-known technology.In addition, by using the sound source separation technology, it is possible to easily collect sound source materials to be processed.

[0079] The control unit 111 may also select a predetermined frequency band from multiple frequency bands. For example, the control unit 111 may select one frequency band from delta waves, theta waves, alpha waves, beta waves, and gamma waves. The control unit 111 may select one frequency band by default, or may select frequency bands in a predetermined order at predetermined times or at predetermined timings. The control unit 111 may also select a frequency band in response to a user selection via the user interface 150. The control unit 111 may generate stimulating music data with a different number of stimulating frequencies depending on the security level described above. For example, when a security level is set in response to a user operation, the control unit 111 may identify the number of frequencies according to the set security level, generate different stimulating instrument sound data equal to the identified number of frequencies, and synthesize these to generate stimulating music data.

[0080] The above processing makes it possible to change the frequency band that the user wants to stimulate or enhance, and when selecting according to user preference, the user can select the frequency band according to their current status or desired status.

[0081] <Example> Next, a specific example of processing will be described. FIG. 3 is a diagram showing an example of generating a sine wave according to the first embodiment. The example shown in FIG. 3 shows an example of generating a sine wave, which is a pure tone. The "Brain Wave" shown in FIG. 3 is a specific frequency that is desired to stimulate the user, and indicates 46.3 Hz. For example, if the "Brain Wave" is set to 40 Hz, it becomes possible to generate a 40 Hz sine wave pure tone. That is, the control unit 111 accepts a user operation, acquires the frequency input to the "Brain Wave," and generates a sine wave pure tone of this frequency. The "interval" indicates the interval between sounds, and indicates 1 second / 40 Hz = 25.0 ms.

[0082] "Interval from Br" indicates the "interval" adjusted by "Brain Wave", and the interval between sounds is adjusted by 46.3Hz. "Length" indicates the length of the sound, for example, 25.0ms. The original sound carrier is 10KHz.

[0083] In the example shown in Fig. 3, the basic setting is to repeat ON / OFF every 25 ms for the original sound carrier frequency of 10 kHz to generate 40 Hz, but the control unit 111 may change the ON / OFF interval to "interval from Br" so that the specific frequency of 46.3 Hz is generated from the sound. According to the above-mentioned process, the interval between sounds varies in accordance with the fluctuation of the specific frequency, and is adjusted so that the interval between sounds becomes the specific frequency. According to the example shown in Fig. 3, the control unit 111 can generate stimulation sound data that stimulates the specific frequency from scratch, and generate stimulation instrument sound data based on this stimulation sound data.

[0084] FIG. 4 is a diagram showing an example of processing A according to the first embodiment. In the example shown in FIG. 4, the control unit 111 performs envelope processing P10 on stimulation sound data W10 to make the stimulation sound data resemble the sound of a bass drum. The stimulation sound data W10 is a pure tone that itself generates fluctuations. The control unit 111 can automatically perform envelope processing by storing or having a machine learning model learn characteristics of the bass drum, such as the magnitude and timing of the attack and the amount of decay.

[0085] The control unit 111 also synthesizes sound data N10 to resemble the characteristics of a snare or hi-hat with the stimulation sound data W10. The control unit 111 may also process both envelope processing and sound data synthesis to resemble the sound of a predetermined instrument such as a bass drum, snare, or hi-hat.

[0086] Fig. 5 is a diagram showing an example of processing B according to the first embodiment. In the example shown in Fig. 5, the control unit 111 performs LFO processing on the instrument sound data W12, modulating it using a low frequency so as to emit a specific frequency W14. As a result, stimulating instrument sound data that stimulates the user with the specific frequency W14 is generated.

[0087] 6 is a diagram showing an example of processing C according to the first embodiment. In the example shown in FIG. 6, the control unit 111 specifies a peak frequency FP at a peak P12 for a graph G12 of frequency components of the musical instrument sound data W12. Next, the control unit 111 calculates a difference frequency F1 between the peak frequency FP and a specific frequency FT that is desired to stimulate the user, and adds the difference frequency F1 in the graph G14 to the frequency components of the musical instrument sound data W12 in the graph G12. As a result, after the addition of the difference frequency F1, stimulating musical instrument sound data that stimulates the user at the specific frequency FT is generated.

[0088] FIG. 7 is a diagram showing an example of remix processing according to the first embodiment. In the example shown in FIG. 7, music data M20 of an original song is music data that does not have, as its main component, a component in a predetermined frequency band that stimulates the user. The control unit 111 separates the sound source data for each instrument using a known sound source separation technique. For example, sound source data W20 represents a guitar waveform 1, sound source data W22 represents a bass waveform 2, and sound source data W24 represents a drum waveform 3.

[0089] For example, if it is difficult to convert the waveform 1 of the sound source data W20 into MIDI (Musical Instrument Digital Interface) information, the control unit 111 processes the sound source data W20 using LFO technology, thereby generating sound source data WS2 that can enhance specific frequencies.

[0090] Furthermore, the control unit 111 converts, for example, waveform 2 of the sound source data W22 into MIDI1 information. For example, the control unit 111 processes the MIDI1 information to produce a specific frequency (e.g., 40 Hz). For example, the control unit 111 performs LFO processing to oscillate at 40 Hz, and generates sound source data WS4 that maintains the waveform of the bass sound as much as possible.

[0091] The control unit 111 also converts, for example, waveform 3 of the sound source data W24 into MIDI2 information. For example, the control unit 111 uses the MIDI2 information to generate sound source data WS6 that maintains the waveform of the drum sound as much as possible through envelope processing and sound data synthesis.

[0092] The control unit 111 synthesizes the sound source data WS2, WS4, and WS6 to generate stimulating music data MS2 in which a predetermined frequency band or a specific frequency is enhanced. This allows the control unit 111 to reconstruct stimulating music data from existing music data that can stimulate the user in a predetermined frequency band or a specific frequency. Note that the instrument sounds corresponding to the above-mentioned instrument sounds are merely examples, and other processing may be performed.

[0093] By the above process, it is possible to generate stimulation music data that can stimulate the user's brain with specific frequencies that are different from one or more predetermined frequency bands that appear in the user's brain activity.

[0094] (Authentication processing) 2, the acquisition unit 113 acquires an electroencephalogram signal of a user who is listening to the stimulation music data output by the output unit 112 from an electroencephalogram measuring device worn by the user. For example, the acquisition unit 113 sequentially acquires electroencephalogram signals measured by bioelectrodes included in the earphone set 20 described below. The electroencephalogram measuring device is not limited to the earphone set 20.

[0095] The generation unit 114 generates an authentication model for authenticating a user from the user's electroencephalogram signal based on feature data of the user's electroencephalogram signal during stimulation by the stimulating music data. For example, the authentication model may be a model generated by the above-mentioned experiment, the analysis method may be Sweep StimPeak, and the machine learning may learn the low frequency and / or the frequency shift stimulated by the stimulating music data.

[0096] Through the above process, the user can listen to the stimulating music data while listening to it, which stimulates specific frequencies in the user's brain, making it possible to generate an authentication model for personal authentication without the user being aware of the recognition process. Therefore, no authentication task is required for authentication, improving usability. Furthermore, the stimulating music data that stimulates the user's brain with different frequencies may be stimulating music data that stimulates the user's brain with at least one frequency.

[0097] The output unit 112 may also output stimulus music data obtained by combining first stimulus music data that stimulates a first frequency and second stimulus music data that stimulates a second frequency. For example, the output unit 112 may output stimulus music data obtained by combining first stimulus instrument sound data that stimulates a first frequency and second stimulus instrument sound data that stimulates a second frequency, as in the stimulus music data used in the experiment. Note that the output unit 112 is not limited to outputting two stimulus instrument sound data, and may output stimulus music data obtained by combining three or more stimulus instrument sound data. The first frequency and the second frequency may be in the same frequency band as a predetermined frequency band in brain activity, or may be in a different frequency band.

[0098] Through the above process, the user can listen to stimulating music that synthesizes instrument sound data generated for each stimulating frequency, thereby generating an authentication model for personal authentication using more natural music and performing authentication processing.

[0099] Furthermore, the generation unit 114 may include, in the feature data, frequency data of each of different frequencies that stimulate the user's brain, among frequency data obtained by frequency-converting the electroencephalogram signal. For example, the generation unit 114 may identify different frequencies (individual-specific frequencies) that are likely to reveal the user's characteristics as frequency data to be used in the authentication model, as will be described later, and include the frequency data of these different frequencies in the feature data.

[0100] Through the above processing, it is possible to generate an authentication model using frequencies that are more likely to reveal characteristics of the user, and to use these as authentication data.

[0101] Furthermore, when an authentication model is generated from the stimulating music data output in the first stage, the output unit 112 may output stimulating music data that stimulates the user's brain with a different frequency in the second stage after the first stage. The output unit 112 may output stimulating music data in the learning stage, and then output stimulating music data in the subsequent authentication stage.

[0102] The acquiring unit 113 may also acquire the user's electroencephalogram signal in a second stage from the electroencephalogram measuring device. The second stage may be a stage after a predetermined period of time has elapsed, in which an authentication model is created, following the first stage, which is a learning stage in which the user's electroencephalogram signal is learned. The second stage may also be a time period (e.g., a different day) that is completely different from the first stage.

[0103] The authentication unit 115 inputs feature data of the user's electroencephalogram signal in the second stage into an authentication model, acquires the user's authentication result, and performs authentication. For example, if the authentication unit 115 knows the specific frequency stimulated by the stimulating music data, it inputs frequency data corresponding to this specific frequency into the authentication model as feature data. Note that the authentication unit 115 may input the user's electroencephalogram signal acquired in the second stage into the authentication model and output the authentication result.

[0104] Through the above process, it becomes possible to input the user's EEG signal and perform personal authentication using an authentication model generated using the user's own EEG signal. Furthermore, since the stimulation music data stimulates specific frequencies and is likely to activate the user's brain activity, it is possible to improve authentication accuracy by performing personal authentication using frequency data of these specific frequencies.

[0105] Furthermore, the authentication unit 115 may perform authentication processing using frequency data analyzed using the analysis method (Sweep StimPeak) used in this experiment. For example, the authentication unit 115 may use frequency data for a specific frequency to be stimulated, and may use average frequency data obtained by averaging surrounding frequency data including the specific frequency (frequency data within a predetermined range including the specific frequency) for frequencies other than the specific frequency.

[0106] The output unit 112 may also stop (or halt) the output of the stimulus music data when the authentication result by the authentication unit 115 indicates success. For example, the output unit 112 stops the output of the stimulus music data to indicate that the authentication is complete.

[0107] Through the above process, the user can understand that the authentication has ended because the output of the stimulus music data has stopped.

[0108] The output unit 112 may also combine audio data indicating the authentication result with the stimulating music data. For example, the output unit 112 reduces the volume of the stimulating music data and combines the audio data indicating the authentication result with the stimulating music data.

[0109] Through the above process, the user can understand the authentication result while being continuously stimulated by the stimulating music data.

[0110] The authentication unit 115 may also determine that authentication has failed if authentication is not successful within a predetermined period of time after starting the authentication process in the second stage. For example, even if authentication fails within the predetermined period of time, the authentication unit 115 may perform authentication processing using the user's electroencephalogram signals that are continuously acquired, and continue performing the authentication processing until authentication is successful. In this case, the predetermined period becomes the authentication time limit, and if authentication does not succeed even once within the predetermined period of time and continues to fail, it is finally determined that authentication has failed.

[0111] The above process eliminates the need to notify the user of a single authentication failure and allows authentication processing to be performed using the user's continuously acquired and updated EEG signals. In this case, to prevent the authentication processing from being performed endlessly, a predetermined time period may be set as a time limit, and the authentication processing may be performed continuously within the predetermined time period. Note that, since it is known that the probability of erroneous authentication can be reduced to nearly zero by using a method of rejecting a user if the user's identity is uncertain, this method can also be applied to real-world environments (similar to facial authentication in smartphones). Since it is known from Consideration 3 above that authentication accuracy is poor in certain situations such as threats, the authentication unit 115 may determine not to perform authentication if the acquired EEG signals of the user indicate a state that adversely affects authentication accuracy. Regarding the determination of a specific state (such as threats) using EEG signals, a machine learning model trained on EEG signals measured in advance during certain states can be used to determine whether the user is in a certain state.

[0112] <Operation> Next, the operation of the information processing device 10 according to the first embodiment will be described with reference to Fig. 8 and Fig. 9. Fig. 8 is a flowchart showing an example of the process of the information processing device 10 during learning according to the first embodiment.

[0113] In step S102, the output unit 112 outputs stimulation music data that stimulates the user's brain with different frequencies. Note that the output unit 112 may output from another device.

[0114] In step S104, the acquisition section 113 acquires an electroencephalogram signal of the user from the electroencephalogram measuring device worn by the user.

[0115] In step S106, the generation unit 114 generates an authentication model for authenticating a user from the user's electroencephalogram signal based on feature data of the user's electroencephalogram signal while the user is being stimulated by the stimulating music data. For example, frequency data obtained by frequency-converting the user's electroencephalogram signal may be input to a machine learning model to learn the feature data of the user's electroencephalogram signal. Alternatively, frequency data in a predetermined frequency range may be extracted from the user's frequency data, and the extracted frequency data may be input to the machine learning model to learn the feature data of the user's electroencephalogram.

[0116] By the above process, it is possible to generate an authentication model using the EEG signal that indicates the brain activity of the user's brain stimulated by the stimulus music data. This allows the user to perform active tasks for authentication in a natural way.

[0117] Fig. 9 is a flowchart showing an example of a process related to authentication according to the first embodiment. In the example shown in Fig. 9, the authentication process may be performed using the authentication model generated according to Fig. 8.

[0118] In step S202, the output unit 112 outputs stimulation music data that stimulates the user's brain with different frequencies. Note that the output unit 112 may output from another device.

[0119] In step S204, the acquisition unit 113 acquires the electroencephalogram signal of the user from the electroencephalogram measuring device worn by the user. For example, if the stage acquired in the process shown in Fig. 8 is the first stage, the acquisition unit 113 performs processing in the second stage after the first stage in the process shown in Fig. 9.

[0120] In step S206, the authentication unit 115 inputs the user's electroencephalogram signal into the authentication model. For example, the authentication unit 115 inputs frequency data obtained by frequency-converting the user's electroencephalogram signal into the authentication model. Furthermore, if the authentication unit 115 knows a specific frequency stimulated by the stimulating music data, it may input frequency data corresponding to this specific frequency into the authentication model as feature data.

[0121] In step S208, the authentication unit 115 uses the electroencephalogram signal and the authentication model to obtain the authentication result of the user and perform authentication. Note that the authentication unit 115 may input the electroencephalogram signal of the user obtained in the second stage into the authentication model and obtain the authentication result. Note that the generated authentication model may be stored in an external device (for example, stored in a server on the cloud), and the electroencephalogram signal may be output to the external device, and the authentication result may be obtained from the external device to perform authentication processing.

[0122] In step S210, the output unit 112 outputs the acquired authentication result. For example, the output unit 112 may output audio data indicating the authentication result from a speaker, earphones, etc., or may output the authentication result by performing control to indicate whether the authentication result is success or failure. For example, if the authentication is success, the output unit 112 may stop outputting the stimulation music data, and if the authentication is failure, the output unit 112 may continue outputting the stimulation music data, thereby outputting the authentication result. Furthermore, the output unit 112 may output the authentication result so as to display it on a display unit.

[0123] With the above processing, personal authentication can be realized by having the user listen to stimulating music that stimulates the user's brain with a specific frequency during personal authentication, and personal authentication can also be performed by listening to music in the background, improving usability.

[0124] [Second embodiment] In the second embodiment, the information processing device 10 may acquire an electroencephalogram signal of the user measured by an earphone set 20 described below, and execute the processing disclosed below using the electroencephalogram signal of the user. <Earphone set composition> 10 and 11 provide an overview of the earphone set 20 according to the second embodiment. Note that the earphone set 20 is not limited to the examples shown in FIGS. 10 and 11, and any earphones that can sense brain waves from the ear canal and output them to an external device can be applied to the technology of the present disclosure.

[0125] Fig. 10 is a diagram showing an example of an earphone set 20 according to the second embodiment. The earphone set 20 shown in Fig. 10 has a pair of earphones 200R, 200L and a neck strap 210. Each earphone 200R, 200L is connected to the neck strap 210 using a cable capable of signal communication, but may also be connected using wireless communication. Hereinafter, RL will be omitted when there is no need to distinguish between left and right.

[0126] The neck strap part 210 has a central member that fits along the back of the neck, and rod-shaped members (arms) 212R, 212L that curve along both sides of the neck. Electrodes 222, 224 that sense EEG signals are provided on the surface of the central member that comes into contact with the neck on the back side. Electrodes 222, 224 are an electrode that is connected to earth and a reference electrode, respectively. This allows the electrodes 222, 224 to be spaced apart from the elastic electrodes provided on the ear tips of the earphones, as will be described later, making it possible to acquire EEG signals with high accuracy. The neck strap part 210 may also have a processing unit that processes EEG signals and a communication device that communicates with the outside, but these processing units and communication units may be provided in the earphones 200.

[0127] Furthermore, the tip sides of the rod-shaped members 212R, 212L on both sides of the neck strap 210 are heavier than the base sides (center member sides), which allows the electrodes 222, 224 to be properly pressed against the neck of the wearer. For example, weights are provided on the tip sides of the rod-shaped members 212R, 212L. However, the positions of the electrodes 222, 224 are not limited to these positions.

[0128] Fig. 11 is a diagram showing an example of a schematic cross section of earphone 200R according to an embodiment. Earphone 200R shown in Fig. 11 may have elastic member 208 (e.g., urethane) between speaker 202 and nozzle 204, for example. Providing this elastic member 208 makes it difficult for vibrations from speaker 202 to be transmitted to the elastic electrode of ear tip 206, preventing sound interference between the elastic electrode of ear tip 206 and speaker 202.

[0129] Furthermore, ear tip 206, which includes an elastic electrode, is located at the sound guide port, but the elasticity of the elastic electrode itself makes it possible to prevent interference from sound vibrations. Also, by using an elastic material for the housing, this elastic material makes it difficult for sound vibrations to be transmitted to the elastic electrode of ear tip 206, making it possible to prevent interference from sound vibrations.

[0130] The earphone 200 includes an audio sound processor.

[0131] Furthermore, the ear tip 206 conducts the brain wave signal sensed from the ear canal to the contact of an electrode provided in the nozzle 204. The brain wave signal is transmitted from the ear tip 206 via the contact to a biosensor (not shown) inside the earphone 200. The biosensor outputs the sequentially acquired brain wave signals via a cable to a processing device provided in the neck strap 210 or transmits them to an external device. Furthermore, the ear tip 206 may be insulated from the housing containing the biosensor and audio sound processor.

[0132] By using the above-described earphone set 20, it is possible to measure the user's electroencephalogram signal in real time, modulate the personal frequency identified from this real-time electroencephalogram signal, and output stimulation sound data from the earphone set 20.

[0133] The user may wear, for example, headgear that measures brain waves using the International 10 / 20 system as the brain wave measuring device. The brain wave measuring device may include devices capable of measuring brain waves, such as a measuring device that uses scalp electrodes, a measuring device that measures brain activity using intracranial electrodes, a measuring device that measures brain activity using functional magnetic resonance imaging (fMRI), or a measuring device that measures brain activity using near-infrared spectroscopy (NIRS). In this case, the acquiring unit 113 may acquire a personal frequency based on the brain wave signal acquired from the brain wave measuring device, and the output unit 112 may output stimulating music data via a speaker or the like.

[0134] The acquisition unit 113 of the information processing device 10 acquires an electroencephalogram signal from an electroencephalogram measuring device (for example, an earphone set 20) worn by the user.

[0135] The authentication unit 115 performs authentication processing based on the electroencephalogram signal acquired by the acquisition unit 113 and an authentication model generated using the user's electroencephalogram signal. In addition, in the disclosed technology, stimulation music data is used to stimulate the user's brain, and personal authentication is performed by utilizing the fact that the degree of stimulation varies from user to user.

[0136] The output unit 112 outputs to the user the result of authentication performed by the authentication unit 115. For example, the output unit 112 may output to a display unit of the information processing device 10 the authentication result based on the electroencephalogram signal of the user stimulated by the stimulation music data.

[0137] Through the above process, the user can grasp the result of authentication while being stimulated by the stimulating music data.

[0138] The electroencephalogram measuring device is included in the earphone 200, and the output unit 112 outputs the stimulation sound data from the earphone 200 via a wired or wireless connection.

[0139] <Modification> The above-described embodiments are examples for explaining the technology of the present disclosure, and are not intended to limit the technology of the present disclosure to only the embodiments and examples. The technology of the present disclosure can be modified in various ways without departing from the spirit of the present disclosure. Note that each sound data, such as the predetermined sound data, instrument sound data, stimulating instrument sound data, and sound source data, may be referred to as nth sound data (n is an integer equal to or greater than 1). Each processing step in the control unit 111 of the present disclosure may be independently implemented as a program to be executed by a computer, thereby enabling only the program corresponding to the required processing step to be installed and executed on the computer. <Variation 1> In the above embodiment, it has been explained that the frequency to stimulate the user (for example, gamma waves of 30 to 50 Hz) can be set as appropriate, but in variant example 1, a personally characteristic frequency (also called an "individual specific frequency") that stimulates each user appropriately is identified, and this individual specific frequency is set as the target frequency.

[0140] For example, the gamma wave band is said to be 30 Hz to 100 Hz, and the inventors have found through the above-mentioned experiment that the gamma waves that are appropriately stimulated vary from user to user. For example, when each user is asked to listen to a series of sounds containing gamma waves in the range of 30 Hz to 50 Hz, the EEG signal of user A may peak at 42 Hz, while the EEG signal of user B may peak at 39 Hz.

[0141] In light of the above, in Modification 1, as a pre-processing step, the user is asked to listen to sounds in a predetermined frequency band to which stimulation is to be applied while the frequencies are changed in sequence, and the frequency within the predetermined frequency band at which the user receives an appropriate stimulation is identified. For example, in the case of the theta wave frequency band, sounds having frequencies from 30 Hz to 50 Hz are output from the earphone set 20 to the user and the user is asked to listen to the sounds. While the user is listening to the sounds, the earphone set 20 measures the user's electroencephalogram (EEG) signals and outputs the results to the information processing device 10.

[0142] The information processing device 10 frequency-converts the EEG signals sequentially acquired from the earphone set 20, and identifies, for example, frequencies between 30 Hz and 50 Hz at which the spectral power value peaks as individual-specific frequencies. For example, the acquisition unit 113 sequentially acquires EEG signals from the earphone set 20, and the control unit 111 frequency-converts the EEG signals generated sequentially, and identifies individual-specific frequencies based on each frequency signal in a predetermined frequency band that has been frequency-converted. The control unit 111 may use a band-pass filter to extract each frequency signal in the predetermined frequency band and then identify the individual-specific frequencies. The control unit 111 sets the identified individual-specific frequencies to target frequencies that are desired to stimulate the user.

[0143] Once the target frequency is set by the control unit 111, other processes are the same as those disclosed in the embodiment. For example, the control unit 111 generates stimulating music data related to instrument sounds that stimulate the user using the above-described processing based on the individual specific frequency in the predetermined frequency band.

[0144] Furthermore, the control unit 111 generates stimulation sound data to stimulate the user using the above-mentioned processing etc. based on the individual specific frequencies extracted using the spectral power values ​​of the electroencephalogram signals acquired in sequence. As a specific example, the control unit 111 may perform envelope processing including processing related to attack and / or processing related to decay on the predetermined sound data, as described above.

[0145] The control unit 111 may also synthesize sound data associated with a predetermined musical instrument sound with the predetermined sound data. The control unit 111 may also generate stimulation sound data using a low-frequency oscillator.

[0146] In addition, when the control unit 111 acquires predetermined sound data including musical instrument sound data, the control unit 111 may also synthesize the differential frequency between the frequency acquired by converting the predetermined sound data and the personal identification frequency into the predetermined sound data.

[0147] This makes it possible to identify a characteristic frequency to which the user's brain responds appropriately within a predetermined frequency band to stimulate the user, and to stimulate the user using that frequency. Furthermore, the electroencephalogram measuring device is not limited to the earphone set 20. By using this frequency to generate an authentication model and perform authentication processing, it becomes possible to perform personal authentication.

[0148] <Variation 2> In Modification 2, the personal authentication of the present disclosure may be applied to personal authentication processing in a digital space. For example, when an avatar receives a predetermined service in the metaverse space, authentication processing of the user who uses the avatar may be performed as authentication processing for the avatar, and personal authentication may be performed by playing stimulating music data to the user. [Explanation of symbols]

[0149] 10. Information processing equipment 20 earphone sets 110 processors 111 Control Unit 112 Output section 113 Acquisition Department 114 Generation part 115 Authentication Department 120 Network Communication Interface 130 Storage device 150 User Interface

Claims

1. A processor included in the information processing device Outputting stimulation music data that stimulates the user's brain with different frequencies; acquiring an electroencephalogram signal of the user from an electroencephalogram measuring device worn by the user; generating an authentication model for authenticating the user from the electroencephalogram signal of the user based on feature data of the electroencephalogram signal of the user during stimulation with the stimulation music data; An information processing method that performs the above.

2. The outputting step includes:

2. The information processing method according to claim 1, further comprising outputting stimulus music data obtained by combining first stimulus music data that stimulates the first frequency and second stimulus music data that stimulates the second frequency.

3. The generating step comprises: The information processing method according to claim 1 , wherein the feature data includes at least the frequency data of each of the different frequencies among the frequency data obtained by frequency-converting the electroencephalogram signal.

4. When the authentication model is generated by the stimulus music data output in the first step, The outputting step includes: a second step after the first step, in which stimulation music data is outputted to stimulate the user's brain with the different frequencies; The obtaining includes: acquiring an electroencephalogram signal of the user at the second stage from the electroencephalogram measuring device; The information processing method according to claim 1 , further comprising: authenticating the user based on the electroencephalogram signal of the user in the second stage and the authentication model.

5. The outputting step includes: The information processing method according to claim 4 , further comprising: stopping output of the stimulus music data when the result of the authentication indicates success.

6. The outputting step includes:

5. The information processing method according to claim 4, further comprising synthesizing audio data indicating the authentication result with the stimulating music data.

7. The authentication is performed by:

5. The information processing method according to claim 4, further comprising determining that the authentication has failed if the authentication is not successful within a predetermined period of time after the start of the authentication process in the second stage.

8. The information processing method according to claim 1 , wherein the number of frequencies of the stimulation music data that stimulates the user's brain varies depending on a security level.

9. The authentication is performed by: The information processing method according to claim 4 , wherein the authentication is not performed when the acquired electroencephalogram signal of the user is determined to be in a specific state.

10. the electroencephalogram measuring device is included in an earphone; The outputting step includes: The information processing method according to claim 1 , further comprising outputting the stimulus music data from the earphone.

11. A processor included in the information processing device Outputting stimulation music data that stimulates the user's brain with different frequencies; acquiring an electroencephalogram signal of the user from an electroencephalogram measuring device worn by the user; extracting feature data of the electroencephalogram signal of the user during stimulation with the stimulation music data; generating an authentication model for authenticating the user from the electroencephalogram signal of the user based on the feature data; A program that executes the following.

12. An information processing device including a processor, the processor: Outputting stimulation music data that stimulates the user's brain with different frequencies; acquiring an electroencephalogram signal of the user from an electroencephalogram measuring device worn by the user; extracting feature data of the electroencephalogram signal of the user during stimulation with the stimulation music data; generating an authentication model for authenticating the user from the electroencephalogram signal of the user based on the feature data; An information processing device that executes the above.

Images