Information processing method, recording media, and information processing device

Abstract

To reduce sense of discomfort as an instrument sound, even if the instrument sound stimulates a user with frequencies in a specified frequency band.SOLUTION: For making it possible to reduce sense of discomfort as a musical instrument sound, even if a sound of a musical instrument stimulates a user with a frequency in a predetermined frequency band, an information processing method causes a processor (110) included in an information processing device to execute: generating stimulus musical instrument sound data related to a musical instrument sound that stimulates the user with the predetermined frequency band (S104); and outputting the stimulus musical instrument sound data (S106).SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to an information processing method, a recording medium, and an information processing device. [Background technology]

[0002] In recent years, it has been shown that 40 Hz gamma waves can improve Alzheimer's disease, as a result of halving the amount of amyloid β accumulated in the brains of mice by stimulating them with 40 Hz gamma waves (see, for example, Non-Patent Document 1). In addition, a technology is known that generates music corresponding to the amplitude and frequency of biometric data and expresses the user's current state through music (see, for example, Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Martorell et al., "Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-Associated Pathology and Improves Cognition", [online], Cell 177, 256-271. April 4, 2019, [Retrieved July 13, 2020], Internet<https: / / www.cell.com / cell / pdf / S0092-8674(19)30163-1.pdf> [Patent documents]

[0004] [Patent Document 1] U.S. Pat. No. 10,369,323 Summary of the Invention [Problem to be solved by the invention]

[0005] According to the experimental results of Non-Patent Document 1, when considering providing a person with a gamma wave sound stimulus at 40 Hz, it is difficult for the person to continuously listen to a monotonous sound, and it is mentally burdensome. In addition, the same problem occurs when making a person listen to a sound that is composed of a specific frequency, not limited to 40 Hz. In addition, in Patent Document 1, music is generated based on the user's biometric data, and relaxing music is generated when the user is in a relaxed state, so the specific frequency is not continuously played.

[0006] That is, in the prior art, there was no technology that could continuously stimulate a user with sounds having frequencies in a specific frequency band while reducing mental stress. Also, when frequencies in a specific frequency band are synthesized with music, the sound image of the existing music is lost, and it feels strange to listen to it as music.

[0007] Therefore, one aspect of the disclosed technology aims to provide an information processing method, a recording medium, and an information processing device that make it possible to reduce the sense of incongruity as an instrument sound, even if the instrument sound has frequencies in a specific frequency band that stimulate the user. [Means for solving the problem]

[0008] An information processing method according to one aspect of the disclosed technology includes a processor included in an information processing device generating stimulating instrument sound data relating to an instrument sound that stimulates a user in a predetermined frequency band, and outputting the stimulating instrument sound data. Effect of the Invention

[0009] According to one aspect of the disclosed technique, even if the frequency of a musical instrument sound is in a predetermined frequency band and stimulates the user, it is possible to reduce the sense of discomfort felt as if it were a musical instrument sound. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of an information processing apparatus according to an embodiment; [Diagram 2] FIG. 13 is a diagram illustrating an example of generation of a sine wave according to an embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of processing process A according to the embodiment. [Figure 4] FIG. 11 is a diagram showing an example of processing process B according to the embodiment. [Diagram 5] FIG. 11 is a diagram showing an example of a processing process C according to the embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of a remix process according to the embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of time division according to the embodiment. [Figure 8] 10 is a flowchart illustrating an example of processing of the information processing device according to the embodiment. [Figure 9] 10 is a flowchart illustrating an example of processing related to the embodiment. [Figure 10] FIG. 13 is a diagram showing the average value of PLI in each frequency band in the electroencephalogram measured from each subject. [Figure 11] FIG. 13 is a diagram showing the average value of 40 Hz PLI in the electroencephalogram measured from each subject. [Figure 12] FIG. 13 is a diagram showing the average 40 Hz power of electroencephalograms measured from each subject. [Figure 13] FIG. 13 shows the results of a questionnaire given to each subject. [Figure 14] FIG. 1 is a diagram showing an example of a feature of a gammaband. [Figure 15] FIG. 13 is a diagram showing an example of a drum feature. [Figure 16] FIG. 13 is a diagram showing an example of bass characteristics. [Figure 17] FIG. 13 is a diagram showing an example of the features of Keyb (Synth). [Figure 18] FIG. 13 is a diagram showing an example of a feature of a CtrlBand. [Figure 19] FIG. 13 is a diagram showing an example of the characteristics of CtrlBand+Star. [Figure 20] FIG. 13 is a diagram showing an example of the characteristics of CtrlBand+NMethods as a reference example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and application of techniques not specified below. In other words, the present invention can be implemented with various modifications within the scope of the gist of the present invention. In addition, in the following description of the drawings, the same or similar parts are represented by the same or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. The drawings may also include parts with different dimensional relationships and ratios.

[0012] [Embodiment] An outline of an embodiment of the present disclosure will be described below. <Summary of the Disclosed Technology> In the disclosed technology, in order to enhance a specific frequency band in electroencephalograms, musical instrument sound data (also referred to as "stimulating musical instrument sound data") is generated that stimulates the user with frequencies in the specific frequency band. By generating the stimulating musical instrument sound data, it becomes possible to generate music using the stimulating musical instrument sound data. In addition, by having the user listen to the generated music, it becomes possible to provide the user with a stimulation of the specific frequency band. This makes it possible to stimulate the user with the specific frequency band while reducing the sense of discomfort felt by the user, thereby exerting the effect of the specific frequency band. The generation of the above-mentioned musical instrument sound data will be described below with reference to the drawings.

[0013] <Configuration example of information processing device 10> 1 is a block diagram showing an example of an information processing device 10 according to an embodiment. The information processing device 10 is, for example, a personal computer, and may be composed of one or more devices. The information processing device 10 processes sound data and generates, for example, musical instrument sound data that stimulates a user in a predetermined frequency band. The information processing device 10 is not necessarily a personal computer, and may be a server, a smartphone having information processing capabilities, a tablet terminal, or the like.

[0014] The information processing device 10 includes one or more processors (CPU: Central Processing Unit) 110, one or more network communication interfaces 120, a storage device 130, a user interface 150, and one or more communication buses 170 for interconnecting these components.

[0015] User interface 150 includes a display device (not shown) and input devices (not shown), such as a keyboard and / or a mouse or some other pointing device.

[0016] The storage device 130 may be, for example, a high-speed random access memory such as a DRAM, SRAM, DDR RAM, or other random access solid-state storage device, or may be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The storage device 130 may be, for example, a computer-readable non-transitory recording medium that records a program that causes a processor to execute the processes described below.

[0017] As another example, storage device 130 may be one or more storage devices located remotely from processor 110. In one embodiment, storage device 130 stores the following programs, modules and data structures, or a subset thereof:

[0018] The one or more processors 110 read and execute a program from the storage device 130 as necessary. For example, the one or more processors 110 configure a control unit 111 that executes the processing of the disclosed technology by executing a program stored in the storage device 130. The control unit 111 may also configure an acquisition unit 112, a generation unit 113, an output unit 114, an extraction unit 115, and a division unit 116.

[0019] The acquiring unit 112 acquires the predetermined sound data. For example, the acquiring unit 112 may acquire the predetermined sound data stored in the storage device 130, may acquire the predetermined sound data from an external device via the network communication interface 120, or may acquire the predetermined sound data generated by the user operating the user interface 150.

[0020] The generating unit 113 generates stimulating instrument sound data related to an instrument sound that stimulates a predetermined frequency band to a user. For example, the generating unit 113 may use the user interface 150 (see FIG. 2, etc., described later) to generate stimulating instrument sound data based on stimulating sound data, described later. The generating unit 113 may include first to third processing units 113a to c, and process the predetermined sound data acquired by the acquiring unit 112 into stimulating instrument sound data. For example, the first to third processing units 113a to c perform a predetermined processing process on the predetermined sound data, and generate stimulating instrument sound data including a predetermined frequency in a predetermined frequency band as a main component.

[0021] The predetermined frequency band includes at least one of the frequency bands of Δ (delta) waves (0.5 to 4 Hz), θ (theta) waves (4 to 8 Hz), α (alpha) waves (8 to 12 Hz), β (beta) waves (12 to 30 Hz), and γ (gamma) waves (30 Hz to 100 Hz) related to brain waves generated by brain activity. The generating unit 113 may generate stimulating instrument sound data including a specific predetermined frequency as a main component among the predetermined frequency bands. For example, the generating unit 113 may generate stimulating instrument sound data including a 40 Hz frequency component as a main component among gamma waves, with 40 Hz being the predetermined frequency. This makes it possible to stimulate the user with a specific frequency, thereby stimulating the user with a specific frequency band including the specific frequency.

[0022] The predetermined processing by the generation unit 113 includes processing the predetermined sound data so that it has the characteristics of each musical instrument sound such as drums, bass, guitar, keyboard (piano), etc. For example, the processing includes processing the envelope of the predetermined sound data, and synthesizing sound data having the characteristics of the musical instrument sound. The generation unit 113 may also select a processing process corresponding to a musical instrument sound designated by a user or the like based on processing information in which a processing process is set for each musical instrument sound, and process the predetermined sound data by the selected processing process.

[0023] The output unit 114 outputs the stimulating instrument sound data generated by the generation unit 113. For example, the output unit 114 outputs the generated stimulating instrument sound data to a speaker and controls the speaker to output the stimulating instrument sound. The output unit 114 may also synthesize the generated stimulating instrument sound data with other sound data and output the synthesized data. The speaker may be provided in the information processing device 10, or may be connected to the information processing device 10 via a network.

[0024] The above processing makes it possible to reduce the sense of incongruity of an instrument sound even if the instrument sound stimulates the user with a frequency in a predetermined frequency band. For example, the processing by the generation unit 113 makes it possible to generate stimulating instrument sound data that has a frequency in a predetermined frequency band but has characteristics similar to those of the instrument sound. This allows the listener to listen to the stimulating instrument sound data without feeling much incongruity as an instrument sound. Below, the processing will be described using three methods as examples.

[0025] (Processing A) The predetermined sound data may include stimulation sound data including a pure tone that stimulates a predetermined frequency band to a user. For example, if the predetermined frequency band is a gamma wave and the specific frequency is 40 Hz, the pure tone may be a 40 Hz sine wave, or a sine wave having a frequency of 10,000 Hz that oscillates at a period of 40 Hz. A pure tone is also called a single tone, a base tone, or a carrier, and represents a sound generated by an oscillator or a vibration that becomes a base tone.

[0026] When the stimulation sound data includes the above-mentioned pure tone, the first processing unit 113a may perform envelope processing including processing related to attack and / or processing related to decay on the stimulation sound data.

[0027] For example, the first processing unit 113a may process the stimulation sound data into a sound similar to a bass drum (base drum). In this case, the first processing unit 113a may calculate an envelope for the stimulation sound data, adjust the timing at which the volume reaches a maximum value as processing related to the attack of the envelope, and process how the sound attenuates as processing related to the decay.

[0028] By carrying out the above-mentioned processing, the envelope processing is performed on the stimuli sound data so that the stimuli sound data has the characteristics of a bass drum, so that the stimuli sound data becomes similar to the sound of a bass drum.

[0029] The first processing unit 113a may process the stimulation sound data into a sound similar to a snare or hi-hat of a drum. In this case, the first processing unit 113a may include synthesizing sound data associated with a predetermined instrument sound with respect to the stimulation sound data. For example, the first processing unit 113a may store preset sound data for a snare and / or sound data for a hi-hat, and synthesize the sound data for the snare or the sound data for the hi-hat with the stimulation sound data according to the purpose or user designation to generate stimulation instrument sound data related to the snare and / or the hi-hat.

[0030] By synthesizing the sound data with the characteristics of the snare or hi-hat of the drums, the stimulus sound data becomes similar to the sound of the snare or hi-hat of the drums. Also, by synthesizing the bass drum, snare, and hi-hat of the drums, it becomes possible to produce a more natural drum sound.

[0031] (Processing B) The predetermined sound data may also include instrument sound data that does not include a specific frequency (e.g., 40 Hz) in the predetermined frequency band as a main component. For example, the instrument sound data includes sound data of drums, bass, keyboard, guitar, keyboard, etc., which are normally played or generated.

[0032] In the case of the above-mentioned instrument sound data, the second processing unit 113b may process the instrument sound data using a low frequency oscillator (LFO). For example, when it is desired to stimulate a specific frequency of 40 Hz to the user, the second processing unit 113b modulates the instrument sound data by oscillating 40 Hz with the LFO so that a vibration of 40 Hz is generated.

[0033] Through the above process, it becomes possible to stimulate the user with a specific frequency band or a specific frequency by modulating normal instrument sound data using an LFO.

[0034] (Processing C) In addition, when the specified sound data includes instrument sound data that does not include a specific frequency in a specified frequency band (e.g., 40 Hz) as a main component, the third processing unit 113c may include synthesizing the differential frequency between the peak frequency of the instrument sound data and the specific frequency in the specified frequency band into the instrument sound data.

[0035] Here, we will explain why processing process C, in which the differential frequency between the peak frequency of instrument sound data and a specific frequency in a specified frequency band is synthesized with the instrument sound data, can stimulate the user to a specific frequency.

[0036] First, we will explain why the above-mentioned LFO can stimulate a specific frequency. In the following, θ1 is the peak frequency of the original sound data, and θ2 is the frequency at which the LFO oscillates. At this time, the frequency of the sound data modulated based on the frequency θ2 oscillated by the LFO is expressed by (Equation 1).

number

[0037] Here, in order to examine the time waveform of the modulated sound data, the modulated sound data is converted into an analytic signal by a Hilbert transform or the like, and the envelope is found by finding the absolute value (Equation 2).

number

[0038] When (Equation 2) is expanded using (Equation 3), it ultimately becomes √(3 / 2+2cosθ2+1 / 2cos(2θ2)).

number

[0039] As described above, the envelope of the modulated sound is affected only by the frequency θ2 oscillated by the LFO, with the original peak frequency θ1 disappearing. From this, the inventors thought that it might be possible to stimulate a specific frequency in a user who hears the synthetic sound by simply synthesizing a specific frequency with the peak frequency of the original sound data. In the following, the peak frequency θ1 is set to the specific frequency θ2. (Equation 4) represents the frequency of synthetic sound data obtained by synthesizing the peak frequency θ1 with the specific frequency θ2.

number

[0040] Here, in order to examine the time waveform of the synthetic voice data, the synthetic voice data is transformed into an analytic signal by a Hilbert transform or the like, and the envelope is found by finding the absolute value (Equation 5).

number

[0041] When (Equation 5) is expanded using (Equation 6), it finally becomes √(2+2cos(θ1-θ2)).

number

[0042] Therefore, when it is desired to stimulate the user with 40Hz as a specific frequency, if the peak frequency θ1 of the original sound data is, for example, 70Hz, by setting the predetermined frequency (differential frequency) θ2 to 30Hz, the frequency of 40Hz can be stimulated to the user with θ1-θ2=70Hz-30Hz. In other words, if the specific frequency to stimulate the user is determined, the differential frequency between the peak frequency and the specific frequency of the original sound data is obtained, and this differential frequency is synthesized into the sound data, so that the specific frequency can be stimulated to the user. Therefore, even a simple process such as the processing process C can stimulate the user with the specific frequency.

[0043] The acquiring unit 112 also acquires predetermined music data. The predetermined music data may be existing music data or may be newly generated music data by a generating AI (Artificial Intelligence) or the like. The acquiring unit 112 may acquire predetermined music data selected by a user from a database in which music data is stored, or may acquire predetermined music data distributed by a music distribution service.

[0044] When the acquisition unit 112 acquires the predetermined music data, the extraction unit 115 analyzes the predetermined music data to extract one or more instrument sound data. For example, the extraction unit 115 separates the sound source data from the existing music data using a publicly known AI technology for performing sound source separation. The sound source separation AI can input music data and output each sound source data by learning the characteristics of each sound source. Each sound source data is, for example, vocals, drums, bass, guitar, keyboard, etc. The extraction unit 115 may be provided in a processing device capable of data communication with the information processing device 10 via a network, for example, a server on a cloud.

[0045] Furthermore, the extraction unit 115 may select at least one piece of instrument sound data from among one or more pieces of instrument sound data as the predetermined sound data. As a result, the selected instrument sound data is acquired by the acquisition unit 112 as the predetermined sound data.

[0046] The generation unit 113 may also process at least one of the sound source-separated instrument sound data into stimulating instrument sound data. For example, the generation unit 113 performs the above-mentioned processing B or processing C on one instrument sound data.

[0047] By the above processing, for example, it becomes possible to perform sound source separation of existing musical instrument data to extract each sound source data, and generate stimulating musical instrument sound data in which a specific frequency is stimulated from the extracted sound source data.

[0048] In addition, the generating unit 113 may synthesize the stimulating instrument sound data with the predetermined music data. For example, the generating unit 113 may replace the sound source data separated from the existing music with the stimulating instrument sound data or may superimpose the sound source data on the original sound source data to generate the music data.

[0049] The output unit 114 may output the synthesized music data including the stimulating instrument sound data. This makes it possible to extract sound source data of a predetermined instrument sound from existing music data, replace this sound source data with the stimulating instrument sound data, or superimpose the stimulating instrument sound data on the original sound source data, to generate and output music data that stimulates a predetermined frequency band or a specific frequency to the user.

[0050] Through the above process, it is possible to realize a remix of a song in a derivative creative manner by reconstructing the composition of the sound source using the well-known sound source separation technology. In addition, by using the sound source separation technology, it is possible to easily collect the sound source material to be processed.

[0051] Furthermore, the division unit 116 divides the predetermined music data by time. For example, the division unit 116 divides the predetermined music data for each predetermined time period or each development of the song (such as the A melody, B melody, and chorus) and divides it into a plurality of music data.

[0052] In this case, the generating unit 113 may process each of the plurality of music data divided by the dividing unit 116 into stimulating instrument sound data. For example, the third processing unit 113c may identify a peak frequency in each of the plurality of music data divided by the dividing unit 116. Furthermore, the third processing unit 113c may calculate a difference frequency between the peak frequency identified in each of the plurality of music data and a specific frequency. The third processing unit 113c may execute processing process C using the difference frequency. Furthermore, the generating unit 113 may execute processing process B on each of the plurality of music data.

[0053] By the above processing, it is possible to time-divide music data in real time and stimulate a specific frequency in the user in real time according to the characteristics of the divided music data. For example, by performing appropriate processing on each time-divided music data of streaming music, it is possible to change it into music that can stimulate a specific frequency in real time.

[0054] The control unit 111 may also select a predetermined frequency band from among a plurality of frequency bands. For example, the control unit 111 may select one frequency band from among delta waves, theta waves, alpha waves, beta waves, and gamma waves. The control unit 111 may select one frequency band by default, or may select a frequency band in a preset order at predetermined times or at predetermined timings. The control unit 111 may also select a frequency band in response to a user selection via the user interface 150.

[0055] The above processing makes it possible to change the frequency band to which the user wishes to be stimulated or enhanced, and also makes it possible for the user to select the frequency band according to his / her current status or desired status when selecting according to user preference.

[0056] Furthermore, the generating unit 113 may specify the type of the predetermined sound data acquired by the acquiring unit 112, and perform processing according to the type of the predetermined sound data. For example, the generating unit 113 performs frequency analysis on the predetermined sound data and / or obtains similarity with musical instrument sound data to specify a pure tone, a stimulating sound, or a musical instrument sound that does not stimulate the predetermined frequency band to the user. The generating unit 113 may perform processing A if the predetermined sound data is pure tone data or stimulating sound data, and may perform processing B or C if the predetermined sound data is musical instrument sound data. This makes it possible to automatically generate appropriate stimulating musical instrument sound data by inputting the predetermined sound data.

[0057] <Example> Next, a specific example of processing will be described. FIG. 2 is a diagram showing an example of generating a sine wave according to an embodiment. In the example shown in FIG. 2, an example of generating a sine wave, which is a pure tone, is shown. The "Brain Wave" shown in FIG. 2 is a specific frequency that is desired to stimulate the user, and indicates 46.3 Hz. For example, if the "Brain Wave" is set to 40 Hz, it becomes possible to generate a sine wave pure tone of 40 Hz. That is, the generating unit 113 accepts a user operation, acquires the frequency input to the "Brain Wave", and generates a sine wave pure tone of this frequency. "Interval" indicates the interval between sounds, and indicates 1 second / 40 Hz=25.0 ms.

[0058] "Interval from Br" indicates the "interval" adjusted by "Brain Wave", and the interval between sounds is adjusted by 46.3Hz. "Length" indicates the length of the sound, for example, 25.0ms. The original sound carrier is 10KHz.

[0059] In the example shown in Fig. 2, the basic setting is to repeat ON / OFF every 25 ms for the frequency of the original sound carrier of 10 KHz to generate 40 Hz, but the generation unit 113 may change the ON / OFF interval to "interval from Br" so that a specific frequency of 46.3 Hz is generated from the sound. According to the above-mentioned process, the interval between sounds varies according to the fluctuation of the specific frequency, and is adjusted so that the interval between sounds becomes the specific frequency. According to the example shown in Fig. 2, the generation unit 113 can generate stimulating sound data that stimulates the specific frequency from scratch, and generate stimulating instrument sound data based on this stimulating sound data.

[0060] FIG. 3 is a diagram showing an example of processing A according to the embodiment. In the example shown in FIG. 3, the first processing unit 113a can make the stimulating sound data W10 resemble the sound of a bass drum by performing envelope processing P10 on the stimulating sound data W10. The stimulating sound data W10 is a pure tone itself that fluctuates. The first processing unit 113a can automatically perform envelope processing by storing or having a machine learning model learn the magnitude and timing of attack, the amount of decay, and the like, which are characteristics of the bass drum, in advance.

[0061] The first processing unit 113a synthesizes sound data N10 to resemble the characteristics of a snare or a hi-hat with respect to the stimulation sound data W10. The first processing unit 113a may also process both envelope processing and sound data synthesis to resemble a predetermined musical instrument sound such as a bass drum, snare, or hi-hat.

[0062] Fig. 4 is a diagram showing an example of processing process B according to the embodiment. In the example shown in Fig. 4, the second processing unit 113b performs LFO processing on the instrument sound data W12, modulating it using a low frequency so as to emit a specific frequency W14. As a result, stimulating instrument sound data that stimulates the user with the specific frequency W14 is generated.

[0063] FIG. 5 is a diagram showing an example of processing process C according to the embodiment. In the example shown in FIG. 5, the third processing unit 113c specifies a peak frequency FP at the peak P12 for the graph G12 of the frequency components of the musical instrument sound data W12. Next, the third processing unit 113c calculates a difference frequency F1 between the peak frequency FP and a specific frequency FT to be stimulated by the user, and adds the difference frequency F1 in the graph G14 to the frequency components of the musical instrument sound data W12 in the graph G12. As a result, after the addition of the difference frequency F1, stimulating musical instrument sound data that stimulates the specific frequency FT to the user is generated.

[0064] FIG. 6 is a diagram showing an example of a remix process according to the embodiment. In the example shown in FIG. 6, music data M20 of an original song is music data that does not have a component in a predetermined frequency band that stimulates the user as a main component. The extraction unit 115 separates the sound source data for each instrument using a known sound source separation technique. For example, the sound source data W20 represents a guitar waveform 1, the sound source data W22 represents a bass waveform 2, and the sound source data W24 represents a drum waveform 3.

[0065] For example, when it is difficult to convert the waveform 1 of the sound source data W20 into MIDI (Musical Instrument Digital Interfac) information, the generating unit 113 processes the sound source data W20 using an LFO technique, thereby generating sound source data WS2 capable of boosting a specific frequency.

[0066] Furthermore, the generating unit 113 converts, for example, the waveform 2 of the sound source data W22 into MIDI1 information. For example, the generating unit 113 processes the MIDI1 information to produce a specific frequency (e.g., 40 Hz). For example, the generating unit 113 performs LFO processing to oscillate at 40 Hz, and generates sound source data WS4 that maintains the waveform of the bass sound as much as possible.

[0067] The generator 113 also converts, for example, the waveform 3 of the sound source data W24 into MIDI2 information. For example, the generator 113 uses the MIDI2 information to generate sound source data WS6 that maintains the waveform of the drum sound as much as possible through envelope processing and sound data synthesis.

[0068] The generator 113 synthesizes the sound source data WS2, WS4, and WS6, respectively, to generate music data MS2 in which a predetermined frequency band or a specific frequency is enhanced. This allows the generator 113 to reconstruct music data that can stimulate a user with a predetermined frequency band or a specific frequency from existing music data. Note that the instrument sounds corresponding to the above-mentioned instrument sounds are merely examples, and other processing may be performed.

[0069] FIG. 7 is a diagram showing an example of time division according to the embodiment. In the example shown in FIG. 7, an example of time-dividing music data into fixed predetermined times T10 to T20 is shown. As shown in FIG. 7, the waveforms of the time-divided music data are different from one another, and the frequencies that become peaks in each piece of music data may differ. In such a case, by applying the processing C, it becomes possible to stimulate a specific frequency continuously in real time to the user. In addition, the time-divided section may be appropriately changed according to the development of the music, etc.

[0070] <Operation> Next, the operation of the information processing device 10 according to the embodiment will be described. Fig. 8 is a flowchart showing an example of the process of the information processing device 10 according to the embodiment.

[0071] In step S102, the acquisition unit 112 acquires predetermined sound data. The predetermined sound data may be either stimulation sound data or musical instrument sound data. The acquisition unit 112 may acquire pure tone data that does not stimulate a predetermined frequency band.

[0072] In step S104, the generating unit 113 processes the acquired predetermined sound data into stimulating instrument sound data related to an instrument sound that stimulates a predetermined frequency band to the user. The generating unit 113 may select and apply any one of the processing processes A to C according to the characteristics of the predetermined sound data. The generating unit 113 may also start from the process of generating a pure tone (e.g., FIG. 2) and generate stimulating instrument sound data from the beginning.

[0073] In step S106, the output unit 114 outputs the stimulating instrument sound data generated or processed by the generation unit 113 via a speaker or the like.

[0074] Fig. 9 is a flowchart showing an example of processing related to the embodiment. In the example shown in Fig. 9, the processing is divided according to the type of acquired predetermined sound data, but the acquisition unit 112 or the generation unit 113 may use the characteristics of the predetermined sound data to determine whether it is stimulating sound data or musical instrument sound data, and automatically select and execute the processing.

[0075] In step S202, the acquisition unit 112 acquires stimulation sound data.

[0076] In step S204, the first processing unit 113a performs envelope processing set to the characteristics of a predetermined musical instrument sound (drums, bass, guitar, keyboard, etc.) on the acquired stimulation sound data.

[0077] In step S206, the first processing unit 113a synthesizes sound data to the acquired stimulating sound data so as to produce a predetermined musical instrument sound. Note that only one of the processes in step S204 and step S206 may be performed, and the order of the two processes does not matter. Furthermore, both processes may be performed in two stages.

[0078] In step S302, the acquisition unit 112 acquires predetermined instrument sound data.

[0079] In step S304, the second processing unit 113b executes an LFO processing on the acquired musical instrument sound data.

[0080] In step S402, the acquisition unit 112 acquires predetermined instrument sound data.

[0081] In step S404, the third processing unit 113c identifies a peak frequency of the acquired musical instrument sound data.

[0082] In step S406, the third processing unit 113c calculates the difference frequency between the peak frequency and the specific frequency (target frequency).

[0083] In step S408, the third processing unit 113c synthesizes the difference frequency with the frequency component of the original musical instrument sound data.

[0084] At least one process may be executed in steps S200, S300, and S400, or a plurality of processes may be executed in parallel or in time series.

[0085] The above processing makes it possible to reduce the sense of incongruity of musical instrument sounds even when the frequency of a specific frequency band is stimulated by the user. Note that when LFO processing is performed on music itself, the timbre of the original music is destroyed, making the music sound unnatural. However, according to the disclosed technology, even if a specific frequency is stimulated for the musical instrument sound that is the sound source of the music, it is possible to continuously stimulate the user with a specific frequency band or a specific frequency while enjoying the music as music, while reducing the sense of incongruity felt by the user by generating it to resemble a musical instrument sound.

[0086] <Experiment> In order to verify the effects of the disclosed technology, the inventors conducted an experiment in which subjects listened to the following sound data generated using the disclosed technology. In this experiment, gamma waves in electroencephalograms were used as the predetermined frequency band, and 40 Hz was used as the specific frequency.

[0087] Drum: A 40Hz sine wave. The bass drum is envelope processed, and the snare and hi-hat are synthesized using sound data synthesis. Bass: A 40Hz sine wave, with stimulus instrument sound data generated by envelope processing to resemble a bass sound. Synth: A 40Hz sine wave is used to create a stimulus instrument sound similar to the electronic sound of a keyboard, using envelope processing etc. Gammaband: A combination of drum, bass, and synth sounds (each stimulus instrument contains gamma vibrations) Ctrlband: Sound data with gamma components removed from Gammaband Star: A simple 40Hz click sound (https: / / www.sciencedirect.com / science / article / pii / S0896627319303460) + Ctrlband

[0088] Each subject was fitted with an electroencephalograph Polymate Mini (Polymate (registered trademark)) manufactured by Miyuki Giken Co., Ltd., and electrodes were placed at the FPz (frontal pole midline), FC3, FCz, and FC4 positions according to the international 10 / 10 system. While listening to the musical instrument sounds and comparison sounds generated by the disclosed technology, the brain waves of each subject were measured by the Polymate Mini. The subjects were 14 students in total, 7 males and 7 females.

[0089] Figure 10 shows the average PLI of each frequency band in the EEG measured from each subject. PLI (Phase Locking Index) is a method to examine the degree to which vibration components are synchronized between trials (https: / / www.jstage.jst.go.jp / article / isciesci / 59 / 9 / 59_KJ00010048188 / _pdf / -char / ja). If the PLI is large, it indicates that the 40 Hz frequency component of the measured EEG is generated in synchronization with the sound data.

[0090] In the example shown in Fig. 10, the average value of PLI in each frequency band of 14 subjects at each measurement position is shown. As shown in Fig. 10, in each sound data other than the Ctrlband that does not have a gamma component, the PLI peaks at 40 Hz. This indicates that each sound data generated by the disclosed technology stimulates the brain of the subject, and an electroencephalogram including a 40 Hz frequency component is generated in synchronization with each sound data.

[0091] In addition, at each measurement position, the PLI value for Synth is equal to or greater than that of Star, which is a conventional technology, and it can be seen that the stimulation effect of Synth is high. Next, the effect of Gammaband is high, and Drum and Bass are less effective than Gammaband, but are significantly more effective than Ctrlband, which does not have a gamma component.

[0092] Fig. 11 is a diagram showing the average value of PLI at 40 Hz in the electroencephalogram measured from each subject. As shown in Fig. 11, the average value of PLI for each sound data having gamma components is larger for the 14 subjects than for Ctrlband, which does not have gamma components, at any measurement position. As described above in Fig. 10, the average value of PLI for Synth is equal to or higher than that of Star, which is a click sound synthesized by the conventional technology, even though it is a sound that resembles a musical instrument sound.

[0093] Fig. 12 is a diagram showing the average power of 40 Hz of the electroencephalogram measured from each subject. The results shown in Fig. 12, like those shown in Fig. 11, show that the power value of the 40 Hz frequency component of the electroencephalogram stimulated by Synth is large. In addition, the Gammaband, Drum, and Bass also have significantly larger power values ​​at 40 Hz than the Ctrlband, which does not have a gamma component.

[0094] 13 is a diagram showing the results of the questionnaire for each subject. After the subjects listened to each sound data, they completed a questionnaire with the following answer items for each sound data. Relax: How calm or arousing did this music make you feel? (0: Calm, 1: Arousing) Immersion: How immersed were you in this music? (0: Highly immersive, 1: Not immersed) Awake: Are you awake or sleepy? (0: awake, 1: sleepy) Emotion: Are you currently in a comfortable or uncomfortable state? (0: comfortable, 1: uncomfortable) Preference:Do you like this music? (0: Like, 1: Don't like) Emotions towards music: Did you feel comfortable or uncomfortable listening to this music? (0: comfortable, 1: uncomfortable) Discomfort: Did you feel anything strange about this music? (0: Yes, 1: No)

[0095] As one subject did not answer either the item on feelings toward music or on discomfort, the average of the 13 subjects is shown for these two items. For items other than the items on feelings toward music and on discomfort, the average of the answers of the 14 subjects is shown.

[0096] As shown in Fig. 13, Synth was the most relaxing sound and was the favorite sound. As for discomfort, Gammaband, Drum, Bass, and Synth, which have gamma components generated by the disclosed technology, were hardly felt to be uncomfortable. On the other hand, Star (a conventional simple 40Hz click sound + Ctrlband) was uncomfortable and not to the taste because the click sound was mixed into the music, and it felt uncomfortable.

[0097] From the results shown in Fig. 13, the stimulating instrument sound data generated by the disclosed technology can reduce the sense of discomfort as an instrument sound, even if the instrument sound stimulates the user with frequencies in a predetermined frequency band. In addition, compared to Star, the answers regarding preferences were also more positively perceived for the sound data generated by the disclosed technology, so it is possible to listen to it for a long time and to continuously stimulate the brain.

[0098] The characteristics of each sound data used in the experiment will be described below. Fig. 14 is a diagram showing an example of the characteristics of the gammaband. The upper part of Fig. 14 shows an example of the amplitude in the time component of the gammaband. The middle part of Fig. 14 shows the power in each frequency band of the gammaband. A peak is observed at 40 Hz in the power of the spectrum. The lower part of Fig. 14 shows the envelope of the gammaband. It can be seen that the envelope also has a peak at 40 Hz.

[0099] FIG. 15 is a diagram showing an example of the characteristics of a drum. The top part of FIG. 15 shows an example of the amplitude of the time component of a drum. The middle part of FIG. 15 shows the power in each frequency band of the drum. There is a peak at 40 Hz in the power of the spectrum. The bottom part of FIG. 15 shows the envelope of the drum. It is believed that there is no peak at 40 Hz in the envelope due to the synthesis of three sounds, the bass drum, snare, and hi-hat.

[0100] Fig. 16 is a diagram showing an example of bass characteristics. The upper part of Fig. 16 shows an example of the amplitude of the bass in the time component. The middle part of Fig. 16 shows the power of the bass in each frequency band. The lower part of Fig. 16 shows the bass envelope. It can be seen that the envelope has a peak at 40 Hz.

[0101] FIG. 17 is a diagram showing an example of the characteristics of a Keyb (Synth). The top part of FIG. 17 shows an example of the amplitude of the time component of the Keyb. The middle part of FIG. 17 shows the power of the Keyb in each frequency band. There is a peak around 40 Hz, and there is also a peak at a frequency n times higher than this reference frequency. The bottom part of FIG. 17 shows the envelope of the Keyb. It can be seen that there is a peak at 40 Hz in the envelope.

[0102] FIG. 18 is a diagram showing an example of the characteristics of the CtrlBand. The upper part of FIG. 18 shows an example of the amplitude in the time component of the CtrlBand. The middle part of FIG. 18 shows the power in each frequency band of the CtrlBand. Since the CtrlBand has had the gamma component removed, there is no peak around 40 Hz. The lower part of FIG. 18 shows the envelope of the CtrlBand. There is also no peak at 40 Hz in the envelope.

[0103] Fig. 19 is a diagram showing an example of the characteristics of CtrlBand+Star. The upper part of Fig. 19 shows an example of the amplitude of the time component of CtrlBand+Star. The middle part of Fig. 18 shows the power of CtrlBand+Star in each frequency band. CtrlBand+Star is modulated at 40 Hz, but the power is not large compared to the other frequency components, so it has the same characteristics as shown in the middle part of Fig. 18. The lower part of Fig. 19 shows the envelope of CtrlBand+Star. In the envelope, the carrier is modulated to 40 Hz, so there are peaks at 40 Hz and at double the frequency, 80 Hz.

[0104] FIG. 20 is a diagram showing an example of the characteristics of CtrlBand+NMethods as a reference example. NMethods indicates processing in processing C. The upper part of FIG. 20 shows an example of the amplitude in the time component of CtrlBand+NMethods. The middle part of FIG. 20 shows the power in each frequency band of CtrlBand+NMethods. The peak of CtrlBand+NMethods is shifted compared to the power shown in FIG. 18 by adding the difference frequency. The lower part of FIG. 20 shows the envelope of CtrlBand+NMethods. It can be seen that the envelope has a peak at 40 Hz compared to the envelope shown in FIG. 18.

[0105] Therefore, it can be said that the processing C also has the effect of stimulating a specific frequency to the user. In addition, the processing C is more preferable because it is easier to identify the difference frequency when there is one peak in the frequency of the sound data than when there are multiple peaks. Therefore, the generation unit 113 may execute the processing C if the number of peaks exceeding the threshold in the frequency of the musical instrument sound data is less than a predetermined value, and execute the processing B if the number of peaks is equal to or greater than the predetermined value.

[0106] <Modification> The above-mentioned embodiment and examples are examples for explaining the technology of the present disclosure, and are not intended to limit the technology of the present disclosure to only the embodiment and examples. The technology of the present disclosure can be modified in various ways without departing from the gist of the technology. Each sound data such as the above-mentioned predetermined sound data, instrument sound data, stimulating instrument sound data, and sound source data may be referred to as n-th sound data (n is an integer of 1 or more). Each processing process in the generation unit 113 of the present disclosure may be independently implemented as a program to be executed by a computer, so that only the program corresponding to the required processing process can be installed and executed in the computer. [Explanation of symbols]

[0107] 10. Information processing device 110 Processor 111 Control section 112 Acquisition Department 113 Generation part 113a~c 1st~3rd processing section 114 Output section 115 Extraction part 116 Division 120 Network Communication Interface 130 Storage device 150 User Interface

Claims

1. A processor included in the information processing device generating stimulating instrument sound data relating to an instrument sound that stimulates at least one predetermined frequency band of delta waves, theta waves, alpha waves, beta waves, and gamma waves related to electroencephalograms generated by brain activity in the user's brain; outputting the stimulus instrument sound data; An information processing method for performing the above.

2. The processor further executes acquiring predetermined sound data including stimulation sound data including a pure tone that stimulates the predetermined frequency band to the user's brain; The generating step comprises: The information processing method according to claim 1 , further comprising performing envelope processing including processing related to an attack and / or processing related to a decay on the stimulation sound data.

3. The processing is carried out as follows: The information processing method according to claim 2 , further comprising synthesizing sound data associated with a predetermined musical instrument sound with the stimulating sound data.

4. The processor further executes acquiring predetermined sound data including musical instrument sound data that does not include a specific frequency in the predetermined frequency band as a main component; The generating step comprises:

2. The method of claim 1, further comprising processing the musical instrument sound data using a low frequency oscillator.

5. The processor further executes acquiring predetermined sound data including musical instrument sound data that does not include a specific frequency in the predetermined frequency band as a main component; The generating step comprises:

2. The information processing method according to claim 1, further comprising: synthesizing a difference frequency between a peak frequency of the musical instrument sound data and a specific frequency in the predetermined frequency band into the musical instrument sound data.

6. Obtaining predetermined music data; Analyzing the predetermined music data to extract one or more instrument sound data; The processor further executes the steps of: selecting at least one instrument sound data from the one or more instrument sound data; The generating step comprises: The information processing method according to claim 1 , further comprising: processing the at least one instrument sound data into the stimulating instrument sound data.

7. The processor further executes synthesizing the stimulus instrument sound data with the predetermined music data; The outputting step includes: The information processing method according to claim 6, further comprising outputting synthesized music data including the stimulating instrument sound data.

8. Obtaining predetermined music data; The processor further executes time-sharing of the predetermined music data; The generating step comprises:

2. The information processing method according to claim 1, further comprising the step of processing each of a plurality of time-shared music data into said stimulating instrument sound data.

9. The method of claim 1 , further comprising the processor selecting the predetermined frequency band from among a plurality of frequency bands.

10. A processor included in the information processing device generating stimulating instrument sound data relating to an instrument sound that stimulates at least one predetermined frequency band of delta waves, theta waves, alpha waves, beta waves, and gamma waves related to electroencephalograms generated by brain activity in the user's brain; outputting the stimulus instrument sound data; A non-transitory computer-readable recording medium for causing the computer to execute the method.

11. An information processing device including a processor, The processor, generating stimulating instrument sound data relating to an instrument sound that stimulates at least one predetermined frequency band of delta waves, theta waves, alpha waves, beta waves, and gamma waves related to electroencephalograms generated by brain activity in the user's brain; outputting the stimulus instrument sound data; An information processing device that executes the above.

Images