Information processing method, recording medium, and information processing device

Abstract

The present invention makes it possible to reduce incongruous sounding instrument sounds, even for instrument sounds that stimulate users at a frequency in a prescribed frequency band. In an information processing method according to the present invention, a processor (110) included in an information processing device generates stimulus instrument sound data pertaining to the instrument sounds for stimulating a user in a prescribed frequency band (S104), and outputs the stimulus instrument sound data (S106).

Description

Information processing method, recording medium, and information processing device

[0001] The present invention relates to an information processing method, a recording medium, and an information processing device.

[0002] In recent years, it has been shown that 40 Hz gamma waves can improve Alzheimer's disease, as a result of the results showing that amyloid beta accumulation in the brains of mice was halved by stimulating them with 40 Hz gamma waves (see, for example, Non-Patent Document 1). Also, a technology is known that generates music corresponding to the amplitude and frequency of biometric data and expresses the user's current state through music (see, for example, Patent Document 1).

[0003] Martorell et al., “Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-Associated Pathology and Improves Cognition”, [online], Cell 177, 256-271. April 4, 2019, [Retrieved July 13, 2020], Internet <https: / / www.cell.com / cell / pdf / S0092-8674(19)30163-1.pdf>

[0004] U.S. Pat. No. 1,036,9323

[0005] According to the experimental results of Non-Patent Document 1, when considering providing a person with a gamma wave sound stimulus at 40 Hz, it is difficult for the person to continuously listen to a monotonous sound, which causes a mental burden. Furthermore, similar problems arise when playing sounds of a specific frequency, not just 40 Hz. Furthermore, in Patent Document 1, music is generated based on the user's biometric data, and relaxing music is generated when the user is in a relaxed state, so the specific frequency is not played continuously.

[0006] In other words, in the prior art, there was no technology that could continuously stimulate a user by playing sounds having frequencies in a predetermined frequency band while reducing mental stress. Also, when frequencies in a predetermined frequency band are synthesized with music, the sound image of the existing music is lost, making it feel unnatural to listen to as music.

[0007] Therefore, one aspect of the disclosed technology aims to provide an information processing method, recording medium, and information processing device that make it possible to reduce the sense of discomfort felt as an instrument sound, even if the instrument sound has frequencies in a specified frequency band that stimulate the user.

[0008] An information processing method according to one aspect of the disclosed technology includes a processor included in an information processing device generating stimulating instrument sound data relating to an instrument sound that stimulates a user in a predetermined frequency band, and outputting the stimulating instrument sound data.

[0009] According to one aspect of the disclosed technology, even if the frequency of a predetermined frequency band is an instrument sound that stimulates the user, it is possible to reduce the sense of discomfort felt as an instrument sound.

[0010] 1 is a block diagram showing an example of an information processing device according to an embodiment. FIG. 1 is a diagram showing an example of generation of a sine wave according to an embodiment. FIG. 2 is a diagram showing an example of processing process A according to an embodiment. FIG. 3 is a diagram showing an example of processing process B according to an embodiment. FIG. 4 is a diagram showing an example of processing process C according to an embodiment. FIG. 5 is a diagram showing an example of remix processing according to an embodiment. FIG. 6 is a diagram showing an example of time division according to an embodiment. FIG. 7 is a flowchart showing an example of processing by an information processing device according to an embodiment. FIG. 8 is a flowchart showing an example of processing related to an embodiment. FIG. 9 is a diagram showing an average value of PLI in each frequency band in the electroencephalogram measured from each subject. FIG. 10 is a diagram showing an average value of PLI at 40 Hz in the electroencephalogram measured from each subject. FIG. 11 is a diagram showing an average value of 40 Hz power in the electroencephalogram measured from each subject. FIG. 12 is a diagram showing the results of a questionnaire for each subject. FIG. 13 is a diagram showing an example of Gammaband characteristics. FIG. 14 is a diagram showing an example of Drum characteristics. FIG. 15 is a diagram showing an example of Bass characteristics. FIG. 16 is a diagram showing an example of Keyb (Synth) characteristics. FIG. 17 is a diagram showing an example of CtrlBand characteristics. FIG. 18 is a diagram showing an example of CtrlBand+Star characteristics. FIG. 19 is a diagram showing an example of CtrlBand+NMethods characteristics as a reference example.

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or applications of techniques not explicitly described below. In other words, the present invention can be implemented with various modifications within the scope of its spirit. Furthermore, in the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. Parts in the drawings may have different dimensional relationships or ratios.

[0012] [Embodiments] An outline of an embodiment of the disclosed technology will be described below. <Outline of the Disclosed Technology> In the disclosed technology, in order to enhance a predetermined frequency band in electroencephalograms, instrument sound data (also referred to as "stimulating instrument sound data") that stimulates a user with frequencies in a predetermined frequency band is generated. By generating the stimulating instrument sound data, it becomes possible to generate music using the stimulating instrument sound data. Furthermore, by having the user listen to the generated music, it becomes possible to provide the user with stimulation in the predetermined frequency band. This makes it possible to stimulate the user with the predetermined frequency band while reducing the discomfort felt by the user, thereby achieving the effect of the predetermined frequency band. The generation of the above-mentioned instrument sound data will be described below with reference to the drawings.

[0013] <Configuration Example of Information Processing Device 10> Fig. 1 is a block diagram showing an example of an information processing device 10 according to an embodiment. The information processing device 10 is, for example, a personal computer and may be configured by one or more devices. The information processing device 10 processes sound data and generates, for example, instrument sound data that stimulates a user in a predetermined frequency band. Note that the information processing device 10 does not necessarily have to be a personal computer, but may also be a server, a smartphone, a tablet terminal, or the like that has information processing capabilities.

[0014] The information processing device 10 includes one or more processors (CPUs: Central Processing Units) 110, one or more network communication interfaces 120, a storage device 130, a user interface 150, and one or more communication buses 170 for interconnecting these components.

[0015] The user interface 150 includes a display device (not shown) and an input device (not shown), such as a keyboard and / or a mouse or some other pointing device.

[0016] The storage device 130 may be, for example, a high-speed random access memory such as a DRAM, an SRAM, a DDR RAM, or other random access solid-state storage device, or may be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The storage device 130 may be, for example, a computer-readable non-transitory recording medium that records a program that causes a processor to execute the processes described below.

[0017] Another example of storage device 130 may be one or more storage devices located remotely from processor 110. In one embodiment, storage device 130 stores the following programs, modules, and data structures, or a subset thereof:

[0018] The one or more processors 110 read and execute a program from the storage device 130 as needed. For example, the one or more processors 110 configure a control unit 111 that executes the processing of the disclosed technology by executing a program stored in the storage device 130. The control unit 111 may also configure an acquisition unit 112, a generation unit 113, an output unit 114, an extraction unit 115, and a division unit 116.

[0019] The acquisition unit 112 acquires the predetermined sound data. For example, the acquisition unit 112 may acquire the predetermined sound data stored in the storage device 130, may acquire the predetermined sound data from an external device via the network communication interface 120, or may acquire the predetermined sound data generated by the user operating the user interface 150.

[0020] The generation unit 113 generates stimulating instrument sound data related to an instrument sound that stimulates a predetermined frequency band to the user. For example, the generation unit 113 may use a user interface 150 (see FIG. 2, etc., described later) to generate stimulating instrument sound data based on stimulating sound data described later. The generation unit 113 may also include first to third processing units 113a to 113c, and process the predetermined sound data acquired by the acquisition unit 112 into stimulating instrument sound data. For example, the first to third processing units 113a to 113c perform predetermined processing on the predetermined sound data to generate stimulating instrument sound data that includes, as main components, predetermined frequencies in a predetermined frequency band.

[0021] The predetermined frequency band includes, for example, at least one of the following frequency bands related to brain waves generated by brain activity: delta (Δ) waves (0.5 to 4 Hz), theta (θ) waves (4 to 8 Hz), alpha (α) waves (8 to 12 Hz), beta (β) waves (12 to 30 Hz), and gamma (γ) waves (30 to 100 Hz). The generator 113 may also generate stimulating instrument sound data that includes a specific predetermined frequency as a main component within the predetermined frequency band. For example, the generator 113 may generate stimulating instrument sound data that includes a 40 Hz frequency component as a main component of gamma waves, with 40 Hz being the predetermined frequency. This makes it possible to stimulate the user with a specific frequency, thereby stimulating the user with a specific frequency band that includes the specific frequency.

[0022] The predetermined processing performed by the generation unit 113 includes processing the predetermined sound data so that it has the characteristics of each instrument sound, such as a drum, bass, guitar, keyboard (piano), etc. For example, the processing includes processing the envelope of the predetermined sound data or synthesizing sound data having the characteristics of the instrument sound. The generation unit 113 may also select a processing process corresponding to an instrument sound specified by a user or the like based on processing information in which processing processes are set for each instrument sound, and process the predetermined sound data using the selected processing process.

[0023] The output unit 114 outputs the stimulating instrument sound data generated by the generation unit 113. For example, the output unit 114 outputs the generated stimulating instrument sound data to a speaker and controls the speaker to output the stimulating instrument sound. The output unit 114 may also combine the generated stimulating instrument sound data with other sound data and output the combined data. The speaker may be provided in the information processing device 10, or may be connected to the information processing device 10 via a network.

[0024] The above processing makes it possible to reduce the sense of incongruity felt as an instrument sound, even for an instrument sound that stimulates the user with frequencies in a predetermined frequency band. For example, the processing by the generation unit 113 makes it possible to generate stimulating instrument sound data that has characteristics similar to those of an instrument sound, while still having frequencies in a predetermined frequency band. This allows the listener to listen to the stimulating instrument sound data without feeling much incongruity as an instrument sound. Below, the processing will be described using three examples.

[0025] (Processing Process A) The predetermined sound data may include stimulation sound data including a pure tone that stimulates a predetermined frequency band to the user. For example, if the predetermined frequency band is a gamma wave and the specific frequency is 40 Hz, the pure tone may be a 40 Hz sine wave, or a 10,000 Hz sine wave that oscillates at a period of 40 Hz. A pure tone is also called a single tone, a master tone, or a carrier, and represents a sound generated by an oscillator or a vibration that becomes a master tone.

[0026] When the stimulation sound data includes the above-mentioned pure tones, the first processing unit 113a may perform envelope processing on the stimulation sound data, including processing related to attack and / or processing related to decay.

[0027] For example, the first processing unit 113a may process the stimulation sound data into a sound similar to a bass drum (bass drum). In this case, the first processing unit 113a may calculate an envelope for the stimulation sound data, and process the envelope in terms of attack by adjusting the timing at which the volume reaches a maximum, and in terms of decay by adjusting how the sound attenuates.

[0028] By performing the above processing, the stimulus sound data is envelope-processed so that it has the characteristics of a bass drum, and the stimulus sound data becomes similar to the sound of a bass drum.

[0029] The first processing unit 113a may also process the stimulation sound data into a sound similar to a snare drum or a hi-hat. In this case, the first processing unit 113a may combine sound data associated with a predetermined instrument sound with the stimulation sound data. For example, the first processing unit 113a may store preset snare sound data and / or hi-hat sound data, and combine the snare sound data or hi-hat sound data with the stimulation sound data according to the application or user designation to generate stimulation instrument sound data related to the snare and / or hi-hat.

[0030] By performing the above process, the stimulus sound data is synthesized to have the characteristics of a snare or hi-hat, so that the stimulus sound data resembles the sounds of a snare or hi-hat. Furthermore, by synthesizing the bass drum, snare, and hi-hat, it becomes possible to produce a more natural drum sound.

[0031] (Processing Process B) The predetermined sound data may also include instrument sound data that does not include a specific frequency (e.g., 40 Hz) in a predetermined frequency band as a main component. For example, the instrument sound data may include sound data of a drum, bass, keyboard, guitar, keyboard, etc., generated by a normal performance or generation method.

[0032] In the case of the above-described instrument sound data, the second processing unit 113b may process the instrument sound data using a low frequency oscillator (LFO). For example, if it is desired to stimulate the user with a specific frequency of 40 Hz, the second processing unit 113b oscillates the LFO at 40 Hz to modulate the instrument sound data so that a 40 Hz vibration is generated.

[0033] Through the above process, by oscillating a specific frequency with an LFO, it is possible to modulate normal instrument sound data and stimulate the user with a specific frequency band or a specific frequency.

[0034] (Processing Process C) Furthermore, when the predetermined sound data includes instrument sound data that does not include a specific frequency (e.g., 40 Hz) in a predetermined frequency band as a main component, the third processing unit 113c may include synthesizing a differential frequency between the peak frequency of the instrument sound data and the specific frequency in the predetermined frequency band into the instrument sound data.

[0035] Here, we will explain why processing C, which synthesizes the differential frequency between the peak frequency of instrument sound data and a specific frequency in a specified frequency band into instrument sound data, can stimulate the user to a specific frequency.

[0036] First, we will explain why the above-mentioned LFO can stimulate a specific frequency. Hereinafter, θ1 is the peak frequency of the original sound data, and θ2 is the frequency at which the LFO oscillates. In this case, the frequency of the sound data modulated based on the frequency θ2 oscillated by the LFO is expressed by (Equation 1).

[0037] Here, in order to examine the time waveform of the modulated sound data, the modulated sound data is converted into an analytic signal by a Hilbert transform or the like, and the envelope is found by finding the absolute value (Equation 2).

[0038] When (Equation 2) is expanded using (Equation 3), it finally becomes √(3 / 2+2cos θ2+½cos(2θ2)).

[0039] As described above, the envelope of the modulated sound loses the original peak frequency θ1 and is affected only by the frequency θ2 generated by the LFO. This led the inventors to consider whether it might be possible to stimulate a specific frequency in a user listening to a synthesized sound by simply synthesizing a specific frequency with the peak frequency of the original sound data. Hereinafter, the peak frequency θ1 is set to the specific frequency θ2. Equation 4 represents the frequency of the synthesized sound data obtained by synthesizing the peak frequency θ1 with the specific frequency θ2.

[0040] Here, in order to examine the time waveform of the synthetic sound data, the synthetic sound data is converted into an analytic signal by a Hilbert transform or the like, and the envelope is found by finding the absolute value (Equation 5).

[0041] When (Equation 5) is expanded using (Equation 6), it finally becomes √(2+2cos(θ1−θ2)).

[0042] Therefore, if you want to stimulate the user with a specific frequency of 40 Hz, and the peak frequency θ1 of the original sound data is, for example, 70 Hz, by setting the predetermined frequency (differential frequency) θ2 to 30 Hz, you can stimulate the user with a frequency of 40 Hz, which is θ1-θ2=70 Hz-30 Hz. In other words, once the specific frequency to stimulate the user has been determined, you can calculate the differential frequency between the peak frequency of the original sound data and the specific frequency, and synthesize this differential frequency into the sound data, thereby stimulating the user with the specific frequency. Therefore, even a simple process like processing process C can stimulate the user with the specific frequency.

[0043] The acquisition unit 112 also acquires predetermined music data. The predetermined music data may be music data of existing music or music data of music newly generated by artificial intelligence (AI) or the like. The acquisition unit 112 may acquire predetermined music data selected by a user from a database in which music data is stored, or may acquire predetermined music data distributed by a music distribution service.

[0044] When the acquisition unit 112 acquires predetermined music data, the extraction unit 115 analyzes the predetermined music data and extracts one or more instrument sound data. For example, the extraction unit 115 separates sound source data from existing music data using a publicly known AI technology for sound source separation. The sound source separation AI can input music data and output each sound source data by learning the characteristics of each sound source. Each sound source data may be, for example, vocals, drums, bass, guitar, keyboard, etc. Note that the extraction unit 115 may be provided in a processing device capable of data communication with the information processing device 10 via a network, such as a cloud server.

[0045] Furthermore, the extraction unit 115 may select at least one piece of instrument sound data from among one or more pieces of instrument sound data as the predetermined sound data, so that the selected instrument sound data is acquired by the acquisition unit 112 as the predetermined sound data.

[0046] The generation unit 113 may also process at least one of the sound source-separated instrument sound data into stimulating instrument sound data. For example, the generation unit 113 performs the above-described processing B or processing C on one instrument sound data.

[0047] By the above processing, it becomes possible to separate existing musical instrument data into sound sources, extract each piece of sound source data, and generate stimulating musical instrument sound data in which a specific frequency is stimulated from the extracted sound source data.

[0048] The generating unit 113 may also synthesize the stimulating instrument sound data with predetermined music data. For example, the generating unit 113 may replace sound source data separated from existing music with the stimulating instrument sound data or superimpose the sound source data on the original sound source data to generate music data.

[0049] The output unit 114 may also output synthesized music data including stimulating instrument sound data. This makes it possible to extract sound source data of a predetermined instrument sound from existing music data, replace this sound source data with stimulating instrument sound data, or superimpose the stimulating instrument sound data on the original sound source data, thereby generating and outputting music data that stimulates a predetermined frequency band or a specific frequency to the user.

[0050] By using the above-mentioned processing, it is possible to realize remixes that create derivative music by reconstructing the structure of the sound source using the sound source separation technology, which is a well-known technology.In addition, by using the sound source separation technology, it is possible to easily collect sound source materials to be processed.

[0051] The dividing unit 116 also divides the predetermined music data by time. For example, the dividing unit 116 divides the predetermined music data for each predetermined time period or for each progression of the song (verse 1, verse 2, chorus, etc.), and divides it into multiple pieces of music data.

[0052] In this case, the generation unit 113 may process each of the plurality of music data time-divided by the division unit 116 into stimulating instrument sound data. For example, the third processing unit 113c may identify a peak frequency for each of the plurality of music data time-divided by the division unit 116. Furthermore, the third processing unit 113c may calculate a differential frequency between the peak frequency identified for each of the plurality of music data and the specific frequency. The third processing unit 113c may perform processing process C using the differential frequency. Furthermore, the generation unit 113 may perform processing process B for each of the plurality of music data.

[0053] By the above process, music data is time-divided in real time, and it is possible to stimulate a specific frequency in the user in real time according to the characteristics of the divided music data. For example, by performing appropriate processing on each time-divided music data of streaming music, it is possible to change it into music that can stimulate a specific frequency in real time.

[0054] The control unit 111 may also select a predetermined frequency band from multiple frequency bands. For example, the control unit 111 may select one frequency band from delta waves, theta waves, alpha waves, beta waves, and gamma waves. The control unit 111 may select one frequency band by default, or may select frequency bands in a preset order at predetermined time intervals or at predetermined timings. The control unit 111 may also select a frequency band in response to a user selection via the user interface 150.

[0055] The above processing makes it possible to change the frequency band that the user wants to stimulate or enhance, and when selecting according to user preference, the user can select the frequency band according to their current status or desired status.

[0056] Furthermore, the generation unit 113 may identify the type of predetermined sound data acquired by the acquisition unit 112 and perform processing according to the type of predetermined sound data. For example, the generation unit 113 may perform frequency analysis on the predetermined sound data and / or determine similarity with instrument sound data to identify a pure tone, a stimulating sound, or an instrument sound that does not stimulate the user in a predetermined frequency band. The generation unit 113 may perform processing A if the predetermined sound data is pure tone data or stimulating sound data, and may perform processing B or C if the predetermined sound data is instrument sound data. This makes it possible to automatically generate appropriate stimulating instrument sound data by inputting predetermined sound data.

[0057] <Specific Example> Next, a specific example of processing will be described. FIG. 2 is a diagram showing an example of generating a sine wave according to an embodiment. The example shown in FIG. 2 shows an example of generating a sine wave, which is a pure tone. "Brain Wave" shown in FIG. 2 is a specific frequency that is desired to stimulate the user, and indicates 46.3 Hz. For example, if "Brain Wave" is set to 40 Hz, it is possible to generate a 40 Hz sine wave pure tone. That is, the generation unit 113 accepts a user operation, acquires the frequency input to "Brain Wave," and generates a sine wave pure tone of this frequency. "Interval" indicates the interval between sounds, and indicates 1 second / 40 Hz = 25.0 ms.

[0058] "Interval from Br" indicates the "interval" adjusted by "Brain Wave," and the interval between sounds is adjusted by 46.3 Hz. "Length" indicates the length of the sound, for example, 25.0 ms. The original sound carrier is 10 kHz.

[0059] In the example shown in Figure 2, the basic setting is to repeatedly turn on and off the original sound carrier at a frequency of 10 kHz every 25 ms to generate 40 Hz, but the generation unit 113 may change the ON / OFF interval to "interval from Br" so that the specific frequency of 46.3 Hz is generated from the sound. According to the above-mentioned process, the interval between sounds varies in accordance with the fluctuation of the specific frequency, and is adjusted so that the interval between sounds becomes the specific frequency. According to the example shown in Figure 2, the generation unit 113 can generate stimulation sound data that stimulates the specific frequency from scratch, and generate stimulation instrument sound data based on this stimulation sound data.

[0060] 3 is a diagram showing an example of processing process A according to the embodiment. In the example shown in FIG. 3, the first processing unit 113a performs envelope processing P10 on stimulation sound data W10 to make the sound resemble a bass drum. The stimulation sound data W10 is a pure tone that itself generates fluctuations. The first processing unit 113a can automatically perform envelope processing by storing or training a machine learning model to learn characteristics of the bass drum, such as the magnitude and timing of the attack and the amount of decay.

[0061] The first processing unit 113a synthesizes sound data N10 that resembles the characteristics of a snare or hi-hat with the stimulation sound data W10. The first processing unit 113a may also perform both envelope processing and sound data synthesis to resemble the sound of a predetermined instrument such as a bass drum, snare, or hi-hat.

[0062] 4 is a diagram showing an example of processing process B according to the embodiment. In the example shown in FIG. 4, the second processing unit 113b performs LFO processing on the instrument sound data W12, modulating it using a low frequency so as to emit a specific frequency W14. As a result, stimulating instrument sound data that stimulates the user with the specific frequency W14 is generated.

[0063] 5 is a diagram showing an example of processing process C according to the embodiment. In the example shown in FIG. 5, the third processing unit 113c identifies a peak frequency FP at peak P12 for a graph G12 of frequency components of the instrument sound data W12. Next, the third processing unit 113c calculates a differential frequency F1 between the peak frequency FP and a specific frequency FT that is desired to stimulate the user, and adds the differential frequency F1 in the graph G14 to the frequency components of the instrument sound data W12 in the graph G12. As a result, after the addition of the differential frequency F1, stimulating instrument sound data that stimulates the user at the specific frequency FT is generated.

[0064] FIG. 6 is a diagram illustrating an example of remix processing according to an embodiment. In the example illustrated in FIG. 6, music data M20 of an original song is music data that does not have, as its main component, a component in a predetermined frequency band that stimulates the user. The extraction unit 115 separates the sound source data for each instrument using a known sound source separation technique. For example, sound source data W20 represents a guitar waveform 1, sound source data W22 represents a bass waveform 2, and sound source data W24 represents a drum waveform 3.

[0065] For example, if it is difficult to convert the waveform 1 of the sound source data W20 into MIDI (Musical Instrument Digital Interface) information, the generation unit 113 processes the sound source data W20 using LFO technology, thereby generating sound source data WS2 that can enhance specific frequencies.

[0066] The generator 113 also converts, for example, waveform 2 of the sound source data W22 into MIDI1 information. For example, the generator 113 processes the MIDI1 information to produce a specific frequency (e.g., 40 Hz). For example, the generator 113 performs LFO processing to oscillate at 40 Hz, generating sound source data WS4 that maintains the waveform of the bass sound as much as possible.

[0067] The generator 113 also converts, for example, waveform 3 of the sound source data W24 into MIDI2 information. For example, the generator 113 uses the MIDI2 information to generate sound source data WS6 that maintains the waveform of the drum sound as much as possible through envelope processing and sound data synthesis.

[0068] The generator 113 synthesizes the sound source data WS2, WS4, and WS6 to generate music data MS2 in which a predetermined frequency band or a specific frequency is enhanced. This allows the generator 113 to reconstruct, from existing music data, music data that can stimulate a user in a predetermined frequency band or a specific frequency. Note that the instrument sounds corresponding to the above-described instrument sounds are merely examples, and other processing may be performed.

[0069] FIG. 7 is a diagram showing an example of time division according to an embodiment. The example shown in FIG. 7 shows an example of time-dividing music data into fixed predetermined times T10 to T20. As shown in FIG. 7, the waveforms of the time-divided music data are different, and the peak frequencies of each piece of music data may differ. In such cases, applying processing C makes it possible to stimulate a specific frequency continuously in real time to the user. Furthermore, the time-divided sections may be changed as appropriate depending on the development of the music, etc.

[0070] <Operation> Next, a description will be given of the operation of the information processing apparatus 10 according to the embodiment. Fig. 8 is a flowchart showing an example of processing by the information processing apparatus 10 according to the embodiment.

[0071] In step S102, the acquisition unit 112 acquires predetermined sound data. The predetermined sound data may be either stimulation sound data or instrument sound data. The acquisition unit 112 may also acquire pure tone data that does not stimulate the predetermined frequency band.

[0072] In step S104, the generation unit 113 processes the acquired predetermined sound data into stimulating instrument sound data relating to an instrument sound that stimulates the user in a predetermined frequency band. The generation unit 113 may also select and apply any one of the processing steps A to C depending on the characteristics of the predetermined sound data. The generation unit 113 may also start with a process for generating pure tones (e.g., see FIG. 2 ) and generate stimulating instrument sound data from scratch.

[0073] In step S106, the output unit 114 outputs the stimulating instrument sound data generated or processed by the generation unit 113 via a speaker or the like.

[0074] 9 is a flowchart showing an example of a processing process according to the embodiment. In the example shown in Fig. 9, the processing is divided depending on the type of acquired predetermined sound data, but the acquisition unit 112 or the generation unit 113 may determine whether the predetermined sound data is stimulating sound data or musical instrument sound data using the characteristics of the predetermined sound data, and automatically select and execute the processing.

[0075] In step S202, the acquisition unit 112 acquires stimulation sound data.

[0076] In step S204, the first processing unit 113a performs envelope processing on the acquired stimulation sound data, the envelope processing being set to the characteristics of a predetermined musical instrument sound (drums, bass, guitar, keyboard, etc.).

[0077] In step S206, the first processing unit 113a synthesizes sound data from the acquired stimulation sound data to produce a predetermined musical instrument sound. Note that the processing in steps S204 and S206 may be performed in either order, or both processes may be performed in two stages.

[0078] In step S302, the acquisition unit 112 acquires predetermined instrument sound data.

[0079] In step S304, the second processing unit 113b performs LFO processing on the acquired instrument sound data.

[0080] In step S402, the acquisition unit 112 acquires predetermined instrument sound data.

[0081] In step S404, the third processing unit 113c identifies the peak frequency of the acquired instrument sound data.

[0082] In step S406, the third processing unit 113c calculates the differential frequency between the peak frequency and the specific frequency (target frequency).

[0083] In step S408, the third processing unit 113c synthesizes the difference frequency with the frequency component of the original instrument sound data.

[0084] At least one process may be executed in steps S200, S300, and S400, or a plurality of processes may be executed in parallel or in time series.

[0085] The above processing makes it possible to reduce the sense of incongruity felt by musical instruments, even when the instrument sounds stimulate the user with frequencies in a predetermined frequency band. Note that if LFO processing is performed on music itself, the timbre of the original music will be destroyed, resulting in a sense of incongruity as music. However, according to the technology disclosed herein, even if a predetermined frequency is stimulated in the musical instrument sound that is the sound source of the music, by generating a sound that resembles a musical instrument sound, it is possible to reduce the sense of incongruity felt by the user, and to continuously stimulate the predetermined frequency band or a specific frequency while providing enjoyment as music.

[0086] <Experiment> In order to verify the effects of the disclosed technology, the inventors conducted an experiment in which subjects listened to the following sound data generated using the disclosed technology. Note that in this experiment, gamma waves in electroencephalograms were used as the predetermined frequency band, and 40 Hz was used as the specific frequency.

[0087] ・Drum: Stimulus instrument sound data synthesized from stimulus instrument sound data generated by envelope processing for the bass drum and sound data synthesis for the snare and hi-hat against a 40Hz sine wave. ・Bass: Stimulus instrument sound data generated by envelope processing, etc., against a 40Hz sine wave to resemble a bass sound. ・Synth: Stimulus instrument sound data generated by envelope processing, etc., against a 40Hz sine wave to resemble the electronic sound of a keyboard. ・Gamma band: Sound synthesized from drum, bass, and synth (each stimulus instrument sound data contains gamma vibrations). ・Ctrl band: Sound data with the gamma component removed from gamma band. ・Star: A conventional simple 40Hz click sound (https: / / www.sciencedirect.com / science / article / pii / S0896627319303460) + Ctrl band

[0088] Each subject wore a Polymate Mini (Polymate (registered trademark) electroencephalograph manufactured by Miyuki Giken Co., Ltd.) with electrodes placed at FPz (medial frontal pole), FC3, FCz, and FC4 positions according to the International 10 / 10 system. While listening to the instrument sounds and comparison sounds generated by the disclosed technology, each subject's EEG was measured using the Polymate Mini. The subjects consisted of 14 students, seven males and seven females.

[0089] Figure 10 shows the average PLI values ​​for each frequency band in the EEG measured from each subject. PLI (Phase Locking Index) is a method for determining the degree of synchronization of oscillatory components between trials (https: / / www.jstage.jst.go.jp / article / isciesci / 59 / 9 / 59_KJ00010048188 / _pdf / -char / ja). A large PLI indicates that the 40 Hz frequency component of the measured EEG is generated in synchronization with the sound data.

[0090] The example shown in Figure 10 shows the average PLI values ​​in each frequency band of 14 subjects at each measurement position. As shown in Figure 10, for each sound data except for the Ctrl band, which does not have a gamma component, a PLI peak occurs at 40 Hz. This indicates that each sound data generated by the disclosed technology stimulates the subject's brain, generating brain waves containing a 40 Hz frequency component in synchronization with each sound data.

[0091] Furthermore, at each measurement position, the PLI value for Synth was equal to or greater than that of Star, a conventional technology, indicating a high stimulation effect by Synth. Next, the effect of Gamma band was high, and Drum and Bass were less effective than Gamma band, but were significantly more effective than Ctrl band, which does not have a gamma component.

[0092] 11 shows the average PLI values ​​at 40 Hz in the electroencephalograms measured for each subject. As shown in FIG. 11, at every measurement position, the average PLI values ​​for the 14 subjects for each sound data containing gamma components were greater than those for the Ctrl band, which does not contain gamma components. As described above in FIG. 10, the average PLI value for Synth, despite being a sound that resembles an instrument, is equal to or greater than that of Star, which is a conventional technology that synthesizes click sounds.

[0093] Figure 12 shows the average power of the 40 Hz EEG measured from each subject. Similar to the results shown in Figure 11, the results shown in Figure 12 also show that the power of the 40 Hz frequency component of the EEG stimulated by Synth is large. Furthermore, the gamma band, drum, and bass also have significantly greater power values ​​at 40 Hz than the Ctrl band, which does not have a gamma component.

[0094] 13 shows the results of the questionnaire for each subject. After the subjects listened to each sound data, they completed a questionnaire with the following answer items for each sound data. Relaxation: To what extent did listening to this music make you feel calm or excited? (0: calm, 1: excited) Immersion: To what extent were you immersed in this music? (0: deeply immersed, 1: not immersed) Arousal: Are you currently awake? Are you sleepy? (0: awake, 1: sleepy) Emotion: Are you currently in a comfortable state? Or an uncomfortable state? (0: comfortable, 1: uncomfortable) Preference: Is this music to your liking? (0: like, 1: not like) Emotions toward the music: Did you feel comfortable or uncomfortable listening to this music? (0: comfortable, 1: uncomfortable) Uncomfortable: Did you feel uncomfortable about this music? (0: remembered, 1: not remembered)

[0095] As one subject did not respond to both the items on feelings toward music and discomfort, the average scores for these two items are shown for 13 subjects. For items other than the items on feelings toward music and discomfort, the average scores for 14 subjects are shown.

[0096] As shown in Figure 13, Synth was the most relaxing and preferred sound. Furthermore, the subjects felt almost no discomfort with the Gamma band, Drum, Bass, and Synth, which have gamma components generated by the disclosed technology. On the other hand, Star (a conventional simple 40 Hz click sound + Ctrl band) was unpleasant and undesirable due to the click sound mixed into the music, resulting in a feeling of discomfort.

[0097] 13, the stimulating instrument sound data generated by the disclosed technology can reduce the sense of discomfort felt as an instrument sound, even for instrument sounds that stimulate the user with frequencies in a predetermined frequency band. Furthermore, compared to Star, responses regarding preferences for the sound data generated by the disclosed technology were more positive, making it possible to listen to the data for long periods of time and providing continuous stimulation to the brain.

[0098] The characteristics of each sound data used in the experiment will be explained below. Fig. 14 is a diagram showing an example of the characteristics of Gammaband. The upper part of Fig. 14 shows an example of the amplitude in the time component of Gammaband. The middle part of Fig. 14 shows the power in each frequency band of Gammaband. A peak appears at 40 Hz in the power of the spectrum. The lower part of Fig. 14 shows the envelope of Gammaband. It can be seen that the envelope also has a peak at 40 Hz.

[0099] FIG. 15 is a diagram showing an example of drum characteristics. The top part of FIG. 15 shows an example of the amplitude of the drum's time component. The middle part of FIG. 15 shows the power in each frequency band of the drum. A peak appears at 40 Hz in the spectral power. The bottom part of FIG. 15 shows the drum's envelope. It is believed that the envelope does not have a peak at 40 Hz because three sounds, bass drum, snare, and hi-hat, are combined.

[0100] Fig. 16 is a diagram showing an example of bass characteristics. The upper part of Fig. 16 shows an example of the amplitude of the bass in the time component. The middle part of Fig. 16 shows the power of the bass in each frequency band. The lower part of Fig. 16 shows the bass envelope. It can be seen that the envelope has a peak at 40 Hz.

[0101] FIG. 17 is a diagram showing an example of the characteristics of Keyb (Synth). The upper part of FIG. 17 shows an example of the amplitude of the time component of Keyb. The middle part of FIG. 17 shows the power of Keyb in each frequency band. A peak is observed around 40 Hz, and a peak also occurs at a frequency n times the reference frequency. The lower part of FIG. 17 shows the envelope of Keyb. It can be seen that a peak is observed at 40 Hz in the envelope.

[0102] FIG. 18 is a diagram showing an example of the characteristics of the CtrlBand. The top part of FIG. 18 shows an example of the amplitude of the time component of the CtrlBand. The middle part of FIG. 18 shows the power in each frequency band of the CtrlBand. Since the CtrlBand has had the gamma component removed, there is no peak around 40 Hz. The bottom part of FIG. 18 shows the envelope of the CtrlBand. There is also no peak at 40 Hz in the envelope.

[0103] FIG. 19 is a diagram showing an example of the characteristics of CtrlBand+Star. The upper part of FIG. 19 shows an example of the amplitude of the time component of CtrlBand+Star. The middle part of FIG. 18 shows the power of CtrlBand+Star in each frequency band. CtrlBand+Star is modulated at 40 Hz, but its power is not large compared to the other frequency components, so it has the same characteristics as those shown in the middle part of FIG. 18. The lower part of FIG. 19 shows the envelope of CtrlBand+Star. In the envelope, since the carrier is modulated to 40 Hz, peaks are present at 40 Hz and twice its frequency, 80 Hz.

[0104] FIG. 20 is a diagram showing an example of the characteristics of CtrlBand+NMethods as a reference example. NMethods indicates processing in processing process C. The upper part of FIG. 20 shows an example of the amplitude of the time component of CtrlBand+NMethods. The middle part of FIG. 20 shows the power in each frequency band of CtrlBand+NMethods. By adding the differential frequency, the peak of CtrlBand+NMethods is shifted compared to the power shown in FIG. 18. The lower part of FIG. 20 shows the envelope of CtrlBand+NMethods. It can be seen that the envelope has a peak at 40 Hz compared to the envelope shown in FIG. 18.

[0105] Therefore, it can be said that processing C also has the effect of stimulating a specific frequency to the user. Furthermore, processing C is more preferable when the frequency of sound data has one peak than when there are multiple peaks, because it is easier to identify the difference frequency. Therefore, the generation unit 113 may execute processing C if the number of peaks exceeding the threshold in the frequency of the musical instrument sound data is less than a predetermined value, and may execute processing B if the number of peaks is equal to or greater than the predetermined value.

[0106] <Modifications> The above-described embodiments and examples are merely examples for explaining the technology of the present disclosure, and are not intended to limit the technology of the present disclosure to only these embodiments and examples. The technology of the present disclosure can be modified in various ways without departing from the spirit of the present disclosure. Note that each sound data, such as the above-described predetermined sound data, instrument sound data, stimulating instrument sound data, and sound source data, may be referred to as nth sound data (n is an integer equal to or greater than 1). Each processing step in the generation unit 113 of the present disclosure may be independently implemented as a program to be executed by a computer, thereby enabling only the program corresponding to the required processing step to be installed and executed on the computer.

[0107] 10 Information processing device 110 Processor 111 Control unit 112 Acquisition unit 113 Generation unit 113a to c First to third processing units 114 Output unit 115 Extraction unit 116 Division unit 120 Network communication interface 130 Storage device 150 User interface

Claims

1. An information processing method, comprising: a processor included in an information processing device; generating stimulating instrument sound data relating to an instrument sound that stimulates a user in a specified frequency band; and outputting the stimulating instrument sound data.

2. The information processing method of claim 1, wherein the processor further executes acquiring predetermined sound data including stimulation sound data including a pure tone that stimulates the predetermined frequency band to the user, and the generating step includes performing envelope processing on the stimulation sound data including processing related to attack and / or processing related to decay.

3. The information processing method according to claim 2, wherein said processing includes synthesizing sound data associated with a predetermined musical instrument sound with respect to said stimulation sound data.

4. The information processing method of claim 1, wherein the processor further executes acquiring predetermined sound data including musical instrument sound data that does not include a specific frequency in the predetermined frequency band as a main component, and the generating step includes processing the musical instrument sound data using a low-frequency oscillator.

5. The information processing method of claim 1, wherein the processor further executes acquiring specified sound data including instrument sound data that does not include a specific frequency in the specified frequency band as a main component, and the generating includes synthesizing a differential frequency between a peak frequency of the instrument sound data and the specific frequency in the specified frequency band into the instrument sound data.

6. The information processing method of claim 1, wherein the processor further executes the steps of: acquiring specified music data; analyzing the specified music data to extract one or more instrument sound data; and selecting at least one instrument sound data from the one or more instrument sound data; and wherein the generating includes processing the at least one instrument sound data into the stimulating instrument sound data.

7. The information processing method according to claim 6, wherein the processor further executes a step of synthesizing the stimulating instrument sound data with the predetermined music data, and the outputting step includes outputting the synthesized music data including the stimulating instrument sound data.

8. The information processing method of claim 1, wherein the processor further executes the steps of: acquiring specified music data; and time-dividing the specified music data; and the generating step includes processing each of the time-divided multiple music data into the stimulating instrument sound data.

9. The information processing method of claim 1, wherein the processor further executes the step of selecting the predetermined frequency band from among a plurality of frequency bands.

10. A computer-readable non-transitory recording medium that causes a processor included in an information processing device to execute the following: generating stimulating instrument sound data relating to an instrument sound that stimulates a user in a specified frequency band; and outputting the stimulating instrument sound data.

11. An information processing device having a processor, wherein the processor executes the following: generating stimulating instrument sound data relating to an instrument sound that stimulates a user in a predetermined frequency band; and outputting the stimulating instrument sound data.

Images