Audio signal processing method, program, and audio signal processing device.

Abstract

A sound signal processing method according to one aspect of this disclosure comprises adjusting the signal level of a second sound signal (for example, additional signal) corresponding to a second content according to the signal level of a specific frequency range of a first sound signal (for example, sound source signal) corresponding to a first content, the second sound signal including only a component of the specific frequency range (for example, step S103 to step S104), superimposing the first sound signal and the adjusted second sound signal (for example, step S105), and outputting the signals (for example, step S106).

Description

Technical Field

[0001] The present disclosure relates to a sound signal processing method, a program, and a sound signal processing apparatus.

Background Art

[0002] Patent Document 1 discloses a device for treating dementia or Alzheimer's disease by combining auditory and visual stimuli.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides a sound signal processing method and the like that can preferably output sound in a specific frequency band as an object.

Means for Solving the Problems

[0005] A sound signal processing method according to an aspect of the present disclosure adjusts the signal level of a second sound signal corresponding to a second content and including components in the specific frequency band according to the signal level of the signal in the specific frequency band in a first sound signal corresponding to a first content, and superimposes and outputs the first sound signal and the adjusted second sound signal.

[0006] Furthermore, an audio signal processing method according to another aspect of the present disclosure adjusts a plurality of first audio signals corresponding to a plurality of different audio contents in a first content containing a plurality of distinct audio contents to increase the signal level of a specific frequency band of each of the plurality of audio contents, corrects each of the adjusted plurality of first audio signals corresponding to the plurality of audio contents to reduce the phase difference of the specific frequency band in each of the adjusted plurality of first audio signals, and outputs the corrected plurality of first audio signals.

[0007] Furthermore, a program relating to one aspect of this disclosure is a program that causes a computer to execute the above-described sound signal processing method.

[0008] An audio signal processing device according to one aspect of the present disclosure comprises a processor and a memory, wherein the processor uses the memory to adjust the signal level of a second audio signal corresponding to a second content, which includes a component of the specific frequency band, according to the signal level of a specific frequency band in a first audio signal corresponding to a first content, and outputs the first audio signal and the adjusted second audio signal superimposed on each other.

[0009] Furthermore, an audio signal processing device according to another aspect of the present disclosure comprises a processor and a memory, wherein the processor uses the memory to adjust the signal level of a specific frequency band of each of a plurality of first audio signals corresponding to a plurality of different audio contents in a first content containing a plurality of the audio contents, corrects each of the adjusted plurality of first audio signals corresponding to the plurality of audio contents to reduce the phase difference of the specific frequency band in each of the adjusted plurality of first audio signals, and outputs the corrected plurality of first audio signals. [Effects of the Invention]

[0010] According to one aspect of the present disclosure, a sound signal processing method, etc., can suitably output sound in a specific frequency band. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a block diagram showing the configuration of the sound signal processing device according to Embodiment 1. [Figure 2] Figure 2 shows a first tone signal and a tone signal in which a second tone signal is superimposed on the first tone signal. [Figure 3] Figure 3 is a flowchart showing the processing procedure of the sound signal processing device according to Embodiment 1. [Figure 4] Figure 4 is a diagram illustrating the calculation results of the envelope derived from the first tone signal. [Figure 5] Figure 5 is a flowchart showing the processing procedure of the sound signal processing device according to Modification 1 of Embodiment 1. [Figure 6] Figure 6 is a flowchart showing the processing procedure of the sound signal processing device according to a modified example 2 of Embodiment 1. [Figure 7] Figure 7 is a diagram illustrating the signal obtained from the calculation result of the FFT of the first tone signal. [Figure 8] Figure 8 is a flowchart showing the processing procedure of the sound signal processing device according to Modification 3 of Embodiment 1. [Figure 9] Figure 9 is a block diagram showing the configuration of an audio signal processing device according to a modified example 4 of Embodiment 1. [Figure 10] Figure 10 is a flowchart showing the processing procedure of the sound signal processing device according to Modification 4 of Embodiment 1. [Figure 11] Figure 11 is a block diagram showing the configuration of an audio signal processing device according to a modified example 5 of Embodiment 1. [Figure 12] Figure 12 is a flowchart showing the processing procedure of the sound signal processing device according to Modification 5 of Embodiment 1. [Figure 13] Figure 13 is a block diagram showing the configuration of an audio signal processing device according to a modified example 6 of Embodiment 1. [Figure 14] Figure 14 is a flowchart showing the processing procedure of the sound signal processing device according to a modified example 6 of Embodiment 1. [Figure 15]FIG. 15 is a block diagram showing the configuration of the audio signal processing apparatus according to Modification Example 7 of Embodiment 1. [Figure 16] FIG. 16 is a flowchart showing the processing procedure of the audio signal processing apparatus according to Modification Example 7 of Embodiment 1. [Figure 17] FIG. 17 is a block diagram showing the configuration of the audio signal processing apparatus according to Embodiment 2. [Figure 18] FIG. 18 is a flowchart showing the processing procedure of the audio signal processing apparatus according to Embodiment 2. BEST MODE FOR CARRYING OUT THE INVENTION

[0012] (Background Leading to the Present Disclosure) Conventionally, it has been known that patients with Alzheimer's type dementia accumulate a protein called amyloid-β generated in the brain without being excreted. The accumulated amyloid-β destroys the brain cells that are the subjects of memory. As a result, dementia patients are prone to forgetting things.

[0013] Here, it is known that by exciting gamma waves in the brain by light, sound, or the like, the production amount of amyloid-β decreases and microglia take in amyloid-β, so that the accumulation amount of amyloid-β decreases. In Patent Document 1 described above, a device that performs treatment, prevention, or alleviation of symptoms (hereinafter also referred to as improvement) of dementia or Alzheimer's disease (hereinafter simply referred to as dementia, etc.) by combining auditory stimulation and visual stimulation by utilizing such an effect is disclosed.

[0014] Gamma waves are about 30 Hz to 90 Hz. If gamma waves are excited in the brain by sound, for example, it is conceivable to let a subject listen to a sound having a frequency of about 30 Hz to 90 Hz. However, there is a problem that sounds having a frequency of about 30 Hz to 90 Hz are likely to be felt unpleasant by many people.

[0015] Therefore, in view of these problems, the present inventors provide an audio signal processing method and the like that can suitably output a sound in a specific frequency band to a subject.

[0016] The embodiments and modifications described below will be explained in detail with reference to the drawings. The embodiments and modifications described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the embodiments and modifications below are examples only and are not intended to limit the scope of this disclosure. Furthermore, any components in the embodiments and modifications below that are not described in an independent claim will be described as optional components.

[0017] Please note that each figure is a schematic diagram and not necessarily a strictly accurate representation. Furthermore, in each figure, substantially identical components are denoted by the same reference numerals, and redundant explanations may be omitted or simplified.

[0018] (Embodiment 1) [composition] First, the configuration of the sound signal processing device according to Embodiment 1 will be described.

[0019] Figure 1 is a block diagram showing the configuration of the sound signal processing device 100 according to Embodiment 1.

[0020] The sound signal processing device 100 is a device (playback system) that outputs (plays back) sound signals based on sound content (sound information) such as music stored in the storage device 110. The sound signal processing device 100 is, for example, a portable or stationary audio device with an earphone-type device. The sound signal processing device 100 only needs to be able to process and output sound signals as described later, and may be, for example, a personal computer, smartphone, tablet terminal, etc., each having its own speaker.

[0021] The sound signal processing device 100 may not have a built-in speaker, and a speaker may be attached externally. For example, the sound signal processing device 100 may be configured to output the analog sound signal output from the amplifier 160 to an external device such as a speaker or earphones.

[0022] The sound signal processing device 100 specifically includes a storage device 110, a DSP 120, a CPU 130, a memory 140, a DAC 150, an amplifier 160, and a speaker 170.

[0023] The storage device 110 is a storage device that stores sound content such as music content. Specifically, the sound content stored in the storage device 110 is an example of the first content, and the signal based on the sound content (sound source signal, described later) is an example of the first sound signal. The storage device 110 is implemented by, for example, an HDD (Hard Disk Drive), flash memory, etc.

[0024] The DSP120 is a Digital Signal Processor that performs various processes by executing a control program stored in the memory 140. Specifically, the DSP120 reads an audio signal (first audio signal) from the storage device 110 and performs signal processing on the read audio signal. More specifically, the DSP120 performs processing to increase the level (signal level) of a component in a specific frequency band (hereinafter referred to as the specific frequency band) of the audio signal in order to increase the sound pressure of that component. The specific frequency band is, for example, 10Hz to 100kHz. The specific frequency band may also be 40Hz to 100Hz. Alternatively, the specific frequency band may be 60Hz to 100Hz. Alternatively, the specific frequency band may be 40Hz ± 10Hz (i.e., 30Hz to 50Hz). Furthermore, the specific frequency band may be a range such as 30Hz to 50Hz, or a specific frequency such as 40Hz.

[0025] The CPU 130 is a central processing unit that performs various processes by executing control programs stored in memory 140. For example, the CPU 130 retrieves additional information 200 from memory 140.

[0026] Memory 140 is a memory that stores additional information 200. Memory 140 is implemented, for example, by semiconductor memory.

[0027] The memory 140 may also store control programs executed by the DSP 120 and CPU 130. Furthermore, the memory 140 may store information (threshold information) indicating thresholds and other parameters necessary for processing performed by the DSP 120 and CPU 130.

[0028] The additional information 200 is sound content that includes a specific frequency band. Specifically, the additional information 200 is sound content that includes only a specific frequency band. For example, if the specific frequency band is a single-frequency signal such as 40Hz, the sound signal based on the additional information 200 is a signal that includes a sine wave of that single frequency. Alternatively, if the specific frequency band is a signal with a bandwidth such as 40Hz ± 10Hz, the sound signal based on the additional information 200 is a signal that includes sine waves of multiple frequencies within the range of that bandwidth. Note that the additional information 200 is an example of the second content, and the additional signal based on the additional information 200 is an example of the second sound signal.

[0029] The CPU 130 outputs an audio signal (second audio signal) based on the acquired additional information 200 to the DSP 120.

[0030] The DSP120 outputs a signal (superimposed signal) to the DAC150, which is a superimposed signal of the first and second audio signals.

[0031] Figure 2 shows a first sound signal and a sound signal in which the second sound signal is superimposed on the first sound signal. Specifically, Figure 2 shows a graph obtained by the Fourier transform of the first sound signal (first power spectrum) and a graph obtained by the Fourier transform of the sound signal in which the second sound signal is superimposed on the first sound signal (second power spectrum). In the graphs shown in Figure 2, the horizontal axis is frequency (unit: Hz) and the vertical axis is sound pressure (unit: dB). In Figure 2, the first power spectrum is shown as a solid line and the second power spectrum is shown as a dashed line.

[0032] As shown by the dashed line in Figure 2, in the second power spectrum, for example, the sound pressure in the 30Hz to 50Hz range, and especially the sound pressure at 40Hz, can be seen to have increased. In this way, the DSP120 generates a signal in which the signal in a specific frequency band is amplified by superimposing the first and second sound signals. Specifically, the DSP120 adjusts the signal level of the second sound signal, which corresponds to the second content and contains components in a specific frequency band, according to the signal level of a specific frequency band in the first sound signal corresponding to the first content, and outputs the first sound signal and the adjusted second sound signal superimposed. For example, the second sound signal may contain components other than those in the specific frequency band, or it may contain only components in the specific frequency band. Alternatively, for example, the second sound signal may contain only components in the specific frequency band and the harmonic components of that specific frequency band, as described later.

[0033] The method by which the DSP120 adjusts the signal level of the second tone signal is not particularly limited. For example, one method is to calculate the envelope (envelope value) of the first tone signal and adjust the signal of the second tone signal based on the calculated envelope. Another example is to calculate the signal level in a specific frequency band of the first tone signal by performing a Fourier transform (i.e., short-time Fourier transform) on the first tone signal in a short predetermined time, and then adjust the signal level of the second tone signal according to the calculated signal level. Specific adjustment methods will be described later.

[0034] The DAC150 is a digital-to-analog converter that converts the signal acquired from the DSP120 from a digital signal to an analog signal. The DAC150 outputs the analog signal to the amplifier 160.

[0035] Amplifier 160 is an amplifier that amplifies analog signals. Amplifier 160 outputs the amplified analog signal to speaker 170.

[0036] Speaker 170 outputs sound based on the analog signal acquired from amplifier 160. Speaker 170 may be a speaker that is worn in the ear canal or a stationary speaker. Speaker 170 may also be a speaker that emits sound waves toward the eardrum or a bone conduction speaker.

[0037] Note that the processing of DSP120 and CPU130 is handled by DSP12 0 and beyond The DSP120 may be executed by either the DSP120 or the CPU130. The DSP120 and CPU130 may be implemented by a single processor. The DSP120 and CPU130 may be implemented by a single microcontroller (MCU) or by multiple MCUs. The DSP120, CPU130, memory 140, and DAC150 may be implemented by a single SoC (System-on-a-Chip) or by multiple SoCs. The DSP120, CPU130, memory 140, and DAC150 may be implemented by any combination of the above configurations.

[0038] [Processing Procedure] Next, the processing procedure of the sound signal processing device 100 will be explained.

[0039] Figure 3 is a flowchart showing the processing procedure of the sound signal processing device 100 according to Embodiment 1. Specifically, Figure 3 is a flowchart showing the processing procedure of the DSP 120.

[0040] First, the DSP120 obtains an additional signal from the CPU130, which is an audio signal based on the additional information 200 that the CPU130 has obtained from the memory 140 (S101).

[0041] Next, the DSP120 reads the audio content from the storage device 110 and acquires an audio source signal, which is an audio signal based on that audio content (S102).

[0042] Next, the DSP120 calculates the envelope of the sound source signal using the Hilbert transform (S103).

[0043] Figure 4 is a diagram illustrating the calculation results of the envelope derived from the first tone signal. Specifically, Figure 4 is a graph showing the time change in the signal level of the first tone signal and the envelope calculated from that first tone signal. 4 In the graph shown, the horizontal axis represents time (in seconds), and the vertical axis represents sound pressure (in dB). In Figure 4, the first sound signal is shown as a solid line, and the envelope is shown as a dashed line.

[0044] The envelope calculated from the first tone signal is a line (n-th degree function / n: natural number) that is tangent to multiple local maximums of the first tone signal. The envelope calculated from the first tone signal may be tangent to all local maximums of the first tone signal, or it may be tangent to any two or more local maximums.

[0045] Referring again to Figure 3, after step S103, the DSP120 multiplies the calculated envelope by the additional signal (S104). Specifically, the DSP120 generates a signal (multiplied signal) by multiplying the calculated envelope by the additional signal.

[0046] For example, an additional signal is a signal whose signal level in a specific frequency band is m (m>0), and whose signal levels in other frequency bands are zero. m can be set arbitrarily and is not particularly limited. For example, m=1. When such an additional signal is multiplied with the envelope, the envelope in the specific frequency band (more specifically, the signal level of the envelope) is multiplied by m, and the envelope in the other frequency bands becomes zero. In other words, in this case, the multiplied signal has a signal level in the specific frequency band that is m times the envelope, and a signal level in the other frequency bands that is zero. Thus, the multiplied signal has a value corresponding to the signal level of the sound source signal because the envelope of the multiplied signal has a value corresponding to the signal level of the sound source signal.

[0047] Next, the DSP120 superimposes (adds) the generated multiplication signal and the sound source signal (S105). Specifically, the DSP120 generates a signal (superimposed signal) by superimposing the generated multiplication signal and the sound source signal. As a result, the superimposed signal is a signal in which the signal level of a specific frequency band in the sound source signal has been corrected according to the signal level of the sound source signal. In other words, the DSP120 can add a signal to the sound source signal with a signal level corresponding to the signal level of the sound source signal.

[0048] Next, the DSP120 outputs (transmits) the generated signal (superimposed signal) to the DAC150 (S106).

[0049] The superimposed signal is transmitted from the DAC 150 to the speaker 170 via the amplifier 160. The speaker 170 outputs sound based on the superimposed signal.

[0050] [Differentiation] The following describes each modified example. The following explanation will focus on the differences between these examples and Embodiment 1 described above or the modified examples described later.

[0051] <Example 1> The sound signal processing device according to Modification 1 has the same configuration as the sound signal processing device 100 shown in Figure 1, but the processing procedure is different. Specifically, the DSP 120 according to Modification 1 sets the signal level of the second sound signal to a predetermined level when the signal level of the first sound signal is below a threshold. As a result, even when the signal level of the first sound signal is too low, a second sound signal at a predetermined level is superimposed, which can prevent the signal level in a specific frequency band from dropping too low.

[0052] Figure 5 is a flowchart showing the processing procedure of the sound signal processing device according to Modification 1 of Embodiment 1. Specifically, Figure 5 is a flowchart showing the processing procedure of the DSP 120 provided in the sound signal processing device according to Modification 1.

[0053] First, the DSP120 obtains an additional signal from the CPU130, which is an audio signal based on the additional information 200 that the CPU130 has obtained from the memory 140 (S101).

[0054] Next, the DSP120 reads the audio content from the storage device 110 and acquires an audio source signal, which is an audio signal based on that audio content (S102).

[0055] Next, the DSP120 calculates the envelope of the sound source signal using the Hilbert transform (S103).

[0056] Next, the DSP 120 determines whether the calculated envelope is greater than a threshold (first threshold) (S201). The first threshold can be arbitrarily determined in advance and is not particularly limited. First threshold information indicating the first threshold is stored in memory 140, for example. The DSP 120 obtains the first threshold information obtained by the CPU 130 from memory 140, for example, from the CPU 130.

[0057] If the DSP120 determines that the calculated envelope is below the threshold (No in S201), it changes the values ​​of the envelope to the threshold value (S202). Specifically, the DSP120 changes each of the values ​​that make up the envelope so that values ​​greater than the threshold remain as they are, and values ​​less than or equal to the threshold become the threshold value.

[0058] This prevents the envelope value from becoming too small.

[0059] If the answer is Yes in step S201, or if the answer is Yes in step S202, the DSP120 generates a multiplied signal by multiplying the calculated envelope by the additional signal (S104).

[0060] Next, the DSP120 generates a signal (superimposed signal) by superimposing the generated multiplication signal with the sound source signal (S105).

[0061] Next, the DSP120 outputs the generated signal (superimposed signal) to the DAC150 (S106).

[0062] The superimposed signal is transmitted from the DAC 150 to the speaker 170 via the amplifier 160. The speaker 170 outputs sound based on the superimposed signal.

[0063] As described above, the DSP120 according to Modification 1 adjusts the signal level of the second tone signal, for example, if the signal level of a specific frequency band in the first tone signal is greater than a threshold, it increases the signal level of the second tone signal by a predetermined ratio to the signal level of the specific frequency band in the first tone signal, and if the signal level of the first tone signal is below the threshold, it sets the signal level of the second tone signal to a predetermined level.

[0064] The predetermined ratio and level may be arbitrarily determined in advance and are not particularly limited. In this example, the predetermined ratio is determined by the envelope. For example, in step S104, the envelope may be further multiplied and / or added to a predetermined value.

[0065] <Modification 2> The audio signal processing device according to Modification 2 has the same configuration as the audio signal processing device 100 shown in Figure 1, but the processing procedure is different. Specifically, the DSP 120 according to Modification 2 generates a multiplied signal by multiplying the signal obtained from the calculation result of performing an FFT on the sound source signal (corresponding signal) with an additional signal, instead of using an envelope. This also allows a signal with a signal level corresponding to the sound source signal to be superimposed on the sound source signal, similar to the case where an envelope is used.

[0066] Figure 6 is a flowchart showing the processing procedure of an audio signal processing device according to Modification 2 of Embodiment 1. Specifically, Figure 6 is a flowchart showing the processing procedure of the DSP 120 provided in the audio signal processing device according to Modification 2.

[0067] First, the DSP120 obtains an additional signal from the CPU130, which is an audio signal based on the additional information 200 that the CPU130 has obtained from the memory 140 (S101).

[0068] Next, the DSP120 reads the audio content from the storage device 110 and acquires an audio source signal, which is an audio signal based on that audio content (S102).

[0069] Next, the DSP120 aggregates the signal levels in a specific frequency band (in other words, the frequency band corresponding to the additional signal) at predetermined time intervals and generates a corresponding signal based on the aggregated signal levels (S301).

[0070] Figure 7 is , the This diagram illustrates the signal (corresponding signal) obtained from the calculation result of performing an FFT on a single tone signal. Specifically, Figure 7 is a graph showing the time change in the signal level of the first tone signal and the signal obtained from the calculation result of performing an FFT on the same first tone signal. In the graph shown in Figure 7, the horizontal axis is time (in seconds), and the vertical axis is sound pressure (in dB). In Figure 7, the first tone signal is shown as a solid line, and the signal obtained from the FFT calculation result is shown as a dashed line.

[0071] For example, DSP120 performs an FFT on the first tone signal at predetermined time intervals. Next, DSP120 determines the signal level of a specific frequency band in the calculation result of the FFT of the first tone signal to be the signal level for each predetermined time interval. Then, DSP120 generates a signal with the signal level determined for each predetermined time interval. As a result, DSP120 generates a corresponding signal, such as the signal shown by the dashed line in Figure 7, in which the signal level remains constant within the predetermined time interval.

[0072] The predetermined time can be arbitrarily set in advance and is not particularly limited. The predetermined time is, for example, on the order of a few milliseconds. In the example shown in Figure 7, the predetermined time is 2.5 milliseconds. Time information indicating the predetermined time is stored in advance in, for example, memory 140. The DSP 120 obtains the time information obtained by the CPU 130 from memory 140 from the CPU 130, for example.

[0073] Furthermore, the sound signal processing device according to Modification 2 may be equipped with a timing unit such as an RTC (Real Time Clock) to measure time.

[0074] Referring again to Figure 6, after step S301, the DSP120 generates a multiplied signal by multiplying the generated corresponding signal and the additional signal (S302).

[0075] Next, the DSP120 generates a signal (superimposed signal) by superimposing the generated multiplication signal with the sound source signal (S105).

[0076] Next, the DSP120 outputs the generated signal (superimposed signal) to the DAC150 (S106).

[0077] The superimposed signal is transmitted from the DAC 150 to the speaker 170 via the amplifier 160. The speaker 170 outputs sound based on the superimposed signal.

[0078] <Variation 3> The sound signal processing device according to Modification 3 has the same configuration as the sound signal processing device 100 shown in Figure 1, but the processing procedure is different. Specifically, the DSP 120 according to Modification 3 controls the signal level of the harmonic frequency band of a specific frequency band of the first sound signal to increase. Harmonics in a specific frequency band are also known to be effective in improving dementia and other conditions, similar to sounds in a specific frequency band. Therefore, a sound that is even more effective in improving dementia and other conditions is output.

[0079] Figure 8 is a flowchart showing the processing procedure of the sound signal processing device according to Modification 3 of Embodiment 1. Specifically, Figure 8 is a flowchart showing the processing procedure of the DSP 120 provided in the sound signal processing device according to Modification 3.

[0080] First, the DSP120 obtains an additional signal from the CPU130, which is an audio signal based on the additional information 200 that the CPU130 has obtained from the memory 140 (S101).

[0081] Next, the DSP120 reads the audio content from the storage device 110 and acquires an audio source signal, which is an audio signal based on that audio content (S102).

[0082] Next, the DSP120 calculates the envelope of the sound source signal using the Hilbert transform (S103).

[0083] Next, the DSP120 generates a multiplied signal by multiplying the calculated envelope, the additional signal, and the harmonics (harmonic signals) of the additional signal (S401).

[0084] For example, if the specific frequency band is 40Hz ± 10Hz, the DSP120 generates a harmonic signal of 80Hz ± 20Hz. Note that the harmonics of the specific frequency band do not have to be twice the frequency of the specific frequency band, but can be p times (p: a natural number). Also, the harmonic signal may contain only one frequency signal or multiple frequencies. For example, the harmonic signal may contain a signal at twice the frequency of the specific frequency band and a signal at three times the frequency of the specific frequency band. Harmonic information indicating the harmonic signal may be pre-stored in memory 140. In this case, the DSP120 obtains the harmonic information obtained by the CPU 130 from memory 140 from the CPU 130.

[0085] Next, the DSP120 generates a signal (superimposed signal) by superimposing the generated multiplication signal with the sound source signal (S105).

[0086] Next, the DSP120 outputs the generated signal (superimposed signal) to the DAC150 (S106).

[0087] The superimposed signal is transmitted from the DAC 150 to the speaker 170 via the amplifier 160. The speaker 170 outputs sound based on the superimposed signal.

[0088] As described above, the DSP120 in Modification 4, for example, when adjusting the signal level of the second tone signal, further controls (adjusts) the signal level of the harmonic frequency band of a specific frequency band to increase it.

[0089] <Modification 4> In Modification 4, the signal level of the additional signal is controlled based on control information. This allows, for example, the output of a sound in a specific frequency band at a volume desired by the user, by receiving control information from the user.

[0090] Figure 9 is a block diagram showing the configuration of the sound signal processing device 101 according to a modified example 4 of Embodiment 1.

[0091] The sound signal processing device 101 includes a storage device 110, a DSP 120, a CPU 130, a memory 141, a DAC 150, an amplifier 160, a speaker 170, and a communication IF 180.

[0092] Memory 141 is a memory that stores additional information 200 and amplitude value information 201. Memory 141 is implemented, for example, by a semiconductor memory.

[0093] The amplitude value information 201 is an example of control information and is information for determining the sound pressure of the additional information 200 (more specifically, the signal level of the additional signal based on the additional information 200). The amplitude value information 201 is information indicating the processing content, such as "ON", "OFF", "UP", or "DOWN".

[0094] For example, if the amplitude value information 201 indicates "ON", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 1.0, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150.

[0095] Alternatively, for example, if the amplitude value information 201 indicates "OFF", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 0, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150. In other words, in this case, for example, the DSP 120 outputs the additional signal based on the additional information 200 to the DAC 150 without adding it to the sound source signal.

[0096] Alternatively, for example, if the amplitude value information 201 indicates "UP", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 1.1, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150.

[0097] Alternatively, for example, if the amplitude value information 201 indicates "DOWN", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 0.9, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150.

[0098] As described above, the DSP120 acquires control information indicating the signal level of a specific frequency band, and in adjusting the signal level of the second tone signal, it adjusts the signal level of the specific frequency band based on the control information. Specifically, the DSP120 switches the on / off status of the additional signal (i.e., whether or not to superimpose the additional signal on the sound source signal) or switches the signal level based on the amplitude value information 201.

[0099] Note that the amplitude value information 201 may also be information that shows a numerical value.

[0100] For example, if the amplitude value information 201 indicates "1.0", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 1.0, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150.

[0101] Alternatively, for example, if the amplitude value information 201 indicates "0.0", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 0, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150. In other words, in this case, for example, the DSP 120 outputs the additional signal based on the additional information 200 to the DAC 150 without adding it to the sound source signal.

[0102] Alternatively, for example, if the amplitude value information 201 indicates "1.1", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 1.1, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150.

[0103] Alternatively, for example, if the amplitude value information 201 indicates "0.9", the DSP 120 multiplies the signal level of the additional signal based on the additional information 200 by 0.9, adds it to the sound source signal stored in the storage device 110, and outputs it to the DAC 150.

[0104] As described above, the amplitude value information 201 only needs to be information that indicates how to set the signal level of the additional signal.

[0105] The amplitude value information 201 is obtained, for example, from an external terminal 300 via the communication IF 180.

[0106] The communication interface (IF) 180 is a communication interface (IF) for communication between the sound signal processing device 101 and the external terminal 300. For example, when the sound signal processing device 101 and the external terminal 300 communicate wirelessly, the communication interface 180 is implemented by an antenna and a wireless communication circuit. Alternatively, when the sound signal processing device 101 and the external terminal 300 communicate via wired connection, the communication interface 180 is implemented by a connector to which the communication line is connected.

[0107] The communication standard used for communication may be Bluetooth® or BLE (Bluetooth® Low Energy), or a proprietary communication standard; it is not particularly limited.

[0108] The external terminal 300 is a communication terminal operated by the user. The external terminal 300 is, for example, a control console or a smartphone. The user transmits amplitude value information 201 to the sound signal processing device 101 by operating the external terminal 300. This allows the user to switch the on / off status of additional signals based on additional information 200 or switch the signal level by operating the external terminal 300. The CPU 130 stores the amplitude value information 201 acquired from the external terminal 300 via the communication IF 180 in the memory 141. The CPU 130 updates the amplitude value information 201 stored in the memory 141 each time it acquires amplitude value information 201 from the external terminal 300 via the communication IF 180.

[0109] Memory 141 may also store control programs executed by the DSP 120 and CPU 130. Furthermore, memory 141 may store information (threshold information) indicating thresholds and other parameters necessary for processing executed by the DSP 120 and CPU 130.

[0110] Figure 10 is a flowchart showing the processing procedure of the sound signal processing device 101 according to Modification 4 of Embodiment 1. Specifically, Figure 10 is a flowchart showing the processing procedure of the DSP 120 provided in the sound signal processing device 101 according to Modification 4.

[0111] First, the DSP120 obtains an additional signal from the CPU130, which is an audio signal based on the additional information 200 obtained by the CPU130 from the memory 141 (S101). Also, for example, the DSP120 obtains amplitude value information 201 from the CPU130, which was obtained by the CPU130 from the memory 141.

[0112] Next, the DSP120 reads the audio content from the storage device 110 and acquires an audio source signal, which is an audio signal based on that audio content (S102).

[0113] Next, the DSP120 calculates the envelope of the sound source signal using the Hilbert transform (S103).

[0114] Next, the DSP120 generates a multiplied signal by multiplying the calculated envelope, the additional signal, and the numerical value indicated by, for example, the amplitude value information 201 (S501).

[0115] Next, the DSP120 generates a signal (superimposed signal) by superimposing the generated multiplication signal and the sound source signal (S105).

[0116] Next, the DSP120 outputs the generated signal (superimposed signal) to the DAC150 (S106).

[0117] The superimposed signal is transmitted from the DAC 150 to the speaker 170 via the amplifier 160. The speaker 170 outputs sound based on the superimposed signal.

[0118] As described above, for example, the DSP120 acquires control information (e.g., amplitude value information 201) indicating the signal level in a specific frequency band, and adjusts (controls) the signal level of the second tone signal based on the control information.

[0119] <Modification 5> In Modification 5, the signal level of the additional signal is controlled based on the user's biometric information. This allows, for example, the output of sound in a specific frequency band at a volume that suits the user's comfort level.

[0120] Figure 11 is a block diagram showing the configuration of the sound signal processing device 102 according to a modified example 5 of Embodiment 1.

[0121] The sound signal processing device 102 includes a storage device 110, a DSP 120, a CPU 130, a memory 142, a DAC 150, an amplifier 160, a speaker 170, and a communication IF 180.

[0122] Memory 142 is a memory that stores the additional information 200 and the pNN information 202. Memory 142 is implemented, for example, by a semiconductor memory.

[0123] pNN information 202 is information used to determine the sound pressure of additional information 200. Specifically, pNN information 202 is an example of biological information and indicates the pNN50 value. The pNN50 value is the percentage of heartbeats in which the difference between consecutive adjacent RR intervals exceeds 50 ms.

[0124] The CPU 130 repeatedly acquires pNN information 202 from the heart rate monitor 310 via the communication IF 180, for example, and stores it in the memory 142. As a result, the memory 142 stores pNN information 202 that shows the time change of the user's pNN 50 value.

[0125] The heart rate monitor 310 measures the user's heart rate, calculates the pNN50 value, and repeatedly transmits the pNN information 202, which indicates the calculation result, to the sound signal processing device 102.

[0126] The time interval at which the heart rate monitor 310 repeatedly transmits pNN information 202 can be arbitrarily determined in advance and is not particularly limited.

[0127] The DSP120 switches the signal level of the additional signal based on the pNN information 202. For example, if the pNN50 value indicated by the pNN information 202 decreases, the DSP120 reduces the signal level of the additional signal.

[0128] The pNN50 value is known to reflect the user's comfort / discomfort (whether the user is comfortable or not). For example, if the pNN50 value decreases when the user is given some kind of stimulus, it is highly likely that the user is feeling uncomfortable. Therefore, the DSP120 acquires, for example, the user's biometric information and, in adjusting the signal level of the second sound signal, controls (adjusts) the signal level of a specific frequency band based on the biometric information. Specifically, for example, if the pNN50 value indicated by the pNN information 202 decreases, the DSP120 reduces the signal level of the additional signal.

[0129] For example, if the pNN50 value indicated by the pNN information 202 increases, the DSP120 may increase the signal level of the additional signal.

[0130] Furthermore, the CPU 130 may, for example, acquire information indicating the user's heart rate, such as an electrocardiogram, from the heart rate monitor 310 via the communication IF 180, calculate the user's pNN50 value based on the acquired information, and store the information indicating the calculation result as pNN information 202 in the memory 142.

[0131] Figure 12 is a flowchart showing the processing procedure of the sound signal processing device 102 according to Modification 5 of Embodiment 1. Specifically, Figure 12 is a flowchart showing the processing procedure of the DSP 120 provided in the sound signal processing device 102 according to Modification 5.

[0132] First, the DSP120 obtains an additional signal from the CPU130, which is an audio signal based on the additional information 200 that the CPU130 has obtained from the memory 142 (S101).

[0133] Next, the DSP120 reads the audio content from the storage device 110 and acquires an audio source signal, which is an audio signal based on that audio content (S102).

[0134] Next, the DSP120 calculates the envelope of the sound source signal using the Hilbert transform (S103).

[0135] The DSP120 obtains pNN information 202 from the CPU130, which has acquired it from memory 142, and determines whether the current pNN50 value is greater than or equal to the previous pNN50 value based on the acquired pNN information 202 (S601). Specifically, the DSP120 determines whether the latest pNN value is greater than or equal to the pNN50 value immediately preceding the current pNN value.

[0136] If the DSP120 determines that the current pNN50 value is less than the previous pNN50 value (No in S601), it reduces the signal level of the additional signal based on the additional information 200 (S602). The signal level that the DSP120 reduces may be arbitrarily determined in advance. Alternatively, the signal level that the DSP120 reduces may be determined based on the difference between the previous pNN50 value and the current pNN50 value. For example, the larger the difference, the greater the reduction in the signal level of the additional signal that the DSP120 may make.

[0137] If the answer is Yes in step S601, or if the answer is Yes in step S602, the DSP120 generates a multiplied signal by multiplying the calculated envelope by the additional signal (S104).

[0138] Next, the DSP120 generates a signal (superimposed signal) by multiplying the generated multiplication signal with the sound source signal (S105).

[0139] Next, the DSP120 outputs the generated signal (superimposed signal) to the DAC150 (S106).

[0140] The superimposed signal is transmitted from the DAC 150 to the speaker 170 via the amplifier 160. The speaker 170 outputs sound based on the superimposed signal.

[0141] In this example, the process of changing the signal level of the additional signal is performed based on the pNN information 202 acquired from the heart rate monitor 310. The process of changing the signal level of the additional signal may be performed based on other biometric information instead of the pNN information 202. Biometric information is information that indicates the degree to which the user feels comfortable. Such other biometric information may be, for example, information indicating the user's respiratory rate, information indicating the user's body temperature, information indicating the user's sweating amount, information indicating the user's brainwaves, or information indicating the user's facial expression (e.g., image information). The process of changing the signal level of the additional signal may be performed based on this biometric information of the user. For example, if the biometric information indicates that the user is feeling uncomfortable (specifically, that the degree to which the user feels comfortable has decreased), the DSP120 reduces the level of the second sound signal.

[0142] For example, the CPU 130 may acquire the user's biometric information from a device that acquires the user's biometric information, such as a thermometer, electroencephalograph, or camera, via the communication IF 180, and store it in the memory 142. The DSP 120 may adjust the signal level of the additional signal based on the time change of the user's biometric information.

[0143] <Variation 6> Variation 6 Next, the sound output from speaker 170 is captured, and the signal level of the second tone signal is adjusted based on the captured sound. This allows speaker 170 to output sound at an appropriate volume depending on the installation environment of speaker 170.

[0144] Figure 13 is a block diagram showing the configuration of the sound signal processing device 103 according to a modified example 6 of Embodiment 1.

[0145] The sound signal processing device 103 includes a DSP 120, a CPU 130, a memory 141, a DAC 150, an amplifier 160, a speaker 170, and a microphone 190.

[0146] Thus, for example, the sound signal processing device 103 does not have a storage device 110. In such a case, for example, the sound signal processing device 103 acquires sound source signals, etc., from a storage device 320 of an external device such as a server device that is connected to it in a communicative manner. Of course, the sound signal processing device 103 may also have a communication interface for communicating with the server device, etc.

[0147] According to this, the sound signal processing device 103 can be realized without having large components such as the memory device 110. For example, the sound signal processing device 103 can be miniaturized and implemented as an earphone.

[0148] Microphone 190 is a microphone that picks up sound output from speaker 170 and outputs an audio signal (hereinafter also referred to as microphone signal) based on the picked-up sound. Microphone 190 is, for example, a condenser microphone, a dynamic microphone, or a MEMS (Micro Electro Mechanical Systems) microphone. Microphone 190 is a so-called earphone microphone that is housed inside the housing of the earphone, for example, when the audio signal processing device 103 is an earphone.

[0149] Microphone 190 outputs a microphone signal based on the sound it has picked up to CPU 130.

[0150] The CPU 130 updates the amplitude value information 201 based on the microphone signal, for example. In other words, the CPU 130 acquires a microphone signal (output sound signal) from the microphone 190, which is based on the sound (output sound) output from the speaker 170 detected by the microphone 190.

[0151] The DSP120 adjusts the signal level of the second tone signal based on the amplitude value information 201. In other words, the DSP120 further controls (adjusts) the signal level of a specific frequency band in the second tone signal based on the output sound signal derived from the output sound output from speaker 170.

[0152] Figure 14 is a flowchart showing the processing procedure of the sound signal processing device 103 according to Modification 6 of Embodiment 1. Specifically, Figure 14 is a flowchart showing the processing procedure for updating amplitude value information 201, which is executed by the CPU 130 of the sound signal processing device 103 according to Modification 6.

[0153] First, CPU130 acquires the microphone signal (output sound signal) from microphone190 (S701).

[0154] Next, the CPU 130 checks the signal level in the bandwidth corresponding to the additional signal (specific frequency band) and performs a short-time Fourier transform on the microphone signal (S702). The time for the Fourier transform can be arbitrarily determined and is not particularly limited.

[0155] The CPU 130 determines whether the signal level in a specific frequency band of the microphone signal is lower than a predetermined threshold (second threshold) (S703). The second threshold information indicating the second threshold can be stored in memory 141 beforehand, for example, and is not particularly limited. The second threshold information indicating the second threshold is stored in memory 140 beforehand, for example. The DSP 120 obtains the second threshold information obtained by the CPU 130 from memory 140, for example.

[0156] The first threshold and the second threshold mentioned above may be the same value or different values.

[0157] If the CPU 130 determines that the signal level in a specific frequency band of the microphone signal is lower than a predetermined threshold (Yes in S703), it updates the amplitude value information 201 (more specifically, the control amplitude value indicated by the amplitude value information 201) (S704).

[0158] Depending on the installation environment of speaker 170, it may not be possible to output sound in a specific frequency band at the expected volume. Therefore, microphone 190 is used to capture the sound actually output from speaker 170, and the amplitude value information 201 is updated using the microphone signal based on the captured sound, that is, the sound pressure (volume) of the specific frequency band is controlled.

[0159] The DSP120 uses the updated amplitude value information 201 as described above to perform the processing shown in Figure 10, for example.

[0160] <Example 7> When using earphones or similar devices to allow subjects to hear sounds with frequencies of approximately 30Hz to 90Hz through both ears, a problem arises where improvement in conditions such as dementia is difficult to achieve if there is a phase difference between, for example, the sound emitted from the speaker in the right ear and the sound emitted from the speaker in the left ear. Therefore, in Modification 7, a process is performed to reduce the phase difference between the two sound signals (more specifically, to align their phases). This makes it easier to achieve improvement in conditions such as dementia.

[0161] Figure 15 is a block diagram showing the configuration of the sound signal processing device 104 according to a modified example 7 of Embodiment 1.

[0162] The sound signal processing device 104 includes a DSP 120, a CPU 130, a memory 143, a DAC 150, an amplifier 160, a speaker 171, a speaker 172, and a communication IF 180.

[0163] Memory 143 is a memory that stores the enhanced information 203. Memory 143 is implemented, for example, by a semiconductor memory.

[0164] Memory 143 may also store control programs executed by the DSP 120 and CPU 130. Furthermore, memory 143 may store information (threshold information) indicating thresholds and other parameters necessary for processing executed by the DSP 120 and CPU 130.

[0165] The enhancement information 203 is information used by the DSP 120 to increase (enhance) the signal level of a specific frequency band in the sound signal acquired by the DSP 120. The enhancement information 203 includes, for example, information indicating a specific frequency band and information indicating a predetermined signal level. Based on the enhancement information 203, the DSP 120 performs a process to increase the signal level of a specific frequency band in the ambient sound signal, as described later, to a predetermined level.

[0166] Speakers 171 and 172 each output sound based on an analog signal obtained from amplifier 160. Speakers 171 and 172 are, for example, speakers shaped to be worn in the ear canal (i.e., earphone-type speakers). For example, speaker 171 is an earphone worn in the left ear, and speaker 172 is an earphone worn in the right ear.

[0167] For example, the storage device 320 stores the audio content of the sound signal output from the earphone worn in the left ear (hereinafter also referred to as Lch) and the audio content of the sound signal output from the earphone worn in the right ear (hereinafter also referred to as Rch).

[0168] The DSP120 processes the Lch and Rch signals in a specific frequency band to align their phases (in other words, to match the phases, eliminate the phase difference, or make the phase difference zero), and then outputs the Lch and Rch signals to the DAC150. Alternatively, for example, the DSP120 processes the Lch and Rch signals to reduce the phase difference in a specific frequency band, and then outputs the Lch and Rch signals to the DAC150. For example, the DSP120 performs a short-time Fourier transform (more specifically, a short-time FFT) on each of the Lch and Rch signals to align their phases in a specific frequency band, and then further aligns their phases by performing an inverse Fourier transform (more specifically, an inverse short-time FFT).

[0169] Furthermore, when the DSP120 aligns the phases of the Lch and Rch, it may change the phase of only the Lch, only the Rch, or both the Lch and Rch.

[0170] Furthermore, setting the phase difference to zero means making it effectively zero; it doesn't have to be perfectly zero, and a slight phase shift is acceptable.

[0171] Thus, when the first content includes multiple different audio contents, the DSP120 corrects, for example, each of the multiple first audio signals corresponding to the multiple audio contents, to reduce the phase difference in a specific frequency band in each of the multiple first audio signals.

[0172] Furthermore, speakers 171 and 172 may be speakers that emit sound waves toward the eardrum, or they may be bone conduction speakers.

[0173] Microphone 330 is a microphone that picks up ambient sounds in an environment where sound is output by speakers 171 and 172, and outputs an audio signal based on the picked-up ambient sounds (hereinafter also referred to as the ambient sound signal) to the CPU 130 via the communication IF 180.

[0174] For example, the CPU 130 acquires an ambient sound signal based on ambient sound from the microphone 330 via the communication IF 180, generates additional information (second content) such that a component of a specific frequency band in the ambient sound signal becomes a second sound signal, and stores it in the memory 143.

[0175] The DSP120 enhances the signal level of a specific frequency band in the ambient sound signal acquired from the microphone 330 via the communication IF180, for example, based on the enhancement information 203.

[0176] Furthermore, the DSP120 superimposes, for example, the augmented ambient sound signal and the sound source signal acquired from the storage device 320, and outputs them to the DAC150. In other words, the ambient sound signal is an example of a second sound signal and is used to be superimposed on the sound source signal, similar to the additional signal described above.

[0177] The microphone 330 is housed in the housing of the earphone, for example, if the sound signal processing device 104 is an earphone.

[0178] Figure 16 is a flowchart showing the processing procedure of the sound signal processing device 104 according to Modification 7 of Embodiment 1. Specifically, Figure 16 is a flowchart showing the processing procedure of the DSP 120 provided in the sound signal processing device 104 according to Modification 7.

[0179] First, the CPU 130 acquires an ambient sound signal from the microphone 330 (S801).

[0180] Next, the CPU 130 generates additional information based on the ambient sound signal (S802). This additional information may, for example, be information indicating the ambient sound signal, or it may be information indicating a signal that includes only a specific frequency band of the ambient sound signal as a narrowband filter is applied to the ambient sound signal. Here, we will explain assuming that the CPU 130 generates additional information indicating the ambient sound signal and stores it in memory 143.

[0181] The sound signal processing device 104 may also include a filter circuit that functions as a narrowband filter.

[0182] Next, the DSP120 obtains the enhancement information 203 that the CPU130 has acquired from memory 143 (S803).

[0183] Next, the DSP120 applies a narrowband filter to the ambient sound signal and enhances the signal level in a specific frequency band based on the enhancement information 203 (S804).

[0184] The sound signal processing device 104 may also include a filter circuit that functions as a narrowband filter.

[0185] Next, the DSP120 reads the audio content from the storage device 320 and acquires the audio source signal, which is the audio signal based on that audio content (S102). For example, the DSP120 acquires the Lch and Rch as the audio source signal.

[0186] Next, the DSP120 generates a signal (corrected signal) by correcting at least one of the Lch and Rch so that the phases of the Lch and Rch in a specific frequency band are aligned (S805). The DSP120 may also correct at least one of the Lch and Rch so as to reduce the phase difference between the Lch and Rch in a specific frequency band.

[0187] Next, the DSP120 generates a signal (superimposed signal) by superimposing the corrected signal (corrected signal) with the ambient sound signal (S806).

[0188] Next, the DSP120 outputs the generated signal (superimposed signal) to the DAC150 (S106).

[0189] The superimposed signals corresponding to the left and right channels are transmitted from the DAC 150 to the speakers 171 and 172 via the amplifier 160. Speakers 171 and 172 then output sound based on these signals.

[0190] [Effects, etc.] As described above, an audio signal processing method according to one aspect of the present disclosure adjusts the signal level of a second audio signal (e.g., an additional signal) corresponding to a second content, which includes a component of a specific frequency band, according to the signal level of a specific frequency band in a first audio signal (e.g., a sound source signal) corresponding to a first content (e.g., steps S103 to S104), and outputs the first audio signal and the adjusted second audio signal superimposed (e.g., step S105) (e.g., step S106).

[0191] According to this, along with a sound based on a first sound signal corresponding to a first content such as music, it is possible to output sounds in a specific frequency band that are effective in improving dementia and other conditions. Therefore, since the user hears sounds in the specific frequency band along with the music, it is possible to suppress the user from feeling uncomfortable due to sounds in the specific frequency band. In other words, according to one aspect of the present disclosure, sounds in a specific frequency band can be suitably output to the target (for example, the user who listens to the sound output by the sound signal processing method).

[0192] Furthermore, for example, in the adjustment described above, if the signal level of a specific frequency band in the first tone signal is greater than the threshold (for example, Yes in step S201), the signal level of the second tone signal is increased by a predetermined ratio to the signal level of the specific frequency band in the first tone signal (for example, step S104), and if the signal level of the first tone signal is less than the threshold (for example, No in step S201), the signal level of the second tone signal is set to a predetermined signal level (for example, steps S202 and S104). In the modified example 1 described above, the DSP120 calculates the envelope of the sound source signal (first tone signal), and if the calculated envelope is greater than the threshold, it multiplies the value of the envelope by the second tone signal to increase the signal level of the second tone signal by a predetermined ratio. On the other hand, if the calculated envelope is less than or equal to the threshold, the DSP120 multiplies the value of the envelope by the second tone signal to a value corresponding to a predetermined signal level to set the signal level of the second tone signal to a predetermined signal level.

[0193] According to this, if the signal level of a specific frequency band in the first tone signal is sufficiently high, the signal level of the second tone signal will also increase accordingly. Therefore, even if the signal level of the second tone signal is increased, it is possible to suppress the user from feeling uncomfortable due to sounds in a specific frequency band. Also, if the signal level of a specific frequency band in the first tone signal is sufficiently high If it's small By setting the signal level of the second tone signal to a predetermined level, it is possible to prevent the signal level of the second tone signal from becoming too low, thus reducing its effectiveness for dementia and other conditions.

[0194] Furthermore, for example, the first content includes multiple different sound contents, and the sound signal processing method further corrects each of the multiple first sound signals corresponding to the multiple sound contents to reduce the phase difference in a specific frequency band in each of the multiple first sound signals (for example, step S805).

[0195] According to this, by correcting to reduce the phase difference in a specific frequency band in each of the multiple first sound signals (for example, the Rch and Lch mentioned above), the signal levels in that specific frequency band can be brought closer together when a second sound signal is superimposed on each. Here, for example, when a user is made to listen to sounds in a specific frequency band from both ears using earphones, there is a problem that if there is a phase difference between the sound emitted from the speaker worn in the right ear and the sound emitted from the speaker worn in the left ear, it will be difficult to obtain the effect of improving dementia, etc. Therefore, for example, when the speakers provided in the sound signal processing device are implemented as earphones such as speakers 171 and 172, by reducing the phase difference between the sound emitted from speaker 171 worn in the left ear and the sound emitted from speaker 172 worn in the right ear, a sound that is effective in improving dementia, etc. can be output.

[0196] Furthermore, for example, an audio signal processing method according to one aspect of the present disclosure further acquires an ambient sound signal based on ambient sound (for example, step S801), and generates a second content such that a component of a specific frequency band in the ambient sound signal becomes a second audio signal (for example, step S802).

[0197] According to this, even if the first sound signal does not contain a signal in a specific frequency band, additional information 200 can be generated using ambient sound, and the second sound signal can be superimposed on the first sound signal and output. Furthermore, for example, since the second sound signal is generated based on ambient sound, even if the sound in a specific frequency band becomes louder, the discomfort caused by the difference from the ambient sound can be suppressed because the sound is similar to the ambient sound, thus preventing the user from feeling uncomfortable due to the sound in a specific frequency band.

[0198] Furthermore, in the adjustment described above, processing is performed to increase the signal level of the harmonics in a specific frequency band (for example, step S401).

[0199] Harmonics in specific frequency bands are also known to be effective in improving conditions such as dementia, similar to sounds in those specific frequency bands. Therefore, according to this method, sounds that are even more effective in improving conditions such as dementia can be emitted. Furthermore, for example, small speakers may not be able to output sounds in specific frequency bands, such as the low frequency band of around 40Hz. Even in such cases, harmonics are likely to be emitted, so an effect on improving conditions such as dementia can be expected.

[0200] Furthermore, the signal level of the harmonics in a specific frequency band in the first tone signal may be increased, or the signal level of the harmonics in a specific frequency band in the second tone signal may be increased.

[0201] Furthermore, in one embodiment of the present disclosure, for example, control information (e.g., amplitude value information 201) indicating the signal level in a specific frequency band is acquired, and in the above adjustment, the signal level in the specific frequency band is controlled based on the control information (e.g., step S501).

[0202] According to this, users can hear sounds in a specific frequency band at their desired volume.

[0203] Furthermore, in one embodiment of the present disclosure, the sound signal processing method further acquires the user's biometric information (e.g., pNN information 202), and in the above-described adjustment, the signal level of a specific frequency band is controlled based on the biometric information (e.g., step S602).

[0204] According to this, for example, the system determines whether the user is comfortable based on biometric information, and adjusts the signal level of the second sound signal based on the determination result, allowing the user to easily hear comfortable sounds without having to make any adjustments themselves.

[0205] Furthermore, in one embodiment of the present disclosure, for example, an audio signal processing method is further performed to acquire an output sound signal based on the output sound (more specifically, the sound output from speakers 170, 171, and 172) output by the above-mentioned output (for example, step S701), and to control the signal level of a specific frequency band based on the output sound signal (for example, steps S702 to S704).

[0206] Depending on the installation environment of the speaker 170 that outputs the sound produced by the sound signal processing method, for example, in the case of earphones, depending on how the earphones are worn by the user, there may be cases where sounds in a specific frequency band are not output from the speaker 170 at the expected volume. Therefore, as described above, for example, the sound actually output from the speaker 170 is captured by the microphone 190, and the signal level of the specific frequency band is adjusted using the microphone signal based on the captured sound. As a result, a suitable sound is output from the speaker 170.

[0207] Furthermore, a program relating to one aspect of this disclosure is, for example, a program that causes a computer to execute an audio signal processing method relating to one aspect of this disclosure.

[0208] Furthermore, an audio signal processing device according to one aspect of the present disclosure comprises a processor and a memory, the processor using the memory to adjust the signal level of a second audio signal corresponding to a second content, which includes a component in a specific frequency band, according to the signal level of a specific frequency band in a first audio signal corresponding to a first content, and outputs the first audio signal and the adjusted second audio signal superimposed.

[0209] According to this, the same effects as the sound signal processing method according to one aspect of the present disclosure are achieved.

[0210] The term "processor" here refers to, for example, at least one of the DSP120 and the CPU130, and may be implemented using only the DSP120, only the CPU130, or both the DSP120 and the CPU130. The term "memory" here refers to, for example, memory modules 140, 141, 142, and 143, but may be implemented using the storage device 110, or using memory modules 140, 141, 142, and 143 and the storage device 110. Furthermore, "using memory" means, for example, that the processor performs various operations using programs and information stored in memory.

[0211] (Embodiment 2) Next, we will describe the sound signal processing device according to Embodiment 2. In the following description, we will focus on the differences between Embodiment 1 and its various modifications, and substantially similar components will be denoted by the same reference numerals, and some explanations may be simplified or omitted.

[0212] The sound signal processing device according to Embodiment 2 adjusts the signal level of the first sound signal in a specific frequency band instead of superimposing the second sound signal on the first sound signal. Furthermore, it performs a correction to reduce the phase difference between multiple first sound signals whose signal levels in specific frequency bands have been adjusted. As a result, the phase difference between multiple sound signals is reduced, making it easier to obtain the effect of improving dementia and other conditions.

[0213] [composition] First, the configuration of the sound signal processing device according to Embodiment 2 will be described.

[0214] Figure 17 is a block diagram showing the configuration of the sound signal processing device 105 according to Embodiment 2.

[0215] The sound signal processing device 105 includes a storage device 110, a DSP 120, a CPU 130, a memory 143, a DAC 150, an amplifier 160, a speaker 171, and a speaker 172.

[0216] For example, the DSP120 performs a process to raise the signal level of a specific frequency band of the sound signal acquired from the storage device 110 to a predetermined level, based on the enhancement information 203.

[0217] For example, both the sound signal processing device 100 and the sound signal processing device 105 perform a process on the acquired sound signal (first sound signal) to adjust (more specifically, increase) the signal level in a specific frequency band and output it.

[0218] The sound signal processing device 100 adjusts the signal level of a specific frequency band in the output signal by superimposing a signal (second sound signal) that includes only a specific frequency band onto the acquired sound signal (first sound signal).

[0219] On the other hand, the sound signal processing device 105 adjusts the signal level of a specific frequency band in the output signal by augmenting the signal in a specific frequency band with respect to the acquired sound signal (first sound signal).

[0220] Specifically, the DSP120 adjusts the signal level of each of the multiple first-order audio signals corresponding to the multiple different audio contents in the first-order audio content, which contains multiple audio contents, by increasing the signal level of a specific frequency band. The DSP120 also corrects the phase difference in a specific frequency band for each of the adjusted first-order audio signals corresponding to the multiple audio contents. The DSP120 then outputs the corrected multiple first-order audio signals.

[0221] <Processing Procedure> Next, the processing procedure of the sound signal processing device 105 will be explained.

[0222] Figure 18 is a flowchart showing the processing procedure of the sound signal processing device 105 according to Embodiment 2. Specifically, Figure 18 is a flowchart showing the processing procedure of the DSP 120 provided in the sound signal processing device 105.

[0223] First, the DSP120 obtains the enhancement information 203 that the CPU130 has acquired from memory 143 (S801).

[0224] Next, the DSP120 reads the audio content from the storage device 110 and acquires the sound source signal, which is the audio signal based on that audio content (S102). For example, the DSP120 acquires the Lch and Rch as the sound source signal.

[0225] Next, the DSP120 applies a narrowband filter to the sound source signal and enhances the signal level of a specific frequency band based on the enhancement information 203 (S901). For example, the DSP120 identifies specific frequency bands for the Lch and Rch respectively and increases the signal levels of the Lch and Rch respectively according to the predetermined signal levels indicated by the enhancement information 203.

[0226] The sound signal processing device 105 may also include a filter circuit that functions as a narrowband filter.

[0227] Next, the DSP120 generates a signal (corrected signal) by correcting at least one of the Lch and Rch so that the phases of the Lch and Rch in a specific frequency band are aligned (S805). The DSP120 may also correct at least one of the Lch and Rch so as to reduce the phase difference between the Lch and Rch in a specific frequency band.

[0228] Next, the DSP120 outputs a signal with the phase difference corrected to zero (corrected signal) to the DAC150 (S106).

[0229] The correction signal is transmitted from the DAC 150 to the speakers 171 and 172 via the amplifier 160. Speakers 171 and 172 output sound based on this correction signal.

[0230] [Effects, etc.] As described above, an audio signal processing method according to another aspect of the present disclosure adjusts the signal level of a specific frequency band of each of a plurality of first audio signals (e.g., Lch and Rch) corresponding to a plurality of different audio contents in a first content containing a plurality of different audio contents (e.g., step S901), corrects each of the adjusted plurality of first audio signals corresponding to the plurality of audio contents to reduce the phase difference in a specific frequency band in each of the adjusted plurality of first audio signals (e.g., step S804), and outputs the corrected plurality of first audio signals (e.g., step S106).

[0231] In recent years, for example, with earphones, the sound output to the right ear and the sound output to the left ear may differ, for reasons such as enhancing the sense of realism. However, it is known that if, for example, a sound in a specific frequency band differs when output to the right ear and when output to the left ear, it may be difficult to obtain the desired effect on dementia, etc. Therefore, by correcting to reduce the phase difference in a specific frequency band in each of the multiple first sound signals (for example, the Rch and Lch mentioned above), the difference in signal levels in that specific frequency band can be reduced. As a result, a sound that is effective in improving dementia, etc., can be output. In other words, according to another embodiment of this disclosure, a sound signal processing method can suitably output a sound in a specific frequency band.

[0232] Furthermore, a program relating to one aspect of this disclosure may also be a program that causes a computer to execute an audio signal processing method relating to another aspect of this disclosure.

[0233] Furthermore, an audio signal processing device according to another aspect of the present disclosure comprises a processor and a memory, the processor using the memory to adjust the signal level of a specific frequency band of each of a plurality of first sound signals corresponding to a plurality of different sound contents in a first content containing a plurality of sound contents, the processor corrects each of the adjusted plurality of first sound signals corresponding to the plurality of sound contents to reduce the phase difference of a specific frequency band in each of the adjusted plurality of first sound signals, and outputs the corrected plurality of first sound signals.

[0234] According to this, the same effects as the sound signal processing method according to another aspect of the present disclosure are achieved.

[0235] The processor referred to here is, for example, at least one of the DSP120 and the CPU130, and may be implemented using only the DSP120, only the CPU130, or both the DSP120 and the CPU130. Furthermore, the memory referred to here is, for example, memory 143, but may be implemented using the storage device 110, or using both memory 143 and storage device 110.

[0236] (Other embodiments) Although each embodiment and each variation has been described above, this disclosure is not limited to the above embodiments and each variation.

[0237] For example, each of the above embodiments and modifications may be implemented by combining them as needed. For example, in the configuration (processing procedure) of Modification 2 described above, the sound signal processing device according to this disclosure may change the value of the corresponding signal to the value of the threshold if the corresponding signal is smaller than the threshold. More specifically, in the processing procedure shown in Figure 6, the sound signal processing device according to this disclosure may add processing similar to that shown in steps S201 and S202 in Figure 5 between steps S301 and S302 shown in Figure 6. Also, for example, the function of the sound signal processing device according to Modification 3 described above (e.g., harmonic multiplication processing) may be implemented in combination with the function of the sound signal processing device according to Embodiment 1 (e.g., envelope multiplication processing), or in combination with the function of other sound signal processing devices. Also, for example, the function of the sound signal processing device according to Modification 4 described above (e.g., signal level adjustment processing based on control information) may be implemented in combination with the function of other sound signal processing devices. Furthermore, for example, the function of the sound signal processing device according to the modified example 5 described above (for example, the signal level adjustment process based on biological information) may be realized in combination with the functions of other sound signal processing devices.

[0238] Furthermore, for example, the signal level of a specific frequency band in the first tone signal may be adjusted based on at least one of the control information and the biological information, or the signal level of a specific frequency band in the second tone signal may be adjusted. In other words, the relative relationship between the signal levels of the first tone signal and the second tone signal in a specific frequency band may be adjusted based on at least one of the control information and the biological information. Alternatively, the signal level of a specific frequency band in the superimposed signal may be adjusted.

[0239] Furthermore, for example, the signal level of the harmonic frequency band of a specific frequency band in the first tone signal may be adjusted based on at least one of the control information and the biological information, or the signal level of the harmonic frequency band of a specific frequency band in the second tone signal may be adjusted. In other words, the relative relationship between the signal levels of the first tone signal and the second tone signal in the harmonic frequency band of a specific frequency band may be adjusted based on at least one of the control information and the biological information. Alternatively, the signal level of the harmonic frequency band of a specific frequency band in the superimposed signal may be adjusted.

[0240] Furthermore, for example, if the signal level of the first tone signal is zero (i.e., silent), the second tone signal may or may not be superimposed. Also, for example, if the signal level of the first tone signal is zero, the signal level of a specific frequency band in the first tone signal may or may not be increased.

[0241] Furthermore, the configurations of the sound signal processing devices according to each of the above embodiments and modifications are merely examples. For example, the sound signal processing device may include components not shown, such as a D / A converter or a filter.

[0242] Furthermore, in each of the above embodiments, the multiple first sound signals corresponding to multiple sound contents were described using two first sound signals, for example, Lch and Rch. However, the number of first sound signals may be three or more.

[0243] Furthermore, in the above embodiment, the sound signal processing device may be implemented as multiple devices (i.e., a system) or as a single device. When the sound signal processing device is implemented by multiple devices, the functional components of the sound signal processing device may be distributed among the multiple devices in any way. For example, a mobile terminal may provide some or all of the functional components of the sound signal processing device.

[0244] Furthermore, the communication method between devices in each of the above embodiments and modifications is not particularly limited. When two devices communicate in each of the above embodiments and modifications, a relay device (not shown) may be interposed between the two devices.

[0245] Furthermore, the order of processing described in each of the above embodiments and modifications is merely an example. The order of multiple processing steps may be changed, and multiple processing steps may be executed in parallel. Also, processing performed by one processing unit may be performed by another processing unit. In addition, some of the digital signal processing described in each of the above embodiments and modifications may be implemented by analog signal processing.

[0246] Furthermore, in each of the above embodiments and modifications, each component may be realized by executing a software program suitable for that component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0247] Furthermore, each component may be implemented by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or they may be separate circuits. Also, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0248] Furthermore, the general or specific aspects of this disclosure may be implemented as a system, apparatus, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM. They may also be implemented in any combination of systems, apparatus, methods, integrated circuits, computer programs, and recording media. For example, this disclosure may be implemented as a method executed by a computer, such as an audio signal processing device or a portable terminal, or as a program to cause a computer to execute such a method. Furthermore, this disclosure may be implemented as a computer-readable non-temporary recording medium on which such a program is recorded. The program as used herein includes application programs to cause a general-purpose portable terminal to function as a portable terminal of each of the embodiments and modifications described above.

[0249] Furthermore, this disclosure also includes forms obtained by applying various modifications to each embodiment and each modification that a person skilled in the art could conceive of, or forms realized by arbitrarily combining the components and functions of each embodiment and each modification without departing from the spirit of this disclosure. [Industrial applicability]

[0250] The sound signal processing device described herein can be applied to a device that outputs sounds capable of improving dementia and other related conditions. [Explanation of Symbols]

[0251] 100, 101, 102, 103, 104, 105 Audio signal processing device 110, 320 storage device 120 DSP 130 CPU 140, 141, 142, 143 memory 150 DAC 160 Amplifier 170, 171, 172 speakers 180 Communication IF 190, 330 microphones 200 Additional Information 201 Amplitude Value Information 202 pNN information 203 Emphasis information 300 External terminals 310 Heart Rate Monitor

Claims

1. Depending on the signal level of a specific frequency band in the first audio signal corresponding to the first content, the signal level of the second audio signal corresponding to the second content, which includes components of the said specific frequency band, is adjusted. The first sound signal and the adjusted second sound signal are superimposed and output. In the aforementioned adjustment, The envelope of the signal level of the first sound signal is calculated, and if the value of the envelope is greater than a threshold, the signal level of the specific frequency band in the second sound signal is adjusted to the signal level obtained by multiplying the signal level of the specific frequency band in the second sound signal by the value of the envelope. If the envelope value is less than or equal to the threshold, the signal level of the specific frequency band in the second tone signal is adjusted to the signal level obtained by multiplying the signal level of the specific frequency band in the second tone signal by the threshold. Audio signal processing method.

2. Depending on the signal level of a specific frequency band in the first audio signal corresponding to the first content, the signal level of the second audio signal corresponding to the second content, which includes components of the said specific frequency band, is adjusted. The first sound signal and the adjusted second sound signal are superimposed and output. In the aforementioned adjustment, If the signal level of the specific frequency band in the first sound signal is greater than a threshold, the signal level of the second sound signal is increased by a predetermined ratio with respect to the signal level of the specific frequency band in the first sound signal. The first content includes multiple audio contents that are different from each other, The multiple first audio signals corresponding to the multiple audio contents include a right channel and a left channel, and at least one of the right channel and the left channel is corrected to make the phase difference between the right channel and the left channel in the specific frequency band zero. Audio signal processing method.

3. moreover, We acquire ambient sound signals based on ambient sounds, The second content is generated such that the component of the specific frequency band in the ambient sound signal becomes the second sound signal. The sound signal processing method according to claim 1 or 2.

4. In the above adjustment, control is further performed to increase the signal level of the harmonic frequency band of the specific frequency band. The sound signal processing method according to claim 1 or 2.

5. Furthermore, control information indicating the signal level in the specified frequency band is acquired. In the adjustment described above, the signal level of the specific frequency band is controlled based on the control information. The sound signal processing method according to claim 1 or 2.

6. Furthermore, the system acquires the user's biometric information. In the adjustment described above, the signal level of the specific frequency band is controlled based on the biological information. The sound signal processing method according to claim 1 or 2.

7. moreover, The output sound signal is acquired based on the output sound output from the aforementioned output. Based on the output sound signal, the signal level of the specific frequency band is controlled. The sound signal processing method according to claim 1 or 2.

8. To cause a computer to execute the sound signal processing method described in claim 1 or 2. program.

9. Processor and Equipped with memory, The processor uses the memory to: Depending on the signal level of a specific frequency band in the first audio signal corresponding to the first content, the signal level of the second audio signal corresponding to the second content, which includes components of the said specific frequency band, is adjusted. The first sound signal and the adjusted second sound signal are superimposed and output. In the aforementioned adjustment, The envelope of the signal level of the first sound signal is calculated, and if the value of the envelope is greater than a threshold, the signal level of the specific frequency band in the second sound signal is adjusted to the signal level obtained by multiplying the signal level of the specific frequency band in the second sound signal by the value of the envelope. If the envelope value is less than or equal to the threshold, the signal level of the specific frequency band in the second tone signal is adjusted to the signal level obtained by multiplying the signal level of the specific frequency band in the second tone signal by the threshold. Sound signal processing device.

10. Processor and Equipped with memory, The processor uses the memory to: Depending on the signal level of a specific frequency band in the first audio signal corresponding to the first content, the signal level of the second audio signal corresponding to the second content, which includes components of the said specific frequency band, is adjusted. The first sound signal and the adjusted second sound signal are superimposed and output. In the aforementioned adjustment, If the signal level of the specific frequency band in the first sound signal is greater than a threshold, the signal level of the second sound signal is increased by a predetermined ratio with respect to the signal level of the specific frequency band in the first sound signal. The first content includes multiple audio contents that are different from each other, The multiple first audio signals corresponding to the multiple audio contents include a right channel and a left channel, and at least one of the right channel and the left channel is corrected to make the phase difference between the right channel and the left channel in the specific frequency band zero. Sound signal processing device.

Images