Parameter determination method, parameter determination device, and program

Abstract

This parameter determination method is for determining a parameter of a filter that outputs a second sound signal in which a frequency characteristic is changed with respect to a first sound signal. The parameter determination method includes: calculating a first acoustic feature amount of a first sound signal; calculating a second acoustic feature amount of a second sound signal; and determining a parameter of a filter so that a difference in a specific acoustic feature amount related to audibility between the first acoustic feature amount and the second acoustic feature amount is no greater than a predetermined value.

Description

Parameter determination method, parameter determination device, and program

[0001] An embodiment of the present invention relates to a parameter determination method, a parameter determination device, and a program.

[0002] Patent Document 1 describes a digital mixer that allows multiple channels and multiple parameters to be displayed, operated, and changed simultaneously.

[0003] Japanese Patent Application Laid-Open No. 2022-77497

[0004] A user (for example, an operator of an audio mixer) had to determine the filter processing parameters as a pre-setting.

[0005] An object of one embodiment of the present invention is to provide a sound processing method that reduces the workload required of a user.

[0006] A parameter determination method according to an embodiment of the present invention is a parameter determination method for determining parameters of a filter that outputs a second sound signal having a changed frequency characteristic with respect to a first sound signal, the method comprising the steps of: calculating a first acoustic feature of the first sound signal; calculating a second acoustic feature of the second sound signal; and determining the parameters of the filter such that a difference between a specific acoustic feature related to audibility between the first acoustic feature and the second acoustic feature is equal to or less than a predetermined value.

[0007] According to one embodiment of the present invention, the workload required of the user can be reduced.

[0008] It is a block diagram showing the hardware configuration of the audio mixer 1. It is a diagram showing the functional configuration of a signal processing block. It is a flowchart showing a filter parameter determination method. It is a flowchart showing a filter parameter determination method according to a first modified example.

[0009] Fig. 1 is a block diagram showing the hardware configuration of an audio mixer 1. Fig. 2 is a diagram showing the functional configuration of a signal processing block of the audio mixer 1.

[0010] The audio mixer 1 is an example of a parameter determination device of the present invention. The audio mixer 1 receives an audio signal from an audio device such as a microphone or a musical instrument. The audio mixer 1 performs signal processing such as mixing or effects on the input audio signal. The audio mixer 1 transmits the processed audio signal to an audio device such as a speaker.

[0011] The audio mixer 1 includes a touch panel display 101, an operation unit 102, an audio I / O (Input / Output) 103, a signal processing unit (DSP) 104, a network I / F (interface) 105, a USB I / F 106, a CPU 107, a flash memory 108, and a RAM 109. These components are connected via a bus 210.

[0012] The CPU 107 is a control unit that controls the operation of the audio mixer 1. The CPU 107 performs various operations by reading programs stored in a flash memory 108, which is a storage medium, into the RAM 109 and executing the programs. Note that the programs do not need to be stored in the flash memory 108 of the device itself. For example, the programs may be downloaded from another device such as a server and read into the RAM 109 each time.

[0013] The CPU 107 controls input and output of sound signals at the audio I / O 103, or controls mixing processing and effect processing at the signal processing unit 104, and changes the values ​​of parameters related to these.

[0014] The touch panel display 101 is composed of, for example, an LCD (Liquid Crystal Display) and a touch panel laminated on the LCD. The touch panel display 101 displays various information under the control of the CPU 107. The touch panel display 101 also accepts operations from the user.

[0015] The operation unit 102 is made up of various controls such as switches, knobs, and faders, and receives operations on the audio mixer 1 from the user.

[0016] The signal processing unit 104 is composed of a DSP (Digital Signal Processor) for performing various signal processing such as mixing processing, effect processing, etc. The signal processing unit 104 performs signal processing such as mixing processing, effect processing, etc. on sound signals input from the audio I / O 103, the network I / F 105, or the USB I / F 106. The signal processing unit 104 outputs the processed sound signals via the audio I / O 103, the network I / F 105, or the USB I / F 106.

[0017] 2, the signal processing block 901 is composed of input channels 951, buses 952, and output channels 953. In this example, the input channels 951 have eight channels (1-8). The buses 952 have various buses such as a stereo bus, a MIX bus, and a MATRIX bus. The output channels 953 are blocks that process the sound signals sent from each bus.

[0018] Each channel of the input channels 951 performs signal processing such as high-pass filtering, low-pass filtering, equalization, or compression on the input sound signal. Each channel of the input channels 951 sends the processed sound signal to a subsequent bus 952. The level of the sound signal sent by the input channel is adjusted by a control such as a fader.

[0019] The bus 952 mixes the sound signals sent from each input channel and outputs the mixed signal to the output channel 953 .

[0020] The output channel 953 applies effect processing such as an equalizer or a compressor to the input sound signal. The sound signal after the signal processing is supplied to the audio I / O 103, the network I / F 105, or the USB I / F 106.

[0021] That is, the signal processing block 901 functions as a filter that outputs a second sound signal in which the frequency characteristics of the first sound signal are changed.

[0022] The above signal processing is controlled based on the values ​​of various parameters. The CPU 107 stores the current values ​​(current data) of the various parameters in the RAM 109. The CPU 107 updates the current data when the user operates the touch panel display 101 or the operation unit 102.

[0023] 3 is a flowchart showing a method for determining the filter parameters, which is one of the operations of the CPU 107. The CPU 107 calculates a first acoustic feature of a first sound signal (S11) and calculates a second acoustic feature of a second sound signal (S12). Then, the CPU 107 determines the filter parameters so that the difference between a specific acoustic feature related to auditory sensation, between the first acoustic feature and the second acoustic feature, is equal to or smaller than a predetermined value (S13).

[0024] The filter may be, for example, a high-pass filter or a low-pass filter, and the parameter determined by the CPU 107 is a cutoff frequency. The cutoff frequencies of the high-pass filter and the low-pass filter must be appropriately set according to the acoustic characteristics of the audio device that outputs the first sound signal. For example, a high-pass filter is used to remove unwanted low-frequency sounds, such as humming noise from an electric instrument, background noise in the surrounding environment, or wind or breathing noise (pop noise) heard by a microphone. The higher the cutoff frequency of the high-pass filter, the more it can remove unwanted low-frequency sounds. However, it also removes the main frequency bands of each audio device, which has a greater impact on the auditory sense. The user sets an appropriate cutoff frequency that removes unwanted low-frequency sounds and minimizes the impact on the auditory sense according to the acoustic characteristics of the audio device.

[0025] Furthermore, the audio mixer 1 has multiple (eight in this embodiment) input channels. Therefore, the user must appropriately set the cutoff frequencies of the high-pass filters and low-pass filters for each of the multiple input channels according to the acoustic characteristics of the audio equipment for each channel. The user (e.g., the audio mixer operator) must determine the cutoff frequencies for all channels as a pre-setting, which places a significant workload on the user.

[0026] The parameter determination method of this embodiment determines the cutoff frequency through the processes of S11 to S13, thereby reducing the workload of the user. The CPU 107 calculates a first acoustic feature of the first sound signal and a second acoustic feature of the second sound signal. The first acoustic feature and the second acoustic feature are, for example, frequency characteristics, and more specifically, spectral centroids or spectral envelopes.

[0027] For example, the CPU 107 transforms the sound signal into a frequency axis using a short-time Fourier transform to acquire an amplitude spectrum of the sound signal. The calculation unit 52 averages the amplitude spectrum over a specific period to acquire an average spectrum. The calculation unit 52 removes a bias (zeroth-order component of the cepstrum), which is an energy component, from the average spectrum to acquire a spectral envelope. Note that either the averaging in the time axis direction or the removal of the bias may be performed first. That is, the CPU 107 may first remove the bias from the amplitude spectrum and then acquire the average spectrum averaged in the time axis direction as the spectral envelope. The CPU 107 calculates the spectral center of gravity by dividing a value obtained by weighting the entire spectral envelope by the frequency by the entire spectral envelope. In the process of S11, the CPU 107 calculates, for example, a first spectral center of gravity S1 as a first acoustic feature of the first sound signal. In the process of S12, the CPU 107 calculates, for example, a second spectral center of gravity S2 as a second acoustic feature of the second sound signal, similar to S11.

[0028] In the process of S13, the CPU 107 determines a cutoff frequency such that the ratio (S2 / S1) of the first spectral centroid S1 to the second spectral centroid S2 is a predetermined value. The predetermined value is a value (slightly greater than 1) that does not significantly change the auditory sensation but is capable of cutting out unnecessary low frequencies.

[0029] As described above, while a high-pass filter can cut out unnecessary low frequencies more effectively as the cutoff frequency increases, the impact on auditory perception increases. In a high-pass filter, the higher the cutoff frequency, the higher the spectral centroid, and therefore the higher the value of the ratio (S2 / S1) between the first spectral centroid S1 and the second spectral centroid S2. Therefore, this spectral centroid ratio (S2 / S1) is an example of a specific acoustic feature related to auditory perception. The CPU 107 calculates the ratio (S2 / S1) between the first spectral centroid S1 and the second spectral centroid S2 every time a specific period elapses, and updates the cutoff frequency using a predetermined algorithm, such as Least Mean Square (LMS) or Recursive Least Squares (RLS), so that the value of this ratio approaches a predetermined value (e.g., 1.02).

[0030] Alternatively, in the process of S13, the CPU 107 may determine a cutoff frequency such that the ratio between the spectral envelope of the first sound signal and the spectral envelope of the second sound signal becomes a predetermined value. The higher the cutoff frequency, the higher the value of the ratio between the spectral envelope of the first sound signal and the spectral envelope of the second sound signal. Therefore, such a ratio of spectral envelopes is also an example of a specific acoustic feature related to auditory sensation. The CPU 107 may calculate the ratio between the spectral envelope of the first sound signal and the spectral envelope of the second sound signal every time a specific period elapses, and update the cutoff frequency using a predetermined algorithm so that the value of the ratio approaches a predetermined value.

[0031] The CPU 107 updates the current data with the determined cutoff frequency of the high-pass filter (S14). Thereafter, signal processing in the signal processing block 901 is performed based on the current data updated by the CPU 107. That is, the filter in the signal processing block 901 performs sound processing to change the frequency characteristics of the first sound signal using the parameters determined by the CPU 107 and output a second sound signal.

[0032] As a result, the parameter determination method of this embodiment can automatically set parameters for various filters, such as high-pass filters, to a degree that does not significantly change the auditory sensation. Therefore, the workload required of the user is reduced. In particular, in a case with a large number of channels, such as an audio mixer, the workload of determining the cutoff frequencies of multiple channels is significantly reduced, allowing the user to enjoy the experience of being able to perform pre-settings in an extremely short time, which was previously impossible.

[0033] (Modification 1) Fig. 4 is a flowchart showing a filter parameter determination method according to Modification 1. Components common to Fig. 3 are assigned the same reference numerals, and description thereof will be omitted. The CPU 107 displays the determined cutoff frequency of the high-pass filter on the touch panel display 101 (S104).

[0034] In this case, the user sets the cutoff frequency by operating touch panel display 101 or operation unit 102, referring to the cutoff frequency of each channel displayed on touch panel display 101. The user can set the cutoff frequency determined by CPU 107 as is, or can change and set the cutoff frequency determined by CPU 107. For example, the user can reflect the automatic determination of parameters by CPU 107, or can adjust the parameters according to their own preferences.

[0035] (Modification 2) The CPU 107 according to Modification 2 determines parameters using a trained model that trains the relationship between the first acoustic feature amount, the second acoustic feature amount, and the filter parameters.

[0036] As described above, the first acoustic feature is, for example, the first spectral centroid, and the second acoustic feature is, for example, the second spectral centroid.

[0037] In the training stage, the audio mixer 1 acquires a large number of data sets (training data) including first and second spectral centroids and a parameter for the cutoff frequency of a high-pass filter, and trains a predetermined training model to determine the relationship between the first and second spectral centroids and the parameter for the cutoff frequency of the high-pass filter using a predetermined algorithm.

[0038] The algorithm for training the training model is not limited, and any machine training algorithm such as a convolutional neural network (CNN) or a recurrent neural network (RNN) can be used. The machine training algorithm may be supervised training, unsupervised training, semi-supervised training, reinforcement training, inverse reinforcement training, active training, or transfer training. Furthermore, the calculation unit 52 may train the training model using a machine training model such as a hidden Markov model (HMM) or a support vector machine (SVM).

[0039] The appropriate cutoff frequency has an optimal value for removing unnecessary low frequencies and suppressing their impact on auditory perception. In other words, there is a correlation between the first spectral centroid, the second spectral centroid, and the cutoff frequency. Therefore, the audio mixer 1 can train a predetermined training model on the relationship between the first spectral centroid, the second spectral centroid, and the cutoff frequency parameters of the high-pass filter, thereby generating a trained model.

[0040] In the initial stage, the audio mixer 1 inputs the first acoustic feature (first spectral centroid) of the first sound signal and the second acoustic feature (second spectral centroid) of the second sound signal into the trained model, and outputs the corresponding cutoff frequency parameters of the high-pass filter.

[0041] In this way, the parameter determination method of this embodiment can determine parameters for various filters such as high-pass filters that do not significantly change the auditory sensation using a trained model that has been trained using settings previously made by the user.

[0042] The description of the present embodiment should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above-described embodiments. Furthermore, the scope of the present invention includes the scope equivalent to the claims.

[0043] 1: Audio mixer, 52: Calculation unit, 101: Touch panel display, 102: Operation unit, 103: Audio I / O, 104: Signal processing unit, 105: Network I / F, 107: CPU, 108: Flash memory, 109: RAM, 210: Bus, 901: Signal processing block, 951: Input channel, 952: Bus, 953: Output channel

Claims

1. A parameter determination method for determining parameters of a filter that outputs a second sound signal having a changed frequency characteristic with respect to a first sound signal, the parameter determination method comprising the steps of: calculating a first acoustic feature of the first sound signal; calculating a second acoustic feature of the second sound signal; and determining the parameters of the filter such that a difference between a specific acoustic feature related to audibility between the first acoustic feature and the second acoustic feature is equal to or smaller than a predetermined value.

2. The parameter determination method according to claim 1, wherein the specific acoustic feature includes a spectral centroid, and the parameter is determined such that a ratio of a first spectral centroid of the first acoustic feature and a second spectral centroid of the second acoustic feature becomes a predetermined value.

3. The parameter determination method according to claim 1, wherein the specific acoustic feature includes a spectral envelope, and the parameter is determined such that a ratio between a first spectral envelope of the first acoustic feature and a second spectral envelope of the second acoustic feature becomes a predetermined value.

4. The parameter determination method according to any one of claims 1 to 3, further comprising displaying the determined parameter.

5. A parameter determination method according to any one of claims 1 to 3, wherein the filter performs sound processing to change a frequency characteristic of the first sound signal using the determined parameters and output the second sound signal.

6. A parameter determination method according to any one of claims 1 to 3, wherein the parameters are determined using a trained model that trains the relationship between the first acoustic feature, the second acoustic feature, and the parameters of the filter.

7. A parameter determination method according to any one of claims 1 to 3, wherein the filter is a high-pass filter or a low-pass filter, and the parameters include a cutoff frequency.

8. A parameter determination device comprising a processor that determines parameters of a filter that outputs a second sound signal that has a changed frequency characteristic with respect to a first sound signal, wherein the processor calculates a first acoustic feature of the first sound signal, calculates a second acoustic feature of the second sound signal, and determines the parameters of the filter such that a difference between a specific acoustic feature related to audibility between the first acoustic feature and the second acoustic feature is equal to or smaller than a predetermined value.

9. The parameter determination device according to claim 8, wherein the specific acoustic feature includes a spectral centroid, and the processor determines the parameter such that a ratio of a first spectral centroid of the first acoustic feature and a second spectral centroid of the second acoustic feature becomes a predetermined value.

10. The parameter determination device according to claim 8, wherein the specific acoustic feature includes a spectral envelope, and the processor determines the parameter such that a ratio between a first spectral envelope of the first acoustic feature and a second spectral envelope of the second acoustic feature becomes a predetermined value.

11. The parameter determination device according to any one of claims 8 to 10, wherein the processor displays the determined parameter on a display.

12. The parameter determination device according to any one of claims 8 to 10, wherein the filter performs sound processing to change a frequency characteristic of the first sound signal using the determined parameters and output the second sound signal.

13. The parameter determination device according to any one of claims 8 to 10, wherein the processor determines the parameters using a trained model that trains a relationship between the first acoustic feature, the second acoustic feature, and the parameters of the filter.

14. The parameter determination device according to any one of claims 8 to 10, wherein the filter is a high-pass filter or a low-pass filter, and the parameters include a cutoff frequency.

15. A program causing an information processing device to execute a process of determining parameters of a filter that outputs a second sound signal with a changed frequency characteristic with respect to a first sound signal, the program causing the information processing device to execute the processes of: calculating a first acoustic feature of the first sound signal; calculating a second acoustic feature of the second sound signal; and determining the parameters of the filter such that a difference between a specific acoustic feature related to auditory sensation among the first acoustic feature and the second acoustic feature is equal to or smaller than a predetermined value.

Images