Sound signal processing method and sound signal processing device

Abstract

To perform appropriate processing for each sound signal and output the processed signal to an appropriate output destination while avoiding complication of equipment connection.SOLUTION: A sound signal processing method is provided in which a processing unit A performs ambient sound enhancement processing on a sound signal Si by a post-processing unit A-2 to generate a sound signal Sg while outputting the sound signal Sg to a mix bus 34, a processing unit C performs C processing on a sound signal Sa to generate a sound signal Se while outputting the sound signal Se to the mix bus 34 and a first output destination (communication terminal device 28 or PC 29), and the mix bus 34 generates a sound signal Sh by mixing the sound signal Sg with the signal Se while outputting the sound signal Sh to the second output destination (headphone 31 or speaker 32).SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a sound signal processing method and a sound signal processing device. [Background technology]

[0002] Conventionally, there is known a sound signal processing device that performs various processes on sound signals and mixes them together to enhance sound effects. For example, Patent Documents 1 and 2 convert a stereo sound signal into a sum signal and a difference signal, and after performing processes such as sound effects on the sum signal and the difference signal, convert them back into a stereo sound signal. This makes it possible to easily and diversify the sound effects of the stereo sound signal. Alternatively, it is possible to provide a sound that is less unnatural. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-116153 A [Patent Document 2] JP 2017-220789 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the processing content to be applied, such as effects, and the output destination to which the sound signal is to be output vary depending on the sound signal. Therefore, when many sound signals are acquired from multiple systems, the number of conversion devices increases in order to apply appropriate processing to each sound signal, and the device connections may become complicated.

[0005] One object of the present invention is to provide an audio signal processing method and an audio signal processing device that can perform appropriate processing on each audio signal and output it to an appropriate output destination while avoiding complicated device connections. [Means for solving the problem]

[0006] According to one aspect of the present invention, a first processing unit performs a first processing on a first sound signal to generate a second sound signal and outputs the second sound signal to a mix bus, and a second processing unit: output from the microphone To the third sound signal , including effects and level adjustments performing a second process different from the first process to generate a fourth sound signal, and outputting the fourth sound signal to the mix bus and a first output destination; a fourth processing unit performs a fourth processing step different from both the first processing step and the second processing step on the seventh sound signal to generate an eighth sound signal, and outputs the eighth sound signal to the mix bus; The mix bus includes the second sound signal and the fourth sound signal. The eighth sound signal; and outputting the fifth sound signal to a second output destination. the eighth sound signal is mixed with the second sound signal and the fourth sound signal in the mix bus without being mixed with the second sound signal or the fourth sound signal upstream of the mix bus, the first sound signal is a stereo signal generated by a third processing unit performing a third process to stereoize a sixth sound signal having three or more channels output from a game device, the first processing unit generates a sum signal and a difference signal from the stereo signal, the first processing unit generates a processed sum signal by performing a process on the sum signal using a DeEsser to reduce sound pressure in a mid-range in order to emphasize surrounding sounds and make them easier to hear, and generates a processed difference signal by performing a process on the difference signal using a PEQ (parametric equalizer) to increase sound pressure in a predetermined band, and generates the second sound signal by converting the processed sum signal and the processed difference signal into stereo signals. A method for processing an audio signal is provided. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to perform appropriate processing on each sound signal and output the signal to an appropriate output destination while avoiding the complication of device connections. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a block diagram of a sound signal processing device. [Diagram 2] FIG. 2 is a block diagram showing a configuration relating to signal processing. [Diagram 3] FIG. 2 is a detailed block diagram of a processing unit. [Figure 4] FIG. 2 is a diagram showing a signal flow in a binaural conversion unit. [Diagram 5] FIG. 2 is a diagram illustrating a detailed configuration of a DeEsser. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] FIG. 1 is a block diagram of a sound signal processing device according to an embodiment of the present invention. The sound signal processing device 100 is configured as a mixer device as an example. The sound signal processing device 100 has a CPU 11, which exchanges information with a plurality of components via a bus 10. The sound signal processing device 100 has a ROM 12, a RAM 13, a storage unit 14, a display unit 15, and a setting operation unit 16. The sound signal processing device 100 also has various interfaces for connecting external devices, and signals are input and output between the connected external devices and the CPU 11 via the various interfaces. The various interfaces include a communication unit 17, a first microphone input terminal 18, a second microphone input terminal 19, an AUX terminal 20, a USB terminal 21, an HDMI (registered trademark) input terminal 22, an HDMI output terminal 23, a headphone output terminal 24, and a speaker output terminal 25.

[0011] A timer (not shown) is connected to the CPU 11. The CPU 11 controls the entire device. The ROM 12 stores the control program executed by the CPU 11, various table data, etc. The RAM 13 stores various data. The memory unit 14 stores various application programs including the above-mentioned control program and various data. The setting operation unit 16 accepts input of various information from the user. The display unit 15 displays various information. The communication unit 17 may include a LAN interface or a MIDI (Musical Instrument Digital Interface).

[0012] An external microphone can be connected to the first microphone input terminal 18 and the second microphone input terminal 19. For example, a first microphone 26 is connected to the first microphone input terminal 18, and a second microphone 27 set with a headphone 31 is connected to the second microphone input terminal 19. A communication terminal device 28 such as a smartphone is connected to the AUX terminal 20. A USB device, for example, a PC 29 which is a personal computer, is connected to the USB terminal 21. An HDMI device can be connected to the HDMI input terminal 22 and the HDMI output terminal 23. For example, a game device 30 is connected to the HDMI input terminal 22, and a display monitor 42 is connected to the HDMI output terminal 23. A headphone 31 is connected to the headphone output terminal 24. A speaker 32 is connected to the speaker output terminal 25. The speaker 32 is not limited to a 2-channel speaker, and may be a surround speaker, and the number of sound channels is, for example, 8 channels (7.1ch).

[0013] 2 is a block diagram showing a configuration related to signal processing in the sound signal processing device 100. The sound signal processing device 100 includes processing units A, B, and C, a mix bus 34, and switching units 33 and 35. The functions of the processing units A, B, and C are realized mainly by cooperation of a CPU 11, a ROM 12, and a RAM 13 in addition to necessary hardware. The operations of the mix bus 34 and the switching units 33 and 35 are controlled by the CPU 11.

[0014] The signal Sd generated by the game device 30 and input from the HDMI input terminal 22 is output to the HDMI output terminal 23 and supplied to the display monitor 42. The signal Sd includes a video signal, and a video based on the signal Sd is displayed on the display monitor 42. The signal Sd may also include an audio signal, and when an audio device is connected to the HDMI output terminal 23, audio based on the signal Sd is generated from the audio device.

[0015] A sound signal generated by the game device 30 and input from the HDMI input terminal 22, or a sound signal generated by the PC 29 and input from the USB terminal 21, is input to the processing unit A as a sound signal Sc. The sound signal Sc is a sound signal with three or more channels, and in this embodiment, it is an eight-channel signal. The processing unit A performs processing A (described later) on the sound signal Sc to generate a sound signal Sg, and outputs the sound signal Sg to the mix bus 34.

[0016] A sound signal generated by the communication terminal device 28 and input from the AUX terminal 20, or a sound signal generated by the PC 29 and input from the USB terminal 21, is input to the processing unit B as a sound signal Sb. An example of the sound signal Sb is a sound signal for voice chat. The sound signal Sb is generated by, for example, application software executed by the communication terminal device 28 or the PC 29. The processing unit B performs B processing on the sound signal Sb to generate a sound signal Sf, and outputs the generated sound signal Sf to the mix bus 34.

[0017] The switching unit 33 exclusively selects between a sound signal collected by the first microphone 26 and input from the first microphone input terminal 18 and a sound signal collected by the second microphone 27 and input from the second microphone input terminal 19. For example, a microphone input terminal to which a microphone is connected is selected, and when microphones are connected to both microphone input terminals, the second microphone input terminal 19 is preferentially selected. Then, the switching unit 33 outputs the sound signal from the selected microphone input terminal to the processing unit C as a sound signal Sa.

[0018] The processing unit C applies C processing to the sound signal Sa to generate a sound signal Se, and outputs the sound signal Se to the mix bus 34. The C processing includes, for example, effects and level adjustment. The processing unit C further outputs the generated sound signal Se to the AUX terminal 20 and the USB terminal 21. At this time, processing such as D / A conversion and two-channelization may be performed. Therefore, the sound signal Se is output to the communication terminal device 28 and the PC 29 connected to the AUX terminal 20 and the USB terminal 21, respectively. Depending on the settings of the communication terminal device 28 and the PC 29, sound corresponding to the sound signal Se is generated.

[0019] Note that the A process, the B process, and the C process are different processes. Also, although the sound signal Sa is input from a microphone, it may be a signal input from a system different from the input system of the sound signal Sc. Also, the output destination of the sound signal Se that is output without passing through the mix bus 34 (connected to the AUX terminal 20 or the USB terminal 21) may be different from the output destination of the sound signal Sh (connected to the headphone output terminal 24 or the speaker output terminal 25).

[0020] The mix bus 34 mixes at least the sound signal Sg and the sound signal Se to generate the sound signal Sh. In this embodiment, the sound signal Sg, the sound signal Se, and the sound signal Sf are mixed to generate the sound signal Sh. The mix bus 34 outputs the sound signal Sh to the headphone output terminal 24 or the speaker output terminal 25 via the switching unit 35. Here, the sound signal Sh includes a signal Sh-1 for headphones and a signal Sh-2 for speakers. The switching unit 35 exclusively selects the output destination of the sound signal Sh. For example, when the headphones 31 are connected to the headphone output terminal 24, the headphone output terminal 24 is selected as the output destination, and when the headphones 31 are not connected to the headphone output terminal 24, the speaker output terminal 25 is selected as the output destination. When the headphone output terminal 24 is selected, the signal Sh-1 for headphones is output to the headphones 31. When the speaker output terminal 25 is selected, the signal Sh-2 for speakers is output to the speaker 32. When the switching unit 35 outputs the sound signal Sh, the signal may be subjected to level adjustment and D / A conversion. In particular, when the signal Sh-1 for headphones is output, the output characteristics may be adjusted to match the headphones.

[0021] FIG. 3 is a detailed block diagram of the processing unit A. The processing unit A is composed of a pre-processing unit A-1 and a post-processing unit A-2. The A processing is composed of a binauralization process by the pre-processing unit A-1 and an ambient sound enhancement process by the post-processing unit A-2. The pre-processing unit A-1 includes a binauralization unit 36. The post-processing unit A-2 includes an MS conversion unit 37, a DeEsser 38, a PEQ 39, an LR conversion unit 40, and a PEQ 41. Both the PEQ 39 and 41 are, for example, 4-band parametric equalizers. However, the PEQ 39 and 41 do not have to be 4-band, and do not have to be PEQs, and may be, for example, graphic equalizers (GEQs).

[0022] 4 is a diagram showing the flow of signals in the binaural conversion unit 36. The sound signal Sc is, for example, an audio signal configured with 7.1ch (C: center, L: front left, R: front right, SL: surround left, SR: surround right, BL: surround back left, BR: surround back right, LFE: subwoofer).

[0023] The head-related transfer function (HRTF) is an impulse response that expresses the difference in the loudness, arrival time, and frequency characteristics of a sound from a virtual speaker installed at a certain position to the left and right ears. For signals other than the LFE, an HRTF and reverb according to each signal are applied. Reverb is applied to reproduce the sense of distance from the sound source. However, since it is difficult to sense the position of the LFE, HRTF and reverb are not applied to the LFE. The 8-channel sound signal Sc is stereo-converted through binaural processing by the binaural conversion unit 36, that is, converted into a two-channel stereo signal of L and R. This stereo signal is input as the sound signal Si to the MS conversion unit 37 in the post-processing unit A-2.

[0024] In the post-processing unit A-2 (FIG. 3), the MS conversion unit 37 generates a sum signal Sj and a difference signal Sk from the sound signal Si. If the left signal of the sound signal Si is represented by L and the right signal by R, the sum signal Sj is calculated by Sj=(L of Si) / 2+(R of Si) / 2. The difference signal Sk is calculated by Sk=(L of Si) / 2-(R of Si) / 2.

[0025] 5 is a diagram showing the detailed configuration of the DeEsser 38. The DeEsser 38 includes an HPF (high pass filter), an LPF (low pass filter), a COMP (compressor), and the like, and generates a processed sum signal Sl by performing processing on the sum signal Sj to reduce the sound pressure in the midrange. The PEQ 39 generates a processed difference signal Sm by performing processing on the difference signal Sk to increase the sound pressure in a predetermined band.

[0026] For example, when playing a shooting game, the DeEsser 38 reduces the sound of gunfire caused by the game user's operation, reducing the loudness. Also, the PEQ 39 emphasizes side sounds, making it easier to hear the footsteps of an enemy shooting. Therefore, although the specified band is not limited, it is desirable to have a frequency band that is easy to sense directionality and that mainly includes surrounding sounds in order to make it easier to hear surrounding sounds.

[0027] The LR conversion unit 40 generates a sound signal Sn by converting the processed sum signal Sl and the processed difference signal Sm into a stereo signal. If the left signal of the sound signal Sn is represented by L and the right signal by R, the L signal of the sound signal Sn is calculated by Sl+Sm. The R signal of the sound signal Sn is calculated by Sl-Sm. The PEQ 41 increases or decreases the sound pressure of each band for the sound signal Sn according to the settings. As a result, a sound signal Sg with the overall sound quality adjusted is output from the processing unit A. Note that it is not essential to provide the PEQ 41.

[0028] The processing B that the processing unit B (FIG. 2) applies to the sound signal Sb includes, as an example, binaural processing for voice chat. For example, at least one of HRTF and reverb is applied to the sound signal Sb to generate a sound signal Sf, which is a stereo signal. Note that the CPU 11 may switch between the HRTF and the reverb to be applied in response to a switching instruction from the user. Also, there may be cases where neither the HRTF nor the reverb is applied in response to a switching instruction from the user.

[0029] According to this embodiment, the processing unit A (first processing unit) performs ambient sound emphasis processing (first processing) on ​​the sound signal Si (first sound signal) by the post-processing unit A-2 to generate a sound signal Sg (second sound signal), and outputs the sound signal Sg to the mix bus 34. The processing unit C (second processing unit) performs C processing (second processing) on ​​the sound signal Sa (third sound signal) to generate a sound signal Se (fourth sound signal), and outputs the sound signal Se to the mix bus 34 and the first output destination (the communication terminal device 28 or the PC 29). The mix bus 34 mixes at least the sound signal Sg (second sound signal) and the sound signal Se (fourth sound signal) to generate a sound signal Sh (fifth sound signal), and outputs the sound signal Sh to the second output destination (the headphones 31 or the speaker 32). Therefore, it is possible to perform appropriate processing on each sound signal and output it to an appropriate output destination while avoiding the complication of device connections. For example, even if many sound signals are acquired from multiple systems, there is no need to connect many conversion devices to perform appropriate processing on each sound signal, and complicated connections are avoided.

[0030] In addition, the sound signal Si is converted into a stereo sound signal Sg through the ambient sound enhancement process (first process) to generate a sum signal Sj, a difference signal Sk, and a processed sum signal Sl and a processed difference signal Sm, thereby making it easier to hear ambient sounds.

[0031] Moreover, the sound signal Si is generated by subjecting an 8-channel sound signal Sc (sixth sound signal) to binaural processing (third processing) in the pre-processing unit A-1 to stereo. Therefore, a sense of realism can be reproduced. From this viewpoint, the sound signal Sc may be a sound signal having three or more channels. The pre-processing unit A-1 is not essential as long as it is desired to avoid the complication of device connections and to perform appropriate processing on each sound signal and output the signal to an appropriate output destination. Therefore, the sound signal Si (first sound signal) may be a sound signal input to the sound signal processing device 100 as a stereo signal.

[0032] Moreover, the processing unit B (fourth processing unit) performs B processing (fourth processing) on ​​the sound signal Sb (seventh sound signal) to generate a sound signal Sf (eighth sound signal), and outputs the sound signal Sf to the mix bus 34. The mix bus 34 generates a sound signal Sh (fifth sound signal) by further mixing the sound signal Sg (second sound signal) and the sound signal Se (fourth sound signal) with the sound signal Sf (eighth sound signal). Therefore, sounds such as voice chat sounds can be output together. Note that it is not essential to provide the processing unit B. Therefore, the sound signal Sh (fifth sound signal) may be a signal obtained by mixing the sound signal Sg (second sound signal) and the sound signal Se (fourth sound signal) without including the sound signal Sf.

[0033] In addition, since the sound signal Sa (third sound signal) is a sound signal output from the microphones (26, 27), it is possible to process the voice of the user playing the game, etc. Note that the source of acquisition of the sound signal Sa is not limited to the microphone.

[0034] Furthermore, if the sound signal Sc (sixth sound signal) is a sound signal output from the game device 30, game sounds and the like can be processed together. Furthermore, if the sound signal Sb (seventh sound signal) is, for example, a sound signal output via an application, sounds such as voice chat sounds can be processed together.

[0035] Furthermore, if the output destination of the sound signal Se that is output without passing through the mix bus 34 is a computer such as the communication terminal device 28 or the PC 29, sound that has been subjected to the C processing can be output to the computer. Furthermore, if the output destination of the sound signal Sh is an audio output device such as the headphones 31 or the speakers 32, sounds that have been subjected to the A, B, and C processing can be mixed and output to the audio output device. The headphones 31 are an example of an ear-mounted audio output device, and may be earphones.

[0036] The present invention has been described above in detail based on preferred embodiments thereof, but the present invention is not limited to these specific embodiments, and various forms that do not deviate from the gist of the present invention are also included in the present invention. [Explanation of symbols]

[0037] A, B, C processing section, Si, Sa, Sg, Se, Sh sound signal, 11 CPU, 28 communication terminal device, 29 PC, 31 headphones, 32 speaker, 34 mix bus

Claims

1. a first processing unit performing a first processing on the first sound signal to generate a second sound signal, and outputting the second sound signal to a mix bus; a second processing unit performs a second processing step, which includes an effect and a level adjustment step, different from the first processing step, on the third sound signal output from the microphone to generate a fourth sound signal, and outputs the fourth sound signal to the mix bus and a first output destination; a fourth processing unit performs a fourth processing step different from both the first processing step and the second processing step on the seventh sound signal to generate an eighth sound signal, and outputs the eighth sound signal to the mix bus; the mix bus mixes the second sound signal, the fourth sound signal, and the eighth sound signal to generate a fifth sound signal, and outputs the fifth sound signal to a second output destination; the eighth sound signal is mixed with the second sound signal and the fourth sound signal at the mix bus without being mixed with the second sound signal or the fourth sound signal upstream of the mix bus; the first sound signal is a stereo signal generated by a third processing unit performing a third process to stereoize a sixth sound signal having three or more channels output from the game device, The first processing unit generates a sum signal and a difference signal from the stereo signal, The first processing unit generates a processed sum signal by processing the sum signal using a DeEsser to reduce the sound pressure in the mid-range in order to emphasize surrounding sounds and make them easier to hear, and generates a processed difference signal by processing the difference signal using a PEQ (parametric equalizer) to increase the sound pressure in a specified band, and generates the second sound signal by converting the processed sum signal and the processed difference signal into stereo signals.

2. The sound signal processing method according to claim 1 , wherein an input system for the first sound signal is different from an input system for the third sound signal.

3. The sound signal processing method according to claim 1 , wherein the first sound signal is a signal that has been subjected to binaural processing.

4. The sound signal processing method according to claim 1 , wherein the sixth sound signal is input through an HDMI terminal.

5. The sound signal processing method according to claim 1 , wherein the seventh sound signal is a sound signal output via an application.

6. The sound signal processing method according to claim 5 , wherein the seventh sound signal is input through an AUX terminal or a USB terminal.

7. the first output destination is a computer; The sound signal processing method according to claim 1 , wherein the second output destination is an audio output device.

8. Mix bus and a first processing unit that performs a first processing on a first sound signal to generate a second sound signal and outputs the second sound signal to the mix bus; a second processing unit that performs a second processing step, which includes applying an effect and a level adjustment to a third sound signal output from a microphone, and is different from the first processing step to generate a fourth sound signal, and outputs the fourth sound signal to the mix bus and a first output destination; a fourth processing unit that performs a fourth processing step different from both the first processing step and the second processing step on the seventh sound signal to generate an eighth sound signal, and outputs the eighth sound signal to the mix bus; the mix bus mixes the second sound signal, the fourth sound signal, and the eighth sound signal to generate a fifth sound signal, and outputs the fifth sound signal to a second output destination; the eighth sound signal is mixed with the second sound signal and the fourth sound signal at the mix bus without being mixed with the second sound signal or the fourth sound signal upstream of the mix bus; the first sound signal is a stereo signal generated by a third processing unit performing a third process to stereoize a sixth sound signal having three or more channels output from the game device, The first processing unit generates a sum signal and a difference signal from the stereo signal, The first processing unit generates a processed sum signal by processing the sum signal using a DeEsser to reduce the sound pressure in the mid-range in order to emphasize surrounding sounds and make them easier to hear, and generates a processed difference signal by processing the difference signal using a PEQ (parametric equalizer) to increase the sound pressure in a specified band, and generates the second sound signal by converting the processed sum signal and the processed difference signal into stereo signals.

9. The sound signal processing device according to claim 8 , wherein an input channel of the first sound signal is different from an input channel of the third sound signal.

10. The sound signal processing device according to claim 8 , wherein the first sound signal is a signal that has been subjected to binaural processing.

11. The audio signal processing device according to claim 8 , wherein the sixth audio signal is input through an HDMI terminal.

12. The sound signal processing device according to claim 8 , wherein the seventh sound signal is a sound signal output via an application.

13. The audio signal processing device according to claim 12 , wherein the seventh audio signal is input through an AUX terminal or a USB terminal.

14. the first output destination is a computer; The sound signal processing device according to claim 8 , wherein the second output destination is an audio output device.

Images