Audio signal processing method and audio signal processing apparatus

Abstract

To provide a sound signal processing method and a sound signal processor, which achieve a richer sound image and space spreading.SOLUTION: A sound signal processing method includes the steps of: acquiring a plurality of sound signals collected by a plurality of microphones (13A to 13D, 14A to 14D) arranged in a prescribed space 620; level-adjusting the plurality of sound signals in accordance with arranging positions of the plurality of microphones; mixing the plurality of adjusted sound signals; and generating reflection sound by using a mixed mixing signal.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] One embodiment of the present invention relates to a sound signal processing method and a sound signal processing apparatus for processing an acquired sound signal.

Background Art

[0002] In facilities such as concert halls, various genres of music are played or speeches such as lectures are given. Such facilities are required to have various acoustic characteristics (for example, reverberation characteristics). For example, a relatively long reverberation is required for performances, and a relatively short reverberation is required for speeches.

[0003] However, in order to physically change the reverberation characteristics in the hall, for example, it is necessary to change the size of the space by moving the ceiling or the like, and very large-scale equipment is required.

[0004] Therefore, for example, a sound field control device as shown in Patent Document 1 generates a reverberant sound by processing the sound acquired by a microphone with a FIR (Finite Impulse Response) filter, and outputs the reverberant sound from a speaker installed in the hall, thereby performing processing to assist the sound field.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, simply adding a reverberant sound blurs the sense of localization. Recently, it has been desired to realize a richer sound image and an expansion of space.

[0007] Therefore, an embodiment of the present invention aims to provide a sound signal processing method and a sound signal processing apparatus that perform sound image localization according to the position of a sound source in space to realize a richer sound image and the expansion of space. **Means for Solving the Problems**

[0008] The sound signal processing method acquires a plurality of sound signals respectively picked up by a plurality of microphones arranged in a predetermined space, performs level adjustment on each of the plurality of sound signals according to the arrangement position of each of the plurality of microphones, mixes the adjusted plurality of sound signals, and generates a reflected sound using the mixed mixing signal. **Effects of the Invention**

[0009] The sound signal processing method can realize sound image localization according to the position of a sound source in space. **Brief Description of the Drawings**

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

[0011] [Embodiment 1] FIG. 1 is a perspective view schematically showing a room 62 constituting a space. FIG. 2 is a block diagram showing the configuration of the sound field support system 1.

[0012] The room 62 constitutes a substantially rectangular parallelepiped space. The sound source 61 exists on the stage 60 in the front of the room 62. The rear of the room 62 corresponds to the auditorium where the listener sits. Note that the shape of the room 62 and the arrangement of the sound source are not limited to the example of FIG. 1. The sound signal processing method and the sound signal processing device of the present invention can provide a desired sound field in any shaped space and can realize a richer sound image and a wider spread of the space than before.

[0013] The sound field support system 1 includes a directional microphone 11A, a directional microphone 11B, a directional microphone 11C, an omnidirectional microphone 12A, an omnidirectional microphone 12B, an omnidirectional microphone 12C, a speaker 51A, a speaker 51B, a speaker 51C, a speaker 51D, a speaker 61A, a speaker 61B, a speaker 61C, a speaker 61D, a speaker 61E, and a speaker 61F in the room 62.

[0014] The speakers 61A, 61B, 61C, 61D, 61E, and 61F correspond to the first speakers that output reverberation sound control signals. The speakers 51A, 51B, 51C, and 51D correspond to the second speakers that output early reflection sound control signals.

[0015] The number of the directional microphones and omnidirectional microphones shown in FIG. 1 is three each. However, the sound field support system 1 only needs to include at least one microphone. Also, the number of speakers is not limited to the number shown in FIG. 1. The sound field support system 1 only needs to include at least one speaker.

[0016] The directional microphones 11A, 11B, and 11C mainly pick up the sound of the sound source 61 on the stage.

[0017] The omnidirectional microphones 12A, 12B, and 12C are installed on the ceiling. The omnidirectional microphones 12A, 12B, and 12C pick up the overall sound in the room 62, including the direct sound of the sound source 61 and the reflected sound in the room 62.

[0018] The speakers 51A, 51B, 51C, and 51D are installed on the wall surface of the room 62. The speakers 61A, 61B, 61C, 61D, 61E, and 61F are installed on the ceiling of the room 62. However, in the present invention, the installation positions of the microphones and speakers are not limited to this example.

[0019] In FIG. 2, in addition to the configuration shown in FIG. 1, the sound field support system 1 includes a sound signal processing unit 10 and a memory 31. The sound signal processing unit 10 is mainly composed of a CPU and a DSP (Digital Signal Processor). Functionally, the sound signal processing unit 10 includes a sound signal acquisition unit 21, a gain adjustment unit 22, a mixer 23, a FIR (Finite Impulse Response) filter 24A, a FIR filter 24B, a level setting unit 25A, a level setting unit 25B, a matrix mixer 26, a delay adjustment unit 28, an output unit 27, an impulse response acquisition unit 151, and a level balance adjustment unit 152. The sound signal processing unit 10 is an example of the sound signal processing device of the present invention.

[0020] The CPU that constitutes the sound signal processing unit 10 reads out the operation program stored in the memory 31 and controls each component. The CPU functionally constitutes the impulse response acquisition unit 151 and the level balance adjustment unit 152 according to the operation program. Note that the operation program does not necessarily need to be stored in the memory 31. The CPU may download the operation program from a server (not shown) each time, for example.

[0021] FIG. 3 is a flowchart showing the operation of the sound signal processing unit 10. First, the sound signal acquisition unit 21 acquires a sound signal (S11). The sound signal acquisition unit 21 acquires a sound signal from the directional microphones 11A, 11B, 11C, the omnidirectional microphones 12A, 12B, and 12C. When the sound signal acquisition unit 21 acquires an analog signal, it converts it into a digital signal and outputs it.

[0022] The gain adjustment unit 22 adjusts the gain of the sound signals acquired from the directional microphones 11A, 11B, 11C, the omnidirectional microphones 12A, 12B, and 12C through the sound signal acquisition unit 21. For example, the gain adjustment unit 22 sets a higher gain for the directional microphones located closer to the sound source 61. Note that the gain adjustment unit 22 is not an essential component in Embodiment 1.

[0023] The mixer 23 mixes the sound signals acquired from the directional microphones 11A, 11B, and 11C. Further, the mixer 23 distributes the mixed sound signal to a plurality of signal processing systems. The mixer 23 outputs the distributed sound signal to the FIR filter 24A. Also, the mixer 23 mixes the sound signals acquired from the omnidirectional microphones 12A, 12B, and 12C. The mixer 23 outputs the mixed sound signal to the FIR filter 24B.

[0024] In the example of FIG. 2, the mixer 23 mixes the sound signals acquired from the directional microphones 11A, 11B, and 11C according to the speakers 51A, 51B, 51C, and 51D and mixes them into four signal processing systems. Also, the mixer 23 mixes the sound signals acquired from the omnidirectional microphones 12A, 12B, and 12C into four signal processing systems. This one signal processing system corresponds to the speakers 61A to 61F. Hereinafter, the four signal processing systems corresponding to the speakers 61A to 61F are referred to as the first system. The four signal processing systems corresponding to the speakers 51A, 51B, 51C, and 51D are referred to as the second system.

[0025] Note that the number of signal processing systems is not limited to this example. The sound signals acquired from the omnidirectional microphones 12A, 12B, and 12C may be distributed to six first systems according to the speakers 61A, 61B, 61C, 61D, 61E, and 61F. Note that the mixer 23 is not an essential component in the first embodiment.

[0026] Note that the mixer 23 may have the function of an EMR (Electronic Microphone Rotator). EMR is a method of flattening the frequency characteristics of a feedback loop by temporally changing the transfer function between a fixed microphone and a speaker. EMR is a function of switching the connection relationship between the microphone and the signal processing system at every moment. The mixer 23 outputs to the FIR filter 24A by switching the output destinations of the sound signals acquired from the directional microphones 11A, 11B, and 11C. Alternatively, the mixer 23 outputs to the FIR filter 24B by switching the output destinations of the sound signals acquired from the omnidirectional microphones 12A, 12B, and 12C. Thereby, the mixer 23 can flatten the frequency characteristics of the acoustic feedback system from the speaker to the microphone in the room 62.

[0027] Next, the impulse response acquisition unit 151 sets the filter coefficients of the FIR filter 24A and the FIR filter 24B, respectively (S12).

[0028] Here, the impulse response data to be set as the filter coefficients will be described. FIG. 4(A) is a schematic diagram showing a classification example of the types of sounds in the time waveform of the impulse response used for the filter coefficients, and FIG. 4(B) is a schematic diagram showing the time waveform of the filter coefficients set for the FIR filter 24A. FIGS. 5(A) and 5(B) are schematic diagrams showing the time waveforms of the filter coefficients set for the FIR filter 24B.

[0029] As shown in FIG. 4(A), the impulse response can be distinguished into direct sound, early reflection sound, and reverberation sound arranged on the time axis. Then, as shown in FIG. 4(B), the filter coefficients set for the FIR filter 24A are set by the portion of the early reflection sound excluding the direct sound and the reverberation sound in the impulse response. The filter coefficients set for the FIR filter 24B are set by the reverberation sound excluding the direct sound and the early reflection sound in the impulse response, as shown in FIG. 5(A). Note that the FIR filter 24B may be set by the early reflection sound and the reverberation sound excluding the direct sound in the impulse response, as shown in FIG. 5(B).

[0030] The data of the impulse response is stored in the memory 31. The impulse response acquisition unit 151 acquires the data of the impulse response from the memory 31. However, the data of the impulse response does not necessarily have to be stored in the memory 31. The impulse response acquisition unit 151 may download the data of the impulse response from, for example, a server (not shown) each time.

[0031] The impulse response acquisition unit 151 may acquire the data of the impulse response from which only the early reflection sound has been cut out in advance and set it for the FIR filter 24A. Alternatively, the impulse response acquisition unit 151 may acquire the data of the impulse response including the direct sound, the early reflection sound, and the reverberation sound, cut out only the early reflection sound, and set it for the FIR filter 24A. Similarly, when only the reverberation sound is used, the impulse response acquisition unit 151 may acquire the data of the impulse response from which only the reverberation sound has been cut out in advance and set it for the FIR filter 24B. Alternatively, the impulse response acquisition unit 151 may acquire the data of the impulse response including the direct sound, the early reflection sound, and the reverberation sound, cut out only the reverberation sound, and set it for the FIR filter 24B.

[0032] FIG. 6 is a plan view schematically showing the relationship between the space 620 and the chamber 62. As shown in FIG. 6, the impulse response data is measured in advance in a predetermined space 620 such as a concert hall or a church that is the object to reproduce the sound field. For example, the impulse response data is measured by emitting a test sound (pulse sound) at the position of the sound source 61 and picking up the sound with a microphone.

[0033] The impulse response data may be acquired at any position in the space 620. However, it is preferable to measure the impulse response data of the early reflection sound using a directional microphone installed near the wall surface. The early reflection sound is a reflection sound with a clear arrival direction. Therefore, by measuring the impulse response data with a directional microphone installed near the wall surface, the reflection sound data of the target space can be obtained in detail. On the other hand, the reverberation sound is a reflection sound with an undetermined arrival direction of the sound. Therefore, the impulse response data of the reverberation sound may be measured with the directional microphone installed near the above wall surface, or may be measured using an omnidirectional microphone different from the early reflection sound.

[0034] The FIR filter 24A convolves the impulse response data different from each other with the four sound signals of the second system which is the signal flow at the upper part of FIG. 2. When there are a plurality of signal processing systems, the FIR filter 24A and the FIR filter 24B may be provided for each signal processing system. For example, four FIR filters 24A may be provided.

[0035] As described above, when using a directional microphone installed near the wall surface, the impulse response data is measured with a separate directional microphone for each signal processing system. For example, as shown in FIG. 6, for the signal processing system corresponding to the speaker 51D installed at the right rear toward the stage 60, the impulse response data is measured with the directional microphone 510D installed near the wall surface at the right rear toward the stage 60.

[0036] The FIR filter 24A convolves the impulse response data with each sound signal of the second system (S13). The FIR filter 24B convolves the impulse response data with each sound signal of the first system, which is the signal flow at the bottom of FIG. 2 (S13).

[0037] The FIR filter 24A generates an early reflection sound control signal that reproduces the early reflection sound of a predetermined space by convolving the input sound signal with the data of the impulse response of the set early reflection sound. The FIR filter 24B generates a reverberation sound control signal that reproduces the reverberation sound of a predetermined space by convolving the input sound signal with the data of the impulse response of the set reverberation sound.

[0038] The level setting unit 25A adjusts the level of the early reflection sound control signal (S14). The level setting unit 25B adjusts the level of the reverberation sound control signal (S14).

[0039] The level balance adjustment unit 152 sets the level adjustment amounts of the level setting unit 25A and the level setting unit 25B.

[0040] The level balance adjustment unit 152 refers to the levels of the early reflection sound control signal and the reverberation sound control signal respectively, and adjusts the level balance between the two. For example, the level balance adjustment unit 152 adjusts the balance between the level of the last component in time of the early reflection sound control signal and the level of the first component in time of the reverberation sound control signal. Alternatively, the level balance adjustment unit 152 may adjust the balance between the power of a plurality of components in the latter half in time of the early reflection sound control signal and the power of the component in the first half in time of the reverberation sound control signal. Thereby, the level balance adjustment unit 152 can control the sounds of the early reflection sound control signal and the reverberation sound control signal individually, and control them to an appropriate balance according to the space to which they are applied.

[0041] Next, the matrix mixer 26 distributes the input audio signal to the output systems for each speaker. The matrix mixer 26 distributes the reverberation sound control signal of the first system to each output system of speakers 61A to 61F and outputs it to the delay adjustment unit 28. Since the second system already corresponds to the output system, the matrix mixer 26 outputs the early reflection sound control signal of the second system as it is to the delay adjustment unit 28.

[0042] Note that the matrix mixer 26 may perform gain adjustment of each output system, adjustment of frequency characteristics, etc.

[0043] The delay adjustment unit 28 adjusts the delay time according to the distance between the sound source 61 and the plurality of speakers (S15). For example, the delay adjustment unit 28 sets the delay time to be shorter in the order of the distance between the sound source 61 and the speakers being shorter among the plurality of speakers. Thereby, the delay adjustment unit 28 can adjust the phases of the reverberation sound control signal and the early reflection sound control signal output from the plurality of speakers according to the positions of the plurality of speakers from the sound source 61.

[0044] The output unit 27 converts the early reflection sound control signal and the reverberation sound control signal output from the delay adjustment unit 28 into analog signals. Also, the output unit 27 amplifies the analog signal. The output unit 27 outputs the amplified analog signal to the corresponding speaker (S16).

[0045] With the above configuration, the audio signal processing unit 10 acquires an audio signal, acquires an impulse response, convolves the impulse response of the early reflection sound in the impulse response with the audio signal, and outputs the audio signal convolved with the impulse response of the early reflection sound as an early reflection sound control signal that has been subjected to processing different from the reverberation sound control signal. Thereby, the audio signal processing unit 10 realizes a richer sound image and spatial expansion than before.

[0046] In Embodiment 1, for example, the following configurations are also possible, and the following effects can be achieved in each configuration.

[0047] (1-1)One embodiment of the present invention is a signal processing method that acquires an audio signal, acquires an impulse response, and convolves the impulse response of early reflected sound among the impulse responses with the audio signal to generate an early reflected sound control signal.

[0048] FIG. 7 is a block diagram showing the configuration of an audio signal processing unit 10A corresponding to the above signal processing method. The audio signal processing unit 10A includes an audio signal acquisition unit 21A that acquires an audio signal from a directional microphone 11A, an impulse response acquisition unit 151A that acquires an impulse response, and a processing unit 204A that convolves the impulse response of early reflected sound among the impulse responses with the audio signal and outputs the audio signal convolved with the impulse response of early reflected sound as an early reflected sound control signal that has been subjected to a process different from the reverberation sound control signal to a speaker 51A.

[0049] The audio signal acquisition unit 21A has the same function as the audio signal acquisition unit 21 shown in FIG. 2. The impulse response acquisition unit 151A has the same function as the impulse response acquisition unit 151 in FIG. 2. The processing unit 204A has the functions of the FIR filter 24A and the output unit 27 in FIG. 2.

[0050] Similar to the audio signal processing unit 10 in FIG. 2, the audio signal processing unit 10A realizes a richer sound image and spatial spread than before.

[0051] (1-2) The processing unit may convolve the impulse response of reverberation sound among the impulse responses with the audio signal to generate a reverberation control signal that does not include direct sound, perform different signal processes on the early reflected sound control signal and the reverberation control signal respectively, output the reverberation control signal to a first speaker (the first system speaker described above), and output the early reflected sound control signal to a second speaker (the second system speaker described above).

[0052] However, the actual room is equipped with a larger number of speakers than the example shown in FIG. 1. Among the second speakers (the second type of speakers described above) that output the early reflection sound control signal, the speakers installed near the first speaker (the first type of speakers described above) may output the reverberation sound control signal. That is, among the plurality of second-type speakers, the speakers installed near the first-type speakers may output the reverberation sound control signal in addition to the early reflection sound control signal.

[0053] Conversely, among the first speakers (the first type of speakers described above), the speakers installed near the wall surface may output the early reflection sound control signal. That is, among the plurality of first-type speakers, the speakers installed near the second-type speakers may output the early reflection sound control signal in addition to the reverberation sound control signal.

[0054] Thereby, the sounds of the early reflection sound control signal and the reverberation sound control signal can be adjusted with an appropriate energy balance.

[0055] (1-3) The first speaker may be omni-directional, and the second speaker may be narrow-directional.

[0056] As described above, the early reflection sound is a reflection sound with a clear arrival direction and contributes to the subjective impression. Therefore, it is effective to use a narrow directionality for the second speaker, and the controllability of the early reflection sound in the target space can be enhanced.

[0057] On the other hand, the reverberation sound is a reflection sound with an undetermined arrival direction of the sound and contributes to the resonance of the space. Therefore, it is effective to use an omni-directionality for the first speaker, and the controllability of the reverberation sound in the target space can be enhanced.

[0058] (1-4) It is preferable that the level per unit of the second speaker is higher than that of the first speaker.

[0059] Similar to the above, the initial reflected sound has fewer reflection times compared to the reverberant sound that has undergone multiple reflections within the space. For this reason, the energy of the initial reflected sound is higher than that of the reverberant sound. Therefore, by increasing the level per unit of the second speaker, the subjective impression effect of the initial reflected sound can be improved, and the controllability of the initial reflected sound can be enhanced. (1-5) It is preferable that the number of second speakers is smaller than that of the first speakers.

[0060] Similar to the above, the second speaker can suppress the increase in excessive diffusion sound energy by reducing the number. That is, it is possible to suppress the initial reflected sound output from the second speaker from diffusing and reverberating in the room, and to suppress the reverberant sound of the initial reflected sound from reaching the listener.

[0061] (1-6) It is preferable to install the first speaker on the ceiling of the room and the second speaker on the side of the room.

[0062] By installing the second speaker on the side of the room, which is a position close to the listener, it is easy to control the delivery of the initial reflected sound to the listener, and the controllability of the initial reflected sound can be enhanced. Also, by installing the first speaker on the ceiling of the room, the difference in reverberant sound depending on the position of the listener can be suppressed.

[0063] (1-7) The processing unit preferably adjusts the level balance between the initial reflected sound control signal and the reverberant sound control signal.

[0064] By individually adjusting the level balance, the processing unit can adjust the sounds of the initial reflected sound control signal and the reverberant sound control signal with an appropriate energy balance.

[0065] (1-8) The sound signal acquisition unit preferably acquires separately a first sound signal for generating the reverberant sound control signal and a second sound signal for convolving the impulse response of the early reflected sound. The first sound signal is a sound signal corresponding to the first system described above (the sound signals acquired from the omnidirectional microphones 12A, 12B, and 12C), and the second sound signal is a sound signal corresponding to the second system described above (the sound signals acquired from the directional microphones 11A, 11B, and 11C).

[0066] Reverberant sound is easily affected by the acoustics of the room. Early reflected sound is easily affected by the sound of the sound source. Therefore, it is preferable that the first sound signal picks up the overall sound in the room, for example, and the second sound signal picks up the sound of the sound source with a high signal-to-noise ratio.

[0067] (1-9) It is preferable that the first sound signal is picked up by an omnidirectional microphone and the second sound signal is picked up by a directional microphone.

[0068] Similar to the above, it is preferable that the first sound signal picks up the overall sound in the room using, for example, an omnidirectional microphone. It is preferable that the second sound signal picks up the sound of the sound source with a high signal-to-noise ratio using, for example, a directional microphone.

[0069] (1-10) It is preferable that the directional microphone is closer to the sound source than the omnidirectional microphone.

[0070] Similar to the above, since it is preferable that the second sound signal picks up the sound of the sound source with a high signal-to-noise ratio, it is preferable that the directional microphone is close to the sound source.

[0071] (1-11) It is preferable that the impulse response is acquired using a directional microphone at the edge of a predetermined space.

[0072] By measuring the impulse response with a directional microphone installed near the wall surface, the reflected sound of the target space can be acquired with higher accuracy.

[0073] [Embodiment 2] The sound field support system 1A according to Embodiment 2 will be described with reference to FIGS. 8, 9, 10, and 11. FIG. 8 is a perspective view schematically showing the space 620. FIG. 9 is a plan view of the space 620 in a plan view. FIG. 10 is a block diagram showing the configuration of the sound field support system 1A.

[0074] FIG. 11 is a flowchart showing the operation of the sound signal processing device. In this example, it is assumed that the sound source 61 moves on the stage 60, or a plurality of sound sources 61 exist on the stage 60. For the same configuration as in the above-described Embodiment 1, the same reference numerals are given and the description is omitted.

[0075] As shown in FIGS. 8 and 9, the sound field support system 1A includes speakers 52A, 52B, 52C, 52D, 52E, 53A, 53B, 53C, 53D, and 53E.

[0076] In this example, as shown in FIGS. 8 and 9, the speakers 52A, 52B, 52C, 52D, and 52E belong to the second - 1 speaker group 520 (on the left side of the center toward the stage 60) that outputs the initial reflected sound control signal of the second - 1 system. Also, in this example, the speakers 53A, 53B, 53C, 53D, and 53E belong to the second - 2 speaker group 530 (on the right side of the center toward the stage 60) that outputs the initial reflected sound control signal of the second - 2 system. The dashed - dotted line shown in FIG. 9 indicates the second - 1 speaker group 520, and the two - dashed - dotted line indicates the second - 2 speaker group 530.

[0077] In the following description, the speakers 52A, 52B, 52C, 52D, and 52E of the second - 1 speaker group 520 are collectively referred to as the speakers of the second - 1 speaker group 520. Also, in the following description, the speakers 53A, 53B, 53C, 53D, and 53E of the second - 2 speaker group 530 are collectively referred to as the speakers of the second - 2 speaker group 530.

[0078] As shown in FIGS. 8 and 9, the sound field support system 1A includes a directional microphone 13A, a directional microphone 13B, a directional microphone 13C, a directional microphone 13D, a directional microphone 14A, a directional microphone 14B, a directional microphone 14C, and a directional microphone 14D in the room 62.

[0079] In this example, the directional microphones 13A, 13B, 13C, and 13D are arranged side by side in the X1 direction (left - right direction) shown in FIGS. 8 and 9 and installed on the ceiling. Also, in this example, the directional microphones 14A, 14B, 14C, and 14D are arranged side by side in the X1 direction (left - right direction) shown in FIGS. 8 and 9 and placed on the ceiling. Further, the directional microphones 14A, 14B, 14C, and 14D are arranged behind (on the audience side when looking at the stage 60 from the side) in the Y1 direction (front - back direction) with respect to the directional microphones 13A, 13B, 13C, and 13D.

[0080] The directional microphones 13A, 13C, 14A, and 14C correspond to the speakers of the second - 1 speaker group 520 as shown in FIG. 9. That is, a second - 1 system initial reflected sound control signal is generated based on the sound signals picked up by the directional microphones 13A, 13C, 14A, and 14C. Also, the directional microphones 13B, 13D, 14B, and 14D correspond to the speakers of the second - 2 speaker group 530. That is, a second - 2 system initial reflected sound control signal is generated based on the sound signals picked up by the directional microphones 13B, 13D, 14B, and 14D.

[0081] In the following description, the directional microphones 13A, 13C, 14A, and 14C are collectively referred to as the directional microphones corresponding to the second - 1 speaker group 520. Also, in the following description, the directional microphones 13B, 13D, 14B, and 14D are collectively referred to as the directional microphones corresponding to the second - 2 speaker group 530.

[0082] As shown in FIG. 10, the sound signal processing unit 10B of the sound field assistance system 1A has a configuration obtained by removing the FIR filter 24B and the level setting unit 25B from the sound field assistance system 1 of Embodiment 1. However, even in Embodiment 2, the FIR filter 24B and the level setting unit 25B may be provided to generate a reverberation sound control signal. In that case, the reverberation sound control signal may be output to any one of the speakers 52A to 52E and the speakers 53A to 53E, or may be output from another speaker.

[0083] The sound signal acquisition unit 21 acquires sound signals from the directional microphones corresponding to the second - 1 speaker group 520 and the directional microphones corresponding to the second - 2 speaker group 530 (see FIG. 10).

[0084] The gain adjustment unit 22 adjusts the gains of the sound signals acquired from each of the directional microphones corresponding to the second - 1 speaker group 520 and the directional microphones corresponding to the second - 2 speaker group 530 (see FIG. 11, S101).

[0085] In this example, the gain adjustment unit 22 sets different gains for each of the directional microphones corresponding to the second - 1 speaker group 520 and each of the directional microphones corresponding to the second - 2 speaker group 530.

[0086] The gain adjustment unit 22 sets the gain of the sound signal to be higher in the order of the distances to the speakers of the second - 1 speaker group 520 (for example, speaker 52A) being closer in the left - right direction among the directional microphones corresponding to the second - 1 speaker group 520.

[0087] Further, the gain adjustment unit 22 sets the gain of the sound signal of the directional microphone corresponding to the second-1 speaker group 520 on the front side (right side of the paper surface in FIG. 9) when viewing the stage 60 from the side in the front-rear direction (left-right direction of the paper surface in FIG. 9) to be lower than the gain of the sound signal of the directional microphone on the side closer to the auditorium (left side of the paper surface in FIG. 9).

[0088] Similarly to the above, the gain adjustment unit 22 sets the gain of the sound signal to be higher in the order of the distances to the speakers of the second-2 speaker group 530 (for example, speaker 53A) in the left-right direction among the directional microphones corresponding to the second-2 speaker group 530.

[0089] Further, the gain adjustment unit 22 sets the gain of the sound signal of the directional microphone corresponding to the second-2 speaker group 530 on the front side (right side of the paper surface in FIG. 9) when viewing the stage 60 from the side in the front-rear direction (left-right direction of the paper surface in FIG. 9) to be lower than the gain of the sound signal of the directional microphone on the side closer to the auditorium (left side of the paper surface in FIG. 9).

[0090] For example, the gain adjustment unit 22 sets the gain of the directional microphone 14A to 0 dB, the gain of the directional microphone 13A to -1.5 dB, the gain of the directional microphone 14C to -3.0 dB, and the gain of the directional microphone 13C to -4.5 dB.

[0091] For example, the gain adjustment unit 22 sets the gain of the directional microphone 14D to 0 dB, the gain of the directional microphone 13D to -1.5 dB, the gain of the directional microphone 14B to -3.0 dB, and the gain of the directional microphone 13B to -4.5 dB.

[0092] The mixer 23 mixes the sound signals acquired from each of the directional microphones corresponding to the second first speaker group 520 (see FIG. 11, S102). The mixer 23 distributes the mixed sound signals to a plurality (five in FIGS. 8 and 9) of signal processing systems according to the number of speakers in the second first speaker group 520 (for example, five). Further, the mixer 23 mixes the sound signals acquired from each of the directional microphones corresponding to the second second speaker group 530. The mixer 23 distributes the mixed sound signals to a plurality (five in FIGS. 8 and 9) of signal processing systems according to the number of speakers in the second second speaker group 530 (for example, five).

[0093] In the real space, the sound image localization changes depending on the arrival direction, level, and density of the direct sound and early reflection sounds. That is, the sound image localization of the sound source 61 in the auditorium depends on the position of the sound source 61 on the stage 60. For example, when the sound source 61 moves to the left toward the stage 60, the levels of the direct sound and early reflection sounds arriving from the left direction become relatively high in the auditorium, so the sound image is localized to the left toward the stage 60. The gain adjustment unit 22 controls the level of the early reflection sound according to the position of the sound source 61 on the stage 60 by setting the gain of the sound signal to be high in the order of the distances from the plurality of directional microphones to the speaker being close, and realizes a sound image localization close to the phenomenon in the real space.

[0094] The delay adjustment unit 28 adjusts the delay time according to the distances between the plurality of directional microphones and the speaker. For example, the delay adjustment unit 28 sets the delay time to be small in the order of the distances between the directional microphones and the speaker being short among the plurality of directional microphones. Thereby, the time difference of the early reflection sounds output from the plurality of speakers is reproduced according to the distance between the sound source 61 and the speaker.

[0095] Further, the sound field support system 1A acquires the sound of the sound source 61 in a wide range on the stage 60 by arranging a plurality of directional microphones side by side in the left-right direction. Thereby, the sound field support system 1A can reflect the level of the early reflection sound corresponding to the position of the sound source 61 in a state close to the real space without detecting the position of the sound source 61.

[0096] In the real space, when the distance between the sound source 61 and the audience seats increases, the level of the early reflected sound also decreases. The gain adjustment unit 22 sets the gain of the sound signal of the speaker farther from the audience seats to a lower value in the front-rear direction, thereby realizing the resonance of the sound in the real space.

[0097] Also, in the real space, when the distance between the sound source 61 and the audience seats increases, the time until the sound directly reaches the audience seats from the sound source 61 becomes longer. Therefore, by the delay adjustment unit 28 setting the delay time of the early reflected sound signal output to the speaker farther from the audience seats to a larger value, the sound field support system 1A more accurately realizes the resonance of the sound in the real space.

[0098] As described above, the sound field support system 1A of the second embodiment can generate an early reflected sound control signal corresponding to the position of the sound source 61 without separately acquiring the position information of the sound source 61 by setting the gain of the directional microphone according to the positional relationship between the sound source and the speaker even when the sound source 61 moves on the stage 60 or when there are a plurality of sound sources 61. Therefore, the sound field support system 1 can effectively realize sound image localization and realize a richer sound image and spatial expansion than before.

[0099] Note that the value of the gain of the sound signal of the directional microphone is not limited to this example. Also, although an example in which the gain of the sound signal of the speaker farther from the audience seats is set lower than the gain of the sound signal of the speaker closer to the audience seats has been described, the present invention is not limited to this example.

[0100] Also, in the sound field support system 1A of the second embodiment, eight directional microphones have been used for the description, but the present invention is not limited to this. The number of directional microphones may be less than eight or nine or more. Also, the positions of the directional microphones are not limited to this example.

[0101] In the sound field support system 1A of Embodiment 2, although the speakers of the five second-1 speaker groups 520 and the speakers of the five second-2 speaker groups 530 have been described, it is not limited thereto. The number of speaker groups may be three or more, and the number of speakers belonging to each speaker group may be one or more. Also, the positions of the speakers are not limited to this example.

[0102] In the sound field support system 1A of Embodiment 2, for example, one directional microphone may be associated with both the second-1 speaker group 520 and the second-2 speaker group 530. In this case, the gain of the sound signal corresponding to the second-1 speaker group 520 (second-1 system) and the gain of the sound signal corresponding to the second-2 speaker group 530 (second-2 system) may be different.

[0103] In Embodiment 2, for example, the following configurations are also possible, and the following effects can be achieved in each configuration.

[0104] (2-1) The sound signal processing method acquires a plurality of sound signals respectively picked up by a plurality of microphones arranged in a predetermined space, performs level adjustment on each of the plurality of sound signals according to the arrangement positions of the plurality of microphones, mixes the adjusted plurality of sound signals, and uses the mixed mixing signal to generate a reflected sound.

[0105] FIG. 12 is a block diagram showing the configuration of a sound signal processing unit 10C corresponding to the signal processing method of Embodiment 2. The sound signal processing unit 10C includes a sound signal acquisition unit 21B that acquires a plurality of sound signals respectively picked up by a plurality of directional microphones 13A, 13B, 14A, 14B arranged in a predetermined space, a gain adjustment unit 22B that performs level adjustment on each of the plurality of sound signals according to the arrangement positions of the plurality of directional microphones 13A, 13B, 14A, 14B, a mixer 23B that mixes the adjusted plurality of sound signals, and a reflected sound generation unit 205B that generates a reflected sound for each system using the mixed mixing signal and outputs it to the speaker 52A and the speaker 53A.

[0106] The sound signal acquisition unit 21B has the same function as the sound signal acquisition unit 21 shown in FIG. 10. The gain adjustment unit 22B has the same function as the gain adjustment unit 22 in FIG. 10. The mixer 23B has the same function as the mixer 23 in FIG. 10. The reflected sound generation unit 205B has the same function as the FIR filter 24A and the level setting unit 25A in FIG. 10.

[0107] Similar to the sound signal processing unit 10B in FIG. 10, the sound signal processing unit 10C realizes more effective sound image localization by changing the level of the signal picked up from the sound signal acquisition unit 21B according to the position of the sound source without the need to detect the position of the sound source.

[0108] (2-2) The levels of the plurality of sound signals may be adjusted according to the distances from the respective arrangement positions of the plurality of microphones to the speaker that outputs the reflected sound.

[0109] In the real space, the sound image localization changes depending on the arrival direction, level, and density of the direct sound and the early reflected sound. Therefore, with this configuration, the sound of the real space is reproduced more realistically.

[0110] (2-3) In the level adjustment, the gains for the respective plurality of sound signals may be set high in the order of increasing distance from the respective arrangement positions of the plurality of microphones to the arrangement position of the speaker that outputs the reflected sound.

[0111] With this configuration, by setting the gain of the sound signal high in the order of increasing distance from the directional microphone to the speaker, the attenuation of the reflected sound depending on the distance between the sound source and the wall is reproduced, and furthermore, the sound of the real space is realized.

[0112] (2-4) Delay adjustment may be performed according to the distances from the respective arrangement positions of the plurality of microphones to the speaker that outputs the reflected sound. With this configuration, sound image localization close to the phenomenon in the real space is realized.

[0113] (2-5) As the distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the front pseudo-reflected sound increases, the delay time may be set to be larger.

[0114] In this configuration, the delay of the reflected sound depending on the distance between the sound source and the wall is reproduced.

[0115] (2-6) The sound signal generation device includes a speaker that outputs a reflected sound. The speaker that outputs the reflected sound includes a first 2-1 speaker group of the first 2-1 system and a second 2-2 speaker group of the second 2-2 system. The level adjustment unit adjusts the level for each sound signal for each of the first 2-1 system and the first 2-1 system, and the mixing unit may perform mixing for each of the first 2-1 system and the second 2-2 system.

[0116] With this configuration, sound image localization can be realized more effectively.

[0117] (2-7) The sound signal generation device includes a plurality of microphones arranged in a predetermined space. The plurality of microphones are preferably distinguished into a plurality of first 2-1 microphones corresponding to the first 2-1 speaker group and a plurality of second 2-2 microphones corresponding to the second 2-2 speaker group.

[0118] With this configuration, even when the position of the sound source moves or there are a plurality of sound sources, sound image localization can be realized more effectively.

[0119] (2-8) The reflected sound may include an early reflected sound.

[0120] [Embodiment 3] The sound field support system 1B of Embodiment 3 will be described with reference to FIGS. 13, 14, and 15. FIG. 13 is a perspective view showing the room 62B of Embodiment 3 schematically. FIG. 14 is a block diagram showing the configuration of the sound field support system 1B. FIG. 15 is a flowchart showing the operation of the sound signal processing device of Embodiment 3. Embodiment 3 assumes that the output sounds from the sound sources 611B, 612B, and 613B are line-input. Note that the same components as those in Embodiment 1 described above are denoted by the same reference numerals, and the description thereof is omitted. The line input is not to input the sound output from a sound source such as various musical instruments described later by picking up the sound with a microphone, but to input a sound signal from an audio cable connected to the sound source. On the other hand, the line output means that an audio cable is connected to a sound source such as various musical instruments described later, and the sound source outputs a sound signal using this audio cable.

[0121] The room 62B does not require the directional microphones 11A, 11B, and 11C with respect to the room 62 shown in Embodiment 1. Note that the directional microphones 11A, 11B, and 11C may be arranged.

[0122] The sound sources 611B, 612B, and 613B are, for example, an electronic piano, an electric guitar, etc., and each outputs a sound signal by line. That is, the sound sources 611B, 612B, and 613B are connected to an audio cable, and output a sound signal via the audio cable. Note that in FIG. 13, the number of sound sources is three, but it may be one, or may be two or a plurality of four or more.

[0123] The sound signal processing unit 10D of the sound field support system 1B is different from the sound signal processing unit 10 shown in Embodiment 1 in that it further includes a line input unit 21D, a sound signal acquisition unit 210, a level setting unit 211, a level setting unit 212, a synthesis unit 213, and a mixer 230. Other configurations of the sound signal processing unit 10D are the same as those of the sound signal processing unit 10, and the description of the same parts is omitted.

[0124] The line input unit 21D inputs audio signals from the sound sources 611B, 612B, and 613B (see FIG. 15, S201). That is, the line input unit 21D is connected to the audio cables connected to the sound sources 611B, 612B, and 613B. Then, the line input unit 21D inputs the audio signals from the sound sources 611B, 612B, and 613B via this audio cable. Hereinafter, this audio signal is referred to as a line input signal. The line input unit 21D outputs the line input signals of the respective sound sources to the gain adjustment unit 22.

[0125] The gain adjustment unit 22 corresponds to a volume control unit and performs volume control of the line input signal (see FIG. 15, S202). Specifically, the gain adjustment unit 22 performs volume control on each of the line input signal of the sound source 611B, the line input signal of the sound source 612B, and the line input signal of the sound source 613B using individual gains. The gain adjustment unit 22 outputs the line input signal after volume control to the mixer 23.

[0126] The mixer 23 mixes the line input signal of the sound source 611B after volume control, the line input signal of the sound source 612B after volume control, and the line input signal of the sound source 613B after volume control.

[0127] The mixer 23 distributes the mixed audio signal to a plurality of signal processing systems. Specifically, the mixer 23 distributes the mixed audio signal to a plurality of signal processing systems for early reflection sound and a signal processing system for reverberation sound. Hereinafter, the audio signal distributed to the plurality of signal processing systems for early reflection sound is referred to as a mixing signal for early reflection sound, and the audio signal distributed to the signal processing system for reverberation sound is referred to as a mixing signal for reverberation sound.

[0128] The mixer 23 outputs the mixing signal for early reflection sound to the level setting unit 211. The mixer 23 outputs the mixing signal for reverberation sound to the level setting unit 212.

[0129] The level setting unit 211 adjusts the level of the mixing signal for the initial reflected sound. The level setting unit 212 adjusts the level of the mixing signal for the reverberant sound. The level adjustment of the level setting unit 211 and the level adjustment of the level setting unit 212 are set by the level balance adjustment unit 152 in the same manner as the level setting unit 25A and the level setting unit 25B.

[0130] The level setting unit 211 outputs the mixing signal for the initial reflected sound after level adjustment to the FIR filter 24A. The level setting unit 212 outputs the mixing signal for the reverberant sound after level adjustment to the synthesizing unit 213.

[0131] The sound signal acquisition unit 210 acquires the sound pickup signals from the omnidirectional microphones 12A, 12B, and 12C. The sound signal acquisition unit 210 outputs the acquired sound pickup signals to the mixer 230. The mixer 230 mixes the sound pickup signals from the sound signal acquisition unit 210. The mixer 230 outputs the mixed sound pickup signals to the synthesizing unit 213.

[0132] The synthesizing unit 213 synthesizes (adds) the mixing signal for the reverberant sound after level adjustment from the level setting unit 212 and the mixed sound pickup signals from the mixer 230. The synthesizing unit 213 outputs the synthesized signal to the FIR filter 24B.

[0133] The FIR filter 24A convolves the mixing signal for the initial reflected sound after level adjustment with the impulse response for the initial reflected sound to generate an initial reflected sound control signal. The FIR filter 24B convolves the synthesized signal with the impulse response for the reverberant sound to generate a reverberant sound control signal.

[0134] The level setting unit 25A adjusts the level of the initial reflected sound control signal. The level setting unit 25B adjusts the level of the reverberant sound control signal.

[0135] The matrix mixer 26 distributes the input audio signal to the output systems for each speaker. The matrix mixer 26 distributes the reverberation sound control signal to each output system of speakers 61A to 61F and outputs it to the delay adjustment unit 28. The matrix mixer 26 distributes the early reflection sound control signal to each output system of speakers 51A to 51D and outputs it to the delay adjustment unit 28.

[0136] The delay adjustment unit 28 adjusts the delay time according to the distances between the sound sources 611B, 612B, and 613B and the plurality of speakers. Thereby, the delay adjustment unit 28 can adjust the phases of the reverberation sound control signal and the early reflection sound control signal output from the plurality of speakers according to the positional relationships (distances) between the sound sources 611B, 612B, and 613B and the plurality of speakers.

[0137] The output unit 27 converts the early reflection sound control signal and the reverberation sound control signal output from the delay adjustment unit 28 into analog signals. Further, the output unit 27 amplifies the analog signals. The output unit 27 outputs the amplified analog signals to the corresponding speakers.

[0138] By performing this configuration and processing, the audio signal processing unit 10D can realize a richer sound image and a more expanded space for the line input signal (the audio signal input via the line) than before. Therefore, the audio signal processing unit 10D can realize the desired sound field support for a sound source having a line output such as an electronic musical instrument.

[0139] Furthermore, the audio signal processing unit 10D generates an early reflection sound control signal using the line input signal. The line input signal has a higher S / N ratio than the audio signal picked up by the microphone. Therefore, the audio signal processing unit 10D can generate the early reflection sound control signal without being affected by noise. Thereby, the audio signal processing unit 10D can more reliably realize the desired sound field having a richer sound image and a more expanded space than before.

[0140] Also, the audio signal processing unit 10D controls the volume of the line input signal and generates an early reflection sound control signal using the line input signal after volume control. Electronic musical instruments have different default volume levels respectively. Therefore, without performing volume control, for example, when the electronic musical instrument input via the line is switched, the desired early reflection sound control signal cannot be generated. However, by performing volume control on the line input signal, the audio signal processing unit 10D can make the level of the audio signal for generating the early reflection sound control signal constant. Thereby, the audio signal processing unit 10D can generate the desired early reflection sound control signal even when, for example, the electronic device input via the line is switched.

[0141] Also, the audio signal processing unit 10D controls the volume of a plurality of line input signals and then mixes them. Then, the audio signal processing unit 10D generates an early reflection sound control signal using this mixed audio signal. Thereby, the audio signal processing unit 10D can appropriately adjust the level balance of the plurality of line input signals. Therefore, the audio signal processing unit 10D can generate the desired early reflection sound control signal even when there are a plurality of line input signals.

[0142] Note that the audio signal processing unit 10D can obtain these operational effects not only for the early reflection sound control signal but also for the reverberation sound control signal.

[0143] Also, the audio signal processing unit 10D uses only the line input signal to generate the early reflection sound control signal. On the other hand, the audio signal processing unit 10D uses the line input signal and the sound collection signal collected by an omnidirectional microphone to generate the reverberation sound control signal. By controlling the early reflection sound and the reverberation sound individually, blurring of the sound image is suppressed, and a rich sound image and expansion of the space are realized. Also, by using the sound collection signal collected by an omnidirectional microphone for the reverberation sound control signal, the effect of sound field support can be extended not only to the sound of the sound source such as an electronic musical instrument but also to the sound generated in the space such as the applause of the audience. Therefore, by having this configuration, the audio signal processing unit 10D can realize flexible sound field support.

[0144] Note that the above description does not directly describe the reproduction of sound. However, the sound signal processing unit 10D may include a direct sound processing system as a processing system different from the above-described configuration.

[0145] In this case, for example, the sound signal processing unit 10D performs level adjustment on the output of the mixer 23, that is, the mixed sound signal, and outputs it to a separately installed stereo speaker or the like.

[0146] Also, for example, the sound signal processing unit 10D performs level adjustment on the mixed sound signal and outputs it to the matrix mixer 26. The matrix mixer 26 mixes the direct sound signal, the early reflection sound control signal, and the reverberation sound control signal and outputs them to the output unit 27. At this time, the matrix mixer 26 may set a dedicated speaker for the direct sound signal and mix the direct sound signal, the early reflection sound control signal, and the reverberation sound control signal so as to output the direct sound signal to this dedicated speaker.

[0147] Also, in the above description, the sound sources 611B, 612B, and 613B are taken as electronic musical instruments as an example. However, the sound sources 611B, 612B, and 613B may be arranged near a singer, such as a hand microphone held by the singer or a stand microphone installed near the singer, and may pick up the singer's voice and output a singing sound signal.

[0148] In Embodiment 3, for example, the following configurations are also possible, and the following effects can be achieved in each configuration. In the following description, the same parts as those in the above description will be omitted.

[0149] (3-1) An embodiment of Embodiment 3 of the present invention is a sound signal processing method that inputs a sound signal via a line, controls the volume of the line-input sound signal, and generates an early reflection sound control signal from the volume-controlled sound signal.

[0150] FIG. 16 is a block diagram showing the configuration of a sound signal processing unit 10E corresponding to the above-described sound signal processing method. The sound signal processing unit 10E includes a line input unit 21E, a gain adjustment unit 22E, an early reflection sound control signal generation unit 214, an impulse response acquisition unit 151A, and a delay adjustment unit 28.

[0151] The line input unit 21E receives one line input signal and outputs it to the gain adjustment unit 22E. The gain adjustment unit 22E controls the volume of the line input signal. The gain adjustment unit 22E outputs the volume-controlled line input signal to the early reflection sound control signal generation unit 214.

[0152] The early reflection sound control signal generation unit 214 convolves the volume-controlled line input signal with the data of the impulse response for early reflection sound to generate an early reflection sound control signal. Similar to the above-described embodiment, the early reflection sound control signal generation unit 214, for example, acquires the data of the impulse response from a memory and uses it for convolution. The early reflection sound control signal generation unit 214 outputs the early reflection sound control signal to the delay adjustment unit 28. The delay adjustment unit 28 adjusts the delay time of the early reflection sound control signal and outputs it to the speaker 51A as described above. When there are a plurality of speakers, a matrix mixer 26 may be provided in the same manner as the sound signal processing unit 10 described above. The matrix mixer 26 distributes and outputs the early reflection sound control signal to a plurality of speakers.

[0153] With this configuration and method, the sound signal processing unit 10E can appropriately generate an early reflection sound control signal for one line input signal, and can realize a desired sound field having a richer sound image and spatial expansion than before.

[0154] (3-2) An embodiment of Embodiment 3 of the present invention is a sound signal processing method in which there are a plurality of line inputs, and the volume of a plurality of line input sound signals is controlled for each line input.

[0155] With this configuration and method, the sound signal processing unit can appropriately generate an early reflection sound control signal for a plurality of line input signals, and can realize a desired sound field having a richer sound image and spatial expansion than before. Further, the sound signal processing unit can appropriately adjust the level balance between the plurality of line input signals, and can realize a desired sound field having a rich sound image and spatial expansion.

[0156] (3-3) An embodiment of Embodiment 3 of the present invention is a sound signal processing method for mixing a plurality of line-input sound signals and generating an early reflection sound control signal from the mixed sound signal.

[0157] FIG. 17 is a block diagram showing the configuration of a sound signal processing unit 10F corresponding to the above-described sound signal processing method. The sound signal processing unit 10F includes a line input unit 21F, a gain adjustment unit 22F, a mixer 23F, an early reflection sound control signal generation unit 214, an impulse response acquisition unit 151A, and a delay adjustment unit 28.

[0158] The line input unit 21F receives a plurality of line input signals and outputs them to the gain adjustment unit 22F. The gain adjustment unit 22F controls the volume of the plurality of line input signals. At this time, the gain adjustment unit 22F sets an individual gain for each of the plurality of line input signals and performs volume control. For example, the gain adjustment unit 22F sets an individual gain based on the level balance of the plurality of line input signals. The gain adjustment unit 22F outputs the plurality of volume-controlled line input signals to the mixer 23F.

[0159] The mixer 23F mixes and outputs the plurality of volume-controlled line input signals. The mixer 23F outputs the mixed signal to the early reflection sound control signal generation unit 214.

[0160] The initial reflected sound control signal generation unit 214 convolves the mixing signal with an impulse response for the initial reflected sound to generate an initial reflected sound control signal. The initial reflected sound control signal generation unit 214 outputs the initial reflected sound control signal to the delay adjustment unit 28. The delay adjustment unit 28 adjusts the delay time of the initial reflected sound control signal and outputs it to the speaker 51A in the same manner as described above. When there are a plurality of speakers, a matrix mixer 26 may be provided in the same manner as the above-described sound signal processing unit 10. The matrix mixer 26 distributes and outputs the initial reflected sound control signal to a plurality of speakers.

[0161] With this configuration and method, the sound signal processing unit 10F can generate an initial reflected sound control signal for a mixing signal obtained by mixing a plurality of line input signals, and can realize a desired sound field having a richer sound image and spatial spread than before.

[0162] (3-4) An embodiment of Embodiment 3 of the present invention is a sound signal processing method for adjusting the balance between the level of the initial reflected sound control signal and the level of the sound signal that is the source of the initial reflected sound control signal.

[0163] FIG. 18 is a block diagram showing the configuration of a sound signal processing unit 10G corresponding to the above-described sound signal processing method. The sound signal processing unit 10G includes a line input unit 21G, a gain adjustment unit 22G, a mixer 23G, an initial reflected sound control signal generation unit 214, a level setting unit 216, a level setting unit 217, a synthesis unit 218, an impulse response acquisition unit 151A, a level balance adjustment unit 153, and a delay adjustment unit 28.

[0164] The line input unit 21G, the gain adjustment unit 22G, and the mixer 23G are each the same as the above-described line input unit 21F, gain adjustment unit 22F, and mixer 23F. The mixer 23G outputs the mixing signal to the level setting unit 216 and the level setting unit 217.

[0165] The level balance adjustment unit 153 sets the gain for the direct sound and the gain for the early reflection sound using the level balance between the direct sound and the early reflection sound. The level balance adjustment unit 153 outputs the gain for the direct sound to the level setting unit 216 and outputs the gain for the early reflection sound to the level setting unit 217.

[0166] The level setting unit 216 performs volume control of the mixing signal using the gain for the direct sound. The level setting unit 216 outputs the mixing signal volume-controlled by the gain for the direct sound to the synthesizing unit 218.

[0167] The level setting unit 217 performs volume control of the mixing signal using the gain for the early reflection sound. The mixing signal volume-controlled by the gain for the early reflection sound is output to the early reflection sound control signal generation unit 214.

[0168] The early reflection sound control signal generation unit 214 convolves the mixing signal volume-controlled by the gain for the early reflection sound with the impulse response for the early reflection sound to generate an early reflection sound control signal. The early reflection sound control signal generation unit 214 outputs the early reflection sound control signal to the synthesizing unit 218.

[0169] The synthesizing unit 218 synthesizes the direct sound signal and the early reflection sound control signal and outputs the result to the delay adjustment unit 28. The delay adjustment unit 28 adjusts the delay time of the synthesized signal and outputs the result to the speaker 51A as described above. When there are a plurality of speakers, similar to the above-described sound signal processing unit 10, a matrix mixer 26 may be provided instead of the synthesizing unit 218. The matrix mixer 26 distributes and outputs the synthesized signal of the direct sound signal and the early reflection sound control signal to a plurality of speakers. The matrix mixer 26 sets the assignment of the direct sound signal and the early reflection sound control signal for each speaker and distributes and outputs the direct sound signal and the early reflection sound control signal to a plurality of speakers using this assignment.

[0170] With this configuration and method, the sound signal processing unit 10G can adjust the level balance between the direct sound signal and the early reflection sound control signal. Therefore, the sound signal processing unit 10G can realize a desired sound field with a rich sound image and an expanded space, which has an excellent balance between the direct sound and the early reflection sound.

[0171] (3-5) An embodiment of Embodiment 3 of the present invention is a sound signal processing method for generating a reverberation sound signal from a volume-controlled sound signal.

[0172] FIG. 19 is a block diagram showing the configuration of a sound signal processing unit 10H corresponding to the above-described sound signal processing method. The sound signal processing unit 10H includes a line input unit 21H, a gain adjustment unit 22H, an early reflection sound control signal generation unit 214, a reverberation sound control signal generation unit 219, an impulse response acquisition unit 151A, and a delay adjustment unit 28.

[0173] The line input unit 21H and the gain adjustment unit 22H are the same as the line input unit 21E and the gain adjustment unit 22E, respectively. The gain adjustment unit 22H outputs the volume-controlled line input signal to the early reflection sound control signal generation unit 214 and the reverberation sound control signal generation unit 219. The early reflection sound control signal generation unit 214 also has the same configuration as described above.

[0174] The reverberation sound control signal generation unit 219 convolves the volume-controlled line input signal with an impulse response for reverberation sound to generate a reverberation sound control signal. The reverberation sound control signal generation unit 219 outputs the reverberation sound control signal to the delay adjustment unit 28. The delay adjustment unit 28 adjusts the delay time of the reverberation sound control signal and outputs it to the speaker 61A in the same manner as described above. When there are a plurality of speakers, a matrix mixer 26 may be provided in the same manner as the sound signal processing unit 10 described above. The matrix mixer 26 distributes and outputs the reverberation sound control signal to a plurality of speakers.

[0175] With this configuration and method, the sound signal processing unit 10H can appropriately generate a reverberation sound control signal together with the early reflection sound control signal, and can reproduce a desired sound field with a richer sound image and an expanded space.

[0176] (3-6) An embodiment of Embodiment 3 of the present invention is a sound signal processing method that picks up an output sound including a sound signal and generates a reverberation sound signal using the picked-up signal. That is, the sound signal processing unit picks up the sound output from the speaker, feeds it back, and generates a reverberation sound signal from the picked-up signal.

[0177] With this configuration and method, the sound signal processing unit can generate a reverberation sound signal corresponding to Room 62B during performance, and can realize a desired sound field having a richer sound image and spatial expansion.

[0178] (3-7) An embodiment of Embodiment 3 of the present invention is a sound signal processing method that performs volume control for the reverberation sound on the reverberation sound signal immediately before or immediately after the generation of the reverberation sound signal.

[0179] With this configuration and method, the sound signal processing unit can appropriately adjust the level of the reverberation sound. Thereby, for example, the sound signal processing unit can appropriately adjust the level balance between the early reflection sound and the reverberation sound, and the level balance between the direct sound and the reverberation sound.

[0180] (3-8) An embodiment of Embodiment 3 of the present invention is a sound signal processing method that performs volume control for the early reflection sound on the early reflection sound control signal immediately before or immediately after the generation of the early reflection sound control signal.

[0181] With this configuration and method, the sound signal processing unit can appropriately adjust the level of the early reflection sound. Thereby, for example, the sound signal processing unit can appropriately adjust the level balance between the early reflection sound and the reverberation sound, and the level balance between the direct sound and the early reflection sound.

[0182] (3-9) An embodiment of Embodiment 3 of the present invention is a sound signal processing method that outputs the sound signal and the early reflection sound control signal together.

[0183] With this configuration and method, the sound signal processing unit can output the direct sound and the early reflection sound in the same (single) output system.

[0184] The description of this embodiment is illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above-described embodiments but by the claims. Furthermore, the scope of the present invention is intended to include all modifications within the meaning and scope equivalent to the claims.

Explanation of Reference Numerals

[0185] 1, 1A, 1B... Sound field support system 10, 10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H... Sound signal processing unit 11A, 11B, 11C... Directional microphone 12A, 12B, 12C... Omnidirectional microphone 13A, 13B, 13C, 13D... Directional microphone 14A, 14B, 14C, 14D... Directional microphone 21, 21A, 21B... Sound signal acquisition unit 21D, 21E, 21F, 21G, 21H... Line input unit 22, 22E, 22F, 22G, 22H... Gain adjustment unit 23, 23F, 23G... Mixer 24A... FIR filter 24B... FIR filter 25A... Level setting unit 25B... Level setting unit 26... Matrix mixer 27... Output unit 28... Delay adjustment unit 31... Memory 51A, 51B, 51C, 51D... Speaker 52A, 52B, 52C, 52D... Speaker 53A, 53B, 53C, 53D... Speaker 60... Stage 61, 611B, 612B, 613B... Sound source 61A, 61B, 61C, 61D, 61E, 61F... Speaker 62, 62B... Room 151, 151A... Impulse response acquisition unit 152…Level balance adjustment unit 153…Level balance adjustment unit 204A…Processing unit 210…Sound signal acquisition unit 211, 212…Level setting unit 213…Synthesis unit 214…Initial reflection sound control signal generation unit 219…Reverberation sound control signal generation unit 230…Mixer 510D…Directional microphone 620…Space

Claims

1. Obtaining a plurality of sound signals respectively picked up by a plurality of microphones arranged in a predetermined space, Performing level adjustment on each of the plurality of sound signals according to the arrangement position of each of the plurality of microphones, Mixing the adjusted plurality of sound signals into a first mixing signal of a first system and a second mixing signal of a second system, Generating reverberant sound using the first mixing signal, Generating early reflection sound using the second mixing signal, A sound signal processing method, Performing level adjustment on each of the plurality of sound signals mixed as the second mixing signal according to the distance from the arrangement position of each of the plurality of microphones to a speaker that outputs the early reflection sound, The plurality of microphones are arranged in plural in the front-rear direction and plural in the left-right direction when the predetermined space is viewed in plan, In the level adjustment, among the plurality of microphones, in the left-right direction, the gain for each of the plurality of sound signals mixed as the second mixing signal is set high in the order of increasing distance from each arrangement position to the speaker that outputs the early reflection sound, In the level adjustment, among the plurality of microphones, in the front-rear direction, the gain of the sound signal mixed as the second mixing signal corresponding to the microphone on the side where the distance from each arrangement position to the listening position is close is set higher than the gain of the sound signal mixed as the second mixing signal corresponding to the microphone on the side where the distance to the listening position is far, A sound signal processing method.

2. Performing delay adjustment according to the distance from the arrangement position of each of the plurality of microphones to a speaker that outputs the early reflection sound, The sound signal processing method according to claim 1.

3. Setting the delay time larger as the distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the early reflection sound becomes longer, The sound signal processing method according to claim 2.

4. An acquisition unit that acquires a plurality of sound signals respectively picked up by a plurality of microphones arranged in a predetermined space, A gain adjustment unit that performs level adjustment on each of the plurality of sound signals according to the arrangement position of each of the plurality of microphones, A mixer that mixes the adjusted plurality of sound signals into a first mixing signal of a first system and a second mixing signal of a second system, A reverberant sound generation unit that generates reverberant sound using the first mixing signal, An initial reflected sound generation unit that generates an initial reflected sound using the second mixing signal. A sound signal processing device, The gain adjustment unit adjusts the level of each of the plurality of sound signals according to the distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the initial reflected sound. The plurality of microphones are arranged in a plurality in the front-rear direction and a plurality in the left-right direction when the predetermined space is viewed in plan. In the level adjustment, among the plurality of microphones, in the left-right direction, the gain for each of the plurality of sound signals mixed as the second mixing signal is set higher in the order of increasing distance from the arrangement position of each to the speaker that outputs the initial reflected sound. In the level adjustment, among the plurality of microphones, for the sound signal mixed as the second mixing signal corresponding to the microphone on the side where the distance from the arrangement position to the listening position is closer in the front-rear direction, the gain of the sound signal is set higher than the gain of the sound signal mixed as the second mixing signal corresponding to the microphone on the side where the distance to the listening position is farther. Sound signal processing device. **Claim 5** The sound signal processing device according to claim 4, further comprising a delay adjustment unit that adjusts the delay according to the distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the initial reflected sound. The sound signal processing device according to claim 4. **Claim 6** The delay adjustment unit sets the delay time to be larger as the distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the initial reflected sound increases. The sound signal processing device according to claim 5.

Images