Sound signal processing method and sound signal processing device

Abstract

To provide a sound signal processing method and a sound signal processor that achieve a richer sound image and spatial expansion.SOLUTION: In a sound field support system 1A, a sound signal processing method acquires a plurality of sound signals each collected by using a plurality of microphones (13A-13D, 14A-14D) arranged in a predetermined space (620), adjusts respective levels of the plurality of sound signals according to respective positions of the plurality of microphones, mixes the plurality of sound signals that have been adjusted to produce reflected sound by using the mixed mixing signals, and outputs to the speakers (52A-52E, 53A-53E).SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a sound signal processing method and a sound signal processing device for processing an acquired sound signal. [Background technology]

[0002] Concert halls and other facilities are used for performances of various genres of music and speeches. Such facilities require a variety of acoustic characteristics (e.g., reverberation characteristics). For example, a relatively long reverberation is required for performances, while a relatively short reverberation is required for speeches.

[0003] However, in order to physically change the reverberation characteristics within a hall, it was necessary to change the size of the space, for example by moving the ceiling, which required very large-scale equipment.

[0004] Therefore, for example, a sound field control device such as that shown in Patent Document 1 processes sound picked up by a microphone with an FIR (Finite Impulse Response) filter to generate reverberant sound, and outputs the reverberant sound from speakers installed in the hall, thereby supporting the sound field. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 6-284493 Summary of the Invention [Problem to be solved by the invention]

[0006] However, simply adding reverberation can blur the sense of sound localization. Recently, there has been a demand for richer sound images and a wider spatial spread.

[0007] Therefore, an object of one embodiment of the present invention is to provide a sound signal processing method and a sound signal processing device that localize a sound image according to the position of a sound source in a space, thereby realizing a richer sound image and a wider space. [Means for solving the problem]

[0008] A sound signal processing method acquires a plurality of sound signals picked up by a plurality of microphones arranged in a predetermined space, adjusts the levels of the plurality of sound signals according to the positions of the plurality of microphones, mixes the adjusted plurality of sound signals into a first mixed signal of a first system and a second mixed signal of a second system, generates reverberation sound using the first mixed signal, and generates early reflection sound using the second mixed signal. The sound signal processing method adjusts the levels of the plurality of sound signals to be mixed as the second mixed signal according to the distance from the positions of the plurality of microphones to a speaker that outputs the early reflection sound, and in the level adjustment, sets a higher gain for the plurality of sound signals to be mixed as the second mixed signal in order of decreasing distance from the positions of the plurality of microphones to the speaker that outputs the early reflection sound. [Effects of the Invention]

[0009] The sound signal processing method can realize sound image localization according to the position of the sound source in space. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a transparent perspective view schematically showing a space in the first embodiment. [Figure 2] 1 is a block diagram showing the configuration of a sound field support system according to a first embodiment. [Figure 3] 4 is a flowchart showing the operation of the sound signal processing device. [Figure 4]10A is a schematic diagram showing an example of classification of sound types in the time waveform of an impulse response used for filter coefficients, and FIG. 10B is a schematic diagram showing the time waveform of filter coefficients set in the FIR filter 24A. [Figure 5] FIG. 10 is a schematic diagram showing an impulse response set in an FIR filter 24A. [Figure 6] 10 is a plan view schematically showing the relationship between a space 620 and a chamber 62. FIG. [Figure 7] FIG. 1 is a block diagram showing the minimum configuration of a sound field support system. [Figure 8] FIG. 10 is a transparent perspective view schematically showing a space in a second embodiment. [Figure 9] FIG. 10 is a plan view schematically showing a space in the second embodiment. [Figure 10] FIG. 10 is a block diagram showing the configuration of a sound field support system according to a second embodiment. [Figure 11] 10 is a flowchart showing the operation of the sound signal processing device of the second embodiment. [Figure 12] FIG. 10 is a block diagram showing the minimum configuration of a sound field support system according to a second embodiment. [Figure 13] FIG. 11 is a transparent perspective view schematically showing a space in a third embodiment. [Figure 14] FIG. 1 is a block diagram showing the configuration of a sound field support system. [Figure 15] 10 is a flowchart showing the operation of the sound signal processing device of the third embodiment. [Figure 16] FIG. 2 is a block diagram showing the configuration of a sound signal processing unit. [Figure 17] FIG. 2 is a block diagram showing the configuration of a sound signal processing unit. [Figure 18] FIG. 2 is a block diagram showing the configuration of a sound signal processing unit. [Figure 19] FIG. 2 is a block diagram showing the configuration of a sound signal processing unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] [Embodiment 1] 1 is a see-through perspective view that schematically shows a room 62 that forms a space. FIG. 2 is a block diagram showing the configuration of the sound field support system 1.

[0012] Room 62 forms a space with a roughly rectangular parallelepiped shape. Sound source 61 is located on stage 60 at the front of room 62. The rear of room 62 corresponds to the audience seats where listeners sit. Note that the shape of room 62 and the arrangement of sound sources are not limited to the example shown in Figure 1. The sound signal processing method and sound signal processing device of the present invention can provide a desired sound field in any shape of space, and can achieve a richer sound image and spatial expansion than conventional methods.

[0013] The sound field support system 1 is provided in a room 62 with a directional microphone 11A, a directional microphone 11B, a directional microphone 11C, an omnidirectional microphone 12A, an omnidirectional microphone 12B, an omnidirectional microphone 12C, a speaker 51A, a speaker 51B, a speaker 51C, a speaker 51D, a speaker 61A, a speaker 61B, a speaker 61C, a speaker 61D, a speaker 61E, and a speaker 61F.

[0014] Speakers 61A, 61B, 61C, 61D, 61E, and 61F correspond to first speakers that output a reverberation sound control signal, and speakers 51A, 51B, 51C, and 51D correspond to second speakers that output an early reflection sound control signal.

[0015] The number of directional microphones and the number of omnidirectional microphones shown in Fig. 1 are three. However, the sound field support system 1 only needs to include at least one microphone. The number of speakers is not limited to the number shown in Fig. 1. The sound field support system 1 only needs to include at least one speaker.

[0016] The directional microphones 11A, 11B, and 11C mainly pick up the sound of a sound source 61 on the stage.

[0017] Omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C are installed on the ceiling and pick up the entire sound in room 62, including the direct sound of sound source 61 and the reflected sound in room 62.

[0018] Speakers 51A, 51B, 51C, and 51D are installed on the wall surfaces of room 62. Speakers 61A, 61B, 61C, 61D, 61E, and 61F are installed on the ceiling of room 62. However, in the present invention, the installation positions of the microphones and speakers are not limited to this example.

[0019] 2, the sound field assistance system 1 includes a sound signal processing unit 10 and a memory 31 in addition to the components shown in FIG. 1. The sound signal processing unit 10 is mainly composed of a CPU and a DSP (Digital Signal Processor). Functionally, the sound signal processing unit 10 includes a sound signal acquiring unit 21, a gain adjusting unit 22, a mixer 23, an FIR (Finite Impulse Response) filter 24A, an FIR filter 24B, a level setting unit 25A, a level setting unit 25B, a matrix mixer 26, a delay adjusting unit 28, an output unit 27, an impulse response acquiring unit 151, and a level balance adjusting unit 152. The sound signal processing unit 10 is an example of the sound signal processing device of the present invention.

[0020] The CPU constituting the sound signal processing unit 10 reads out an operation program stored in the memory 31 and controls each component. The CPU functionally configures an impulse response acquisition unit 151 and a level balance adjustment unit 152 by the operation program. Note that the operation program does not need to be stored in the memory 31. The CPU may download the operation program each time from, for example, a server (not shown).

[0021] 3 is a flowchart showing the operation of the sound signal processing unit 10. First, the sound signal acquiring unit 21 acquires sound signals (S11). The sound signal acquiring unit 21 acquires sound signals from the directional microphone 11A, the directional microphone 11B, the directional microphone 11C, the omnidirectional microphone 12A, the omnidirectional microphone 12B, and the omnidirectional microphone 12C. When the sound signal acquiring unit 21 acquires an analog signal, it converts the acquired analog signal into a digital signal and outputs the digital signal.

[0022] Gain adjustment unit 22 adjusts the gain of the sound signals acquired from directional microphone 11A, directional microphone 11B, directional microphone 11C, omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C through sound signal acquisition unit 21. For example, gain adjustment unit 22 sets a high gain for a directional microphone located closer to sound source 61. Note that gain adjustment unit 22 is not an essential component in the first embodiment.

[0023] Mixer 23 mixes the sound signals acquired from directional microphone 11A, directional microphone 11B, and directional microphone 11C. Mixer 23 also distributes the mixed sound signal to multiple signal processing systems. Mixer 23 outputs the distributed sound signal to FIR filter 24A. Mixer 23 also mixes the sound signals acquired from omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C. Mixer 23 outputs the mixed sound signal to FIR filter 24B.

[0024] In the example of FIG. 2, mixer 23 mixes sound signals acquired from directional microphone 11A, directional microphone 11B, and directional microphone 11C into four signal processing systems for speakers 51A, 51B, 51C, and 51D. Mixer 23 also mixes sound signals acquired from omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C into four signal processing systems. Each signal processing system corresponds to speakers 61A to 61F. Hereinafter, the four signal processing systems corresponding to speakers 61A to 61F will be referred to as the first system. The four signal processing systems corresponding to speakers 51A, 51B, 51C, and 51D will be referred to as the second system.

[0025] The number of signal processing systems is not limited to this example. Sound signals acquired from omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C may be distributed to six first systems in accordance with speakers 61A, 61B, 61C, 61D, 61E, and 61F. Mixer 23 is not an essential component in the first embodiment.

[0026] The mixer 23 may also have an EMR (Electronic Microphone Rotator) function. The EMR is a technique for flattening the frequency characteristics of a feedback loop by changing the transfer function between a fixed microphone and a speaker over time. The EMR function constantly switches the connection between the microphones and the signal processing system. The mixer 23 outputs the sound signals acquired from the directional microphone 11A, the directional microphone 11B, and the directional microphone 11C to the FIR filter 24A by switching the output destination. Alternatively, the mixer 23 outputs the sound signals acquired from the omnidirectional microphone 12A, the omnidirectional microphone 12B, and the omnidirectional microphone 12C to the FIR filter 24B by switching the output destination. This allows the mixer 23 to flatten the frequency characteristics of the acoustic feedback system from the speaker to the microphone in the room 62.

[0027] Next, the impulse response acquisition unit 151 sets the filter coefficients of the FIR filter 24A and the FIR filter 24B (S12).

[0028] Here, we will explain the impulse response data set in the filter coefficients. Fig. 4(A) is a schematic diagram showing an example of classification of sound types in the time waveform of the impulse response used in the filter coefficients, and Fig. 4(B) is a schematic diagram showing the time waveform of the filter coefficients set in the FIR filter 24A. Fig. 5(A) and Fig. 5(B) are schematic diagrams showing the time waveform of the filter coefficients set in the FIR filter 24B.

[0029] As shown in Fig. 4(A), an impulse response can be divided into a direct sound, an early reflection sound, and a reverberant sound, which are arranged on the time axis. The filter coefficients set in the FIR filter 24A are set based on the portion of the early reflection sound excluding the direct sound and the reverberant sound in the impulse response, as shown in Fig. 4(B). The filter coefficients set in the FIR filter 24B are set based on the reverberant sound excluding the direct sound and the early reflection sound in the impulse response, as shown in Fig. 5(A). The FIR filter 24B may also be set based on the early reflection sound and the reverberant sound excluding the direct sound in the impulse response, as shown in Fig. 5(B).

[0030] The impulse response data is stored in the memory 31. The impulse response acquisition unit 151 acquires the impulse response data from the memory 31. However, the impulse response data does not need to be stored in the memory 31. The impulse response acquisition unit 151 may download the impulse response data each time, for example, from a server (not shown) or the like.

[0031] The impulse response acquisition unit 151 may acquire impulse response data from which only the early reflection sound has been extracted in advance, and set the data in the FIR filter 24A. Alternatively, the impulse response acquisition unit 151 may acquire impulse response data containing direct sound, early reflection sound, and reverberation sound, extract only the early reflection sound, and set the extracted data in the FIR filter 24A. Similarly, when only reverberation sound is used, the impulse response acquisition unit 151 may acquire impulse response data from which only the reverberation sound has been extracted in advance, and set the extracted data in the FIR filter 24B. Alternatively, the impulse response acquisition unit 151 may acquire impulse response data from which the direct sound, early reflection sound, and reverberation sound have been extracted, and set the extracted data in the FIR filter 24B.

[0032] Fig. 6 is a plan view showing a schematic relationship between a space 620 and a room 62. As shown in Fig. 6, impulse response data is measured in advance in a predetermined space 620, such as a concert hall or a church, in which a sound field is to be reproduced. For example, the impulse response data is measured by emitting a test sound (pulse sound) at the position of a sound source 61 and collecting the sound with a microphone.

[0033] The impulse response data may be acquired at any position in the space 620. However, it is preferable to measure the impulse response data of the early reflection sound using a directional microphone installed near the wall surface. The early reflection sound is a reflection sound from a clear direction of arrival. Therefore, by measuring the impulse response data using a directional microphone installed near the wall surface, it is possible to precisely acquire the reflected sound data of the target space. On the other hand, the reverberation sound is a reflection sound from which the direction of arrival of the sound is not clear. Therefore, the impulse response data of the reverberation sound may be measured using a directional microphone installed near the wall surface, or may be measured using an omnidirectional microphone separate from the microphone used to measure the early reflection sound.

[0034] The FIR filter 24A convolves different impulse response data with the four sound signals of the second system, which is the signal flow at the top of Fig. 2. If there are multiple signal processing systems, the FIR filter 24A and the FIR filter 24B may be provided for each signal processing system. For example, four FIR filters 24A may be provided.

[0035] When directional microphones installed near the wall are used as described above, impulse response data is measured by a separate directional microphone for each signal processing system. For example, as shown in Fig. 6, for a signal processing system corresponding to speaker 51D installed at the rear right as one faces stage 60, impulse response data is measured by directional microphone 510D installed near the wall at the rear right as one faces stage 60.

[0036] The FIR filter 24A convolves the impulse response data with each sound signal of the second system (S13). The FIR filter 24B convolves the impulse response data with each sound signal of the first system, which is the signal flow at the bottom of FIG. 2 (S13).

[0037] The FIR filter 24A generates an early reflection sound control signal that reproduces the early reflection sound of a predetermined space by convolving the input sound signal with the impulse response data of the set early reflection sound.The FIR filter 24B generates a reverberation sound control signal that reproduces the reverberation sound of a predetermined space by convolving the input sound signal with the impulse response data of the set reverberation sound.

[0038] The level setting unit 25A adjusts the level of the early reflection sound control signal (S14), and the level setting unit 25B adjusts the level of the reverberation sound control signal (S14).

[0039] The level balance adjustment section 152 sets the level adjustment amounts of the level setting section 25A and the level setting section 25B.

[0040] The level balance adjustment unit 152 refers to the respective levels of the early reflection sound control signal and the reverberation sound control signal and adjusts the level balance between them. For example, the level balance adjustment unit 152 adjusts the balance between the level of the last component in time of the early reflection sound control signal and the level of the first component in time of the reverberation sound control signal. Alternatively, the level balance adjustment unit 152 may adjust the balance between the power of multiple components in the latter part in time of the early reflection sound control signal and the power of components in the first part in time of the reverberation sound control signal. In this way, the level balance adjustment unit 152 can individually control the sounds of the early reflection sound control signal and the reverberation sound control signal, and adjust the balance appropriately to suit the space to which they are applied.

[0041] Next, the matrix mixer 26 distributes the input sound signal to the output systems for the respective speakers. The matrix mixer 26 distributes the reverberation sound control signal of the first system to each of the output systems for the speakers 61A to 61F and outputs it to the delay adjustment unit 28. Since the second system already corresponds to an output system, the matrix mixer 26 outputs the early reflection sound control signal of the second system directly to the delay adjustment unit 28.

[0042] The matrix mixer 26 may also adjust the gain of each output system, adjust the frequency characteristics, and so on.

[0043] The delay adjustment unit 28 adjusts the delay time according to the distance between the sound source 61 and the plurality of speakers (S15). For example, the delay adjustment unit 28 sets the delay time to be shorter for the plurality of speakers in ascending order of the distance between the sound source 61 and the speaker. This allows the delay adjustment unit 28 to adjust the phases of the reverberation sound control signal and the early reflection sound control signal output from the plurality of speakers according to the positions of the plurality of speakers from the sound source 61.

[0044] The output unit 27 converts the early reflection sound control signal and the reverberation sound control signal output from the delay adjustment unit 28 into analog signals. The output unit 27 also amplifies the analog signals. The output unit 27 outputs the amplified analog signals to the corresponding speakers (S16).

[0045] With the above configuration, the sound signal processing unit 10 acquires a sound signal, acquires an impulse response, convolves the impulse response of the early reflection sound from the impulse response into the sound signal, and outputs the sound signal convolved with the impulse response of the early reflection sound as an early reflection sound control signal that has been processed separately from the reverberation sound control signal. In this way, the sound signal processing unit 10 achieves a richer sound image and a wider spatial spread than conventional sound signals.

[0046] In the first embodiment, for example, the following configurations are also possible, and the following advantageous effects can be achieved in each configuration.

[0047] (1-1) One embodiment of the present invention is a signal processing method that acquires a sound signal, acquires an impulse response, and convolves an impulse response of an early reflection sound from the impulse response into the sound signal to generate an early reflection sound control signal.

[0048] 7 is a block diagram showing the configuration of a sound signal processing unit 10A corresponding to the above-described signal processing method. The sound signal processing unit 10A includes a sound signal acquisition unit 21A that acquires a sound signal from a directional microphone 11A, an impulse response acquisition unit 151A that acquires an impulse response, and a processing unit 204A that convolves the impulse response of an early reflection sound from the impulse responses into the sound signal, and outputs the sound signal convolved with the impulse response of the early reflection sound to a speaker 51A as an early reflection sound control signal that has been processed separately from the reverberation sound control signal.

[0049] The sound signal acquisition unit 21A has the same function as the sound signal acquisition unit 21 shown in Fig. 2. The impulse response acquisition unit 151A has the same function as the impulse response acquisition unit 151 in Fig. 2. The processing unit 204A has the functions of the FIR filter 24A and the output unit 27 in Fig. 2.

[0050] The sound signal processing unit 10A, like the sound signal processing unit 10 in FIG. 2, achieves a richer sound image and a wider space than conventional ones.

[0051] (1-2) The processing unit may convolve an impulse response of a reverberant sound among the impulse responses with the sound signal to generate a reverberation control signal that does not include a direct sound, perform separate signal processing on the early reflection sound control signal and the reverberation control signal, and output the reverberation control signal to a first speaker (the above-mentioned first system speaker) and output the early reflection sound control signal to a second speaker (the above-mentioned second system speaker).

[0052] However, an actual room will have more speakers than the example shown in Fig. 1. Of the second speakers (speakers of the second system described above) that output the early reflection sound control signal, a speaker installed near the first speaker (speaker of the first system described above) may output a reverberation sound control signal. In other words, of the multiple speakers of the second system, a speaker installed near the speaker of the first system may output a reverberation sound control signal in addition to the early reflection sound control signal.

[0053] Conversely, among the first speakers (speakers of the first system described above), a speaker installed near a wall surface may output an early reflection sound control signal. That is, among the multiple speakers of the first system, a speaker installed near a speaker of the second system may output an early reflection sound control signal in addition to a reverberation sound control signal.

[0054] This allows the sounds of the early reflection sound control signal and the reverberation sound control signal to be adjusted with an appropriate energy balance.

[0055] (1-3) The first speaker may have a wide directivity, and the second speaker may have a narrow directivity.

[0056] As mentioned above, early reflections are reflected from a specific direction and contribute to the subjective impression. Therefore, it is effective to use a narrow directivity for the second speaker, which can improve the controllability of the early reflections in the target space.

[0057] On the other hand, reverberation is reflected sound that does not have a fixed direction of arrival and contributes to the reverberation of a space. Therefore, it is effective to use a wide directivity first speaker, which can improve the controllability of the reverberation in the target space.

[0058] (1-4) It is preferable that the level per second speaker is higher than that per first speaker.

[0059] As mentioned above, early reflections occur less frequently than reverberant sounds, which are reflected multiple times within a space. As a result, the energy of early reflections is higher than that of reverberant sounds. Therefore, by increasing the level of each second speaker, it is possible to improve the subjective impression that the early reflections have, and to increase the controllability of the early reflections.

[0060] (1-5) It is preferable that the number of second speakers is smaller than the number of first speakers.

[0061] As mentioned above, by reducing the number of second speakers, it is possible to suppress the increase in unnecessary diffuse sound energy. In other words, it is possible to suppress the early reflection sound output from the second speaker from diffusing and reverberating throughout the room, and to prevent the reverberation of the early reflection sound from reaching the listener.

[0062] (1-6) It is preferable that the first speaker is installed on the ceiling of the room, and the second speaker is installed on the side of the room.

[0063] By placing the second speaker on the side of the room, closer to the listener, it is easier to control the early reflections so that they reach the listener, and the controllability of the early reflections can be improved. Also, by placing the first speaker on the ceiling of the room, it is possible to reduce the difference in reverberation depending on the listener's position.

[0064] (1-7) It is preferable that the processing unit adjusts the level balance between the early reflection sound control signal and the reverberation sound control signal.

[0065] The processing unit can adjust the level balance individually, thereby adjusting the sounds of the early reflection sound control signal and the reverberation sound control signal to an appropriate energy balance.

[0066] (1-8) It is preferable that the sound signal acquisition unit separately acquires a first sound signal for generating the reverberation sound control signal and a second sound signal for convolving the impulse response of the early reflection sound. The first sound signal is the sound signal corresponding to the first system (sound signals acquired from omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C), and the second sound signal is the sound signal corresponding to the second system (sound signals acquired from directional microphone 11A, directional microphone 11B, and directional microphone 11C).

[0067] Reverberation is easily affected by the reverberation in a room. Early reflections are easily affected by the sound of the sound source. Therefore, it is preferable that the first sound signal picks up the entire sound in the room, and the second sound signal picks up the sound of the sound source with a high signal-to-noise ratio.

[0068] (1-9) It is preferable that the first sound signal is picked up by an omnidirectional microphone, and the second sound signal is picked up by a directional microphone.

[0069] As in the above, the first sound signal is preferably obtained by collecting the entire sound in the room using, for example, an omnidirectional microphone, and the second sound signal is preferably obtained by collecting the sound of the sound source with a high signal-to-noise ratio using, for example, a directional microphone.

[0070] (1-10) It is preferable that the directional microphone is closer to the sound source than the omnidirectional microphone.

[0071] As in the above, it is preferable that the second sound signal picks up the sound of the sound source with a high S / N ratio, and therefore it is preferable that the directional microphone is close to the sound source.

[0072] (1-11) It is preferable that the impulse response is acquired by using a directional microphone near a wall of a predetermined space.

[0073] The impulse response can be measured with a directional microphone placed near the wall, allowing for more accurate capture of the reflected sound in the target space.

[0074] [Embodiment 2] A sound field support system 1A of embodiment 2 will be described with reference to Figs. 8, 9, 10, and 11. Fig. 8 is a see-through perspective view schematically showing a space 620. Fig. 9 is a plan view of the space 620. Fig. 10 is a block diagram showing the configuration of the sound field support system 1A.

[0075] 11 is a flowchart showing the operation of the sound signal processing device. In this example, it is assumed that a sound source 61 moves on a stage 60, or that multiple sound sources 61 exist on the stage 60. Note that the same components as those in the first embodiment are given the same reference numerals, and their description will be omitted.

[0076] As shown in FIGS. 8 and 9, the sound field support system 1A includes a speaker 52A, a speaker 52B, a speaker 52C, a speaker 52D, a speaker 52E, a speaker 53A, a speaker 53B, a speaker 53C, a speaker 53D, and a speaker 53E.

[0077] 8 and 9, the speakers 52A, 52B, 52C, 52D, and 52E belong to a 2-1 speaker group 520 (to the left of the center as one faces the stage 60) that outputs an early reflection sound control signal of a 2-1 system. Also, in this example, the speakers 53A, 53B, 53C, 53D, and 53E belong to a 2-2 speaker group 530 (to the right of the center as one faces the stage 60) that outputs an early reflection sound control signal of a 2-2 system. The dashed-dotted line in FIG. 9 indicates the 2-1 speaker group 520, and the dashed-two-dotted line indicates the 2-2 speaker group 530.

[0078] In the following description, speakers 52A, 52B, 52C, 52D, and 52E of 2-1 speaker group 520 are collectively referred to as speakers of 2-1 speaker group 520. In the following description, speakers 53A, 53B, 53C, 53D, and 53E of 2-2 speaker group 530 are collectively referred to as speakers of 2-2 speaker group 530.

[0079] As shown in Figures 8 and 9, the sound field support system 1A is equipped with directional microphone 13A, directional microphone 13B, directional microphone 13C, directional microphone 13D, directional microphone 14A, directional microphone 14B, directional microphone 14C and directional microphone 14D in a room 62.

[0080] In this example, directional microphone 13A, directional microphone 13B, directional microphone 13C, and directional microphone 13D are installed on the ceiling lined up in the X1 direction (left-right direction) shown in Figures 8 and 9. Also, in this example, directional microphone 14A, directional microphone 14B, directional microphone 14C, and directional microphone 14D are installed on the ceiling lined up in the X1 direction (left-right direction) shown in Figures 8 and 9. Also, directional microphone 14A, directional microphone 14B, directional microphone 14C, and directional microphone 14D are installed behind directional microphone 13A, directional microphone 13B, directional microphone 13C, and directional microphone 13D in the Y1 direction (front-to-back direction) (towards the audience seats when looking at stage 60 from the side).

[0081] 9, the directional microphones 13A, 13C, 14A, and 14C correspond to the speakers of the 2-1 speaker group 520. That is, the early reflected sound control signal of the 2-1 system is generated based on the sound signals collected by the directional microphones 13A, 13C, 14A, and 14C. Also, the directional microphones 13B, 13D, 14B, and 14D correspond to the speakers of the 2-2 speaker group 530. That is, the early reflected sound control signal of the 2-2 system is generated based on the sound signals collected by the directional microphones 13B, 13D, 14B, and 14D.

[0082] In the following description, directional microphone 13A, directional microphone 13C, directional microphone 14A, and directional microphone 14C will be collectively referred to as directional microphones corresponding to 2-1 speaker group 520. In the following description, directional microphone 13B, directional microphone 13D, directional microphone 14B, and directional microphone 14D will be collectively referred to as directional microphones corresponding to 2-2 speaker group 530.

[0083] 10, the sound signal processing unit 10B of the sound field support system 1A has a configuration in which the FIR filter 24B and the level setting unit 25B are removed from the sound field support system 1 of embodiment 1. However, embodiment 2 may also include the FIR filter 24B and the level setting unit 25B to generate a reverberation sound control signal. In that case, the reverberation sound control signal may be output to any of the speakers 52A to 52E and the speakers 53A to 53E, or may be output from another speaker.

[0084] The sound signal acquisition unit 21 acquires sound signals from a directional microphone corresponding to the 2-1 speaker group 520 and a directional microphone corresponding to the 2-2 speaker group 530 (see FIG. 10).

[0085] The gain adjustment unit 22 adjusts the gain of the sound signals acquired from the directional microphone corresponding to the 2-1 speaker group 520 and the directional microphone corresponding to the 2-2 speaker group 530 (see S101 in FIG. 11).

[0086] In this example, the gain adjustment unit 22 sets different gains for the directional microphones corresponding to the 2-1 speaker group 520 and the directional microphones corresponding to the 2-2 speaker group 530.

[0087] The gain adjustment unit 22 sets the gain of the sound signal of the directional microphones corresponding to the 2-1 speaker group 520 to be higher in ascending order of distance to the speaker of the 2-1 speaker group 520 (for example, speaker 52A) in the left-right direction.

[0088] In addition, the gain adjustment unit 22 sets the gain of the sound signal of the directional microphone corresponding to the 2-1 speaker group 520 in the front-to-back direction (left-to-right direction on the paper in Figure 9) when looking at the stage 60 from the side to be lower than the gain of the sound signal of the directional microphone on the side closer to the audience seats (left-hand side on the paper in Figure 9).

[0089] As described above, the gain adjustment unit 22 sets the gain of the sound signal to a higher value for the directional microphones corresponding to the 2-2 speaker group 530 in order of decreasing distance to the speaker of the 2-2 speaker group 530 (for example, speaker 53A) in the left-right direction.

[0090] In addition, the gain adjustment unit 22 sets the gain of the sound signal of the directional microphone corresponding to the 2-2 speaker group 530 in the front-to-back direction (left-to-right direction on the paper in Figure 9) when looking at the stage 60 from the side to be lower than the gain of the sound signal of the directional microphone on the side closer to the audience seats (left-hand side on the paper in Figure 9).

[0091] For example, gain adjustment unit 22 sets the gain of directional microphone 14A to 0 dB, the gain of directional microphone 13A to −1.5 dB, the gain of directional microphone 14C to −3.0 dB, and the gain of directional microphone 13C to −4.5 dB.

[0092] For example, gain adjustment unit 22 sets the gain of directional microphone 14D to 0 dB, the gain of directional microphone 13D to −1.5 dB, the gain of directional microphone 14B to −3.0 dB, and the gain of directional microphone 13B to −4.5 dB.

[0093] The mixer 23 mixes the sound signals acquired from the directional microphones corresponding to the 2-1 speaker group 520 (see S102 in FIG. 11). The mixer 23 distributes the mixed sound signals to a plurality of signal processing systems (five in FIGS. 8 and 9) in accordance with the number (e.g., five) of speakers in the 2-1 speaker group 520. The mixer 23 also mixes the sound signals acquired from the directional microphones corresponding to the 2-2 speaker group 530. The mixer 23 distributes the mixed sound signals to a plurality of signal processing systems (five in FIGS. 8 and 9) in accordance with the number (e.g., five) of speakers in the 2-2 speaker group 530.

[0094] In a real space, sound image localization changes depending on the arrival direction, level, and density of the reflected sound of the direct sound and early reflected sound. In other words, the sound image localization of the sound source 61 in the audience seats depends on the position of the sound source 61 on the stage 60. For example, when the sound source 61 moves to the left as viewed from the stage 60, the levels of the direct sound and early reflected sound arriving from the left in the audience seats become relatively higher, and the sound image is localized to the left as viewed from the stage 60. The gain adjustment unit 22 sets the gain of the sound signal to be higher for the multiple directional microphones closest to the speaker, thereby controlling the level of the early reflected sound in accordance with the position of the sound source 61 on the stage 60 and realizing sound image localization that is close to phenomena in a real space.

[0095] The delay adjustment unit 28 adjusts the delay time according to the distance between the multiple directional microphones and the speaker. For example, the delay adjustment unit 28 sets the delay time to be shorter for the multiple directional microphones in ascending order of the distance between the directional microphone and the speaker. As a result, the time difference between the early reflection sounds output by the multiple speakers is reproduced according to the distance between the sound source 61 and the speaker.

[0096] Furthermore, by arranging multiple directional microphones in the left-right direction, the sound field support system 1A acquires the sound of the sound source 61 over a wide range on the stage 60. This allows the sound field support system 1A to reflect the level of the early reflected sound according to the position of the sound source 61 in a state close to the real space, without having to detect the position of the sound source 61.

[0097] In the real space, the level of the early reflected sound decreases as the distance between the sound source 61 and the audience seats increases. The gain adjustment unit 22 reduces the gain of the sound signal from the speaker that is farther from the audience seats in the front-to-rear direction, thereby achieving the reverberation of sound in the real space.

[0098] Furthermore, in the real space, the farther the sound source 61 is from the audience seats, the longer the time it takes for the sound to directly reach the audience seats from the sound source 61. Therefore, by using the delay adjustment unit 28 to increase the delay time of the early reflection sound signal output to the speaker farther from the audience seats, the sound field support system 1A can more accurately reproduce the reverberation of sound in the real space.

[0099] In this way, even when the sound source 61 moves on the stage 60 or when there are multiple sound sources 61, the sound field support system 1A of embodiment 2 can generate an early reflection sound control signal corresponding to the position of the sound source 61 by setting the gain of the directional microphone according to the positional relationship between the sound source and the speaker, without having to separately obtain positional information of the sound source 61. Therefore, the sound field support system 1 can effectively achieve sound image localization and achieve richer sound images and a wider space than ever before.

[0100] The gain value of the sound signal from the directional microphone is not limited to this example. Also, although the example has been described in which the gain of the sound signal from the speaker farther from the audience seats is set lower than the gain of the sound signal from the speaker closer to the audience seats, the present invention is not limited to this example.

[0101] Furthermore, in the sound field support system 1A of the second embodiment, eight directional microphones are used, but the present invention is not limited to this. The number of directional microphones may be less than eight or may be nine or more. Furthermore, the positions of the directional microphones are not limited to this example.

[0102] Furthermore, in the sound field support system 1A of the second embodiment, five speakers in the 2-1 speaker group 520 and five speakers in the 2-2 speaker group 530 have been described, but this is not limiting. The number of speaker groups may be three or more, and the number of speakers belonging to each speaker group may be one or more. Furthermore, the positions of the speakers are not limited to this example.

[0103] Furthermore, in the sound field support system 1A of the second embodiment, one directional microphone may be associated with, for example, both the 2-1 speaker group 520 and the 2-2 speaker group 530. In this case, the gain of the sound signal corresponding to the 2-1 speaker group 520 (2-1 system) may be different from the gain of the sound signal corresponding to the 2-2 speaker group 530 (2-2 system).

[0104] In the second embodiment, for example, the following configurations are also possible, and the following advantageous effects can be achieved in each configuration.

[0105] (2-1) A sound signal processing method acquires a plurality of sound signals picked up by a plurality of microphones arranged in a predetermined space, adjusts the levels of the plurality of sound signals according to the respective positions of the plurality of microphones, mixes the adjusted plurality of sound signals, and generates reflected sound using the mixed signal.

[0106] 12 is a block diagram showing the configuration of a sound signal processing unit 10C corresponding to the signal processing method of embodiment 2. The sound signal processing unit 10C includes a sound signal acquisition unit 21B that acquires a plurality of sound signals picked up by a plurality of directional microphones 13A, 13B, 14A, and 14B arranged in a predetermined space, a gain adjustment unit 22B that adjusts the levels of the plurality of sound signals according to the respective arrangement positions of the plurality of directional microphones 13A, 13B, 14A, and 14B, a mixer 23B that mixes the adjusted plurality of sound signals, and a reflected sound generation unit 205B that uses the mixed signals to generate reflected sounds for each system and output them to speakers 52A and 53A.

[0107] The sound signal acquisition unit 21B has the same function as the sound signal acquisition unit 21 shown in Fig. 10. The gain adjustment unit 22B has the same function as the gain adjustment unit 22 in Fig. 10. The mixer 23B has the same function as the mixer 23 in Fig. 10. The reflected sound generation unit 205B has the same function as the FIR filter 24A and the level setting unit 25A in Fig. 10.

[0108] Similar to the sound signal processing unit 10B in FIG. 10, the sound signal processing unit 10C achieves more effective sound image localization by changing the level of the signal collected from the sound signal acquisition unit 21B according to the position of the sound source, without the need to detect the position of the sound source.

[0109] (2-2) The levels of the plurality of sound signals may be adjusted according to the distance from the respective positions of the plurality of microphones to a speaker that outputs the reflected sounds.

[0110] In a real space, sound image localization changes depending on the direction of arrival of the direct sound and early reflected sound, as well as the level and density of the reflected sound. Therefore, this configuration reproduces the reverberation of sound in a real space more accurately.

[0111] (2-3) In the level adjustment, the gain for each of the plurality of sound signals may be set higher in ascending order of distance from the placement position of each of the plurality of microphones to the placement position of the speaker that outputs the reflected sound.

[0112] In this configuration, the gain of the sound signal is set higher for directional microphones that are closer to the speaker, thereby reproducing the attenuation of reflected sound that depends on the distance between the sound source and the wall, and further realizing the reverberation of sound in a real space.

[0113] (2-4) The delay may be adjusted according to the distance from each of the microphones to the speaker that outputs the reflected sound. This configuration achieves sound image localization that closely resembles a phenomenon occurring in real space.

[0114] (2-5) The delay time may be set to be longer as the distance from each of the microphones to the speaker that outputs the pseudo-reflected sound increases.

[0115] This configuration reproduces the delay of reflected sound that depends on the distance between the sound source and the wall.

[0116] (2-6) The sound signal generating device may include speakers that output reflected sounds, and the speakers that output the reflected sounds may include a 2-1 speaker group of a 2-1 system and a 2-2 speaker group of a 2-2 system, and the level adjustment unit may adjust the level for each sound signal for each of the 2-1 system and the 2-1 system, and the mixing unit may mix each of the 2-1 system and the 2-2 system.

[0117] With this configuration, sound image localization can be achieved more effectively.

[0118] (2-7) The sound signal generating device preferably includes a plurality of microphones arranged in a predetermined space, and the plurality of microphones are preferably divided into a plurality of 2-1 microphones corresponding to the 2-1 speaker group and a plurality of 2-2 microphones corresponding to the 2-2 speaker group.

[0119] With this configuration, even if the position of the sound source moves or there are multiple sound sources, sound image localization can be achieved more effectively.

[0120] (2-8) The reflected sound may include an early reflected sound.

[0121] [Embodiment 3] A sound field support system 1B of embodiment 3 will be described with reference to FIGS. 13, 14, and 15. FIG. 13 is a see-through perspective view schematically showing a room 62B of embodiment 3. FIG. 14 is a block diagram showing the configuration of sound field support system 1B. FIG. 15 is a flowchart showing the operation of the sound signal processing device of embodiment 3. Embodiment 3 assumes that output sounds from sound sources 611B, 612B, and 613B are line-input. Note that the same components as those in embodiment 1 above are given the same reference numerals, and their description will be omitted. Line input does not refer to inputting sounds output from sound sources such as various musical instruments described below by a microphone, but rather refers to inputting a sound signal from an audio cable connected to the sound source. In contrast, line output refers to inputting a sound signal from an audio cable connected to a sound source such as various musical instruments described below.

[0122] Room 62B does not require directional microphone 11A, directional microphone 11B, and directional microphone 11C compared to room 62 shown in embodiment 1. Note that directional microphone 11A, directional microphone 11B, and directional microphone 11C may be provided.

[0123] Sound source 611B, sound source 612B, and sound source 613B are, for example, an electronic piano, an electric guitar, etc., and each outputs a sound signal via a line. That is, sound source 611B, sound source 612B, and sound source 613B are connected to audio cables and output sound signals via the audio cables. Note that although the number of sound sources is three in FIG. 13, it may be one, or two, four, or more.

[0124] The sound signal processing unit 10D of the sound field support system 1B differs from the sound signal processing unit 10 shown in the first embodiment in that it further includes a line input unit 21D, a sound signal acquiring unit 210, a level setting unit 211, a level setting unit 212, a synthesizing unit 213, and a mixer 230. The other configuration of the sound signal processing unit 10D is the same as that of the sound signal processing unit 10, and a description of similar parts will be omitted.

[0125] The line input unit 21D receives input of audio signals from the sound source 611B, the sound source 612B, and the sound source 613B (see S201 in FIG. 15). That is, the line input unit 21D is connected to audio cables connected to the sound source 611B, the sound source 612B, and the sound source 613B. The line input unit 21D receives input of audio signals from the sound source 611B, the sound source 612B, and the sound source 613B via these audio cables. Hereinafter, these audio signals will be referred to as line input signals. The line input unit 21D outputs the line input signals of the respective sound sources to the gain adjustment unit 22.

[0126] Gain adjustment unit 22 corresponds to a volume control unit and controls the volume of the line input signal (see S202 in FIG. 15). Specifically, gain adjustment unit 22 controls the volume of each of the line input signal of sound source 611B, the line input signal of sound source 612B, and the line input signal of sound source 613B using individual gains. Gain adjustment unit 22 outputs the line input signal after volume control to mixer 23.

[0127] The mixer 23 mixes the volume-controlled line input signal of the sound source 611B, the volume-controlled line input signal of the sound source 612B, and the volume-controlled line input signal of the sound source 613B.

[0128] The mixer 23 distributes the mixed sound signal to multiple signal processing systems. Specifically, the mixer 23 distributes the mixed sound signal to multiple signal processing systems for early reflection sounds and a signal processing system for reverberation sounds. Hereinafter, the sound signals distributed to the multiple signal processing systems for early reflection sounds will be referred to as "mixed signals for early reflection sounds," and the sound signals distributed to the signal processing system for reverberation sounds will be referred to as "mixed signals for reverberation sounds."

[0129] The mixer 23 outputs a mixed signal for the early reflection sound to the level setting unit 211. The mixer 23 outputs a mixed signal for the reverberation sound to the level setting unit 212.

[0130] The level setting unit 211 adjusts the level of the mixing signal for the early reflection sound. The level setting unit 212 adjusts the level of the mixing signal for the reverberation sound. The level adjustments of the level setting units 211 and 212 are set by the level balance adjustment unit 152, similar to the level setting units 25A and 25B.

[0131] The level setting unit 211 outputs the level-adjusted mixing signal for the early reflection sound to the FIR filter 24 A. The level setting unit 212 outputs the level-adjusted mixing signal for the reverberation sound to the synthesis unit 213.

[0132] Sound signal acquisition unit 210 acquires picked-up signals from omnidirectional microphone 12A, omnidirectional microphone 12B, and omnidirectional microphone 12C. Sound signal acquisition unit 210 outputs the acquired picked-up signals to mixer 230. Mixer 230 mixes the picked-up signals from sound signal acquisition unit 210. Mixer 230 outputs the mixed picked-up signals to synthesis unit 213.

[0133] The synthesis unit 213 synthesizes (adds) the level-adjusted mixed signal for reverberation sound from the level setting unit 212 and the mixed picked-up signal from the mixer 230. The synthesis unit 213 outputs the synthesized signal to the FIR filter 24B.

[0134] The FIR filter 24A convolves the level-adjusted mixing signal for early reflection sounds with an impulse response for early reflection sounds to generate an early reflection sound control signal.The FIR filter 24B convolves the synthesized signal with an impulse response for reverberation sounds to generate a reverberation sound control signal.

[0135] The level setting unit 25A adjusts the level of the early reflection sound control signal, and the level setting unit 25B adjusts the level of the reverberation sound control signal.

[0136] The matrix mixer 26 distributes the input sound signal to the output system for each speaker. The matrix mixer 26 distributes the reverberation sound control signal to each output system of the speakers 61A to 61F and outputs it to the delay adjustment unit 28. The matrix mixer 26 distributes the early reflection sound control signal to each output system of the speakers 51A to 51D and outputs it to the delay adjustment unit 28.

[0137] Delay adjustment unit 28 adjusts the delay time according to the distances between sound source 611B, sound source 612B, and sound source 613B and the plurality of speakers. This allows delay adjustment unit 28 to adjust the phases of the reverberation sound control signals and early reflection sound control signals output from the plurality of speakers according to the positional relationships (distances) between sound source 611B, sound source 612B, and sound source 613B and the plurality of speakers.

[0138] The output unit 27 converts the early reflection sound control signal and the reverberation sound control signal output from the delay adjustment unit 28 into analog signals. The output unit 27 also amplifies the analog signals. The output unit 27 outputs the amplified analog signals to the corresponding speakers.

[0139] By using this configuration and performing this processing, the sound signal processing unit 10D can realize a richer sound image and a wider space than ever before for a line input signal (a sound signal input through a line). Therefore, the sound signal processing unit 10D can realize the desired sound field support for a sound source with a line output, such as an electronic musical instrument.

[0140] Furthermore, the sound signal processing unit 10D generates an early reflection sound control signal using a line input signal. The line input signal has a higher S / N ratio than a sound signal picked up by a microphone. Therefore, the sound signal processing unit 10D can generate the early reflection sound control signal without being affected by noise. This allows the sound signal processing unit 10D to more reliably realize a desired sound field with a richer sound image and wider spatial extent than ever before.

[0141] The sound signal processing unit 10D also controls the volume of the line input signal and generates the early reflection sound control signal using the volume-controlled line input signal. Each electronic musical instrument has a different default volume level. Therefore, without volume control, it would be impossible to generate the desired early reflection sound control signal, for example, when the electronic musical instrument receiving the line input is switched. However, by controlling the volume of the line input signal, the sound signal processing unit 10D can maintain a constant level of the sound signal used to generate the early reflection sound control signal. This allows the sound signal processing unit 10D to generate the desired early reflection sound control signal, even when, for example, the electronic instrument receiving the line input is switched.

[0142] Furthermore, the sound signal processing unit 10D controls the volume of multiple line input signals and then mixes them. The sound signal processing unit 10D then generates an early reflection sound control signal using this mixed sound signal. This allows the sound signal processing unit 10D to appropriately adjust the level balance of multiple line input signals. Therefore, even if there are multiple line input signals, the sound signal processing unit 10D can generate a desired early reflection sound control signal.

[0143] The sound signal processing unit 10D can obtain these effects not only on the early reflection sound control signal but also on the reverberation sound control signal.

[0144] Furthermore, the sound signal processing unit 10D uses only the line input signal to generate the early reflection sound control signal. On the other hand, the sound signal processing unit 10D uses the line input signal and a sound signal picked up by an omnidirectional microphone to generate the reverberation sound control signal. By separately controlling the early reflection sound and the reverberation sound, blurring of the sound image is suppressed, resulting in a rich sound image and a spacious sound. Furthermore, by using a sound signal picked up by an omnidirectional microphone for the reverberation sound control signal, the effect of sound field support can be expanded not only to sounds from sound sources such as electronic musical instruments, but also to sounds generated within a space, such as applause from an audience. Therefore, with this configuration, the sound signal processing unit 10D can achieve flexible sound field support.

[0145] Although the above description does not address the reproduction of direct sound, the sound signal processing unit 10D may include a direct sound processing system as a processing system separate from the above-described configuration.

[0146] In this case, for example, the sound signal processing unit 10D adjusts the level of the output of the mixer 23, that is, the mixed sound signal, and outputs it to separately installed stereo speakers or the like.

[0147] Furthermore, for example, the sound signal processing unit 10D adjusts the level of the mixed sound signal and outputs it to the matrix mixer 26. The matrix mixer 26 mixes the direct sound signal, the early reflection sound control signal, and the reverberation sound control signal, and outputs the result to the output unit 27. At this time, the matrix mixer 26 may set a dedicated speaker for the direct sound signal and mix the direct sound signal, the early reflection sound control signal, and the reverberation sound control signal so that the direct sound signal is output to this dedicated speaker.

[0148] In the above description, sound source 611B, sound source 612B, and sound source 613B are electronic musical instruments as an example. However, sound source 611B, sound source 612B, and sound source 613B may be a handheld microphone held by the singer, a stand microphone placed near the singer, or the like, that picks up the singer's voice and outputs a singing sound signal.

[0149] In the third embodiment, for example, the following configurations are also possible, and the following advantageous effects can be achieved in each configuration. Note that in the following explanation, explanations of points that are the same as those described above will be omitted.

[0150] (3-1) One embodiment of the third embodiment of the present invention is a sound signal processing method that inputs a sound signal through a line, controls the volume of the line-input sound signal, and generates an early reflection sound control signal from the volume-controlled sound signal.

[0151] 16 is a block diagram showing the configuration of a sound signal processing unit 10E corresponding to the above-described sound signal processing method. The sound signal processing unit 10E includes a line input unit 21E, a gain adjustment unit 22E, an early reflection sound control signal generation unit 214, an impulse response acquisition unit 151A, and a delay adjustment unit 28.

[0152] The line input unit 21E receives one line input signal and outputs it to the gain adjustment unit 22E. The gain adjustment unit 22E controls the volume of the line input signal. The gain adjustment unit 22E outputs the volume-controlled line input signal to the early reflection sound control signal generation unit 214.

[0153] The early reflection sound control signal generation unit 214 convolves the volume-controlled line input signal with impulse response data for the early reflection sound to generate an early reflection sound control signal. As in the above-described embodiment, the early reflection sound control signal generation unit 214 obtains impulse response data from a memory, for example, and uses it for convolution. The early reflection sound control signal generation unit 214 outputs the early reflection sound control signal to the delay adjustment unit 28. As in the above-described description, the delay adjustment unit 28 adjusts the delay time of the early reflection sound control signal and outputs it to the speaker 51A. Note that if there are multiple speakers, a matrix mixer 26 may be provided, as in the above-described sound signal processing unit 10. The matrix mixer 26 distributes the early reflection sound control signal to the multiple speakers and outputs it.

[0154] With this configuration and method, the sound signal processing unit 10E can appropriately generate an early reflection sound control signal for one line input signal, and can realize a desired sound field with a richer sound image and wider space than ever before.

[0155] (3-2) One embodiment of the third embodiment of the present invention is a sound signal processing method in which there are a plurality of line inputs, and sound signals input through the plurality of lines are subjected to volume control for each line input.

[0156] With this configuration and method, the sound signal processing unit can appropriately generate early reflection sound control signals for multiple line input signals, thereby realizing a desired sound field with a richer sound image and a wider spatial spread than ever before. Also, the sound signal processing unit can appropriately adjust the level balance between multiple line input signals, thereby realizing a desired sound field with a richer sound image and a wider spatial spread.

[0157] (3-3) One embodiment of the third embodiment of the present invention is a sound signal processing method for mixing a plurality of line-input sound signals and generating an early reflection sound control signal from the mixed sound signal.

[0158] 17 is a block diagram showing the configuration of a sound signal processing unit 10F corresponding to the above-described sound signal processing method. The sound signal processing unit 10F includes a line input unit 21F, a gain adjustment unit 22F, a mixer 23F, an early reflection sound control signal generation unit 214, an impulse response acquisition unit 151A, and a delay adjustment unit 28.

[0159] The line input unit 21F receives a plurality of line input signals and outputs them to a gain adjustment unit 22F. The gain adjustment unit 22F controls the volume of the plurality of line input signals. In this case, the gain adjustment unit 22F sets an individual gain for each of the plurality of line input signals and controls the volume. For example, the gain adjustment unit 22F sets an individual gain based on the level balance of the plurality of line input signals. The gain adjustment unit 22F outputs the plurality of volume-controlled line input signals to a mixer 23F.

[0160] The mixer 23F mixes a plurality of volume-controlled line input signals and outputs the mixed signal. The mixer 23F outputs the mixed signal to the early reflection sound control signal generation unit 214.

[0161] The early reflection sound control signal generation unit 214 convolves the mixing signal with the impulse response for the early reflection sound to generate an early reflection sound control signal. The early reflection sound control signal generation unit 214 outputs the early reflection sound control signal to the delay adjustment unit 28. As described above, the delay adjustment unit 28 adjusts the delay time of the early reflection sound control signal and outputs it to the speaker 51A. Note that if there are multiple speakers, a matrix mixer 26 may be provided, similar to the sound signal processing unit 10 described above. The matrix mixer 26 distributes the early reflection sound control signal to the multiple speakers and outputs it.

[0162] With this configuration and method, the sound signal processing unit 10F can generate an early reflection sound control signal for a mixed signal obtained by mixing multiple line input signals, thereby realizing a desired sound field with a richer sound image and spatial expansion than before.

[0163] (3-4) An embodiment of the third embodiment of the present invention is a sound signal processing method that adjusts the balance between the level of an early reflection sound control signal and the level of a sound signal that is the source of the early reflection sound control signal.

[0164] 18 is a block diagram showing the configuration of a sound signal processing unit 10G that corresponds to the above-mentioned sound signal processing method. The sound signal processing unit 10G includes a line input unit 21G, a gain adjustment unit 22G, a mixer 23G, an early reflection sound control signal generation unit 214, level setting units 216 and 217, a synthesis unit 218, an impulse response acquisition unit 151A, a level balance adjustment unit 153, and a delay adjustment unit 28.

[0165] The line input unit 21G, the gain adjustment unit 22G, and the mixer 23G are similar to the above-described line input unit 21F, the gain adjustment unit 22F, and the mixer 23F, respectively. The mixer 23G outputs a mixed signal to the level setting unit 216 and the level setting unit 217.

[0166] The level balance adjustment unit 153 sets a gain for the direct sound and a gain for the early reflected sound using the level balance between the direct sound and the early reflected sound. The level balance adjustment unit 153 outputs the gain for the direct sound to the level setting unit 216 and outputs the gain for the early reflected sound to the level setting unit 217.

[0167] The level setting unit 216 controls the volume of the mixed signal using the gain for the direct sound. The level setting unit 216 outputs the mixed signal, the volume of which has been controlled using the gain for the direct sound, to the synthesis unit 218.

[0168] The level setting unit 217 controls the volume of the mixed signal using the gain for the early reflection sound, and outputs the mixed signal whose volume has been controlled using the gain for the early reflection sound to the early reflection sound control signal generation unit 214.

[0169] The early reflection sound control signal generator 214 convolves the impulse response for the early reflection sound with the mixing signal whose volume has been controlled by the gain for the early reflection sound to generate an early reflection sound control signal. The early reflection sound control signal generator 214 outputs the early reflection sound control signal to the synthesizer 218.

[0170] The synthesis unit 218 synthesizes the direct sound signal and the early reflection sound control signal and outputs the synthesized signal to the delay adjustment unit 28. As described above, the delay adjustment unit 28 adjusts the delay time of the synthesized signal and outputs it to the speaker 51A. Note that if there are multiple speakers, a matrix mixer 26 may be provided instead of the synthesis unit 218, as in the above-mentioned sound signal processing unit 10. The matrix mixer 26 distributes and outputs the synthesized signal of the direct sound signal and the early reflection sound control signal to the multiple speakers. The matrix mixer 26 sets an allocation of the direct sound signal and the early reflection sound control signal for each speaker, and uses this allocation to distribute and output the direct sound signal and the early reflection sound control signal to the multiple speakers.

[0171] With this configuration and method, the sound signal processing unit 10G can adjust the level balance between the direct sound signal and the early reflection sound control signal, thereby realizing a desired sound field with a good balance between the direct sound and the early reflection sound, a rich sound image, and a spacious sound field.

[0172] (3-5) An embodiment of the third embodiment of the present invention is a sound signal processing method for generating a reverberation sound signal from a volume-controlled sound signal.

[0173] 19 is a block diagram showing the configuration of a sound signal processing unit 10H corresponding to the above-described sound signal processing method. The sound signal processing unit 10H includes a line input unit 21H, a gain adjustment unit 22H, an early reflection sound control signal generation unit 214, a reverberation sound control signal generation unit 219, an impulse response acquisition unit 151A, and a delay adjustment unit 28.

[0174] The line input unit 21H and the gain adjustment unit 22H are similar to the line input unit 21E and the gain adjustment unit 22E, respectively. The gain adjustment unit 22H outputs the volume-controlled line input signal to the early reflection sound control signal generation unit 214 and the reverberation sound control signal generation unit 219. The early reflection sound control signal generation unit 214 also has the same configuration as described above.

[0175] The reverberation sound control signal generator 219 convolves the volume-controlled line input signal with an impulse response for reverberation sound to generate a reverberation sound control signal. The reverberation sound control signal generator 219 outputs the reverberation sound control signal to the delay adjustment unit 28. As described above, the delay adjustment unit 28 adjusts the delay time of the reverberation sound control signal and outputs it to the speaker 61A. Note that if there are multiple speakers, a matrix mixer 26 may be provided, similar to the sound signal processor 10 described above. The matrix mixer 26 distributes the reverberation sound control signal to the multiple speakers and outputs it.

[0176] With this configuration and method, the sound signal processing unit 10H can appropriately generate a reverberation control signal as well as an early reflection control signal, and can reproduce a desired sound field with a richer sound image and spatial expansion.

[0177] (3-6) One embodiment of the third embodiment of the present invention is a sound signal processing method that collects an output sound including a sound signal and generates a reverberation sound signal using the collected sound signal. That is, the sound signal processing unit collects a sound output from a speaker, feeds it back, and generates a reverberation sound signal from the collected sound signal.

[0178] With this configuration and method, the sound signal processing section can generate a reverberation signal that corresponds to the room 62B during performance, and can realize a desired sound field with a richer sound image and a wider spatial spread.

[0179] (3-7) An embodiment of the third embodiment of the present invention is a sound signal processing method in which volume control for reverberation is performed on the reverberation signal immediately before or immediately after the generation of the reverberation signal.

[0180] This configuration and method enable the sound signal processing unit to appropriately adjust the level of reverberation sound, thereby enabling the sound signal processing unit to appropriately adjust, for example, the level balance between early reflection sound and reverberation sound, and the level balance between direct sound and reverberation sound.

[0181] (3-8) An embodiment of the third embodiment of the present invention is a sound signal processing method in which volume control for early reflection sounds is performed on the early reflection sound control signal immediately before or immediately after generation of the early reflection sound control signal.

[0182] This configuration and method enable the sound signal processing unit to appropriately adjust the level of the early reflection sound, thereby enabling the sound signal processing unit to appropriately adjust, for example, the level balance between the early reflection sound and the reverberant sound, and the level balance between the direct sound and the early reflection sound.

[0183] (3-9) An embodiment of the third embodiment of the present invention is a sound signal processing method that outputs a sound signal and an early reflection sound control signal together.

[0184] With this configuration and method, the sound signal processing unit can output the direct sound and the early reflected sound through the same (single) output system.

[0185] The description of the present embodiment is illustrative in all respects and is not restrictive. The scope of the present invention is defined not by the above-described embodiments but by the claims. Furthermore, the scope of the present invention is intended to include all modifications that are equivalent to the claims and fall within the scope thereof. [Explanation of symbols]

[0186] 1, 1A, 1B...Sound field support system 10, 10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H...Sound signal processing section 11A, 11B, 11C...directional microphone 12A, 12B, 12C...Omnidirectional microphone 13A, 13B, 13C, 13D...directional microphones 14A, 14B, 14C, 14D...directional microphones 21, 21A, 21B...Sound signal acquisition section 21D, 21E, 21F, 21G, 21H...Line input section 22, 22E, 22F, 22G, 22H...Gain adjustment section 23, 23F, 23G...Mixer 24A...FIR filter 24B...FIR filter 25A...Level setting section 25B...Level setting section 26...Matrix mixer 27...Output section 28...Delay adjustment section 31...Memory 51A, 51B, 51C, 51D...Speakers 52A, 52B, 52C, 52D...Speakers 53A, 53B, 53C, 53D...Speakers 60...Stage 61, 611B, 612B, 613B...sound source 61A, 61B, 61C, 61D, 61E, 61F...Speakers Room 62, 62B... 151, 151A...Impulse response acquisition section 152...Level balance adjustment section 153...Level balance adjustment section 204A...Processing section 210...sound signal acquisition unit 211, 212...Level setting section 213...Synthesis section 214... Early reflection sound control signal generation unit 219...Reverberation control signal generation unit 230...Mixer 510D...directional microphone 620…Space

Claims

1. Acquire multiple sound signals picked up by multiple microphones arranged in a specified space, adjusting the levels of the plurality of sound signals in accordance with the respective positions of the plurality of microphones; mixing the adjusted sound signals into a first mixed signal of a first system and a second mixed signal of a second system; generating a reverberant sound using the first mixed signal; generating early reflection sounds using the second mixed signal; A sound signal processing method, comprising: adjusting the levels of the plurality of sound signals to be mixed as the second mixed signal in accordance with the distances from the respective positions of the plurality of microphones to a speaker that outputs the early reflected sounds; a plurality of the speakers are arranged on each of the left and right wall surfaces when the predetermined space is viewed from above; the plurality of microphones are arranged along a left-right direction when the predetermined space is viewed in a plan view, In the level adjustment, a gain for each of the plurality of sound signals to be mixed as the second mixed signal is set to be higher in ascending order of distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the early reflection sound in the left-right direction. Sound signal processing method.

2. performing delay adjustment according to the distance from each of the plurality of microphones to a speaker that outputs the early reflected sounds; The sound signal processing method according to claim 1 .

3. The delay time is set to be longer as the distance from each of the arrangement positions of the plurality of microphones to the speaker that outputs the early reflected sound increases. The sound signal processing method according to claim 2 .

4. an acquisition unit that acquires a plurality of sound signals picked up by a plurality of microphones arranged in a predetermined space; a gain adjustment unit that adjusts the levels of the plurality of sound signals in accordance with the respective positions of the plurality of microphones; a mixer that mixes the adjusted sound signals into a first mixed signal of a first system and a second mixed signal of a second system; a reverberation sound generator that generates reverberation sound using the first mixed signal; an early reflection sound generating unit that generates early reflection sounds using the second mixed signal. A sound signal processing device, the gain adjustment unit adjusts the levels of the plurality of sound signals in accordance with the distances from the respective positions of the plurality of microphones to a speaker that outputs the early reflection sounds; a plurality of the speakers are arranged on each of the left and right wall surfaces when the predetermined space is viewed from above; the plurality of microphones are arranged along a left-right direction when the predetermined space is viewed in a plan view, In the level adjustment, a gain for each of the plurality of sound signals to be mixed as the second mixed signal is set to be higher in ascending order of distance from the arrangement position of each of the plurality of microphones to the speaker that outputs the early reflection sound in the left-right direction. Sound signal processing device.

5. a delay adjusting unit that adjusts a delay in accordance with a distance from each of the plurality of microphones to a speaker that outputs the early reflected sounds; The sound signal processing device according to claim 4 .

6. the delay adjustment unit sets a delay time that increases as the distance from each of the plurality of microphones to the speaker that outputs the early reflected sound increases; The sound signal processing device according to claim 5 .

Images