Sound signal processing method and sound signal processing device

Abstract

To clearly reproduce sound image localization in a virtual space.SOLUTION: In a sound signal processing method, a virtual sound source indicating a reflected sound of a target acoustic space is classified into a first virtual sound source that is located between a position of a speaker and a position of a sound receiving point, and indicating a first sound source of the reflected sound of the target acoustic space and a second virtual sound source that is located outside the speaker and indicates a second sound source of the reflected sound in the target acoustic space, and only in the case of the first virtual sound source, a position of the first virtual sound source is moved to a position in which the reproduction can be performed using the position of the speaker near the first virtual sound source.SELECTED DRAWING: Figure 18

Description

[Technical field]

[0001] An embodiment of the present invention relates to a sound signal processing method and a sound signal processing device for performing a predetermined process on a sound input from a sound source. [Background technology]

[0002] In an acoustic system for a space such as a hall, a sound image is localized relative to a sound source by speakers arranged in the space.

[0003] For example, the audio processing device described in Patent Document 1 outputs the sound of an audio object (sound source) from two or more speakers in the vicinity of the audio object (sound source). At this time, the audio processing device described in Patent Document 1 calculates the gain of the audio signal to be output to each speaker using position information and sound image information of the audio object (sound source). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2016 / 208406 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the above-mentioned conventional configuration, it is not possible to clearly reproduce the sound image localization in the virtual space in which the speakers are installed.

[0006] Therefore, an object of one embodiment of the present invention is to clearly reproduce sound image localization in a virtual space. [Means for solving the problem]

[0007] The sound signal processing method classifies virtual sound sources indicating reflected sound in a target acoustic space into a first virtual sound source that is located between the position of the speaker and the position of the sound receiving point and indicates a first sound source of reflected sound in the target acoustic space, and a second virtual sound source that is located outside the speaker and indicates a second sound source of reflected sound in the target acoustic space, and only in the case of the first virtual sound source, the position of the first virtual sound source is moved to a position where it can be reproduced using the position of a speaker near the first virtual sound source. Effect of the Invention

[0008] The sound signal processing method can clearly reproduce the sound image localization in a virtual space. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a functional block diagram showing the configuration of an audio system including a sound signal processing device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flowchart of a sound signal processing method according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing discrete waveforms of a typical sound including a direct sound, an early reflection sound, and a reverberation sound (late reverberation sound). [Figure 4] 4(A) and 4(B) are diagrams showing the concept of setting a virtual sound source. [Diagram 5] FIG. 5 is a functional block diagram showing an example of the configuration of grouping section 40. As shown in FIG. [Figure 6] FIG. 6 is a flowchart showing a method for grouping sound sources. [Figure 7] FIG. 7 is a diagram showing the concept of grouping multiple sound sources into multiple regions. [Figure 8] FIG. 8(A) is a flowchart showing a method for grouping sound sources using representative points, and FIG. 8(B) is a flowchart showing a method for grouping sound sources using area boundaries. [Figure 9] FIG. 9 is a flowchart showing an example of a grouping method based on the movement of a sound source. [Figure 10]FIG. 10 is a functional block diagram showing an example of the configuration of the early reflection sound control signal generating unit 50. As shown in FIG. [Figure 11] FIG. 11 is a diagram illustrating an example of the GUI. [Figure 12] FIG. 12 is a flowchart showing an example of a process for setting a virtual sound source. [Figure 13] 13(A) and 13(B) are diagrams showing examples of setting imaginary sound sources when the geometric shapes are different. [Figure 14] 14(A), 14(B), and 14(C) are diagrams showing examples of setting an imaginary sound source. [Figure 15] 15(A), 15(B), and 15(C) are diagrams showing examples of setting an imaginary sound source. [Figure 16] FIG. 16 is a flowchart showing the process of assigning imaginary sound sources to speakers. [Figure 17] 17(A) and 17(B) are diagrams showing the concept of allocating imaginary sound sources to speakers. [Figure 18] FIG. 18 is a flowchart showing the coefficient setting process of LDtap. [Figure 19] 19(A) and 19(B) are diagrams for explaining the concept of coefficient setting. [Figure 20] FIG. 20(A) shows an example of the LDtap coefficient when the virtual space shape is large, and FIG. 20(B) shows an example of the LDtap coefficient when the virtual space shape is small. [Figure 21] FIG. 21 is a diagram showing the waveform of the early reflection sound control signal generated by the early reflection sound control signal generating unit 50. As shown in FIG. [Figure 22] FIG. 22 is a functional block diagram showing an example of the configuration of the reverberation control signal generating unit 70. As shown in FIG. [Figure 23] FIG. 23 is a flowchart showing an example of a process for generating a reverberation control signal. [Figure 24] FIG. 24 is a graph showing example waveforms of a direct sound, an early reflection sound control signal, and a reverberation sound control signal. [Diagram 25] FIG. 25 is a diagram showing an example of region setting for reverberation sounds. [Figure 26] FIG. 26 is a functional block diagram showing an example of the configuration of the output adjustment unit 90. As shown in FIG. [Figure 27] FIG. 27 is a flowchart showing an example of the output adjustment process. [Figure 28] FIG. 28 is a diagram showing an example of a GUI for adjusting the output. [Figure 29] 29(A) and 29(B) are diagrams showing example settings for localizing and spreading sound toward the rear of the reproduction space. [Diagram 30] 30(A) and 30(B) are diagrams showing setting examples in which sound is localized and spread in the horizontal direction of the reproduction space. [Diagram 31] FIG. 31 is a diagram showing an image of sound spread when there is a spread in the height direction. [Diagram 32] FIG. 32 is a functional block diagram showing the configuration of a sound signal processing device with a binaural reproduction function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A sound signal processing method and a sound signal processing device according to an embodiment of the present invention will be described with reference to the drawings. Note that in the following embodiment, an overview of the sound signal processing method and the sound signal processing device will be described first, and then the specific content of each process and each configuration will be described.

[0011] In this embodiment, the reproduction space is a space where a user (listener) hears sounds (direct sound, early reflection sound, reverberation sound) from a sound source using a speaker, etc. The virtual space is a space that has a sound field (acoustics) different from the reproduction space, and is a space where the early reflection sound and reverberation sound due to this sound field are reproduced (simulated) in the reproduction space.

[0012] [Outline of the sound signal processing device] FIG. 1 is a functional block diagram showing the configuration of an audio system including a sound signal processing device according to an embodiment of the present invention.

[0013] 1, the sound signal processing device 10 includes an area setting unit 30, a grouping unit 40, an early reflection sound control signal generating unit 50, a mixer 60, a reverberation sound control signal generating unit 70, an adder 80, and an output adjustment unit 90. The sound signal processing device 10 is realized, for example, by electronic circuits that respectively realize the area setting unit 30, the grouping unit 40, the early reflection sound control signal generating unit 50, the mixer 60, the reverberation sound control signal generating unit 70, the adder 80, and the output adjustment unit 90, or by an arithmetic processing device such as a computer. The portion consisting of the adder 80 and the output adjustment unit 90 corresponds to the "output signal generating unit" of the present invention.

[0014] The sound signal processing device 10 is connected to a plurality of speakers SP1 to SP64. Although Fig. 1 shows an embodiment using 64 speakers, the number of speakers is not limited to this.

[0015] Sound signals S1-S96 of a plurality of sound sources OBJ1-OBJ96 are input to the sound signal processing device 10. Note that, although Fig. 1 shows an embodiment in which 96 sound sources are used, the number of sound sources is not limited to this.

[0016] The region setting unit 30 divides the reproduction space into a plurality of regions and sets information (region information) about the divided regions. The region information includes position coordinates that determine the boundaries of the regions and position coordinates of representative points set in the regions.

[0017] The area setting unit 30 outputs area information on the multiple areas Area1 to Area8 that have been set to the grouping unit 40. Note that, although Fig. 1 shows an embodiment in which eight areas are set, the number of areas is not limited to this.

[0018] The grouping unit 40 groups the sound sources OBJ1-OBJ96 into a plurality of areas Area1-Area8. Based on the grouping results, the grouping unit 40 generates area-specific sound signals SA1-SA8 for each area Area1-Area8 using the sound signals S1-S96 of the sound sources OBJ1-OBJ96. For example, the grouping unit 40 mixes the sound signals of the plurality of sound sources grouped into the area Area1 to generate the area-specific sound signal SA1.

[0019] The grouping unit 40 outputs the multiple area sound signals SA1-SA8 to the early reflection sound control signal generating unit 50. The grouping unit 40 also outputs the sound signals S1-S96 of the sound sources OBJ1-OBJ96 to the mixer 60.

[0020] The early reflection sound control signal generator 50 generates early reflection sound control signals ER1-ER64 for each of the speakers SP1-SP64 from the multiple area sound signals SA1-SA8. The early reflection sound control signals ER1-ER64 are signals output to the speakers SP1-SP64, respectively, to simulate the early reflection sounds in the virtual space in the reproduction space. The early reflection sound control signal generator 50 outputs the generated early reflection sound control signals ER1-ER64 to the adder 80.

[0021] In general (detailed configuration and processing will be described later), the early reflection sound control signal generation unit 50 sets an imaginary sound source (virtual sound source) in the reproduction space using the positions of the speakers SP1-SP64 arranged in the reproduction space and the geometric shape of the virtual space. Specific settings of the imaginary sound source will be described later. By using the imaginary sound source, the early reflection sound control signal generation unit 50 generates early reflection sound control signals ER1-ER64 that simulate early reflection sounds in the virtual space. At this time, the early reflection sound control signal generation unit 50 performs desired tone adjustments on the early reflection sound control signals ER1-ER64.

[0022] The mixer 60 is a summing mixer. The mixer 60 mixes the sound signals S1-S96 of the sound sources OBJ1-OBJ96 to generate a reverberation sound generating signal Sr. The mixer 60 outputs the reverberation sound generating signal Sr to the reverberation sound control signal generating unit 70.

[0023] The reverberation sound control signal generator 70 generates reverberation sound control signals REV1-REV64 for each of the multiple speakers SP1-SP64 from the reverberation sound generation signal Sr. The reverberation sound control signals REV1-REV64 are signals output to the speakers SP1-SP64, respectively, to simulate the reverberation sound (rear reverberation sound) in the virtual space in the reproduction space. The reverberation sound control signal generator 70 outputs the generated reverberation sound control signals REV1-REV64 to the adder 80.

[0024] In general (detailed configuration and processing will be described later), the reverberation sound control signal generator 70 divides the reproduction space into a plurality of reverberation sound setting regions, and generates a reverberation sound control signal for each of the plurality of reverberation sound setting regions. The reverberation sound control signal generator 70 assigns a plurality of speakers SP1-SP64 to the plurality of reverberation sound setting regions. Based on this assignment, the reverberation sound control signal generator 70 sets a reverberation sound control signal for each of the reverberation sound setting regions to the plurality of speakers SP1-SP64.

[0025] At this time, the reverberation sound control signal generating unit 70 sets the connection timing between the early reflection sound and the reverberation sound based on the geometric shape of the reproduction space. The reverberation sound control signal generating unit 70 gradually increases the level (amplitude) of the reverberation sound control signal in the period before the connection timing, and gradually decreases the level (amplitude) of the reverberation sound control signal in the period after the connection timing.

[0026] The adder 80 adds the early reflection sound control signal and the reverberation sound control signal generated for each of the multiple speakers SP1-SP64 to generate multiple speaker signals Sat1-Sat64. For example, the adder 80 adds the early reflection sound control signal for the speaker SP1 to the reverberation sound control signal for the speaker SP1 to generate the speaker signal Sat1. The adder 80 outputs the multiple speaker signals Sat1-Sat64 to the output adjustment unit 90.

[0027] The output adjustment unit 90 performs gain control and delay control on the speaker signals Sat1-Sat64 to generate output signals So1-So64. The output adjustment unit 90 outputs the output signals So1-So64 to the speakers SP1-SP64. For example, the output adjustment unit 90 performs gain control and delay control for the speaker SP1 on the speaker signal Sat1 to generate the output signal So1. The output adjustment unit 90 outputs the output signal So1 to the speaker SP1.

[0028] Schematically (detailed configuration and processing will be described later), the output adjustment unit 90 accepts input of acoustic parameters in the reproduction space. The acoustic parameters are parameters for setting, for example, adjustment of the widthwise spatial spread in the sound space, adjustment of the spatial spread behind the sound receiving point in the sound space, adjustment of the spatial spread in the ceiling direction in the sound space, and the like. The output adjustment unit 90 collectively sets the gain values ​​and delay amounts (delay amounts) of the signals Sat1-Sat64 for the multiple speakers based on the position coordinates and acoustic parameters of the multiple speakers SP1-SP64. Setting collectively does not mean setting them individually for each speaker, but means setting the gain values ​​and delay amounts of each speaker by simply inputting the position coordinates of each speaker into a specific calculation formula common to all speakers. The output adjustment unit 90 performs gain control and delay control on the signals Sat1-Sat64 for the multiple speakers using the set gain values ​​and delay values.

[0029] [Outline of sound signal processing method] Fig. 2 is a flowchart of a sound signal processing method according to an embodiment of the present invention. Fig. 2 shows a sound signal processing method realized by the sound signal processing device 10 of Fig. 1. Note that the contents of each process shown in Fig. 2 have been explained in the explanation of Fig. 1 above, so they will be described briefly.

[0030] (Grouping of sound sources OBJ1-OBJ96) The grouping unit 40 groups the multiple sound sources OBJ1-OBJ96 into multiple areas Area1-Area8 (S11).

[0031] (Generation of early reflection control signals) The early reflection sound control signal generating unit 50 sets a tone color for the early reflection sound for each group (SS12). The early reflection sound control signal generating unit 50 sets an imaginary sound source for each group (S13). The early reflection sound control signal generating unit 50 uses the tone color and the imaginary sound source to generate an early reflection sound control signal for each of the multiple speakers SP1-SP64 (S14).

[0032] (Generation of reverberation control signals) The mixer 60 sums the sound signals S1-S96 of the multiple sound sources OBJ1-OBJ96 (S21). The reverberation sound control signal generator 70 sets the connection timing between the early reflection sound and the reverberation sound based on the geometric shape of the reproduction space (S22). The reverberation sound control signal generator 70 generates a reverberation sound control signal using the set connection timing (S23). The reverberation sound control signal generator 70 assigns the generated reverberation sound control signal to the multiple speakers SP1-SP64 based on the position coordinates of the multiple speakers SP1-SP64 in the reproduction space (S24).

[0033] (Output processing to multiple speakers) The adder 80 adds the early reflection sound control signal and the reverberation sound control signal for each of the multiple speakers SP1-SP64 to generate speaker signals Sat1-Sat64 (S31).

[0034] The output adjustment unit 90 generates output signals So1-So64 from the speaker signals Sat1-Sat64 using acoustic parameters that realize the positioning of reverberation in the reproduction space and the spread of the space (S32). The output adjustment unit 90 outputs the output signals So1-So64 to the multiple speakers SP1-SP64 (S33).

[0035] By using the above-mentioned configuration and processing, the sound signal processing device 10 (sound signal processing method) can obtain the following various effects.

[0036] (1) The sound signal processing device 10 (sound signal processing method) groups sound sources for each region obtained by dividing the reproduction space and generates early reflection sounds, thereby realizing clear sound image localization and a rich spatial spread. In this case, the reverberation sound is constant throughout the reproduction space, and only the early reflection sounds change depending on the position of the sound source. Therefore, for example, when the position of the sound source moves, the movement of the sound of this sound source becomes smoother.

[0037] (2) The sound signal processing device 10 (sound signal processing method) generates an early reflection sound control signal using an imaginary sound source, thereby making it possible to more faithfully simulate in the reproduction space the early reflection sounds due to the geometric shape of the virtual space.

[0038] (3) The sound signal processing device 10 (sound signal processing method) adjusts the tone of the early reflection sound control signal, thereby making it possible to eliminate unnaturalness in the tone of the early reflection sound simulated only by an imaginary sound source, for example.

[0039] (4) The sound signal processing device 10 (sound signal processing method) can make the connection from the early reflection sound to the reverberation sound smoother and more natural by setting the connection timing between the early reflection sound control signal and the reverberation sound control signal based on the geometric shape of the reproduction space.

[0040] (5) The sound signal processing device 10 (sound signal processing method) adjusts the gain values ​​and delay amounts of speaker signals Sat1-Sat64, which include early reflection sound control signals and reverberation sound control signals, all at once, thereby enabling the user to realize the sound field desired in the playback space with easier operational input.

[0041] [Detailed explanation of each signal processing unit and each process] The above-mentioned signal processing units and the processes will be described in detail below. First, the early reflection sound, reverberation sound, and imaginary sound source, which are necessary for understanding the present invention, will be described with reference to the drawings.

[0042] [Early reflections and reverberation] Fig. 3 is a diagram showing the discrete waveforms of a typical sound including a direct sound, an early reflection sound, and a reverberation sound (a late reverberation sound). For example, a hall in which a performance or content is played has a closed space surrounded by walls. When a sound is generated in this closed space, the direct sound, the early reflection sound, and the reverberation sound (a late reverberation sound) arrive at the sound receiving point.

[0043] Direct sound is sound that arrives directly from the sound source to the sound receiving point.

[0044] An early reflected sound is a sound that is generated at the source position and reflects off the walls, floor, or ceiling before reaching the sound receiving point. For this reason, early reflected sound arrives at the sound receiving point after the direct sound. The volume (level) of the early reflected sound is smaller than that of the direct sound. If the sound is reflected once, it is a first reflected sound, and if it is reflected n times, it is an nth reflected sound. The direction from which the early reflected sound arrives at the sound receiving point and its volume are greatly affected by the source position of the sound.

[0045] Reverberant sound arrives at the sound receiving point after the initial reflection sound. Reverberant sound is sound that arrives at the sound receiving point after multiple reflections of sound generated at the generation position. In other words, reverberant sound is sound that arrives at the sound receiving point after being further reflected many times and attenuated. Therefore, the volume (level) of reverberant sound is smaller than that of early reflection sound. Furthermore, the arrival direction and volume of reverberant sound are less affected by the sound generation position than those of early reflection sound.

[0046] [imaginary sound source] Figures 4(A) and 4(B) are diagrams showing the concept of setting a virtual sound source. In Figures 4(A) and 4(B), the concept of setting a virtual sound source in two dimensions is shown for ease of explanation, but the virtual sound source can be set in three dimensions using the same concept. That is, in the actual reproduction space, when the sound sources are not aligned on a single plane but are spatially arranged, and the virtual space is set three-dimensionally, the virtual sound source is set in three dimensions.

[0047] In the reproduction space, there exists a sound source SS and a sound receiving point RP. Note that the sound source SS shown in Fig. 4(A) and Fig. 4(B) has a different meaning from the sound source OBJ described above, and refers to something that generates a general sound. In addition, in the reproduction space, a virtual wall IWL that realizes the sound field of the virtual space is set. The virtual wall IW is obtained from the geometric shape of the virtual space.

[0048] The sound source SS and the sound receiving point RP exist in a space surrounded by the virtual wall IWL. The virtual wall IWL includes a virtual wall IWL1, a virtual wall IWL2, a virtual wall IWL3, and a virtual wall IWL4. The virtual walls IWL1 and IWL4 are arranged to sandwich the sound source SS and the sound receiving point RP in the first direction (vertical direction in Fig. 4(A) and Fig. 4(B)) of the reproduction space. The virtual wall IWL1 is arranged closer to the sound source SS than the sound receiving point RP, and the virtual wall IWL4 is arranged closer to the sound receiving point RP than the sound source SS. The virtual walls IWL2 and IWL3 are arranged to sandwich the sound source SS and the sound receiving point RP in the second direction (horizontal direction in Fig. 4(A) and Fig. 4(B)) of the reproduction space. The virtual wall IWL2 is arranged closer to the sound source SS than the sound receiving point RP, and the virtual wall IWL3 is arranged closer to the sound receiving point RP than the sound source SS.

[0049] If virtual walls IWL1, IWL2, IWL3, and IWL4 were actually walls that reflected sound, the sound emitted from the sound source SS would be reflected by virtual walls IWL1, IWL2, and IWL3 and reach the sound receiving point RP, as shown in Fig. 4(B). Note that, although reflection from virtual wall IWL4 is not shown in Fig. 4(B), reflection also occurs from virtual wall IWL4 in the same way as from virtual walls IWL1, IWL2, and IWL3.

[0050] However, the virtual walls IWL1, IWL2, IWL3, and IWL4 do not actually exist in the reproduction space. Therefore, as shown in Fig. 4(A), the sound signal processing device 10 sets imaginary sound sources IS1, IS2, and IS3 by regarding the reflection of sound on the wall surfaces as specular reflection.

[0051] Specifically, the sound signal processing device 10 sets an imaginary sound source IS1 at a position that is line-symmetrical with respect to the sound source SS, with the virtual wall IWL1 as a reference line. The sound signal processing device 10 sets an imaginary sound source IS2 at a position that is line-symmetrical with respect to the sound source SS, with the virtual wall IWL2 as a reference line. The sound signal processing device 10 sets an imaginary sound source IS3 at a position that is line-symmetrical with respect to the sound source SS, with the virtual wall IWL3 as a reference line. Note that by adjusting the acoustic power of each imaginary sound source IS, it is possible to simulate the energy loss due to reflection at the virtual wall IWL.

[0052] By making such settings, the sound generated by imaginary sound source IS1 becomes the same as the sound generated by sound source SS and reflected by virtual wall IW1. The sound generated by imaginary sound source IS2 becomes the same as the sound generated by sound source SS and reflected by virtual wall IW2. The sound generated by imaginary sound source IS3 becomes the same as the sound generated by sound source SS and reflected by virtual wall IW3. Note that although a virtual sound source for virtual wall IWL4 is not shown in Figures 4(A) and 4(B), a virtual sound source can be set for virtual wall IWL4 in the same way as for virtual walls IWL1, IWL2, and IWL3.

[0053] By setting a virtual sound source in this manner, the sound signal processing device 10 can simulate the early reflected sound of a virtual space in a reproduction space where no real walls exist in the virtual space.

[0054] [Configuration and processing of grouping unit 40] Fig. 5 is a functional block diagram showing an example of the configuration of the grouping section 40. Fig. 6 is a flowchart showing a method of grouping sound sources.

[0055] As shown in FIG. 5, the grouping unit 40 includes a sound source position detection unit 41, a region determination unit 42, and a matrix mixer 400.

[0056] The sound source position detection unit 41 detects the position coordinates of multiple sound sources OBJ1-OBJ96 in the reproduction space (FIG. 6: S111). For example, the sound source position detection unit 41 detects the position coordinates of the sound sources OBJ1-OBJ96 by an operational input from the user. Alternatively, the sound source position detection unit 41 includes a position detection sensor for detecting the sound sources OBJ1-OBJ96, and detects the position coordinates of the sound sources OBJ1-OBJ96 by the positions detected by the position detection sensor.

[0057] The sound source position detection unit 41 outputs the position coordinates of the sound sources OBJ1-OBJ96 to the area determination unit .

[0058] The area determination unit 42 groups the sound sources OBJ1-OBJ96 into multiple areas Area1-Area8 using the area information of the multiple areas Area1-Area8 from the area setting unit 30 and the position coordinates of the sound sources OBJ1-OBJ96 from the sound source position detection unit 41 (FIG. 6: S112). More specifically, the area determination unit 42 performs grouping as follows.

[0059] Fig. 7 is a diagram showing the concept of grouping multiple sound sources into multiple regions. In Fig. 7, the upper part of the figure is the front of the hall, which is the reproduction space, and the lower part of the figure is the rear of the hall.

[0060] The region setting unit 30 sets a reference point Pso for region division for the reproduction space. For example, as shown in FIG. 7, the region setting unit 30 sets the center position of a hall that realizes the reproduction space as the reference point Pso. Note that the region setting unit 30 can also use a point (position) set by the user as the reference point. For example, the region setting unit 30 can use a sound receiving point or the like set by the user as the reference point.

[0061] The area setting unit 30 sets eight areas, Area1-Area8, so as to divide the entire circumference on a plane into eight parts centered on the reference point Pso for area division. For example, in the case of FIG. 7, the area setting unit 30 sets multiple areas, Area1, Area2, and Area3, in front of the reference point Pso in the hall (reproduction space). The area setting unit 30 also sets an area, Area4, to the left of the reference point Pso facing forward of the hall, and sets an area, Area5, to the right of the reference point Pso facing forward of the hall. The area setting unit 30 also sets multiple areas, Area6, Area7, and Area8, in the rear of the reference point Pso in the hall (reproduction space).

[0062] Note that this area setting is just an example, and other settings may be used as long as the entire reproduction space is covered by the multiple areas that are set. Also, this description shows the setting of planar areas, but spatial areas can be set in the same manner. For example, the vertical range of area Area1 is also included in area Area1.

[0063] The area setting unit 30 sets representative points RP1-RP8 in each of the areas Area1-Area8. For example, the area setting unit 30 sets the representative points RP1-RP8 at the center positions of the areas Area1-Area8. Alternatively, in the case of an area spreading radially as shown in FIG. 7, for example, the area setting unit 30 sets a representative point at a position a predetermined distance from the reference point Pso on a straight line passing through the center of the corner spreading radially. Note that the method of setting these representative points is merely an example, and other methods may be used as long as one representative point can be set in one area and the sound source grouping process can be reliably performed.

[0064] The area setting unit 30 outputs area information of the multiple areas Area1-Area8 to the area determination unit 42 of the grouping unit 40 and the matrix mixer 400. The area information of the multiple areas Area1-Area8 includes position coordinates of representative points RP1-RP8 of the areas Area1-Area8, coordinate information indicating the boundary lines that define the shapes of the areas Area1-Area8, and the like.

[0065] (Method of grouping sound sources into regions using representative points) FIG. 8A is a flowchart showing a method for grouping sound sources using representative points.

[0066] The area determination unit 42 acquires the position coordinates of the representative points RP1-RP8 from the area information of the multiple areas Area1-Area8 (S131). The area determination unit 42 calculates the distance between the position coordinates of the sound source to be determined for grouping and the position coordinates of the representative points RP1-RP8 (S132). The area determination unit 42 groups the sound sources into an area including the representative point that is the shortest distance away (S133).

[0067] For example, in the case of sound source OBJ1 in the example of FIG. 7, the area determination unit 42 detects the position coordinates of the sound source OBJ1 and obtains the position coordinates of the representative points RP1-RP8. The area determination unit 42 calculates the distance between the sound source OBJ1 and the representative points RP1-RP8 from the position coordinates of the sound source OBJ1 and the position coordinates of the representative points RP1-RP8. The area determination unit 42 detects that the distance between the sound source OBJ1 and the representative point RP1 is shorter than the distance between the sound source OBJ1 and the other representative points RP2-RP8. In other words, the area determination unit 42 detects that the distance between the sound source OBJ1 and the representative point RP1 is the shortest distance. The area determination unit 42 groups the sound source OBJ1 into the area Area1 associated with the representative point RP1.

[0068] (Method of grouping sound sources into regions using region boundaries) FIG. 8B is a flowchart showing a method for grouping sound sources using region boundaries.

[0069] The area determination unit 42 acquires coordinate information (boundary coordinates) representing the boundary line of each of the areas Area1-Area8 from the area information of the areas Area1-Area8 (S136). The area determination unit 42 determines whether the position coordinates of the sound source to be determined for grouping are inside each of the areas Area1-Area8 (S137). For example, the area determination unit 42 uses a crossing number algorithm to determine whether the sound source is inside or outside the area. If the sound source is inside the area (S137: YES), the area determination unit 42 groups the sound source into this area (S138).

[0070] For example, in the case of sound source OBJ1 in the example of Fig. 7, the area determination unit 42 detects the position coordinates of the sound source OBJ1 and acquires coordinate information (boundary coordinates) indicating the boundary lines of the multiple areas Area1-Area8. The area determination unit 42 determines whether the sound source OBJ1 is inside or outside the multiple areas Area1-Area8 from the position coordinates of the sound source OBJ1 and the boundary coordinates of the multiple areas Area1-Area8. The area determination unit 42 detects that the sound source OBJ1 is inside the area Area1. The area determination unit 42 groups the sound source OBJ1 into the area Area1.

[0071] The area determination unit 42 groups the multiple input sound sources OBJ1-OBJ96 into multiple areas Area1-Area 8. For example, in the example of Fig. 7, the area determination unit 42 groups sound sources OBJ1 and OBJ4 into area Area1, groups sound source OBJ2 into area Area2, and groups sound source OBJ3 into area Area5.

[0072] The area determination unit 42 outputs the grouping information to the matrix mixer 400. The grouping information is the above-mentioned information indicating which sound source is grouped into which area.

[0073] The matrix mixer 400 generates area-specific sound signals SA1-SA8 for each of the areas Area1-Area8 using sound signals S1-S96 of the sound sources OBJ1-OBJ96 based on the grouping information. For example, if multiple sound sources are grouped in an area, the matrix mixer 400 mixes the sound signals of the multiple sound sources to generate an area-specific sound signal for the area. The matrix mixer 400 outputs the area-specific sound signal for each area to the early reflection sound control signal generation unit 50. Note that if at least one sound source is grouped in an area, the matrix mixer 400 outputs the sound signal of this sound source to the early reflection sound control signal generation unit 50 as the area-specific sound signal for the area.

[0074] In the example of FIG. 7, sound sources OBJ1 and OBJ4 are grouped in area Area1. The matrix mixer 400 mixes the sound signal S1 of sound source OBJ1 and the sound signal S4 of sound source OBJ4 to generate and output an area-specific sound signal SA1 for area Area1. Moreover, sound source OBJ2 is grouped in area Area2. The matrix mixer 400 outputs the sound signal S2 of sound source OBJ2 as an area-specific sound signal SA2 for area Area2. Moreover, sound source OBJ3 is grouped in area Area5. The matrix mixer 400 outputs the sound signal S3 of sound source OBJ3 as an area-specific sound signal SA5 for area Area5.

[0075] By implementing such a configuration and processing, the sound signal processing device 10 can group a plurality of sound sources for each of a plurality of regions that divide the sound space, and generate an early reflection sound control signal. As a result, the sound signal processing device 10 can reproduce early reflection sounds according to the positions of the sound sources, and can achieve clear sound image localization and a rich spatial spread.

[0076] Although the above description does not specifically describe the case where the sound source moves, when the sound source moves, the grouping unit 40 performs the process shown in Fig. 9. Fig. 9 is a flowchart showing an example of a grouping method based on the movement of the sound source.

[0077] The sound source position detection unit 41 detects the movement of the sound source (S104). For example, the sound source position detection unit 41 detects the movement of the sound source by an operation input from the user. Alternatively, the sound source position detection unit 41 detects the movement of the sound source by continuously detecting the sound source position using a position detection sensor. The sound source position detection unit 41 detects the position coordinates of the sound source after the movement and outputs them to the area determination unit 42.

[0078] The area determination unit 42 uses the position coordinates of the sound source after the movement to perform grouping into a plurality of areas Area1 to Area8, as described above.

[0079] By performing such processing, the sound signal processing device 10 can generate an early reflection sound control signal according to the position of the sound source after the sound source moves, even if the sound source moves. As a result, the sound signal processing device 10 can reproduce the change in the early reflection sound according to the movement of the sound source, and can realize clear sound image localization and rich spatial expansion according to the movement, even if the sound source moves.

[0080] Furthermore, when such a sound source movement occurs, the sound signal processing device 10 can perform cross-fading processing on the early reflection sound control signal before the movement and the early reflection sound control signal after the movement. For example, when a sound source moves, the sound signal processing device 10 gradually lowers the sound signal component of the sound source in the regional sound signal including the sound source before the movement. On the other hand, the sound signal processing device 10 gradually raises the sound signal component of the sound source in the regional sound signal including the sound source after the movement.

[0081] By performing such processing, the sound signal processing device 10 can suppress discontinuous changes in the early reflected sounds when the sound source moves. As a result, when the sound source moves, the sound signal processing device 10 can change the early reflected sounds more smoothly in accordance with the movement of the sound source.

[0082] Furthermore, the matrix mixer 400 outputs sound signals S1-S96 of the multiple sound sources OBJ1-OBJ96 to the mixer 60. As described above, the mixer 60 sums the sound signals S1-S96 to generate a reverberation sound generation signal Sr and outputs the signal to the reverberation sound control signal generator 70. The reverberation sound control signal generator 70 uses the reverberation sound generation signal Sr to generate reverberation sound control signals REV1-REV64.

[0083] By performing such processing, the reverberation sound is not affected by the position or movement of the sound source. Therefore, the sound signal processing device 10 can reproduce the movement of the sound source more clearly by changing the early reflection sound while keeping the reverberation sound in the reproduction space constant even if the sound source moves.

[0084] [Generation of early reflection control signals] Fig. 10 is a functional block diagram showing an example of the configuration of the early reflection sound control signal generating unit 50. Fig. 11 is a diagram showing an example of a GUI.

[0085] 10, the early reflection sound control signal generation unit 50 includes an FIR filter circuit 51, an LDtap circuit 52, an addition processing unit 53, a tone color setting unit 501, an imaginary sound source setting unit 502, and an operation unit 500. The FIR filter circuit 51 includes a plurality of FIR filters 511-518. The LDtap circuit 52 includes a plurality of LDtap circuits 521-528, an output speaker setting unit 5201, and a coefficient setting unit 5202. The order of the FIR filter circuit 51 and the LDtap circuit 52 may be reversed.

[0086] [Early reflection sound tone adjustment] The operation unit 500 receives information designating a tone color to be added to the early reflection sound from a user, and outputs the information to the tone color setting unit 501. The tone color designation information is, for example, information designating emphasis on low-pitched sounds, emphasis on high-pitched sounds, the volume of the early reflection sound, the attenuation characteristics of the early reflection sound, etc. (information representing filter characteristics).

[0087] As a specific example, the operation unit 500 accepts operations through a GUI 100 (Graphical User Interface) as shown in FIG.

[0088] The GUI 110 includes a setting display window 111 , a plurality of controls 112 , a knob 1131 , and an adjustment value display window 1132 .

[0089] The setting display window 111 displays the shape of the virtual wall IWL of the virtual space that is set by the multiple operators 112 and knobs 1131. At this time, the setting display window 111 can display the position of the sound source SS, the position of the speaker SP, the position of the sound receiving point RP, and the coordinate axes of the reproduction space that are separately set, together with the virtual wall IWL.

[0090] The multiple operators 112 are associated with preset samples of virtual space (various halls, rooms, etc.). Although not shown in the figure, an index (e.g., hall name, etc.) that clearly indicates the sample of the virtual space associated with each operator 112 is displayed on each of the multiple operators 112.

[0091] The knob 1131 is for setting the room size. The adjustment value display window 1132 displays the setting value of the room size.

[0092] The GUI 100 accepts various operations for adjusting the tone color. For example, the GUI 100 includes a plurality of operators 112, such as an operator for a low range, an operator for a high range, an operator for adjusting the volume, an operator for adjusting the attenuation characteristics, and the like, and accepts operations using these operators.

[0093] When the user operates a desired operator using the GUI 100, the operation section 500 detects this operation and sets timbre designation information in response to this operation.

[0094] For example, when the operation unit 500 receives a selection of a plurality of operators 112, it acquires designation information of a tone color that is set in advance in the virtual space associated with the operator 112. Furthermore, when the operation unit 500 receives an operation using an operator for a low range, an operator for a high range, an operator for adjusting the volume, an operator for adjusting the attenuation characteristics, or the like, it acquires designation information of the tone color that is set by these operators.

[0095] Although not shown in the figures, the GUI 100 can also display the timbre designation information using, for example, filter coefficients of FIR filters 511-518 (described later), schematic waveforms, etc. In this case, when the GUI 100 receives an adjustment to the timbre designation information, it can also change the display in accordance with the adjustment. For example, the GUI 100 can change the display of the waveform in accordance with the adjustment.

[0096] The timbre setting unit 501 sets the filter coefficients of the FIR filters 511-518 of the FIR filter circuit 51 based on the timbre designation information. For example, when the timbre setting unit 501 receives designation information for emphasizing the low frequency range, it sets the filter coefficients of the FIR filters 511-518 of the FIR filter circuit 51 with the low frequency range boosted. Also, when the timbre setting unit 501 receives designation information for emphasizing the high frequency range, it sets the filter coefficients of the FIR filters 511-518 of the FIR filter circuit 51 with the high frequency range boosted. The timbre setting unit 501 outputs the set filter coefficients to the FIR filter circuit 51. Note that the timbre setting unit 501 is not limited to the filter coefficients, but can also set and adjust the sampling frequency and the filter length as filter characteristics.

[0097] Furthermore, the tone color setting unit 501 sets the gain value of each tap of the FIR filters 511-518 of the FIR filter circuit 51 based on tone color designation information. The tone color setting unit 501 outputs the set gain value to the FIR filter circuit 51.

[0098] The multiple FIR filters 511-518 are filters corresponding to the regional sound signals SA1-SA8, respectively. The regional sound signals SA1-SA8 are input to the FIR filters 511-518. For example, as shown in Fig. 10, the regional sound signal SA1 is input to the FIR filter 511, the regional sound signal SA2 is input to the FIR filter 512, the regional sound signal SA3 is input to the FIR filter 513, and the regional sound signal SA4 is input to the FIR filter 514. The regional sound signal SA5 is input to the FIR filter 515, the regional sound signal SA6 is input to the FIR filter 516, the regional sound signal SA7 is input to the FIR filter 517, and the regional sound signal SA8 is input to the FIR filter 518.

[0099] The multiple FIR filters 511-518 have the same number of taps. For example, the multiple FIR filters 511-518 have 16,000 taps. Note that this number of taps is just an example, and may be set based on the resource conditions of the sound signal processing device 10, the accuracy of the tone of the early reflection sound to be reproduced, etc.

[0100] The multiple FIR filters 511-518 perform filter processing (convolution operation) on the multiple regional sound signals SA1-SA8, respectively, using the filter coefficients and gain values ​​set by the tone color setting unit 501. As a result, the multiple FIR filters 511-518 generate post-filter regional sound signals SA1f-SA8f. For example, the FIR filter 511 performs filter processing (convolution operation) on the regional sound signal SA1 using the filter coefficients and gain values ​​set by the tone color setting unit 501, to generate post-filter regional sound signal SA1f. Similarly, the multiple FIR filters 512-518 individually generate post-filter regional sound signals SA2f-SA8f from the regional sound signals SA2-SA8.

[0101] The multiple FIR filters 511-518 output the filtered regional sound signals SA1f-SA8f to the multiple LD taps 521-528. For example, the FIR filter 511 outputs the filtered regional sound signal SA1f to the LD tap 521. Similarly, the multiple FIR filters 512-518 output the filtered regional sound signals SA2f-SA8f to the multiple LD taps 522-528.

[0102] The timbre designation information is not limited to the emphasis information on the range of tones, but also includes information that sets the waveform of the early reflection sound to a user's desired characteristic. By using such timbre designation information, the sound signal processing device 10 can realize early reflection sounds with more diverse timbres that meet the user's preferences.

[0103] [Virtual sound source settings and LDtap settings] The virtual sound source setting unit 502 sets a virtual sound source based on the position coordinates of the sound receiving point in the reproduction space and the geometric shape of the virtual space.

[0104] 12 is a flowchart showing an example of a setting process of a virtual sound source. The virtual sound source setting unit 502 acquires the position coordinates of the sound receiving point in the reproduction space (S131). For example, the virtual sound source setting unit 502 acquires the position coordinates of the sound receiving point in the reproduction space by an operation input from a user, detection of a position by a position detection sensor, or the like.

[0105] The virtual sound source setting unit 502 acquires the geometric shape of the virtual space (S132). For example, the virtual sound source setting unit 502 acquires the geometric shape of the virtual space by operation input from the user, etc. The geometric shape of the virtual space includes a group of coordinates that indicate the shape of a wall placed in the virtual space, etc.

[0106] The virtual sound source setting unit 502 is connected to the GUI 100. When the user selects a desired operator 112 from the multiple operators 112, the GUI 100 reads and acquires the geometric shape of the virtual space associated with the operator 112. In addition, when the user adjusts the room size using the knob 1131, the GUI 100 acquires the adjustment value of the room size.

[0107] The virtual sound source setting unit 502 acquires the position coordinates of the geometric shape of the virtual space in which the room size is set based on each setting acquired by the GUI 100 in this way. The virtual sound source setting unit 502 also acquires the position coordinates of the sound source SS and the position coordinates of the sound receiving point RP (room center). Using these acquired information, the virtual sound source setting unit 502 sets a virtual sound source as follows. The virtual sound source setting unit 502 matches the coordinate system of the reproduction space with the coordinate system of the virtual space. Using the position coordinates of the sound receiving point in the reproduction space and the geometric shape of the virtual space, the virtual sound source setting unit 502 sets the position coordinates of the virtual sound source in the reproduction space according to the concept using the above-mentioned Figures 4(A) and 4(B) (S133).

[0108] 13(A) and 13(B) are diagrams showing examples of setting virtual sound sources when the geometric shapes are different. In Fig. 13(A), the virtual wall IWL is a square, and in Fig. 13(B), the virtual wall IWLh is a hexagon.

[0109] In this way, when the geometric shape of the virtual space is different, even if the position coordinates of the sound source SSa and the sound receiving point RP do not change, the positional relationship between the sound source SSa and the sound receiving point RP and the virtual wall IWL and the positional relationship between the sound source SSa and the sound receiving point RP and the virtual wall IWLh are different. As a result, the positions of the virtual sound sources IS1a, IS2a, and IS3a set in the case of Fig. 13(A) are different from the positions IS1ah, IS2ah, and IS3ah of the virtual sound sources set in Fig. 13(B).

[0110] Fig. 14(A), Fig. 14(B), and Fig. 14(C) are diagrams showing examples of setting a virtual sound source. Fig. 14(A), Fig. 14(B), and Fig. 14(C) are diagrams showing planar changes. Fig. 14(B) shows a case where the position of the sound source SSa relative to the reference point (sound receiving point RP) is the same as in Fig. 14(A), but the size of the virtual space is different. Fig. 14(C) shows a case where the size of the virtual space is the same as in Fig. 14(A), but the positional relationship between the reference point of the virtual space and the reference point (sound receiving point) of the reproduction space has changed (the room center of the reproduction space has changed).

[0111] As can be seen from the comparison between Fig. 14(A) and Fig. 14(B), the size of the virtual space in the reproduction space (indicated as virtual wall IWL in Fig. 14(A) and virtual wall IWLc in Fig. 14(B)) is different, so the distance and positional relationship between the sound source that is the source of the imaginary sound source and the virtual wall are different. As a result, the positions of the imaginary sound sources IS1a, IS2a, and IS3a set in the case of Fig. 14(A) are different from the positions of the imaginary sound sources IS1c, IS2c, and IS3c set in the case of Fig. 14(B).

[0112] Also, as can be seen from the comparison between Fig. 14(A) and Fig. 14(C), the position of the imaginary sound source in the reproduction space (the position of the imaginary sound source relative to the sound receiving point RP and the speaker) moves as the positional relationship between the reference point in the virtual space and the sound receiving point RP changes. As a result, the positions of the imaginary sound sources IS1a, IS2a, and IS3a set in the case of Fig. 14(A) differ from the positions of the imaginary sound sources IS1as, IS2as, and IS3as set in the case of Fig. 14(C).

[0113] Fig. 15(A), Fig. 15(B), and Fig. 15(C) are diagrams showing examples of setting an imaginary sound source. Fig. 15 is a diagram showing changes in the height direction. Fig. 15(A), Fig. 15(B), and Fig. 15(C) are diagrams showing changes in height.

[0114] The height of the ceiling is different between Figure 15(A) and Figure 15(B). That is, the distance (height) from the virtual wall IWFL on the floor to the virtual wall IWCL on the ceiling in the virtual wall IWL shown in Figure 15(A) is different from the distance (height) from the virtual wall IWFL on the floor to the virtual wall IWCLL on the ceiling in the virtual wall IWLL shown in Figure 15(B).

[0115] As can be seen from the comparison between Fig. 15(A) and Fig. 15(B), the difference in ceiling height causes the distance and positional relationship between the sound source that is the source of the imaginary sound source and the virtual walls IWCL and IWCLL of the ceiling to differ. As a result, the position of the imaginary sound source IS1Ca set in the case of Fig. 15(A) differs from the position of the imaginary sound source IS1CaL set in the case of Fig. 15(B).

[0116] The shapes of the ceiling are different between Fig. 15(A) and Fig. 15(C). That is, the shape of the virtual wall IWCL of the ceiling in the virtual wall IWL shown in Fig. 15(A) is different from the shape of the virtual wall IWCLx of the ceiling in the virtual wall IWLx shown in Fig. 15(C).

[0117] As can be seen from the comparison between Fig. 15(A) and Fig. 15(C), the positional relationship between the sound source that is the source of the imaginary sound source and the virtual walls IWCL and IWCLx of the ceiling differs due to the difference in the shape of the ceiling. As a result, the position of the imaginary sound source IS1Ca set in the case of Fig. 15(A) differs from the position of the imaginary sound source IS1Cax set in the case of Fig. 15(C).

[0118] In this way, the imaginary sound source setting unit 502 can optimally set the position of the imaginary sound source in the reproduction space in accordance with the geometric shape of the virtual space and the positional relationship between the reproduction space and the virtual space. This allows the sound signal processing device 10 to clarify the sound image position of the early reflection sound in accordance with the position coordinates of the speaker in the reproduction space, the geometric shape of the virtual space, and the positional relationship between the reproduction space and the virtual space.

[0119] The imaginary sound source setting unit 502 outputs the position coordinates of the imaginary sound source set for each of the multiple areas Area1 to Area8 to the output speaker setting unit 5201 of the LDtap circuit 52.

[0120] The output speaker setting unit 5201 sets the imaginary sound source IS to be assigned to each speaker based on the position coordinates of the imaginary sound source IS, the position coordinates of the sound receiving point RP, and the position coordinates of the multiple speakers SP1-SP64. Fig. 16 is a flowchart showing the process of assigning the imaginary sound source to the speakers.

[0121] The output speaker setting unit 5201 acquires the position coordinates of the virtual sound source from the virtual sound source setting unit 502 (S141). The output speaker setting unit 5201 acquires the position coordinates of the sound receiving point in the reproduction space, for example, by operation input from the user (S142). The output speaker setting unit 5201 acquires the position coordinates of the multiple speakers SP1-SP64, for example, by operation input from the user (S143).

[0122] The output speaker setting unit 5201 sets the area covered by the imaginary sound source for each speaker based on the positional relationship between the sound receiving point RP and the multiple speakers SP1-SP64 in the reproduction space (S144).

[0123] More specifically, the output speaker setting unit 5201 sets the responsible area of ​​the imaginary sound source for each speaker as follows. Figures 17(A) and 17(B) are diagrams showing the concept of allocating the imaginary sound source to the speakers. Figure 17(A) shows the concept of allocation using the azimuth angle φ, and Figure 17(B) shows the concept of allocation using the elevation / depression angle θ. In the following, the speaker SP1 will be described as an example, but the output speaker setting unit 5201 sets the responsible areas for the other speakers SP2-SP64 in a similar manner.

[0124] The output speaker setting unit 5201 sets a straight line (dashed line in FIG. 17A) passing through the sound receiving point RP and the speaker SP1 using the position coordinates of the sound receiving point RP and the position coordinates of the speaker SP1. As shown in FIG. 17A, the output speaker setting unit 5201 sets an azimuth angle φ that spreads toward the speaker SP1 side on a plane with the sound receiving point RP as a reference point with respect to this straight line (dashed line in FIG. 17A). The azimuth angle φ is an angle in the horizontal direction with respect to the straight line that passes through the sound receiving point RP and the speaker SP1. Also, as shown in FIG. 17B, the output speaker setting unit 5201 sets an elevation / depression angle θ that spreads in the up-down direction perpendicular to the plane with respect to the above-mentioned straight line (dashed line in FIG. 17B). The elevation / depression angle θ is an angle in the vertical direction (direction perpendicular to the horizontal direction) with respect to the straight line that passes through the sound receiving point RP and the speaker SP1.

[0125] The output speaker setting unit 5201 sets the space on the speaker SP1 side of the plane determined by the azimuth angle φ and the elevation / depression angle θ as the region RGSP1 covered by the speaker SP1.

[0126] The output speaker setting unit 5201 acquires the position coordinates of a plurality of imaginary sound sources IS (a plurality of imaginary sound sources ISa-ISg in the case of FIG. 17).

[0127] The output speaker setting unit 5201 uses the position coordinates of the multiple imaginary sound sources ISa-ISg and the coordinates representing the assigned region RGSP1 to determine whether the multiple imaginary sound sources ISa-ISg are within the assigned region RGSP1. This determination can be realized by the same method as the grouping of the sound sources into regions described above.

[0128] By performing this determination process, for example, in the case shown in FIG. 14, the output speaker setting unit 5201 determines that multiple imaginary sound sources ISa, ISb, ISc, and ISd are within the assigned region RGSP1, and determines that multiple imaginary sound sources ISe, ISf, and ISg are outside the assigned region RGSP1.

[0129] The output speaker setting unit 5201 assigns the multiple imaginary sound sources ISa, ISb, ISc, and ISd that are determined to be within the assigned region RGSP1 to the speaker SP1.

[0130] The output speaker setting unit 5201 outputs allocation information of multiple imaginary sound sources to the multiple speakers SP1-SP64 to the coefficient setting unit 5202. At this time, the output speaker setting unit 5201 outputs the position coordinates of the sound receiving point RP, the position coordinates of the multiple speakers SP1-SP64, and the position coordinates of the multiple imaginary sound sources to the coefficient setting unit 5202 together with the allocation information.

[0131] The azimuth angle φ is, for example, 60°, and the elevation / depression angle θ is, for example, 45°. The angles of the azimuth angle φ and the elevation / depression angle θ are merely examples, and can be set and adjusted, for example, by an operational input from the user.

[0132] The coefficient setting unit 5202 sets tap coefficients to be given to the LDtap 521-528 using the distance between the sound receiving point RP and the multiple speakers SP1-SP64 and the distance between the sound receiving point RP and the imaginary sound source IS. The tap coefficients to be given to the LDtap 521-528 are the gain values ​​and delay amounts of the LDtap 521-528.

[0133] Fig. 18 is a flowchart showing the coefficient setting process of LDtap. Fig. 19(A) and Fig. 19(B) are diagrams for explaining the concept of coefficient setting.

[0134] The coefficient setting unit 5202 calculates the distances (speaker distances) between the sound receiving point PR and the multiple speakers SP1-SP64, using the position coordinates of the sound receiving point RP and the position coordinates of the multiple speakers SP1-SP64 (S151).

[0135] The coefficient setting unit 5202 calculates the distances (imaginary sound source distances) between the sound receiving point PR and a plurality of imaginary sound sources IS (S152).

[0136] The coefficient setting unit 5202 compares the speaker distances and the imaginary sound source distances for the multiple speakers SP1-SP64 and the multiple imaginary sound sources IS assigned to these speakers SP1-SP64 (S153). For example, in the example of Fig. 17(A), the speaker distances and the imaginary sound source distances are compared for the speaker SP1 and the multiple imaginary sound sources ISa, ISb, ISc, and ISd.

[0137] If the speaker distance is equal to or less than the imaginary sound source distance (S153: YES), the coefficient setting unit 5202 sets the tap coefficients using the imaginary sound source distance as is (S154).

[0138] For example, in the case shown in FIG. 19(A), the imaginary sound source ISa is farther from the sound receiving point RP than the speaker SP1, and the imaginary sound source distance Lia between the sound receiving point RP and the imaginary sound source ISa is greater than the speaker distance Ls1 between the sound receiving point RP and the speaker SP1.

[0139] In this case, the coefficient setting unit 5202 sets the tap coefficient using the distance Da1 between the imaginary sound source ISa and the speaker SP1. Specifically, the coefficient setting unit 5202 sets the gain value and delay amount to be set for the imaginary sound source ISa according to the distance Da1. The coefficient setting unit 5202 sets the gain value to be smaller as the distance Da1 increases, and sets the delay amount to be larger as the distance Da1 increases.

[0140] If the speaker distance is greater than the imaginary sound source distance (S153: NO), the coefficient setting unit 5202 determines whether to reproduce the imaginary sound source. In other words, the coefficient setting unit 5202 determines whether to reproduce the imaginary sound source on the sound receiving point side of the speaker (S155).

[0141] If an imaginary sound source closer to the sound receiving point than the speaker is to be reproduced (S155: YES), the coefficient setting unit 5202 moves the position of this imaginary sound source (S156). More specifically, the coefficient setting unit 5202 moves the position of the imaginary sound source closer to the sound receiving point than the speaker to a position farther from the sound receiving point than the speaker. At this time, the coefficient setting unit 5202 moves the position of the imaginary sound source using the distance difference between the imaginary sound source and the speaker. The coefficient setting unit 5202 sets the tap coefficient using the position coordinates of the imaginary sound source after the movement (S157).

[0142] For example, in the case shown in FIG. 19(B), the imaginary sound source ISd is closer to the sound receiving point RP than the speaker SP1, and the imaginary sound source distance Lid between the sound receiving point RP and the imaginary sound source ISd is smaller than the speaker distance Ls1 between the sound receiving point RP and the speaker SP1.

[0143] In this case, the coefficient setting unit 5202 moves the imaginary sound source ISd using the distance difference Dd between the imaginary sound source distance Lid and the speaker distance Ls1. More specifically, the coefficient setting unit 5202 moves the imaginary sound source ISd to a position of the distance difference Dd on the straight line passing through the sound receiving point RP and the speaker SP1 and on the opposite side to the sound receiving point RP side with the speaker SP1 as a reference. Then, the coefficient setting unit 5202 sets the tap coefficient using this distance difference Dd. Specifically, the coefficient setting unit 5202 sets the gain value and delay amount to be set for the imaginary sound source ISd according to the distance difference Dd. The coefficient setting unit 5202 sets the gain value to be smaller as the distance difference Dd is larger, and sets the delay amount to be larger as the distance difference Dd is larger. Note that conceptually, the imaginary sound source is moved as described above, but as a setting process of the tap coefficient, the coefficient setting unit 5202 may set the tap coefficient according to the distance between the speaker distance and the imaginary sound source distance.

[0144] That is, the coefficient setting unit 5202 moves only the sound receiving point located between the sound receiving point and the speaker. It is preferable that the imaginary sound source outside the speaker with respect to the sound receiving point is not moved, but this also includes the case where this outside imaginary sound source moves within a predetermined range. For example, even if this outside imaginary sound source moves, it is sufficient that the distance between the outside imaginary sound source and the speaker is within a predetermined range, and within the predetermined range is a range in which the change in the early reflection sound control signal due to the movement does not give a sense of incongruity to the viewer. If the coefficient setting unit 5202 does not reproduce the imaginary sound source closer to the sound receiving point than the speaker (S155: NO), the tap coefficient for this imaginary sound source is not set.

[0145] The coefficient setting unit 5202 sets the tap coefficients set for each speaker SP1-SP64 to a plurality of LDtaps. More specifically, the coefficient setting unit 5202 sets the tap coefficients for each speaker SP1-SP64 to LDtaps 521 based on the imaginary sound source positions set in the area Area1. Similarly, the coefficient setting unit 5202 sets the tap coefficients of the imaginary sound sources assigned to each speaker SP1-SP64 to LDtaps 522-528, respectively, based on the imaginary sound source positions set in the plurality of areas Area2-Area8, respectively.

[0146] The multiple LD taps 521-528 perform gain processing and delay processing on the post-filtered regional sound signals SA1f-SA8f according to the set tap coefficients, and output the signals to the addition processing unit 53. More specifically, the tap coefficients are set according to the combination of the imaginary sound source positions of the multiple regions and each speaker, as described above. Therefore, the multiple LD taps 521-528 set a tap coefficient for each speaker based on the imaginary sound source assigned to the speaker. The multiple LD taps 521-528 perform gain processing and delay processing on the post-filtered regional sound signals SA1f-SA8f for each speaker. The multiple LD taps 521-528 output the signals subjected to the gain processing and delay processing for each speaker.

[0147] For example, when imaginary sound sources ISa, ISb, ISc, and ISd are assigned to the speaker SP1, the LDtap 521 performs gain processing and delay processing on the filtered regional sound signal SA1f using tap coefficients (gain value and delay amount) based on the imaginary sound sources ISa, ISb, ISc, and ISd. Then, the LDtap 521 outputs this signal to the addition processing unit 53 for the speaker SP1. The multiple LDtap 521-528 perform such processing on the imaginary sound sources for which the tap coefficients are set.

[0148] The addition processing unit 53 adds the signals after LDtap processing for each of the multiple speakers SP1-SP64 output from the multiple LDtap 521-528 for each of the multiple speakers SP1-SP64. The addition processing unit 53 outputs these added signals to the adder 80 as early reflection sound control signals ER1-ER64 for each of the multiple speakers SP1-SP64.

[0149] By carrying out such processing, the early reflection sound control signal generating unit 50 can generate an early reflection sound control signal having the following characteristics.

[0150] Figures 20(A) and 20(B) are waveform diagrams showing an example of the relationship between the shape of the virtual space and the components of the early reflection sound control signal realized by LDtap. Figure 20(A) shows the case where the shape of the virtual space is large, and Figure 20(B) shows the case where the shape of the virtual space is small. Figures 20(A) and 20(B) show an example of the components of the early reflection sound control signal when multiple imaginary sound sources are set for one speaker.

[0151] If the positional relationship between the reproduction space and the virtual space does not change, and the positions of the sound receiving points and the speakers do not change, if the shape of the virtual space is large, the distribution of the imaginary sound source spreads over a wider range than when the shape of the virtual space is small. Therefore, as shown in Figures 20(A) and 20(B), if the shape of the virtual space is large, each component set by LDtap521-528 tends to be small, and the distribution range on the time axis also becomes wider.

[0152] In this way, by performing the above-mentioned processing, the early reflection sound control signal generating unit 50 can set optimal tap coefficients in accordance with the shape of the virtual space.

[0153] Furthermore, even if the positional relationship between the virtual space and the reproduction space changes, the speaker position changes, or the sound receiving point changes, the early reflection sound control signal generator 50 can set optimal tap coefficients in response to these changes, just as in the case where the shape of the virtual space changes.

[0154] In this case, the multiple sound sources OBJ1-OBJ96 are optimally assigned to the multiple speakers SP1-SP64 through grouping by the multiple areas Area1-Area8. Then, the multiple imaginary sound sources are optimally set for these multiple speakers SP1-SP64. Therefore, even if there is a change in the relationship between the virtual space and the reproduction space, a change in the sound receiving point RP, a change in the positions of the multiple speakers SP1-SP64, or a change in the positions of the sound sources OBJ1-OBJ96, the sound signal processing device 10 can clarify the sound image localization by the early reflection sound in response to these changes.

[0155] In addition, in the above-mentioned configuration, even if the imaginary sound source IS is located closer to the sound receiving point RP than the speaker SP, the early reflection sound control signal generation unit 50 can reproduce the components of the early reflection sound control signal by the imaginary sound source IS in a pseudo manner. Therefore, for example, when the number of imaginary sound sources set for the early reflection sound control signal is small, the early reflection sound control signal generation unit 50 can use an imaginary sound source closer to the sound receiving point RP than the speaker SP. In this case, the early reflection sound control signal generation unit 50 relocates the imaginary sound source outside the speaker using the distance difference between the imaginary sound source IS and the speaker SP as described above. This allows the early reflection sound control signal generation unit 50 to suppress the strangeness of the early reflection sound caused by moving the position of the imaginary sound source.

[0156] In the above configuration, when the imaginary sound source IS is located closer to the sound receiving point RP than the speaker SP, the early reflection sound control signal generation unit 50 may set the imaginary sound source IS to the position of the speaker SP. This allows the early reflection sound control signal generation unit 50 to reduce the processing load of moving the imaginary sound source IS.

[0157] Furthermore, in the above-mentioned configuration, when the imaginary sound source IS is located closer to the sound receiving point RP than the speaker SP, the early reflection sound control signal generating unit 50 does not need to use this imaginary sound source IS to generate the early reflection sound control signal. This eliminates the need for the processing load of moving the imaginary sound source IS, and reduces the processing load of generating the early reflection sound control signal.

[0158] In the above configuration, the early reflection sound control signal generating unit 50 performs tone adjustment using the FIR filters 511-518 while setting the components of the early reflection sound control signal by the imaginary sound source. The FIR filters 511-518 have the above-mentioned number of taps (for example, 16000 taps), which is greater than the number of taps of the LD taps 521-528. The time interval (depending on the sampling frequency) of the FIR filters 511-518 is shorter than the time interval between the taps of the LD taps 521-528 (depending on the arrangement of the imaginary sound source). Therefore, the components of the early reflection sound control signal generated by the FIR filters 511-518 are arranged more densely on the time axis than the components of the early reflection sound control signal generated by the LD taps 521-528. In other words, the resolution (time resolution) on the time axis of the FIR filters 511-518 is higher than that of the LD taps 521-528, and the number of components per unit time is greater.

[0159] The early reflection sound control signal generating unit 50 multiplies the processing of the FIR filters 511-518 with the LDtap 521-528. Therefore, the early reflection sound control signal generating unit 50 has high resolution on the time axis and can generate early reflection sound control signals ER1-ER64 with more diverse timbres. Fig. 21 is a diagram showing an image of the waveform of the early reflection sound control signal generated by the early reflection sound control signal generating unit 50.

[0160] 21, the early reflection sound control signal generating unit 50 can generate an early reflection sound control signal with higher resolution and capable of handling a variety of timbres while retaining the early reflection sound components due to an imaginary sound source. That is, the sound signal processing device 10 can realize an early reflection sound with a timbre according to the user's preference while clearly maintaining the sound image localization due to the early reflection sound using an imaginary sound source.

[0161] In addition, because the resolution of the FIR filter is high, for example, when the sound source is a short pulse sound, the early reflection sound control signal by the LDtap alone may become coarse, causing an unnatural tone. However, with the above-mentioned configuration and processing, the sound signal processing device 10 can suppress the early reflection sound from becoming coarse and the unnatural tone.

[0162] In the above configuration, the early reflection sound control signal generating unit 50 sets an allocation area for the imaginary sound source IS for each speaker SP, and does not allocate the imaginary sound source IS outside this area to this speaker SP. This allows the early reflection sound control signal generating unit 50 to suppress the generation of excessive early reflection sound components. Therefore, the sound signal processing device 10 can suppress excessive early reflection sounds and realize more natural early reflection sounds according to the virtual space.

[0163] [Generation of reverberation control signals] Fig. 22 is a functional block diagram showing an example of the configuration of the reverberation sound control signal generating unit 70. Fig. 23 is a flowchart showing an example of a process for generating a reverberation sound control signal.

[0164] 22, the reverberation sound control signal generation unit 70 includes a PEQ 71, an FIR filter circuit 72, a router 73, a reverberation sound area setting unit 701, a filter coefficient setting unit 702, a reverberation sound reproduction speaker setting unit 703, and an operation unit 700. The FIR filter circuit 72 includes a plurality of FIR filters 721-728.

[0165] The reverberation sound region setting unit 701 sets a plurality of reverberation sound regions Arr1-Arr8 in the reproduction space. More specifically, the reverberation sound region setting unit 701 sets the reproduction space so as to divide the reproduction space into a plurality of reverberation sound regions Arr1-Arr8 over the entire circumference on a plane, for example, with the center point Psr of the reproduction space as a reference (see FIG. 25 described later).

[0166] The reverberation sound region setting unit 701 outputs coordinate information indicating a plurality of reverberation sound regions Arr1 to Arr8 to the filter coefficient setting unit 702 and the reverberation sound reproduction speaker setting unit 703.

[0167] The filter coefficient setting unit 702 sets a filter coefficient for reverberation sound in response to a user operation or the like. The filter coefficient for reverberation sound is set, for example, based on the actual measurement result of an impulse response in the real space of the virtual space (a different space reproduced in the playback space). The filter coefficient for reverberation sound may be set artificially using the geometric shape of the virtual space, the material of the walls, and the like. At this time, the filter coefficient setting unit 702 sets a filter coefficient for each of the reverberation sound regions Arr1-Arr8 using the coordinate information for each of the reverberation sound regions Arr1-Arr8.

[0168] The filter coefficient setting unit 702 receives input of the volume of the virtual space, the surface area of ​​the virtual space, etc., through a user operation, etc. The filter coefficient setting unit 702 sets a fade-in function for the filter coefficient from parameters such as the volume of the virtual space, the surface area of ​​the virtual space, etc.

[0169] More specifically, the filter coefficient setting unit 702 calculates the mean free path ρ using the volume V of the virtual space and the surface area S of the virtual space. The calculation formula for the mean free path ρ is ρ=4V / S. The mean free path is the average propagation distance in a closed space from when a sound is reflected on a wall to when it is reflected again. By dividing the mean free path by the speed of sound c0, the average time it takes for a sound to be reflected on a wall to when it is reflected again can be calculated.

[0170] The filter coefficient setting unit 702 sets the connection timing tc from the mean free path ρ (FIG. 23: S231). Specifically, the filter coefficient setting unit 702 sets the connection timing tc using the mean free path ρ, the sound speed c0, and the reflection order n. The calculation formula for the connection timing tc is tc=ρ×n / c0.

[0171] As can be seen from this calculation formula, the connection timing tc corresponds to the average time required for n reflections in the virtual space, and corresponds to the time when the transition to reverberation begins when reproducing the nth early reflection sound. In other words, the connection timing tc corresponds to the timing when the component of the early reflection sound control signal generated by the early reflection sound control signal generator 50 disappears.

[0172] By performing such processing, the filter coefficient setting unit 702 can optimally set the connection timing tc between the early reflection sound and the reverberation sound in accordance with the geometric shape of the virtual space.

[0173] The filter coefficient setting unit 702 uses the connection timing tc to set a fade-in function from the following equation (FIG. 23: S232).

[0174]

number

[0175] In this formula, t is the time that has elapsed since the direct sound was generated, and K is set by the following formula.

[0176]

number

[0177] In this formula, G REV is the gain value of the reverberation sound at time t = 0, and can be set by the user. For example, the reverberation time is generally the time required for the sound to decay to -60 dB. REV =-60dB, etc.

[0178] The filter coefficient setting unit 702 sets reverberation sound filter coefficients from the filter coefficients and the fade-in function fin (FIG. 23: S233), and outputs them to the multiple FIR filters 721-728.

[0179] The reverberation sound generating signal Sr output from the mixer 60 is input to the PEQ 71. The PEQ 71 performs predetermined signal processing on the reverberation sound generating signal Sr and outputs the result to a plurality of FIR filters 721-728.

[0180] By performing signal processing by the PEQ 71, the level (signal magnitude) and tone of the reverberation sound generating signal Sr can be adjusted. For example, the PEQ 71 can adjust the level (signal magnitude) of the reverberation sound generating signal Sr by referring to the volume of the early reflection sound control signal, so that the volume of the early reflection sound and the volume of the reverberation sound are approximately the same at the connection timing tc. The PEQ 71 can also adjust the tone, etc., according to settings by the user, etc.

[0181] The FIR filters 721-728 filter the reverberation sound generation signal Sr using the reverberation sound filter coefficients to generate the reverberation sound control signals REVr1-REVr8 for the respective regions. For example, the FIR filter 721 performs a convolution operation on the reverberation sound generation signal Sr using the reverberation sound filter coefficients set for the reverberation sound region Arr1 to generate the reverberation sound control signal REVr1 for the region Arr1. Similarly, the FIR filters 722-728 perform a convolution operation on the reverberation sound generation signal Sr using the reverberation sound filter coefficients set for the reverberation sound regions Arr2-Arr8 to generate the reverberation sound control signals REVr2-REVr8 for the respective regions Arr2-Arr8 (FIG. 23: S234). The FIR filters 721-728 output the reverberation sound control signals REVr1-REVr8 for the respective regions to the router 73.

[0182] By setting the above-mentioned fade-in function, the reverberation sound control signal has a waveform as shown in Fig. 21. Fig. 24 is a graph showing example waveforms of the direct sound, the early reflection sound control signal, and the reverberation sound control signal. For convenience, in Fig. 24, the reverberation sound control signal is illustrated by the envelope of each time component. The vertical axis in Fig. 24 is expressed in dB.

[0183] As shown in Fig. 24(A), the signal level of the reverberation sound control signal gradually increases according to the fade-in function from the output timing of the direct sound to the connection timing tc. More specifically, the signal level of the reverberation sound control signal is -60 dBFs at the output timing of the direct sound, gradually increases until the connection timing tc, and becomes 0 dBFs at the connection timing tc. This level is set based on the signal level of the early reflection sound control signal at the connection timing tc.

[0184] 24, the above-mentioned fade-in function is used to exponentially increase the signal level as the connection timing tc approaches, in other words, the fade-in function has the opposite characteristics to the attenuation curve of the reverberation sound control signal without fade-in processing. Note that the characteristics of the change in the level of the reverberation sound control signal due to fade-in processing are not limited to this, and the user can set the desired characteristics by appropriately setting the fade-in function.

[0185] By performing such processing, the reverberation sound control signal generator 70 can generate a reverberation sound control signal that accurately reproduces the reverberation sound in the virtual space, using the FIR filters 721-728. Also, the signal level of the reverberation sound control signal gradually increases in the section where the early reflection sound control signal exists, reaches a peak value according to the signal level of the early reflection sound control signal at the connection timing tc, and then attenuates.

[0186] As a result, the sound signal processing device 10 can smoothly connect the early reflection sound control signal and the reverberation sound control signal, which are multiple LDtaps that reproduce the imaginary sound source distribution at multiple sound source positions in the virtual space, by the reverberation sound generated by the reverberation sound control signal. Therefore, the sound output from the sound signal processing device 10 and heard by the user is a sound in which the sense of incongruity at the connection from the early reflection sound to the reverberation sound is suppressed.

[0187] The reverberation sound reproduction speaker setting unit 703 groups the multiple speakers SP1-SP64 into reverberation sound areas Arr1-Arr8.

[0188] More specifically, the reverberation sound reproduction speaker setting unit 703 divides the reproduction space into a plurality of reverberation sound regions Arr1-Arr8, for example, around the entire circumference on a plane with the center point Psr of the reproduction space as a reference. The reverberation sound reproduction speaker setting unit 703 groups the plurality of speakers SP1-SP64 with respect to the plurality of reverberation sound regions Arr1-Arr8, using the position coordinates of the plurality of speakers SP1-SP64 and coordinate information indicating the plurality of reverberation sound regions Arr1-Arr8. This grouping can be realized by a method similar to the method of grouping the sound source OBJ described above.

[0189] Fig. 25 is a diagram showing an example of a region setting for reverberation sound. In Fig. 25, a plurality of speakers SP1-SP14 are shown in order to simplify the explanation and make it easier to understand. For example, as shown in Fig. 25, the reverberation sound reproduction speaker setting unit 703 detects that speakers SP6 and SP7 are present in the reverberation sound region Arr1, and groups the speakers SP6 and SP7 into the reverberation sound region Arr1. Similarly, the reverberation sound reproduction speaker setting unit 703 groups the other speakers SP1-SP5 and SP8-SP14 into a plurality of reverberation sound regions Arr2-Arr8, respectively.

[0190] The reverberation sound reproduction speaker setting unit 703 outputs to the router 73 grouping information of the multiple speakers SP1 to SP64 for the multiple reverberation sound areas Arr2 to Arr8.

[0191] The router 73 allocates the region-specific reverberation sound control signals REVr1-REVr8 to the multiple speakers SP1-SP64 using the grouping information from the reverberation sound reproduction speaker setting unit 703. Based on the allocation, the router 73 outputs the region-specific reverberation sound control signals REVr1-REVr8 as reverberation sound control signals REV1-REV48 for each of the multiple speakers SP1-SP64.

[0192] For example, the router 73 extracts from the grouping information that speakers SP6 and SP7 are grouped in area Arr1. The router 73 assigns the area-specific reverberation sound control signal REVr1 of area Arr1 to speakers SP6 and SP7. The router 73 outputs the area-specific reverberation sound control signal REVr1 to speaker SP6 as a reverberation sound control signal REV6 for speaker SP6. The router 73 also outputs the area-specific reverberation sound control signal REVr1 to speaker SP7 as a reverberation sound control signal REV6 for speaker SP7.

[0193] By performing region-by-region reverberation control signals REVr1-REVr8 by the router 73 in this manner, the reverberation control signal generator 70 can output optimal reverberation control signals to each of the multiple speakers SP1-SP64 in accordance with the arrangement of the multiple speakers SP1-SP64.

[0194] [Output Adjustment] Fig. 26 is a functional block diagram showing an example of the configuration of the output adjustment unit 90. Fig. 27 is a flowchart showing an example of the output adjustment process.

[0195] 26, the output adjustment unit 90 includes a gain control unit 91, a delay control unit 92, a gain / delay setting unit 901, an operation unit 900, and a display unit 909. The gain control unit 91 includes a plurality of gain control units 9101-9168 corresponding to the plurality of speakers SP1-SP64. The delay control unit 92 includes a plurality of delay control units 9201-9264 corresponding to the plurality of speakers SP1-SP64.

[0196] The operation unit 900 receives settings of acoustic parameters of the reproduction space through operation input from the user (FIG. 27: S321). The acoustic parameters of the reproduction space are parameters for reproducing a desired sound field in the reproduction space.

[0197] In this case, the acoustic parameters of the playback space are not the individual gain values ​​or delay amounts of the multiple speakers SP1-SP64, but rather a weight value that represents the weighting of the sound in a specific direction in the playback space, and a shape value that represents the spread of the sound in a specific direction in the playback space.

[0198] The weight value is composed of a gain value and a delay amount, and includes weight values ​​for the front and rear of the reproduction space, weight values ​​for the left and right of the reproduction space, and weight values ​​for the up and down direction of the reproduction space. The shape value is composed of a gain value and a delay amount, and includes a shape value for the horizontal direction.

[0199] The display unit 909 includes a GUI. Fig. 28 is a diagram showing an example of the GUI for output adjustment.

[0200] 28, the GUI 100A includes a setting display window 111, an output state display window 115, and a plurality of operators 116. The plurality of operators 116 includes a knob 1161, and an adjustment value display window 1162.

[0201] The multiple operators 116 are operators for setting weight volumes for setting weight values, shape volumes for setting shape values, etc. The operators 116 for weight volumes include operators 116 for setting left / right weights, front / rear weights, and top / bottom weights, each of which includes an operator for setting a gain value and an operator for setting a delay amount. The operators 116 for shape volumes include an operator for setting spread, an operator for setting a gain value, and an operator for setting a delay amount.

[0202] The output status display window 115 graphically and typically displays the sound spread and localization achieved by the weight values ​​and shape values ​​set by the multiple operators 116. This allows the user to easily recognize the sound spread and localization set by the multiple operators 116 as an image.

[0203] The user sets the acoustic parameters (weight values ​​and delay amounts) that he or she wishes to reproduce using the GUI 100A of the display unit 909. The operation unit 900 accepts the settings made using the GUI 100A. The operation unit 900 outputs the settings (each weight value and each delay amount of the acoustic parameters) to the gain / delay setting unit 901.

[0204] The gain delay setting unit 901 sets gain values ​​and delay amounts for the multiple speakers SP1-SP64 based on the weight values ​​and delay amounts of the acoustic parameters. More specifically, the gain delay setting unit 901 performs the following process.

[0205] The gain delay setting unit 901 acquires the position coordinates of the multiple speakers SP1-SP64 arranged in the reproduction space (S322). The position coordinates are expressed in a coordinate system in which, for example, the x-axis is set in the left-right direction of the reproduction space, the y-axis is set in the front-back direction of the reproduction space, and the z-axis is set in the up-down direction.

[0206] The gain delay setting unit 901 extracts the maximum and minimum values ​​of the position coordinates of the multiple speakers SP1-SP64 in each axial direction (S323).

[0207] The gain delay setting unit 901 stores coefficient setting formulas. The coefficient setting formulas include, for example, a weight coefficient setting formula for setting weighting in a predetermined direction in the reproduction space, and a shape coefficient setting formula for setting weighting in a predetermined direction in the reproduction space.

[0208] The coefficient setting formula for weight includes a setting formula for a gain value for weight and a setting formula for a delay amount for weight. The coefficient setting formula for shape includes a setting formula for a gain value for shape and a setting formula for a delay amount for shape.

[0209] The weighting coefficient setting formula includes a front-to-rear coefficient setting formula for setting weighting in the front-to-rear direction of the reproduction space, a left-to-right coefficient setting formula for setting weighting in the left-to-right direction of the reproduction space, and a up-to-down coefficient setting formula for setting weighting in the up-to-down direction of the reproduction space.

[0210] The coefficient setting formula for the shape includes coefficient setting formulas for the left and right directions of the reproduction space.

[0211] The coefficient setting formula for the gain value for weighting is, for example, a linear function combining the gain value of the set weight value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the gain value is to be set (the speaker to be set), and this formula determines the gain value in proportion to the difference between the position coordinates of the speaker to be set and the minimum value of the position coordinates.

[0212] The coefficient setting formula for the delay amount for the weight is, for example, a linear function combining the delay amount of the set weight value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the delay amount is to be set (the speaker to be set), and this formula determines the delay amount in proportion to the difference between the position coordinate of the speaker to be set and the minimum value of the position coordinates.

[0213] The coefficient setting formula for the gain value for shape is, for example, a linear function combining the gain value of the set shape value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the gain value is to be set (the speaker to be set), and is a formula that determines the gain value in proportion to the difference between the position coordinates of the speaker to be set and the minimum value of the position coordinates.

[0214] The coefficient setting formula for the delay amount for shape is, for example, a linear function combining the delay amount of the set shape value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the delay amount is to be set (the speaker to be set), and this formula determines the delay amount in proportion to the difference between the position coordinates of the speaker to be set and the minimum value of the position coordinates.

[0215] The gain delay setting unit 901 calculates a gain value and a delay amount for each speaker to be set, using the set gain value and delay amount (acoustic parameters), the extracted maximum and minimum values ​​of the position coordinates, and the coefficient setting formula (S324).

[0216] By using such processing, the gain delay setting unit 901 can automatically calculate and set the gain values ​​and delay amounts of the multiple speakers SP1-SP64 arranged in the reproduction space using a coefficient setting formula, without having to set them individually and manually.

[0217] The gain delay setting unit 901 outputs the gain values ​​set for each of the multiple speakers SP1-SP64 to the multiple gain control units 9101-9164. The gain delay setting unit 901 outputs the delay amounts set for each of the multiple speakers SP1-SP64 to the multiple delay control units 9201-9264.

[0218] The speaker signals Sat1-Sat64 corresponding to the plurality of speakers SP1-SP64 are input from the adder 80 to the plurality of gain control sections 9101-9164, respectively.

[0219] The multiple gain control units 9101-9164 use the gain values ​​set therein to control the signal levels of the speaker signals Sat1-Sat64 and output them to the multiple delay control units 9201-9264. For example, the gain control unit 9101 uses the gain value set in the gain control unit 9101 to control the signal level of the speaker signal Sat1 and output it to the delay control unit 9201. Similarly, the gain control units 9102-9164 use the gain values ​​set in the gain control units 9102-9164 to control the signal levels of the speaker signals Sat2-Sat64 and output them to the delay control units 9202-9164.

[0220] The multiple delay control units 9201-9164 use the delay amounts set therein to control the signal levels of the signals input from the multiple gain control units 9101-9164, and output the signals to the multiple speakers SP1-SP64. For example, the delay control unit 9201 uses the delay amount set in the delay control unit 9201 to control the signal level of the signal input from the gain control unit 9101, and output the signals to the speaker SP1. Similarly, the delay control units 9202-9164 use the delay amounts set in the delay control units 9202-9164 to control the signal levels of the signals input from the gain control units 9102-9164, and output the signals to the speakers SP1-SP64, respectively.

[0221] With this configuration, the sound signal processing device 10 can easily realize a desired sound field corresponding to the set acoustic parameters by using the early reflection sound control signal and the reverberation sound control signal, without forcing the user to make complicated settings for each of the multiple speakers. As a result, for example, the sound signal processing device 10 can easily realize a sound field that can obtain the Haas effect at a predetermined position in the reproduction space.

[0222] (Example of sound field realization by output control) Fig. 29(A) and Fig. 29(B) are diagrams showing a setting example when weighting the sound to the rear of the reproduction space. Fig. 29(A) is a diagram showing an example of the gain value and delay amount setting, and Fig. 29(B) is a diagram showing an image of the weighting of the sound by the setting of Fig. 25(A). Note that Fig. 29(A) and Fig. 29(B) show a case where 14 speakers SP1-SP14 are arranged in order to simplify the explanation and make it easier to understand.

[0223] 29(A) and 29(B), for example, the gain value and delay amount of the rear end are set as the acoustic parameters. The gain delay setting unit 901 sets the gain value and delay amount of the front end to values ​​with the opposite signs to the gain value and delay amount of the rear end. The gain delay setting unit 901 calculates the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14.

[0224] The gain delay setting unit 901 calculates gain values ​​for the 14 speakers SP1-SP14 using the gain values ​​at the rear and front ends, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula for the front-to-rear direction (for setting gain values) that sets the weighting in the front-to-rear direction of the reproduction space.

[0225] In addition, the gain delay setting unit 901 calculates the delay amounts of the 14 speakers SP1-SP14 using the delay amounts of the rear and front ends, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula for the front-rear direction (for setting the delay amount) that sets the weighting in the front-rear direction of the reproduction space.

[0226] By this processing, the sound signal processing device 10 can automatically and easily set acoustic parameters such that the gain value and delay amount are larger for the speaker at the rear of the reproduction space and the gain value and delay amount are smaller for the speaker at the front, as shown in Fig. 29(A). This enables the sound signal processing device 10 to easily realize a sound field (see Fig. 29(B)) that spreads to the rear of the reproduction space and where reverberation is localized.

[0227] Although an example of the front-rear direction has been shown in this description, the sound signal processing device 10 can also realize a weighted sound field in the left-right direction and the height direction (up-down direction) in the same manner.

[0228] Fig. 30(A) and Fig. 30(B) are diagrams showing a setting example when sound is spread in the horizontal direction of the reproduction space. Fig. 30(A) is a diagram showing an example of gain value and delay amount settings, and Fig. 30(B) is a diagram showing an image of sound spread by the settings of Fig. 30(A). Note that Fig. 30(A) and Fig. 30(B) show a case where 14 speakers SP1-SP14 are arranged in order to simplify the explanation and make it easier to understand.

[0229] 30(A) and 30(B), for example, a value that quantifies the sound spread (spread setting value) is set as the acoustic parameter. The gain delay setting unit 901 calculates the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14.

[0230] The gain delay setting unit 901 calculates gain values ​​for the 14 speakers SP1-SP14 using a spread setting value, maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula for shape (for setting gain values).

[0231] In addition, the gain delay setting unit 901 calculates the delay amounts of the 14 speakers SP1-SP14 using the delay amounts of the rear and front ends, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula for shape (for setting the delay amount).

[0232] By this processing, the sound signal processing device 10 can automatically and easily set acoustic parameters such that the closer to both ends of the horizontal direction of the reproduction space the larger the gain value and delay amount, and the closer to the center of the horizontal direction the smaller the gain value and delay amount, as shown in Fig. 30(A). This enables the sound signal processing device 10 to easily realize a sound field (see Fig. 30(B)) in which the reproduction space is spread in the horizontal direction and reverberation is localized.

[0233] By setting the above-mentioned acoustic parameters, the sound signal processing device 10 can realize weighting and spreading of the reproduction space in the height direction (up and down direction) as well as weighting in the front-rear direction, weighting in the left-right direction, and spreading in the lateral direction. For example, Fig. 31 is a diagram showing an image of the spread of sound when spreading in the height direction.

[0234] The sound signal processing device 10 sets the gain value and delay amount of the speaker SPU on the ceiling side to be larger than the gain value and delay amount of the speakers SPL and SPR close to the floor surface. This allows the sound signal processing device 10 to easily realize a sound field (see FIG. 31) that is more expansive toward the ceiling of the reproduction space and in which reverberation is localized.

[0235] In the above configuration, the output adjustment unit 90 outputs the output signals So1-So64 to the multiple speakers SP1-SP64. However, the sound signal processing device may output the output signals So1-So64 after binaural processing.

[0236] 32 is a functional block diagram showing the configuration of a sound signal processing device with a binaural reproduction function. As shown in FIG. 32, a sound signal processing device 10A with a binaural reproduction function differs from the above-mentioned sound signal processing device 10 in that it includes an output adjustment unit 90A, a reverberation processing unit 97, a selection unit 98, and a binaural processing unit 99.

[0237] The output adjustment section 90A generates a plurality of output signals So1-So64 from the plurality of speaker signals Sat1-Sat64 output from the adder 80 using the same processing as the output adjustment section 90 described above.

[0238] The output adjustment unit 90A can select an output target. The selection of the output target is executed, for example, by an operation input from a user using the above-mentioned GUI. More specifically, the GUI displays an operator that can select speaker output and binaural output, and the output target is selected by operating this operator.

[0239] When speaker output is selected, the output adjustment unit 90A outputs the multiple output signals So1-So64 to the multiple speakers SP1-AP64, respectively (the same processing as the output adjustment unit 90). When binaural output is selected, the output adjustment unit 90A outputs the multiple output signals So1-So64 to the selection unit 98.

[0240] Sound signals S1-S96 of multiple sound sources OBJ1-OBJ96 are input to the reverberation processor 97. The reverberation processor 97 adds an early reflection sound control signal and a reverberation sound control signal to the multiple sound signals S1-S96 and outputs them to a selection unit 98. The early reflection sound control signals for the multiple sound signals S1-S96 are set based on the position coordinates of the multiple sound sources OBJ1-OBJ96. The reverberation processor 97 outputs the multiple reverberation processed sound signals S1'-S96' to the selection unit 98.

[0241] The selection unit 98 receives the output signals So1-So64 and the reverberation-processed sound signals S1'-S96'. The selection unit 98 selects the output signals So1-So64 and the reverberation-processed sound signals S1'-S96', for example, in response to an operation input from a user using the above-mentioned GUI. More specifically, the GUI displays an operator that can select between a sound that has been subjected to acoustic processing by the sound signal processing device 10A and a sound that has been subjected to virtual acoustic processing based on the position coordinates of the sound sources OBJ1-OBJ96, and the output target is selected by operating this operator.

[0242] When sound that has been subjected to acoustic processing by the sound signal processing device 10A is selected, the selection unit 98 selects multiple output signals So1-So64 and outputs them to the binaural processing unit 99. When sound that has been subjected to virtual acoustic processing based on the position coordinates of the sound sources OBJ1-OBJ96 is selected, the selection unit 98 selects multiple post-reverberation sound signals S1'-S96' and outputs them to the binaural processing unit 99.

[0243] The binaural processing unit 99 applies binaural processing to the input sound signal. More specifically, when a plurality of output signals So1-So64 are input, the binaural processing unit 99 applies binaural processing to the plurality of output signals So1-So64. When a plurality of reverberation-processed sound signals S1'-S96' are input, the binaural processing unit 99 applies binaural processing to the plurality of reverberation-processed sound signals S1'-S96'.

[0244] Note that binaural processing uses head-related transfer functions, and the details are known, so a detailed description of the binaural processing will be omitted.

[0245] The binaural processing unit 99 outputs two-channel sound signals that have been subjected to binaural processing.

[0246] This allows the user to listen to the sound generated by the sound signal processing device 10A and the sound that has been subjected to virtual reverberation processing based on the position coordinates of the sound sources OBJ1-OBJ96 through binaural playback. Therefore, the user can easily check, using headphones or the like, whether the sound processing performed by the sound signal processing device 10A can reproduce the sound of the virtual space, without physically constructing a reproduction space. The sound processing performed by the sound signal processing device 10A includes, for example, the grouping of the sound sources, the setting of the early reflection sound control signal, the setting of the reverberation sound control signal, the setting of the output control, and the like. By listening and comparing the sounds in this way, the user can adjust the settings of the above-mentioned sound processing so that the sound of the virtual space can be reproduced more faithfully.

[0247] It should be noted that binaural reproduction is not limited to headphones, but may also be performed using stereo speakers or the like.

[0248] The description of the present embodiment is illustrative in all respects and is not restrictive. The scope of the present invention is indicated by the claims, not by the above-described embodiments. Furthermore, the scope of the present invention is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0249] 10, 10A: Sound signal processing device 30: Area setting section 40: Grouping section 41: Sound source position detection unit 42: Area determination section 50: Early reflection sound control signal generator 51: FIR filter circuit 52: LDtap circuit 53: Addition processing unit 60: Mixer 70: Reverberation control signal generator 71: PEQ 72: FIR filter circuit 73: Router 80: Adder 90, 90A: Output adjustment section 91: Gain control section 92: Delay control section 97: Reverberation processor 98: Selection section 99: Binaural processing section 100, 100A: GUI 400: Matrix Mixer 500:Operation unit 501: Tone setting section 502: Imaginary sound source setting section 511-518: FIR filters 521-528:LDtap 700:Operation unit 701: Reverberation area setting section 702: Filter coefficient setting unit 703: Reverberation playback speaker setting section 721-728: FIR filters 900:Operation unit 901: Gain delay setting section 909: Display section 5201: Output speaker setting section 5202: Coefficient setting section 9101-9164: Gain control section 9201-9264: Delay control section

Claims

1. determining whether a virtual sound source representing a reflected sound of a target acoustic space is within an area defined by a predetermined azimuth angle and elevation / depression angle from the center position of the acoustic space and the position of the speaker; outputting and controlling an early reflection sound control signal of the virtual sound source from the speaker only when the virtual sound source is within the region; A method for processing an audio signal.

2. setting the virtual sound source based on the shape of the acoustic space, the position of the speaker in the acoustic space, and the position of a sound receiving point; The method for processing a sound signal according to claim 1 .

3. A gain value and a delay amount of the early reflection sound control signal are set based on a positional relationship between the virtual sound source, the sound receiving point, and the speaker. The sound signal processing method according to claim 1 or 2.

4. The virtual sound sources are classified into a first virtual sound source that is located between the speaker and the sound receiving point and indicates a first sound source of a reflected sound in a target acoustic space, and a second virtual sound source that is located outside the speaker and indicates a second sound source of a reflected sound in the target acoustic space, A method for setting the positions of the first virtual sound source and the second virtual sound source is made different from each other. The sound signal processing method according to any one of claims 1 to 3.

5. Only when the virtual sound source indicating the reflected sound of the target acoustic space is the first virtual sound source, the position of the first virtual sound source is moved to a position where the reflected sound can be reproduced using a position of a speaker near the first virtual sound source. The sound signal processing method according to claim 4.

6. A speaker that reproduces the reflected sound of the desired acoustic space; an early reflection sound control unit that determines whether a virtual sound source representing a reflected sound of the target acoustic space is within a region formed by a predetermined azimuth angle and elevation / depression angle from the center position of the acoustic space and the position of the speaker, and controls the output of an early reflection sound control signal of the virtual sound source from the speaker only if the virtual sound source is within the region; Equipped with Sound signal processing device.

7. The early reflection sound control unit includes: setting the virtual sound source based on the shape of the acoustic space, the position of the speaker in the acoustic space, and the position of a sound receiving point; The sound signal processing device according to claim 6 .

8. The early reflection sound control unit includes: A gain value and a delay amount of the early reflection sound control signal are set based on a positional relationship between the virtual sound source and the speaker. The sound signal processing device according to claim 6 or 7.

9. The early reflection sound control unit includes: The virtual sound sources are classified into a first virtual sound source that is located between the speaker and the sound receiving point and indicates a first sound source of a reflected sound in a target acoustic space, and a second virtual sound source that is located outside the speaker and indicates a second sound source of a reflected sound in the target acoustic space, A method for setting a gain value and a delay amount of the early reflection sound control signal is made different between the first virtual sound source and the second virtual sound source. The sound signal processing device according to any one of claims 6 to 8.

10. The early reflection sound control unit includes: Only when the virtual sound source is the first virtual sound source, the position of the first virtual sound source is moved to a position where the sound can be reproduced using a position of a speaker near the first virtual sound source. The sound signal processing device according to claim 9 .

Images