Sound signal processing method and sound signal processing device

Abstract

To clearly reproduce sound image localization in a virtual space.SOLUTION: In a sound signal processing method, a virtual sound source indicating a reflected sound of a target acoustic space is classified into a first virtual sound source that is located between a position of a speaker and a position of a sound receiving point, and indicating a first sound source of the reflected sound of the target acoustic space and a second virtual sound source that is located outside the speaker and indicates a second sound source of the reflected sound in the target acoustic space, and only in the case of the first virtual sound source, a position of the first virtual sound source is moved to a position in which the reproduction can be performed using the position of the speaker near the first virtual sound source.SELECTED DRAWING: Figure 18

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a sound signal processing method and a sound signal processing device for performing predetermined processing on sound input from a sound source. [Background technology]

[0002] In an acoustic system for a space such as a hall, a sound image relative to a sound source is localized by speakers arranged in the space.

[0003] For example, the audio processing device described in Patent Document 1 outputs the sound of an audio object (sound source) from two or more speakers near the audio object (sound source). At this time, the audio processing device described in Patent Document 1 calculates the gain of the audio signal to be output to each speaker using position information and sound image information of the audio object (sound source). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2016 / 208406 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the above-described conventional configuration, it is not possible to clearly reproduce the sound image localization in a virtual space in a space where speakers are installed.

[0006] Therefore, an object of one embodiment of the present invention is to clearly reproduce sound image localization in a virtual space. [Means for solving the problem]

[0007] The sound signal processing method classifies virtual sound sources that represent reflected sound in a target acoustic space into a first virtual sound source that is located between the position of the speaker and the position of the sound receiving point and represents a first sound source of reflected sound in the target acoustic space, and a second virtual sound source that is located outside the speaker and represents a second sound source of reflected sound in the target acoustic space, and moves the position of the first virtual sound source only to a position where it can be reproduced using the position of a speaker near the first virtual sound source. [Effects of the Invention]

[0008] The sound signal processing method can clearly reproduce the sound image localization in the virtual space. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a functional block diagram showing the configuration of an acoustic system including a sound signal processing device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart of a sound signal processing method according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing the discrete waveforms of a typical sound including a direct sound, an early reflection sound, and a reverberation sound (late reverberation sound). [Figure 4] 4(A) and 4(B) are diagrams showing the concept of setting an imaginary sound source. [Figure 5] FIG. 5 is a functional block diagram showing an example of the configuration of the grouping unit 40. As shown in FIG. [Figure 6] FIG. 6 is a flowchart showing a method for grouping sound sources. [Figure 7] FIG. 7 is a diagram showing the concept of grouping multiple sound sources into multiple regions. [Figure 8] FIG. 8(A) is a flowchart showing a method for grouping sound sources using representative points, and FIG. 8(B) is a flowchart showing a method for grouping sound sources using area boundaries. [Figure 9] FIG. 9 is a flowchart showing an example of a grouping method based on the movement of a sound source. [Figure 10]FIG. 10 is a functional block diagram showing an example of the configuration of the early reflection sound control signal generator 50. As shown in FIG. [Figure 11] FIG. 11 is a diagram illustrating an example of the GUI. [Figure 12] FIG. 12 is a flowchart showing an example of the setting process of the imaginary sound source. [Figure 13] 13(A) and 13(B) are diagrams showing examples of setting imaginary sound sources when the geometric shapes are different. [Figure 14] 14(A), 14(B), and 14(C) are diagrams showing examples of setting an imaginary sound source. [Figure 15] 15(A), 15(B), and 15(C) are diagrams showing examples of setting an imaginary sound source. [Figure 16] FIG. 16 is a flowchart showing the process of assigning imaginary sound sources to speakers. [Figure 17] 17(A) and 17(B) are diagrams showing the concept of assigning imaginary sound sources to speakers. [Figure 18] FIG. 18 is a flowchart showing the coefficient setting process of LDtap. [Figure 19] 19(A) and 19(B) are diagrams for explaining the concept of coefficient setting. [Figure 20] FIG. 20(A) shows an example of the LDtap coefficient when the virtual space shape is large, and FIG. 20(B) shows an example of the LDtap coefficient when the virtual space shape is small. [Figure 21] FIG. 21 is a diagram showing the waveform of the early reflection sound control signal generated by the early reflection sound control signal generating unit 50. As shown in FIG. [Figure 22] FIG. 22 is a functional block diagram showing an example of the configuration of the reverberation control signal generating unit 70. As shown in FIG. [Figure 23] FIG. 23 is a flowchart showing an example of a process for generating a reverberation control signal. [Figure 24] FIG. 24 is a graph showing example waveforms of a direct sound, an early reflection sound control signal, and a reverberation sound control signal. [Figure 25] FIG. 25 is a diagram showing an example of region setting for reverberation sounds. [Figure 26] FIG. 26 is a functional block diagram showing an example of the configuration of the output adjustment unit 90. As shown in FIG. [Figure 27] FIG. 27 is a flowchart showing an example of the output adjustment process. [Figure 28] FIG. 28 is a diagram showing an example of a GUI for adjusting output. [Figure 29] 29(A) and 29(B) are diagrams showing setting examples in which sound is localized and spread toward the rear of the reproduction space. [Figure 30] 30(A) and 30(B) are diagrams showing setting examples for localizing and spreading sound in the horizontal direction of the reproduction space. [Figure 31] FIG. 31 is a diagram showing an image of sound spread when spreading in the height direction. [Figure 32] FIG. 32 is a functional block diagram showing the configuration of a sound signal processing device with a binaural reproduction function. DETAILED DESCRIPTION OF THE INVENTION

[0010] A sound signal processing method and a sound signal processing device according to an embodiment of the present invention will be described with reference to the drawings. In the following embodiments, an overview of the sound signal processing method and the sound signal processing device will be first described, and then specific details of each process and each configuration will be described.

[0011] In this embodiment, the reproduction space is a space where a user (listener) hears sounds (direct sounds, early reflection sounds, and reverberant sounds) from a sound source using speakers, etc. The virtual space is a space that has a sound field (acoustics) different from that of the reproduction space, and is a space where the early reflection sounds and reverberant sounds due to this sound field are reproduced (simulated) in the reproduction space.

[0012] [Outline of sound signal processing device] FIG. 1 is a functional block diagram showing the configuration of an acoustic system including a sound signal processing device according to an embodiment of the present invention.

[0013] 1, the sound signal processing device 10 includes an area setting unit 30, a grouping unit 40, an early reflection sound control signal generating unit 50, a mixer 60, a reverberation sound control signal generating unit 70, an adder 80, and an output adjustment unit 90. The sound signal processing device 10 is realized, for example, by electronic circuits that respectively realize the area setting unit 30, the grouping unit 40, the early reflection sound control signal generating unit 50, the mixer 60, the reverberation sound control signal generating unit 70, the adder 80, and the output adjustment unit 90, or by an arithmetic processing device such as a computer. The portion consisting of the adder 80 and the output adjustment unit 90 corresponds to the "output signal generating unit" of the present invention.

[0014] The sound signal processing device 10 is connected to a plurality of speakers SP1 to SP64. Although Fig. 1 shows an embodiment using 64 speakers, the number of speakers is not limited to this.

[0015] Sound signals S1-S96 from a plurality of sound sources OBJ1-OBJ96 are input to the sound signal processing device 10. Note that, although Fig. 1 shows an embodiment in which 96 sound sources are used, the number of sound sources is not limited to this.

[0016] The region setting unit 30 divides the reproduction space into a plurality of regions and sets information (region information) about the divided regions. The region information includes position coordinates that determine the boundaries of the regions and position coordinates of representative points set in the regions.

[0017] The area setting unit 30 outputs area information of the multiple areas Area1 to Area8 that have been set to the grouping unit 40. Note that, although Fig. 1 shows an example in which eight areas are set, the number of areas is not limited to this.

[0018] The grouping unit 40 groups the sound sources OBJ1-OBJ96 into a plurality of areas Area1-Area8. Based on the grouping results, the grouping unit 40 generates area-specific sound signals SA1-SA8 for each area Area1-Area8 using the sound signals S1-S96 of the sound sources OBJ1-OBJ96. For example, the grouping unit 40 mixes the sound signals of the plurality of sound sources grouped into area Area1 to generate an area-specific sound signal SA1.

[0019] The grouping unit 40 outputs the multiple area sound signals SA1-SA8 to the early reflection sound control signal generating unit 50. The grouping unit 40 also outputs the sound signals S1-S96 of the sound sources OBJ1-OBJ96 to the mixer 60.

[0020] The early reflection sound control signal generator 50 generates early reflection sound control signals ER1-ER64 for each of the speakers SP1-SP64 from the multiple area sound signals SA1-SA8. The early reflection sound control signals ER1-ER64 are signals that are output to each of the speakers SP1-SP64 in order to simulate the early reflection sounds of the virtual space in the reproduction space. The early reflection sound control signal generator 50 outputs the generated early reflection sound control signals ER1-ER64 to the adder 80.

[0021] In general (detailed configuration and processing will be described later), the early reflection sound control signal generation unit 50 sets an imaginary sound source (virtual sound source) in the reproduction space using the positions of the speakers SP1-SP64 arranged in the reproduction space and the geometric shape of the virtual space. Specific settings of the imaginary sound source will be described later. By using the imaginary sound source, the early reflection sound control signal generation unit 50 generates early reflection sound control signals ER1-ER64 that simulate early reflection sounds in the virtual space. At this time, the early reflection sound control signal generation unit 50 performs desired tone adjustments on the early reflection sound control signals ER1-ER64.

[0022] The mixer 60 is a summing mixer. The mixer 60 mixes the sound signals S1-S96 of the sound sources OBJ1-OBJ96 to generate a reverberation sound generation signal Sr. The mixer 60 outputs the reverberation sound generation signal Sr to the reverberation sound control signal generator 70.

[0023] The reverberation sound control signal generator 70 generates reverberation sound control signals REV1-REV64 for each of the multiple speakers SP1-SP64 from the reverberation sound generation signal Sr. The reverberation sound control signals REV1-REV64 are signals that are output to the speakers SP1-SP64, respectively, to simulate the reverberation sound (late reverberation sound) of the virtual space in the reproduction space. The reverberation sound control signal generator 70 outputs the generated reverberation sound control signals REV1-REV64 to the adder 80.

[0024] In general (detailed configuration and processing will be described later), the reverberation sound control signal generator 70 divides the reproduction space into a plurality of reverberation sound setting regions and generates a reverberation sound control signal for each of the plurality of reverberation sound setting regions. The reverberation sound control signal generator 70 assigns a plurality of speakers SP1-SP64 to the plurality of reverberation sound setting regions. Based on this assignment, the reverberation sound control signal generator 70 sets a reverberation sound control signal for each of the plurality of reverberation sound setting regions to the plurality of speakers SP1-SP64.

[0025] At this time, the reverberation sound control signal generator 70 sets the timing of connecting the early reflection sound and the reverberation sound based on the geometric shape of the reproduction space. The reverberation sound control signal generator 70 gradually increases the level (amplitude) of the reverberation sound control signal in the period before the connection timing, and gradually decreases the level (amplitude) of the reverberation sound control signal in the period after the connection timing.

[0026] The adder 80 adds the early reflection sound control signal and the reverberation sound control signal generated for each of the plurality of speakers SP1 to SP64 to generate a plurality of speaker signals Sat1 to Sat64. For example, the adder 80 adds the early reflection sound control signal for the speaker SP1 to the reverberation sound control signal for the speaker SP1 to generate a speaker signal Sat1. The adder 80 outputs the plurality of speaker signals Sat1 to Sat64 to the output adjustment unit 90.

[0027] The output adjustment unit 90 performs gain control and delay control on the speaker signals Sat1-Sat64 to generate output signals So1-So64. The output adjustment unit 90 outputs the output signals So1-So64 to the speakers SP1-SP64. For example, the output adjustment unit 90 performs gain control and delay control for the speaker SP1 on the speaker signal Sat1 to generate the output signal So1. The output adjustment unit 90 outputs the output signal So1 to the speaker SP1.

[0028] In general (detailed configuration and processing will be described later), the output adjustment unit 90 receives input of acoustic parameters for the reproduction space. The acoustic parameters are parameters that set, for example, the adjustment of the spatial spread in the width direction of the sound space, the adjustment of the spatial spread behind the sound receiving point in the sound space, and the adjustment of the spatial spread in the ceiling direction of the sound space. The output adjustment unit 90 collectively sets the gain values ​​and delay amounts (delay amounts) of the multiple speaker signals Sat1-Sat64 based on the position coordinates of the multiple speakers SP1-SP64 and the acoustic parameters. "Central setting" does not mean setting the gain values ​​and delay amounts for each speaker individually, but means, for example, setting the gain values ​​and delay amounts for each speaker simply by inputting the position coordinates of each speaker into a specific calculation formula common to all speakers. The output adjustment unit 90 performs gain control and delay control on the multiple speaker signals Sat1-Sat64 using the set gain values ​​and delay values.

[0029] [Outline of sound signal processing method] Fig. 2 is a flowchart of a sound signal processing method according to an embodiment of the present invention. Fig. 2 shows a sound signal processing method realized by the sound signal processing device 10 of Fig. 1. Note that the contents of each process shown in Fig. 2 have been explained in the explanation of Fig. 1 above, and therefore will be described briefly.

[0030] (Grouping of sound sources OBJ1-OBJ96) The grouping unit 40 groups the plurality of sound sources OBJ1-OBJ96 into a plurality of areas Area1-Area8 (S11).

[0031] (Generation of early reflection control signals) The early reflection sound control signal generator 50 sets a tone color for the early reflection sound for each group (SS12). The early reflection sound control signal generator 50 sets an imaginary sound source for each group (S13). The early reflection sound control signal generator 50 uses the tone color and the imaginary sound source to generate an early reflection sound control signal for each of the multiple speakers SP1-SP64 (S14).

[0032] (Generation of reverberation control signals) The mixer 60 sums sound signals S1-S96 of multiple sound sources OBJ1-OBJ96 (S21). The reverberation sound control signal generator 70 sets the connection timing between the early reflection sounds and the reverberation sounds based on the geometric shape of the reproduction space (S22). The reverberation sound control signal generator 70 generates a reverberation sound control signal using the connection timing that has been set (S23). The reverberation sound control signal generator 70 assigns the generated reverberation sound control signal to multiple speakers SP1-SP64 based on the position coordinates of the multiple speakers SP1-SP64 in the reproduction space (S24).

[0033] (Output processing to multiple speakers) The adder 80 adds the early reflection sound control signal and the reverberation sound control signal for each of the plurality of speakers SP1 to SP64 to generate speaker signals Sat1 to Sat64 (S31).

[0034] The output adjustment unit 90 generates output signals So1-So64 from the speaker signals Sat1-Sat64 using acoustic parameters that realize the reverberation localization and spatial expansion in the reproduction space (S32). The output adjustment unit 90 outputs the output signals So1-So64 to the multiple speakers SP1-SP64 (S33).

[0035] By using the above-described configuration and processing, the sound signal processing device 10 (sound signal processing method) can obtain the following various effects.

[0036] (1) The sound signal processing device 10 (sound signal processing method) generates early reflection sounds by grouping sound sources into regions obtained by dividing the reproduction space, thereby achieving clear sound image localization and a rich spatial expansion. In this case, the reverberation sound remains constant throughout the entire reproduction space, and only the early reflection sounds change depending on the position of the sound source. Therefore, for example, if the position of the sound source moves, the movement of the sound from this sound source becomes smoother.

[0037] (2) The sound signal processing device 10 (sound signal processing method) generates an early reflection sound control signal using an imaginary sound source, thereby enabling early reflection sounds due to the geometric shape of a virtual space to be more faithfully simulated in a reproduction space.

[0038] (3) The sound signal processing device 10 (sound signal processing method) adjusts the tone of the early reflection sound control signal, thereby eliminating the unnatural tone of the early reflection sound simulated only by an imaginary sound source, for example.

[0039] (4) The sound signal processing device 10 (sound signal processing method) can make the transition from the early reflection sound to the reverberation sound smoother and more natural by setting the timing of the connection between the early reflection sound control signal and the reverberation sound control signal based on the geometric shape of the playback space.

[0040] (5) The sound signal processing device 10 (sound signal processing method) adjusts the gain values ​​and delay amounts of the speaker signals Sat1-Sat64, which include the early reflection sound control signal and the reverberation sound control signal, all at once, thereby realizing the user's desired sound field in the playback space with easier operational input.

[0041] [Specific explanation of each signal processing unit and each process] The following describes in detail each of the signal processing units and the processes described above. First, the early reflections, reverberation, and imaginary sound sources necessary for understanding the present invention will be explained with reference to the drawings.

[0042] [Early reflections and reverberation] Fig. 3 shows the discrete waveforms of a typical sound, including direct sound, early reflections, and reverberation (late reverberation). For example, a hall where a performance or content is played back has a closed space surrounded by walls. When sound is generated in this closed space, the direct sound, early reflections, and reverberation (late reverberation) arrive at the sound receiving point.

[0043] Direct sound is sound that arrives directly from the source to the receiving point.

[0044] Early reflected sound is sound that is generated at the source position and reflects off the walls, floor, and ceiling before reaching the receiving point. Therefore, early reflected sound arrives at the receiving point after the direct sound. The volume (level) of early reflected sound is lower than the volume (level) of the direct sound. If there is one reflection, it is a first-order reflected sound, and if there are n reflections, it is an nth-order reflected sound. The arrival direction and volume of early reflected sound at the receiving point are greatly affected by the source position of the sound.

[0045] Reverberant sound arrives at the sound receiving point after the initial reflections. Reverberant sound is sound that arrives at the sound receiving point after multiple reflections of sound generated at the source location. In other words, reverberant sound is sound that arrives at the sound receiving point after being reflected multiple times and attenuated. Therefore, the volume (level) of reverberant sound is smaller than that of early reflections. Furthermore, the arrival direction and volume of reverberant sound are less affected by the sound generation location than those of early reflections.

[0046] [imaginary sound source] Figures 4(A) and 4(B) are diagrams showing the concept of setting an imaginary sound source. Note that, for ease of explanation, Figures 4(A) and 4(B) show the concept of setting an imaginary sound source in two dimensions, but the imaginary sound source can be set in three dimensions using the same concept. That is, in the actual reproduction space, when the sound sources are not aligned on a single plane but are spatially arranged, and the virtual space is set three-dimensionally, the imaginary sound source is set in three dimensions.

[0047] In the reproduction space, there exists a sound source SS and a sound receiving point RP. Note that the sound source SS shown in Figures 4(A) and 4(B) has a different meaning from the sound source OBJ described above, and refers to something that generates a general sound. Also, in the reproduction space, a virtual wall IWL is set to realize the sound field of the virtual space. The virtual wall IW is obtained from the geometric shape of the virtual space.

[0048] The sound source SS and the sound receiving point RP exist in a space surrounded by a virtual wall IWL. The virtual wall IWL includes virtual wall IWL1, virtual wall IWL2, virtual wall IWL3, and virtual wall IWL4. The virtual walls IWL1 and IWL4 are arranged to sandwich the sound source SS and the sound receiving point RP in a first direction of the reproduction space (the vertical direction in FIGS. 4A and 4B). The virtual wall IWL1 is arranged closer to the sound source SS than the sound receiving point RP, and the virtual wall IWL4 is arranged closer to the sound receiving point RP than the sound source SS. The virtual walls IWL2 and IWL3 are arranged to sandwich the sound source SS and the sound receiving point RP in a second direction of the reproduction space (the horizontal direction in FIGS. 4A and 4B). The virtual wall IWL2 is arranged closer to the sound source SS than the sound receiving point RP, and the virtual wall IWL3 is arranged closer to the sound receiving point RP than the sound source SS.

[0049] If virtual walls IWL1, IWL2, IWL3, and IWL4 were actually walls that reflected sound, the sound emitted from sound source SS would be reflected by virtual walls IWL1, IWL2, and IWL3 and reach sound receiving point RP, as shown in Fig. 4(B). Note that although Fig. 4(B) does not show reflection from virtual wall IWL4, reflection also occurs from virtual wall IWL4 in the same way as from virtual walls IWL1, IWL2, and IWL3.

[0050] However, virtual walls IWL1, IWL2, IWL3, and IWL4 do not actually exist in the reproduction space. Therefore, as shown in Fig. 4(A), the sound signal processing device 10 sets imaginary sound sources IS1, IS2, and IS3 by treating the reflection of sound on the wall surfaces as specular reflection.

[0051] Specifically, the sound signal processing device 10 sets an imaginary sound source IS1 at a position line-symmetrical with respect to the sound source SS, with the virtual wall IWL1 as a reference line. The sound signal processing device 10 sets an imaginary sound source IS2 at a position line-symmetrical with respect to the sound source SS, with the virtual wall IWL2 as a reference line. The sound signal processing device 10 sets an imaginary sound source IS3 at a position line-symmetrical with respect to the sound source SS, with the virtual wall IWL3 as a reference line. Note that by adjusting the acoustic power of each imaginary sound source IS, it is possible to simulate energy loss due to reflection at the virtual wall IWL.

[0052] By making these settings, the sound generated by imaginary sound source IS1 will be the same as the sound generated by sound source SS and reflected by virtual wall IW1. The sound generated by imaginary sound source IS2 will be the same as the sound generated by sound source SS and reflected by virtual wall IW2. The sound generated by imaginary sound source IS3 will be the same as the sound generated by sound source SS and reflected by virtual wall IW3. Note that although the imaginary sound source for virtual wall IWL4 is not shown in Figures 4(A) and 4(B), a virtual sound source can be set for virtual wall IWL4 in the same way as for virtual walls IWL1, IWL2, and IWL3.

[0053] By setting the imaginary sound source in this way, the sound signal processing device 10 can simulate the early reflected sound of the virtual space in a reproduction space where there are no real walls of the virtual space.

[0054] [Configuration and processing of grouping unit 40] Fig. 5 is a functional block diagram showing an example of the configuration of the grouping section 40. Fig. 6 is a flowchart showing a method for grouping sound sources.

[0055] As shown in FIG. 5, the grouping unit 40 includes a sound source position detection unit 41, an area determination unit 42, and a matrix mixer 400.

[0056] The sound source position detection unit 41 detects the position coordinates of multiple sound sources OBJ1-OBJ96 in the reproduction space (FIG. 6: S111). For example, the sound source position detection unit 41 detects the position coordinates of the sound sources OBJ1-OBJ96 based on an operational input from the user. Alternatively, the sound source position detection unit 41 is provided with a position detection sensor for detecting the sound sources OBJ1-OBJ96, and detects the position coordinates of the sound sources OBJ1-OBJ96 based on the positions detected by the position detection sensor.

[0057] The sound source position detection unit 41 outputs the position coordinates of the sound sources OBJ1-OBJ96 to the area determination unit .

[0058] The area determination unit 42 groups the sound sources OBJ1-OBJ96 into multiple areas Area1-Area8 using the area information of the multiple areas Area1-Area8 from the area setting unit 30 and the position coordinates of the sound sources OBJ1-OBJ96 from the sound source position detection unit 41 (FIG. 6: S112). More specifically, the area determination unit 42 performs grouping as follows.

[0059] Fig. 7 is a diagram showing the concept of grouping multiple sound sources into multiple regions. In Fig. 7, the top of the figure is the front of the hall, which is the reproduction space, and the bottom of the figure is the rear of the hall.

[0060] The region setting unit 30 sets a reference point Pso for dividing the reproduction space into regions. For example, as shown in FIG. 7, the region setting unit 30 sets the center position of a hall that realizes the reproduction space as the reference point Pso. Note that the region setting unit 30 can also use a point (position) set by the user as the reference point. For example, the region setting unit 30 can use a sound receiving point or the like set by the user as the reference point.

[0061] The area setting unit 30 sets eight areas, Area1-Area8, so as to divide the entire circumference on a plane into eight parts with the reference point Pso for area division at the center. For example, in the case of FIG. 7, the area setting unit 30 sets multiple areas, Area1, Area2, and Area3, in front of the reference point Pso in the hole (reproduction space). The area setting unit 30 also sets area Area4 to the left of the reference point Pso as facing forward of the hole, and sets area Area5 to the right of the reference point Pso as facing forward of the hole. The area setting unit 30 also sets multiple areas, Area6, Area7, and Area8, in the hole (reproduction space) behind the reference point Pso.

[0062] Note that this area setting is just an example, and other settings are possible as long as the entire playback space is covered by the multiple areas that are set. Also, this explanation shows the setting of planar areas, but spatial areas can also be set in the same way. For example, the vertical range of area Area1 is also included in area Area1.

[0063] The area setting unit 30 sets representative points RP1-RP8 in each of the multiple areas Area1-Area8. For example, the area setting unit 30 sets the multiple representative points RP1-RP8 at the center positions of the multiple areas Area1-Area8. Alternatively, in the case of areas spreading radially as shown in FIG. 7, for example, the area setting unit 30 sets representative points at positions a predetermined distance from the reference point Pso on a line passing through the center of a corner of the area spreading radially. Note that these methods of setting representative points are merely examples, and other methods may be used as long as they can set one representative point in one area and reliably perform grouping processing of sound sources.

[0064] The area setting unit 30 outputs area information of the multiple areas Area1 to Area8 to the area determination unit 42 of the grouping unit 40 and the matrix mixer 400. The area information of the multiple areas Area1 to Area8 includes the position coordinates of the representative points RP1 to RP8 of the areas Area1 to Area8, coordinate information indicating the boundary lines that form the shapes of the areas Area1 to Area8, etc.

[0065] (Method of grouping sound sources into regions using representative points) FIG. 8A is a flowchart showing a method for grouping sound sources using representative points.

[0066] The area determination unit 42 acquires the position coordinates of representative points RP1-RP8 from the area information of multiple areas Area1-Area8 (S131). The area determination unit 42 calculates the distance between the position coordinates of the sound source to be determined for grouping and the position coordinates of representative points RP1-RP8 (S132). The area determination unit 42 groups the sound sources into an area including the representative point that is the shortest distance away (S133).

[0067] For example, in the case of sound source OBJ1 in the example of FIG. 7, the area determination unit 42 detects the position coordinates of the sound source OBJ1 and obtains the position coordinates of multiple representative points RP1-RP8. The area determination unit 42 calculates the distance between the sound source OBJ1 and each of the multiple representative points RP1-RP8 from the position coordinates of the sound source OBJ1 and the position coordinates of the multiple representative points RP1-RP8. The area determination unit 42 detects that the distance between the sound source OBJ1 and the representative point RP1 is shorter than the distance between the sound source OBJ1 and the other representative points RP2-RP8. In other words, the area determination unit 42 detects that the distance between the sound source OBJ1 and the representative point RP1 is the shortest. The area determination unit 42 groups the sound source OBJ1 into area Area1 associated with the representative point RP1.

[0068] (Method of grouping sound sources into regions using region boundaries) FIG. 8B is a flowchart showing a method for grouping sound sources using region boundaries.

[0069] The area determination unit 42 acquires coordinate information (boundary coordinates) representing the boundary lines of each of the areas Area1 to Area8 from the area information of the multiple areas Area1 to Area8 (S136). The area determination unit 42 determines whether the position coordinates of the sound source to be determined for grouping are inside each of the areas Area1 to Area8 (S137). For example, the area determination unit 42 uses a crossing number algorithm to determine whether the sound source is inside or outside the area. If the sound source is inside the area (S137: YES), the area determination unit 42 groups the sound source into this area (S138).

[0070] For example, in the case of sound source OBJ1 in the example of FIG. 7, the area determination unit 42 detects the position coordinates of the sound source OBJ1 and acquires coordinate information (boundary coordinates) that represent the boundary lines of the multiple areas Area1 to Area8. The area determination unit 42 determines whether the sound source OBJ1 is inside or outside the multiple areas Area1 to Area8 based on the position coordinates of the sound source OBJ1 and the boundary coordinates of the multiple areas Area1 to Area8. The area determination unit 42 detects that the sound source OBJ1 is inside the area Area1. The area determination unit 42 groups the sound source OBJ1 into the area Area1.

[0071] The area determination unit 42 groups the input sound sources OBJ1-OBJ96 into multiple areas Area1-Area8. For example, in the example of Fig. 7, the area determination unit 42 groups sound sources OBJ1 and OBJ4 into area Area1, group sound source OBJ2 into area Area2, and group sound source OBJ3 into area Area5.

[0072] The area determination unit 42 outputs the grouping information to the matrix mixer 400. The grouping information is the information indicating which sound sources are grouped into which areas, as described above.

[0073] The matrix mixer 400 generates area-specific sound signals SA1-SA8 for each of the areas Area1-Area8 using sound signals S1-S96 of the multiple sound sources OBJ1-OBJ96 based on the grouping information. For example, if multiple sound sources are grouped into an area, the matrix mixer 400 mixes the sound signals of these multiple sound sources to generate an area-specific sound signal for that area. The matrix mixer 400 outputs the area-specific sound signal for each area to the early reflection sound control signal generation unit 50. Note that if at least one sound source is grouped into an area, the matrix mixer 400 outputs the sound signal of this sound source to the early reflection sound control signal generation unit 50 as the area-specific sound signal for that area.

[0074] In the example of FIG. 7, sound sources OBJ1 and OBJ4 are grouped in area Area1. The matrix mixer 400 mixes the sound signal S1 of sound source OBJ1 and the sound signal S4 of sound source OBJ4 to generate and output a regional sound signal SA1 for area Area1. Furthermore, sound source OBJ2 is grouped in area Area2. The matrix mixer 400 outputs the sound signal S2 of sound source OBJ2 as a regional sound signal SA2 for area Area2. Furthermore, sound source OBJ3 is grouped in area Area5. The matrix mixer 400 outputs the sound signal S3 of sound source OBJ3 as a regional sound signal SA5 for area Area5.

[0075] By realizing such a configuration and processing, the sound signal processing device 10 can group multiple sound sources for each of multiple regions that divide the sound space and generate an early reflection sound control signal. As a result, the sound signal processing device 10 can reproduce early reflection sounds according to the position of the sound source, thereby achieving clear sound image localization and a rich spatial expansion.

[0076] Although the above description does not specifically describe the case where the sound source moves, when the sound source moves, the grouping unit 40 performs the process shown in Fig. 9. Fig. 9 is a flowchart showing an example of a grouping method based on the movement of the sound source.

[0077] The sound source position detection unit 41 detects the movement of the sound source (S104). The sound source position detection unit 41 detects the movement of the sound source, for example, by an operation input from the user. Alternatively, the sound source position detection unit 41 detects the movement of the sound source by continuously detecting the sound source position using a position detection sensor. The sound source position detection unit 41 detects the position coordinates of the sound source after the movement and outputs them to the area determination unit 42.

[0078] The area determination unit 42 uses the position coordinates of the sound source after movement to perform grouping into a plurality of areas Area1 to Area8, as described above.

[0079] By performing such processing, the sound signal processing device 10 can generate an early reflection sound control signal that corresponds to the position of the sound source after it has moved, even if the sound source moves. This allows the sound signal processing device 10 to reproduce changes in the early reflection sound that correspond to the movement of the sound source, and can achieve clear sound image localization and a rich spatial expansion that correspond to the movement, even if the sound source moves.

[0080] Furthermore, when such movement of the sound source occurs, the sound signal processing device 10 can perform cross-fade processing on the early reflection sound control signal before the movement and the early reflection sound control signal after the movement. For example, when the sound source moves, the sound signal processing device 10 gradually lowers the sound signal component of this sound source in the regional sound signal including the sound source before the movement. On the other hand, the sound signal processing device 10 gradually raises the sound signal component of this sound source in the regional sound signal including the sound source after the movement.

[0081] By performing such processing, the sound signal processing device 10 can suppress discontinuous changes in the early reflected sounds when the sound source moves, thereby allowing the sound signal processing device 10 to change the early reflected sounds more smoothly in accordance with the movement of the sound source when the sound source moves.

[0082] Furthermore, the matrix mixer 400 outputs sound signals S1-S96 of the multiple sound sources OBJ1-OBJ96 to the mixer 60. As described above, the mixer 60 sums the sound signals S1-S96 to generate a reverberation sound generation signal Sr and outputs it to the reverberation sound control signal generator 70. The reverberation sound control signal generator 70 uses the reverberation sound generation signal Sr to generate reverberation sound control signals REV1-REV64.

[0083] By performing this processing, the reverberation sound is not affected by the position or movement of the sound source. Therefore, the sound signal processing device 10 can reproduce the movement of the sound source more clearly by changing the early reflection sound while keeping the reverberation sound in the reproduction space constant even if the sound source moves.

[0084] [Generation of early reflection control signals] Fig. 10 is a functional block diagram showing an example of the configuration of the early reflection sound control signal generator 50. Fig. 11 is a diagram showing an example of a GUI.

[0085] 10, the early reflection sound control signal generation unit 50 includes an FIR filter circuit 51, an LD tap circuit 52, an addition processing unit 53, a tone color setting unit 501, an imaginary sound source setting unit 502, and an operation unit 500. The FIR filter circuit 51 includes a plurality of FIR filters 511-518. The LD tap circuit 52 includes a plurality of LD taps 521-528, an output speaker setting unit 5201, and a coefficient setting unit 5202. The order of the FIR filter circuit 51 and the LD tap circuit 52 may be reversed.

[0086] [Early reflection sound tone adjustment] The operation unit 500 receives, from the user, designation information for the tone color to be added to the early reflection sound, and outputs it to the tone color setting unit 501. The designation information for the tone color is information (information representing filter characteristics) that designates, for example, emphasis on low frequencies, emphasis on high frequencies, the volume of the early reflection sound, the attenuation characteristics of the early reflection sound, etc.

[0087] As a specific example, the operation unit 500 accepts operations through a GUI 100 (Graphical User Interface) as shown in FIG.

[0088] The GUI 110 includes a setting display window 111 , a plurality of controls 112 , a knob 1131 , and an adjustment value display window 1132 .

[0089] The setting display window 111 displays the shape of the virtual wall IWL of the virtual space that has been set using the multiple controls 112 and knobs 1131. At this time, the setting display window 111 can display the position of the sound source SS, the position of the speaker SP, the position of the sound receiving point RP, and the coordinate axes of the reproduction space, which have been separately set, together with the virtual wall IWL.

[0090] The multiple operators 112 are associated with preset virtual space samples (various halls, rooms, etc.). Although not shown in the figure, the multiple operators 112 display an index (for example, a hall name) that clearly indicates the virtual space sample associated with each operator 112.

[0091] A knob 1131 is used to set the room size, and an adjustment value display window 1132 displays the setting value of the room size.

[0092] The GUI 100 accepts various operations for adjusting the tone color. For example, the GUI 100 includes a plurality of controls 112, such as a control for a bass range, a control for a treble range, a control for adjusting the volume, and a control for adjusting the attenuation characteristics, and accepts operations using these controls.

[0093] When the user operates a desired operator using the GUI 100, the operation unit 500 detects this operation and sets the timbre designation information in accordance with this operation.

[0094] For example, when the operation unit 500 receives a selection of a plurality of operators 112, it acquires designation information of a tone color that is set in advance in the virtual space associated with the operator 112. Furthermore, when the operation unit 500 receives an operation using an operator for a low frequency range, an operator for a high frequency range, an operator for adjusting the volume, an operator for adjusting the attenuation characteristics, or the like, it acquires designation information of the tone color that is set by these operators.

[0095] Although not shown, GUI 100 can also display the timbre designation information using, for example, the filter coefficients of FIR filters 511-518 (described later), a rough waveform, etc. In this case, when GUI 100 receives an adjustment to the timbre designation information, it can change the display in accordance with the adjustment. For example, GUI 100 can change the waveform display in accordance with the adjustment.

[0096] The timbre setting unit 501 sets the filter coefficients of the FIR filters 511-518 of the FIR filter circuit 51 based on the timbre designation information. For example, when the timbre setting unit 501 receives designation information to emphasize the bass range, it sets filter coefficients that boost the low frequencies of the FIR filters 511-518 of the FIR filter circuit 51. On the other hand, when the timbre setting unit 501 receives designation information to emphasize the treble range, it sets filter coefficients that boost the high frequencies of the FIR filters 511-518 of the FIR filter circuit 51. The timbre setting unit 501 outputs the set filter coefficients to the FIR filter circuit 51. Note that the timbre setting unit 501 is not limited to setting the filter coefficients, but can also set and adjust the sampling frequency and filter length as filter characteristics.

[0097] Furthermore, based on the timbre designation information, the timbre setting unit 501 sets the gain value of each tap of the FIR filters 511-518 of the FIR filter circuit 51. The timbre setting unit 501 outputs the set gain value to the FIR filter circuit 51.

[0098] The plurality of FIR filters 511-518 are filters corresponding to the regional sound signals SA1-SA8, respectively. The regional sound signals SA1-SA8 are input to the FIR filters 511-518. For example, as shown in Fig. 10, the regional sound signal SA1 is input to the FIR filter 511, the regional sound signal SA2 is input to the FIR filter 512, the regional sound signal SA3 is input to the FIR filter 513, and the regional sound signal SA4 is input to the FIR filter 514. The regional sound signal SA5 is input to the FIR filter 515, the regional sound signal SA6 is input to the FIR filter 516, the regional sound signal SA7 is input to the FIR filter 517, and the regional sound signal SA8 is input to the FIR filter 518.

[0099] The multiple FIR filters 511-518 have the same number of taps. For example, the multiple FIR filters 511-518 have 16,000 taps. Note that this number of taps is just an example, and may be set based on the resource conditions of the sound signal processing device 10, the accuracy of the tone color of the early reflection sound to be reproduced, etc.

[0100] The plurality of FIR filters 511-518 perform filtering (convolution operation) on the plurality of regional sound signals SA1-SA8, respectively, using the filter coefficients and gain values ​​set by the timbre setting unit 501. As a result, the plurality of FIR filters 511-518 generate post-filtering regional sound signals SA1f-SA8f. For example, the FIR filter 511 performs filtering (convolution operation) on the regional sound signal SA1 using the filter coefficients and gain values ​​set by the timbre setting unit 501, to generate post-filtering regional sound signal SA1f. Similarly, the plurality of FIR filters 512-518 individually generate post-filtering regional sound signals SA2f-SA8f from the regional sound signals SA2-SA8.

[0101] The plurality of FIR filters 511-518 output the filtered regional sound signals SA1f-SA8f to the plurality of LD taps 521-528. For example, the FIR filter 511 outputs the filtered regional sound signal SA1f to the LD tap 521. Similarly, the plurality of FIR filters 512-518 output the filtered regional sound signals SA2f-SA8f to the plurality of LD taps 522-528.

[0102] The timbre designation information is not limited to the emphasis information on the frequency range, but also includes information for adjusting the waveform of the early reflection sound to a user's desired characteristic. By using such timbre designation information, the sound signal processing device 10 can realize early reflection sounds with a wider variety of timbres that meet the user's preferences.

[0103] [Virtual sound source settings and LDtap settings] The imaginary sound source setting unit 502 sets an imaginary sound source based on the position coordinates of the sound receiving point in the reproduction space and the geometric shape of the virtual space.

[0104] 12 is a flowchart showing an example of the setting process of the imaginary sound source. The imaginary sound source setting unit 502 acquires the position coordinates of the sound receiving point in the reproduction space (S131). For example, the imaginary sound source setting unit 502 acquires the position coordinates of the sound receiving point in the reproduction space by operation input from the user, detection of the position by a position detection sensor, etc.

[0105] The imaginary sound source setting unit 502 acquires the geometric shape of the virtual space (S132). For example, the imaginary sound source setting unit 502 acquires the geometric shape of the virtual space through operation input from the user, etc. The geometric shape of the virtual space includes a group of coordinates that represent the shapes of walls placed in the virtual space, etc.

[0106] The virtual sound source setting unit 502 is connected to the GUI 100. When the user selects a desired operator 112 from the multiple operators 112, the GUI 100 reads and acquires the geometric shape of the virtual space associated with this operator 112. Furthermore, when the user adjusts the room size using the knob 1131, the GUI 100 acquires the adjustment value of this room size.

[0107] The imaginary sound source setting unit 502 acquires the position coordinates of the geometric shape of the virtual space in which the room size is set based on each setting acquired by the GUI 100 in this way. The imaginary sound source setting unit 502 also acquires the position coordinates of the sound source SS and the position coordinates of the sound receiving point RP (room center). Using this acquired information, the imaginary sound source setting unit 502 sets the imaginary sound source as follows: The imaginary sound source setting unit 502 matches the coordinate system of the reproduction space with the coordinate system of the virtual space. Using the position coordinates of the sound receiving point in the reproduction space and the geometric shape of the virtual space, the imaginary sound source setting unit 502 sets the position coordinates of the imaginary sound source in the reproduction space according to the concept using the above-mentioned Figures 4(A) and 4(B) (S133).

[0108] Figures 13(A) and 13(B) show examples of setting imaginary sound sources for different geometric shapes. In Figure 13(A), the imaginary wall IWL is rectangular, and in Figure 13(B), the imaginary wall IWLh is hexagonal.

[0109] In this way, when the geometric shape of the virtual space is different, even if the position coordinates of the sound source SSa and the sound receiving point RP do not change, the positional relationship between the sound source SSa and the sound receiving point RP and the virtual wall IWL and the positional relationship between the sound source SSa and the sound receiving point RP and the virtual wall IWLh are different. As a result, the positions of the imaginary sound sources IS1a, IS2a, and IS3a set in the case of Fig. 13(A) are different from the positions IS1ah, IS2ah, and IS3ah of the imaginary sound sources set in Fig. 13(B).

[0110] Figures 14(A), 14(B), and 14(C) are diagrams showing examples of setting a virtual sound source. Figures 14(A), 14(B), and 14(C) are diagrams showing planar changes. Figure 14(B) shows a case where the position of the sound source SSa relative to the reference point (sound receiving point RP) is the same as in Figure 14(A), but the size of the virtual space is different. Figure 14(C) shows a case where the size of the virtual space is the same as in Figure 14(A), but the positional relationship between the reference point of the virtual space and the reference point (sound receiving point) of the reproduction space has changed (the room center of the reproduction space has changed).

[0111] As can be seen from the comparison between Fig. 14(A) and Fig. 14(B), the size of the virtual space in the reproduction space (shown as virtual wall IWL in Fig. 14(A) and virtual wall IWLc in Fig. 14(B)) is different, which results in different distances and positional relationships between the sound source that is the source of the imaginary sound source and the virtual wall. As a result, the positions of imaginary sound sources IS1a, IS2a, and IS3a set in the case of Fig. 14(A) and the positions of imaginary sound sources IS1c, IS2c, and IS3c set in the case of Fig. 14(B) are different.

[0112] 14(A) and 14(C), the position of the imaginary sound source in the reproduction space (the position of the imaginary sound source relative to the sound receiving point RP and the speaker) moves as the positional relationship between the reference point in the virtual space and the sound receiving point RP changes. As a result, the positions of the imaginary sound sources IS1a, IS2a, and IS3a set in the case of Fig. 14(A) differ from the positions of the imaginary sound sources IS1as, IS2as, and IS3as set in the case of Fig. 14(C).

[0113] Figures 15(A), 15(B), and 15(C) are diagrams showing examples of setting an imaginary sound source. Figure 15 is a diagram showing changes in the height direction. Figures 15(A), 15(B), and 15(C) are diagrams showing changes in height.

[0114] The height of the ceiling is different between Figures 15(A) and 15(B). That is, the distance (height) from the virtual wall IWFL on the floor to the virtual wall IWCL on the ceiling in the virtual wall IWL shown in Figure 15(A) is different from the distance (height) from the virtual wall IWFL on the floor to the virtual wall IWCLL on the ceiling in the virtual wall IWLL shown in Figure 15(B).

[0115] As can be seen from the comparison results between Figure 15(A) and Figure 15(B), the difference in ceiling height causes differences in the distance and positional relationship between the sound source that is the source of the imaginary sound source and the virtual walls IWCL and IWCLL on the ceiling. As a result, the position of the imaginary sound source IS1Ca set in the case of Figure 15(A) and the position of the imaginary sound source IS1CaL set in the case of Figure 15(B) are different.

[0116] The shapes of the ceiling are different between Figure 15(A) and Figure 15(C). That is, the shape of the virtual wall IWCL of the ceiling in the virtual wall IWL shown in Figure 15(A) is different from the shape of the virtual wall IWCLx of the ceiling in the virtual wall IWLx shown in Figure 15(C).

[0117] As can be seen from the comparison results between Fig. 15(A) and Fig. 15(C), the positional relationship between the sound source that is the source of the imaginary sound source and the virtual walls IWCL and IWCLx of the ceiling differs due to the different shapes of the ceiling. As a result, the position of the imaginary sound source IS1Ca set in the case of Fig. 15(A) differs from the position of the imaginary sound source IS1Cax set in the case of Fig. 15(C).

[0118] In this way, the imaginary sound source setting unit 502 can optimally set the position of the imaginary sound source in the reproduction space in accordance with the geometric shape of the virtual space and the positional relationship between the reproduction space and the virtual space. This allows the sound signal processing device 10 to clarify the sound image position of the early reflection sound in accordance with the position coordinates of the speaker in the reproduction space, the geometric shape of the virtual space, and the positional relationship between the reproduction space and the virtual space.

[0119] The imaginary sound source setting unit 502 outputs the position coordinates of the imaginary sound source set for each of the multiple areas Area1 to Area8 to the output speaker setting unit 5201 of the LDtap circuit 52.

[0120] The output speaker setting unit 5201 sets the imaginary sound source IS to be allocated to each speaker based on the position coordinates of the imaginary sound source IS, the position coordinates of the sound receiving point RP, and the position coordinates of the multiple speakers SP1 to SP64. Fig. 16 is a flowchart showing the process of allocating the imaginary sound source to the speakers.

[0121] The output speaker setting unit 5201 acquires the position coordinates of the imaginary sound source from the imaginary sound source setting unit 502 (S141). The output speaker setting unit 5201 acquires the position coordinates of the sound receiving point in the reproduction space, for example, through an operation input from the user (S142). The output speaker setting unit 5201 acquires the position coordinates of the multiple speakers SP1-SP64, for example, through an operation input from the user (S143).

[0122] The output speaker setting unit 5201 sets the area covered by the imaginary sound source for each speaker based on the positional relationship between the sound receiving point RP and the plurality of speakers SP1 to SP64 in the reproduction space (S144).

[0123] More specifically, the output speaker setting unit 5201 sets the responsible area of ​​the imaginary sound source for each speaker as follows. Fig. 17(A) and Fig. 17(B) are diagrams showing the concept of allocating imaginary sound sources to speakers. Fig. 17(A) shows the concept of allocation using the azimuth angle φ, and Fig. 17(B) shows the concept of allocation using the elevation / depression angle θ. Furthermore, although the following description will be given using speaker SP1 as an example, the output speaker setting unit 5201 sets the responsible areas for the other speakers SP2-SP64 in a similar manner.

[0124] The output speaker setting unit 5201 sets a straight line (dashed line in FIG. 17A) passing through the sound receiving point RP and the speaker SP1 using the position coordinates of the sound receiving point RP and the position coordinates of the speaker SP1. As shown in FIG. 17A, the output speaker setting unit 5201 sets an azimuth angle φ, which spreads toward the speaker SP1 on a plane with the sound receiving point RP as the reference point, with respect to this straight line (dashed line in FIG. 17A). The azimuth angle φ is an angle formed in the horizontal direction with respect to the straight line passing through the sound receiving point RP and the speaker SP1. Furthermore, as shown in FIG. 17B, the output speaker setting unit 5201 sets an elevation / depression angle θ, which spreads in the up-down direction perpendicular to the plane, with respect to the above-mentioned straight line (dashed line in FIG. 17B). The elevation / depression angle θ is an angle formed in the vertical direction (direction perpendicular to the horizontal direction) with respect to the straight line passing through the sound receiving point RP and the speaker SP1.

[0125] The output speaker setting unit 5201 sets the space on the speaker SP1 side of the plane determined by the azimuth angle φ and the elevation / depression angle θ as the responsible region RGSP1 of the speaker SP1.

[0126] The output speaker setting unit 5201 acquires the position coordinates of a plurality of imaginary sound sources IS (a plurality of imaginary sound sources ISa-ISg in the case of FIG. 17).

[0127] The output speaker setting unit 5201 determines whether the multiple imaginary sound sources ISa-ISg are within the assigned region RGSP1 by using the position coordinates of the multiple imaginary sound sources ISa-ISg and the coordinates representing the assigned region RGSP1. This determination can be realized by the same method as the grouping of sound sources into regions described above.

[0128] By performing this determination process, for example, in the case shown in FIG. 14, the output speaker setting unit 5201 determines that multiple imaginary sound sources ISa, ISb, ISc, and ISd are within the assigned region RGSP1, and determines that multiple imaginary sound sources ISe, ISf, and ISg are outside the assigned region RGSP1.

[0129] The output speaker setting unit 5201 assigns the plurality of imaginary sound sources ISa, ISb, ISc, and ISd that are determined to be within the assigned region RGSP1 to the speaker SP1.

[0130] The output speaker setting unit 5201 outputs allocation information of multiple imaginary sound sources to the multiple speakers SP1-SP64 to the coefficient setting unit 5202. At this time, the output speaker setting unit 5201 outputs the position coordinates of the sound receiving point RP, the position coordinates of the multiple speakers SP1-SP64, and the position coordinates of the multiple imaginary sound sources to the coefficient setting unit 5202 together with the allocation information.

[0131] The azimuth angle φ is, for example, 60°, and the elevation / depression angle θ is, for example, 45°. The azimuth angle φ and the elevation / depression angle θ are merely examples, and can be set and adjusted by, for example, an operational input from the user.

[0132] The coefficient setting unit 5202 sets tap coefficients to be given to the LD taps 521-528 using the distance between the sound receiving point RP and the plurality of speakers SP1-SP64 and the distance between the sound receiving point RP and the imaginary sound source IS. The tap coefficients to be given to the LD taps 521-528 are the gain values ​​and delay amounts of the LD taps 521-528.

[0133] Fig. 18 is a flowchart showing the coefficient setting process of LDtap Fig. 19(A) and Fig. 19(B) are diagrams for explaining the concept of coefficient setting.

[0134] The coefficient setting unit 5202 calculates the distances (speaker distances) between the sound receiving point PR and the plurality of speakers SP1 to SP64 using the position coordinates of the sound receiving point RP and the position coordinates of the plurality of speakers SP1 to SP64 (S151).

[0135] The coefficient setting unit 5202 calculates the distances (imaginary sound source distances) between the sound receiving point PR and the plurality of imaginary sound sources IS (S152).

[0136] The coefficient setting unit 5202 compares the speaker distances and the imaginary sound source distances for the multiple speakers SP1-SP64 and the multiple imaginary sound sources IS assigned to these speakers SP1-SP64 respectively (S153). For example, in the example of Fig. 17(A), the coefficient setting unit 5202 compares the speaker distances and the imaginary sound source distances for the speaker SP1 and the multiple imaginary sound sources ISa, ISb, ISc, and ISd.

[0137] If the speaker distance is equal to or less than the imaginary sound source distance (S153: YES), the coefficient setting unit 5202 sets the tap coefficients using the imaginary sound source distance as is (S154).

[0138] For example, in the case shown in FIG. 19(A), the imaginary sound source ISa is farther from the sound receiving point RP than the speaker SP1, and the imaginary sound source distance Lia between the sound receiving point RP and the imaginary sound source ISa is greater than the speaker distance Ls1 between the sound receiving point RP and the speaker SP1.

[0139] In this case, the coefficient setting unit 5202 sets the tap coefficients using the distance Da1 between the imaginary sound source ISa and the speaker SP1. Specifically, the coefficient setting unit 5202 sets the gain value and delay amount to be set for the imaginary sound source ISa based on the distance Da1. The coefficient setting unit 5202 sets a smaller gain value as the distance Da1 increases, and sets a larger delay amount as the distance Da1 increases.

[0140] If the speaker distance is greater than the imaginary sound source distance (S153: NO), the coefficient setting unit 5202 determines whether to reproduce this imaginary sound source. In other words, the coefficient setting unit 5202 determines whether to reproduce an imaginary sound source closer to the sound receiving point than the speaker (S155).

[0141] If an imaginary sound source closer to the sound receiving point than the speaker is to be reproduced (S155: YES), the coefficient setting unit 5202 moves the position of this imaginary sound source (S156). More specifically, the coefficient setting unit 5202 moves the position of the imaginary sound source that is closer to the sound receiving point than the speaker to a position farther from the sound receiving point than the speaker. At this time, the coefficient setting unit 5202 moves the position of the imaginary sound source using the difference in distance between the imaginary sound source and the speaker. The coefficient setting unit 5202 sets the tap coefficients using the position coordinates of the imaginary sound source after the movement (S157).

[0142] For example, in the case shown in FIG. 19(B), the imaginary sound source ISd is closer to the sound receiving point RP than the speaker SP1, and the imaginary sound source distance Lid between the sound receiving point RP and the imaginary sound source ISd is smaller than the speaker distance Ls1 between the sound receiving point RP and the speaker SP1.

[0143] In this case, the coefficient setting unit 5202 moves the imaginary sound source ISd using the distance difference Dd between the imaginary sound source distance Lid and the speaker distance Ls1. More specifically, the coefficient setting unit 5202 moves the imaginary sound source ISd to a position at the distance difference Dd on a line passing through the sound receiving point RP and the speaker SP1 and on the opposite side of the sound receiving point RP with the speaker SP1 as a reference. Then, the coefficient setting unit 5202 sets a tap coefficient using this distance difference Dd. Specifically, the coefficient setting unit 5202 sets a gain value and a delay amount to be set for the imaginary sound source ISd based on the distance difference Dd. The coefficient setting unit 5202 sets a smaller gain value as the distance difference Dd is larger, and sets a larger delay amount as the distance difference Dd is larger. Note that conceptually, the imaginary sound source is moved as described above, but in the process of setting the tap coefficients, the coefficient setting unit 5202 may set the tap coefficients based on the distance between the speaker distance and the imaginary sound source distance.

[0144] That is, the coefficient setting unit 5202 moves only the sound receiving point located between the sound receiving point and the speaker. This means that it is preferable not to move an imaginary sound source that is outside the speaker relative to the sound receiving point, but it also includes the case where this outside imaginary sound source moves within a predetermined range. For example, even if this outside imaginary sound source moves, it is sufficient as long as the distance between the outside imaginary sound source and the speaker is within a predetermined range, and this predetermined range is a range within which a change in the early reflection sound control signal due to the movement does not give an uncomfortable feeling to the listener. If the coefficient setting unit 5202 does not reproduce an imaginary sound source that is closer to the sound receiving point than the speaker (S155: NO), it does not set a tap coefficient for this imaginary sound source.

[0145] The coefficient setting unit 5202 sets the tap coefficients set for each of the speakers SP1-SP64 to a plurality of LDtaps. More specifically, the coefficient setting unit 5202 sets the tap coefficients for each of the speakers SP1-SP64 to the LDtaps 521 based on the imaginary sound source positions set in the area Area 1. Similarly, the coefficient setting unit 5202 sets the tap coefficients of the imaginary sound sources assigned to each of the speakers SP1-SP64 to the LDtaps 522-528, respectively, based on the imaginary sound source positions set in the plurality of areas Area 2-Area 8, respectively.

[0146] The multiple LD taps 521-528 perform gain processing and delay processing on the post-filtered regional sound signals SA1f-SA8f according to the set tap coefficients, and output the results to the addition processing unit 53. More specifically, the tap coefficients are set according to the combinations of the imaginary sound source positions of the multiple regions and the respective speakers, as described above. Therefore, the multiple LD taps 521-528 set tap coefficients for each speaker based on the imaginary sound source assigned to that speaker. The multiple LD taps 521-528 perform gain processing and delay processing on the post-filtered regional sound signals SA1f-SA8f for each speaker. The multiple LD taps 521-528 output the signals subjected to the gain processing and delay processing for each speaker.

[0147] For example, when imaginary sound sources ISa, ISb, ISc, and ISd are assigned to the speaker SP1, the LDtap 521 performs gain processing and delay processing on the filtered regional sound signal SA1f using tap coefficients (gain values ​​and delay amounts) based on the imaginary sound sources ISa, ISb, ISc, and ISd. Then, the LDtap 521 outputs this signal to the addition processing unit 53 for the speaker SP1. The multiple LDtap 521-528 perform such processing on the imaginary sound sources for which tap coefficients are set.

[0148] The addition processing unit 53 adds, for each of the speakers SP1 to SP64, the signals after LDtap processing output from the plurality of LDtaps 521 to 528. The addition processing unit 53 outputs these added signals to the adder 80 as early reflection sound control signals ER1 to ER64 for each of the speakers SP1 to SP64.

[0149] By performing such processing, the early reflection sound control signal generating unit 50 can generate an early reflection sound control signal having the following characteristics.

[0150] Figures 20(A) and 20(B) are waveform diagrams showing an example of the relationship between the shape of the virtual space and the components of the early reflection sound control signal realized by LDtap. Figure 20(A) shows the case where the shape of the virtual space is large, and Figure 20(B) shows the case where the shape of the virtual space is small. Figures 20(A) and 20(B) show an example of the components of the early reflection sound control signal when multiple imaginary sound sources are set for one speaker.

[0151] If the positional relationship between the reproduction space and the virtual space does not change, and the positions of the sound receiving points and the speakers do not change, if the shape of the virtual space is large, the distribution of the imaginary sound source will spread over a wider area than if the shape of the virtual space is small. Therefore, as shown in Figures 20(A) and 20(B), if the shape of the virtual space is large, each component set by LDtap521-528 tends to be small, and the distribution range on the time axis will also be wide.

[0152] In this way, by performing the above-described processing, the early reflection sound control signal generator 50 can set optimal tap coefficients according to the shape of the virtual space.

[0153] Furthermore, even if the positional relationship between the virtual space and the reproduction space changes, the speaker position changes, or the sound receiving point changes, the early reflection sound control signal generator 50 can set optimal tap coefficients in response to these changes, just as in the case where the shape of the virtual space changes.

[0154] In this case, the multiple sound sources OBJ1-OBJ96 are optimally assigned to the multiple speakers SP1-SP64 through grouping by the multiple areas Area1-Area8. Then, the multiple imaginary sound sources are optimally set for these multiple speakers SP1-SP64. Therefore, even if there are changes in the relationship between the virtual space and the reproduction space, changes in the sound receiving point RP, changes in the positions of the multiple speakers SP1-SP64, and changes in the positions of the sound sources OBJ1-OBJ96, the sound signal processing device 10 can clarify the sound image localization by the early reflection sounds in accordance with these changes.

[0155] Furthermore, with the above-described configuration, even if the imaginary sound source IS is located closer to the sound receiving point RP than the speaker SP, the early reflection sound control signal generation unit 50 can simulate the components of the early reflection sound control signal due to this imaginary sound source IS. Therefore, for example, when the number of imaginary sound sources set for the early reflection sound control signal is small, the early reflection sound control signal generation unit 50 can use an imaginary sound source that is closer to the sound receiving point RP than the speaker SP. In this case, the early reflection sound control signal generation unit 50 relocates the imaginary sound source outside the speaker using the difference in distance between the imaginary sound source IS and the speaker SP, as described above. In this way, the early reflection sound control signal generation unit 50 can suppress the unnatural feeling of the early reflection sound caused by moving the position of the imaginary sound source.

[0156] In the above configuration, if the imaginary sound source IS is located closer to the sound receiving point RP than the speaker SP, the early reflection sound control signal generator 50 may set the imaginary sound source IS to the position of the speaker SP. This allows the early reflection sound control signal generator 50 to reduce the processing load of moving the imaginary sound source IS.

[0157] Furthermore, in the above-described configuration, if the imaginary sound source IS is located closer to the sound receiving point RP than the speaker SP, the early reflection sound control signal generator 50 does not need to use this imaginary sound source IS to generate the early reflection sound control signal. This eliminates the processing load of moving the imaginary sound source IS, and reduces the processing load of generating the early reflection sound control signal.

[0158] Furthermore, in the above-described configuration, the early reflection sound control signal generation unit 50 performs timbre adjustment using the FIR filters 511-518 while setting the components of the early reflection sound control signal due to the imaginary sound source. The FIR filters 511-518 have the above-described number of taps (for example, 16,000 taps), which is greater than the number of taps of the LD taps 521-528. Furthermore, the time interval between taps of the FIR filters 511-518 (which depends on the sampling frequency) is shorter than the time interval between taps of the LD taps 521-528 (which depends on the arrangement of the imaginary sound source). Therefore, the components of the early reflection sound control signal generated by the FIR filters 511-518 are arranged more densely on the time axis than the components of the early reflection sound control signal generated by the LD taps 521-528. In other words, the resolution on the time axis (time resolution) of the FIR filters 511-518 is higher than that of the LD taps 521-528, and the number of components per unit time is greater.

[0159] The early reflection sound control signal generation unit 50 combines the processing of the FIR filters 511-518 with the LDtap 521-528. Therefore, the early reflection sound control signal generation unit 50 has high resolution on the time axis and can generate early reflection sound control signals ER1-ER64 with a wider variety of timbres. Fig. 21 is a diagram showing an image of the waveforms of the early reflection sound control signals generated by the early reflection sound control signal generation unit 50.

[0160] 21, the early reflection sound control signal generation unit 50 can generate an early reflection sound control signal with higher resolution and capable of handling a variety of timbres while retaining the early reflection sound components due to an imaginary sound source. In other words, the sound signal processing device 10 can realize early reflection sounds with timbres that suit the user's preferences while maintaining clear sound image localization due to early reflection sounds using an imaginary sound source.

[0161] Furthermore, due to the high resolution of the FIR filter, for example, in the case of a short pulse sound from a sound source, the early reflection sound control signal generated by LDtap alone may become coarse, resulting in an unnatural tone. However, with the above-described configuration and processing, the sound signal processing device 10 can suppress the coarseness of the early reflection sound and the unnatural tone.

[0162] Furthermore, in the above-described configuration, the early reflection sound control signal generation unit 50 sets an allocation area for the imaginary sound source IS for each speaker SP, and does not allocate an imaginary sound source IS outside this area to that speaker SP. This allows the early reflection sound control signal generation unit 50 to suppress the generation of excessive early reflection sound components. Therefore, the sound signal processing device 10 can suppress excessive early reflection sounds and realize more natural early reflection sounds that correspond to the virtual space.

[0163] [Generation of reverberation control signals] Fig. 22 is a functional block diagram showing an example of the configuration of the reverberation sound control signal generating unit 70. Fig. 23 is a flowchart showing an example of a process for generating a reverberation sound control signal.

[0164] 22, the reverberation sound control signal generation unit 70 includes a PEQ 71, an FIR filter circuit 72, a router 73, a reverberation sound area setting unit 701, a filter coefficient setting unit 702, a reverberation sound reproduction speaker setting unit 703, and an operation unit 700. The FIR filter circuit 72 includes a plurality of FIR filters 721-728.

[0165] The reverberation sound region setting unit 701 sets a plurality of reverberation sound regions Arr1 to Arr8 in the reproduction space. More specifically, the reverberation sound region setting unit 701 sets the reproduction space so as to divide the reproduction space into a plurality of reverberation sound regions Arr1 to Arr8 over the entire circumference on a plane, for example, with the center point Psr of the reproduction space as a reference (see FIG. 25, which will be described later).

[0166] The reverberation sound region setting unit 701 outputs coordinate information indicating a plurality of reverberation sound regions Arr1 to Arr8 to the filter coefficient setting unit 702 and the reverberation sound reproduction speaker setting unit 703.

[0167] The filter coefficient setting unit 702 sets filter coefficients for reverberation sounds in response to user operations, etc. The filter coefficients for reverberation sounds are set, for example, based on the results of measuring impulse responses in the real space of the virtual space (a different space reproduced in the playback space). Note that the filter coefficients for reverberation sounds may be set artificially using the geometric shape of the virtual space, the material of the walls, etc. In this case, the filter coefficient setting unit 702 sets filter coefficients for each of the reverberation sound regions Arr1-Arr8 using coordinate information for each of the reverberation sound regions Arr1-Arr8.

[0168] The filter coefficient setting unit 702 receives input of the volume of the virtual space, the surface area of ​​the virtual space, etc., through a user operation, etc. The filter coefficient setting unit 702 sets a fade-in function for the filter coefficient based on parameters such as the volume of the virtual space and the surface area of ​​the virtual space.

[0169] More specifically, the filter coefficient setting unit 702 calculates the mean free path ρ using the volume V of the virtual space and the surface area S of the virtual space. The calculation formula for the mean free path ρ is ρ=4V / S. The mean free path is the average propagation distance in a closed space from when sound is reflected off a wall to when it is reflected again. By dividing the mean free path by the speed of sound c0, the average time it takes for sound to reflect off a wall until it is reflected again can be calculated.

[0170] The filter coefficient setting unit 702 sets the connection timing tc from the mean free path ρ (FIG. 23: S231). Specifically, the filter coefficient setting unit 702 sets the connection timing tc using the mean free path ρ, the sound speed c0, and the reflection order n. The calculation formula for the connection timing tc is tc=ρ×n / c0.

[0171] As can be seen from this calculation formula, the connection timing tc corresponds to the average time required for n reflections in the virtual space, and corresponds to the time when the transition to reverberation begins when reproducing an nth-order early reflection sound. In other words, the connection timing tc corresponds to the timing when the components of the early reflection sound control signal generated by the early reflection sound control signal generator 50 disappear.

[0172] By performing such processing, the filter coefficient setting unit 702 can optimally set the connection timing tc between the early reflection sound and the reverberation sound in accordance with the geometric shape of the virtual space.

[0173] The filter coefficient setting unit 702 sets a fade-in function using the connection timing tc according to the following equation (FIG. 23: S232).

[0174]

number

[0175] In this equation, t is the time that has elapsed since the direct sound was generated, and K is set by the following equation.

[0176]

number

[0177] In this equation, G REV is the gain value of the reverberation sound at time t = 0, and can be set by the user. For example, the reverberation time is generally the time required for the sound to decay to -60 dB. REV It is recommended to set it to =-60dB, etc.

[0178] The filter coefficient setting unit 702 sets reverberation sound filter coefficients from the filter coefficients and the fade-in function fin (FIG. 23: S233), and outputs them to the plurality of FIR filters 721-728.

[0179] The reverberation sound generating signal Sr output from the mixer 60 is input to the PEQ 71. The PEQ 71 performs predetermined signal processing on the reverberation sound generating signal Sr and outputs the result to a plurality of FIR filters 721-728.

[0180] By performing signal processing using the PEQ 71, the level (signal magnitude), tone, etc. of the reverberation sound generating signal Sr can be adjusted. For example, the PEQ 71 can refer to the volume, etc. of the early reflection sound control signal and adjust the level (signal magnitude) of the reverberation sound generating signal Sr so that the volume of the early reflection sound and the volume of the reverberation sound are approximately the same at the connection timing tc. The PEQ 71 can also adjust the tone, etc., according to settings by the user, etc.

[0181] The multiple FIR filters 721-728 filter the reverberation sound generation signal Sr using reverberation sound filter coefficients to generate the reverberation sound control signals REVr1-REVr8 for the respective regions. For example, the FIR filter 721 performs a convolution operation on the reverberation sound generation signal Sr using the reverberation sound filter coefficients set for the reverberation sound region Arr1 to generate the reverberation sound control signal REVr1 for the region Arr1. Similarly, the FIR filters 722-728 perform a convolution operation on the reverberation sound generation signal Sr using the reverberation sound filter coefficients set for the reverberation sound regions Arr2-Arr8, respectively, to generate the reverberation sound control signals REVr2-REVr8 for the respective regions Arr2-Arr8 (FIG. 23: S234). The multiple FIR filters 721-728 output the reverberation sound control signals REVr1-REVr8 for the respective regions to the router 73.

[0182] By setting the above-mentioned fade-in function, the reverberation sound control signal will have a waveform as shown in Fig. 21. Fig. 24 is a graph showing example waveforms of the direct sound, early reflection sound control signal, and reverberation sound control signal. For convenience, in Fig. 24, the reverberation sound control signal is illustrated by the envelope of each time component. The vertical axis in Fig. 24 is expressed in dB.

[0183] As shown in Fig. 24(A), the signal level of the reverberation sound control signal gradually increases in accordance with a fade-in function from the output timing of the direct sound to the connection timing tc. More specifically, the signal level of the reverberation sound control signal is -60 dBFS at the output timing of the direct sound, and gradually increases until the connection timing tc, reaching 0 dBFS at the connection timing tc. This level is set based on the signal level of the early reflection sound control signal at the connection timing tc.

[0184] 24, the fade-in function is used to exponentially increase the signal level as the connection timing tc approaches, in other words, the fade-in function has the opposite characteristics to the attenuation curve of the reverberation sound control signal without fade-in processing. Note that the characteristics of the change in the level of the reverberation sound control signal due to fade-in processing are not limited to this, and the user can set the desired characteristics by appropriately setting the fade-in function.

[0185] By performing such processing, the reverberation sound control signal generator 70 can generate a reverberation sound control signal that accurately reproduces the reverberation sound in the virtual space, using the FIR filters 721-728. The signal level of the reverberation sound control signal gradually increases in the section where the early reflection sound control signal exists, reaches a peak value according to the signal level of the early reflection sound control signal at the connection timing tc, and then attenuates.

[0186] As a result, the sound signal processing device 10 can smoothly connect the early reflection sound control signal and the reverberation sound control signal, which are multiple LDtaps that reproduce an imaginary sound source distribution at multiple sound source positions in a virtual space, using reverberation sounds generated by the reverberation sound control signal. Therefore, the sound output from the sound signal processing device 10 and heard by the user is a sound in which the sense of incongruity at the transition from the early reflection sound to the reverberation sound is suppressed.

[0187] The reverberation sound reproduction speaker setting unit 703 groups the multiple speakers SP1 to SP64 into reverberation sound areas Arr1 to Arr8.

[0188] More specifically, the reverberation sound reproduction speaker setting unit 703 sets the reproduction space so as to divide the reproduction space into multiple reverberation sound regions Arr1-Arr8, for example, around the entire circumference on a plane, with the center point Psr of the reproduction space as the reference point. The reverberation sound reproduction speaker setting unit 703 groups the multiple speakers SP1-SP64 into the multiple reverberation sound regions Arr1-Arr8 using the position coordinates of the multiple speakers SP1-SP64 and coordinate information indicating the multiple reverberation sound regions Arr1-Arr8. This grouping can be achieved by a method similar to the method for grouping sound source OBJ described above.

[0189] Fig. 25 is a diagram showing an example of setting an area for reverberation sound. To simplify the explanation and make it easier to understand, Fig. 25 shows multiple speakers SP1-SP14. For example, as shown in Fig. 25, the reverberation sound reproduction speaker setting unit 703 detects that speakers SP6 and SP7 are present in reverberation sound area Arr1, and groups speakers SP6 and SP7 into reverberation sound area Arr1. Similarly, the reverberation sound reproduction speaker setting unit 703 groups the other speakers SP1-SP5 and SP8-SP14 into multiple reverberation sound areas Arr2-Arr8, respectively.

[0190] The reverberation sound reproduction speaker setting unit 703 outputs to the router 73 grouping information of the plurality of speakers SP1 to SP64 for the plurality of reverberation sound areas Arr2 to Arr8.

[0191] The router 73 allocates the region-specific reverberation sound control signals REVr1-REVr8 to the multiple speakers SP1-SP64 using the grouping information from the reverberation sound reproduction speaker setting unit 703. Based on the allocation, the router 73 outputs the region-specific reverberation sound control signals REVr1-REVr8 as reverberation sound control signals REV1-REV48 for each of the multiple speakers SP1-SP64.

[0192] For example, the router 73 extracts from the grouping information that speakers SP6 and SP7 are grouped in area Arr1. The router 73 assigns the area-specific reverberation sound control signal REVr1 of area Arr1 to speakers SP6 and SP7. The router 73 outputs the area-specific reverberation sound control signal REVr1 to speaker SP6 as a reverberation sound control signal REV6 for speaker SP6. The router 73 also outputs the area-specific reverberation sound control signal REVr1 to speaker SP7 as a reverberation sound control signal REV6 for speaker SP7.

[0193] By generating the reverberation sound control signals REVr1-REVr8 for each region using the router 73 in this way, the reverberation sound control signal generator 70 can output the optimal reverberation sound control signal for each of the multiple speakers SP1-SP64 depending on the arrangement of the multiple speakers SP1-SP64.

[0194] [Output Adjustment] Fig. 26 is a functional block diagram showing an example of the configuration of the output adjustment unit 90. Fig. 27 is a flowchart showing an example of the output adjustment process.

[0195] 26, the output adjustment unit 90 includes a gain control unit 91, a delay control unit 92, a gain / delay setting unit 901, an operation unit 900, and a display unit 909. The gain control unit 91 includes a plurality of gain control units 9101-9168 corresponding to a plurality of speakers SP1-SP64. The delay control unit 92 includes a plurality of delay control units 9201-9264 corresponding to a plurality of speakers SP1-SP64.

[0196] The operation unit 900 receives settings of acoustic parameters of the reproduction space through operation input from the user (FIG. 27: S321). The acoustic parameters of the reproduction space are parameters for reproducing a desired sound field in the reproduction space.

[0197] In this case, the acoustic parameters of the playback space are not the individual gain values ​​or delay amounts of the multiple speakers SP1-SP64, but rather weight values ​​that represent the weighting of sound in a specific direction in the playback space, and shape values ​​that represent the spread of sound in a specific direction in the playback space.

[0198] The weight values ​​are made up of gain values ​​and delay amounts, and include weight values ​​for the front and rear of the reproduction space, weight values ​​for the left and right of the reproduction space, and weight values ​​for the up and down direction of the reproduction space. The shape values ​​are made up of gain values ​​and delay amounts, and include a shape value for the horizontal direction.

[0199] The display unit 909 includes a GUI. Fig. 28 is a diagram showing an example of the GUI for output adjustment.

[0200] 28, the GUI 100A includes a setting display window 111, an output state display window 115, and a plurality of controls 116. The plurality of controls 116 includes a knob 1161 and an adjustment value display window 1162.

[0201] The multiple operators 116 are operators that set weight volumes that set weight values, shape volumes that set shape values, etc. The operators 116 for the weight volumes include operators 116 that set left and right weights, front and rear weights, and up and down weights, each of which includes an operator for setting a gain value and an operator for setting a delay amount. The operators 116 for the shape volumes include an operator that sets the spread, and also includes an operator for setting a gain value and an operator for setting a delay amount.

[0202] The output status display window 115 graphically and schematically displays the sound spread and localization achieved by the weight values ​​and shape values ​​set by the multiple operators 116. This allows the user to easily recognize the sound spread and localization set by the multiple operators 116 as an image.

[0203] The user sets the acoustic parameters (weight values ​​and delay amounts) that he or she wishes to reproduce using GUI 100A of this display unit 909. The operation unit 900 accepts the settings made using GUI 100A. The operation unit 900 outputs the settings (each weight value and each delay amount of the acoustic parameters) to the gain delay setting unit 901.

[0204] The gain delay setting unit 901 sets gain values ​​and delay amounts for the plurality of speakers SP1 to SP64 based on the weight values ​​and delay amounts of the acoustic parameters. More specifically, the gain delay setting unit 901 performs the following processing.

[0205] The gain delay setting unit 901 acquires the position coordinates of the multiple speakers SP1-SP64 arranged in the reproduction space (S322). The position coordinates are expressed in a coordinate system in which, for example, the x-axis is set in the left-right direction of the reproduction space, the y-axis is set in the front-back direction of the reproduction space, and the z-axis is set in the up-down direction.

[0206] The gain delay setting unit 901 extracts the maximum and minimum values ​​of the position coordinates of the plurality of speakers SP1 to SP64 in each axial direction (S323).

[0207] The gain delay setting unit 901 stores coefficient setting formulas, which include, for example, a weight coefficient setting formula that sets weighting in a predetermined direction in the reproduction space, and a shape coefficient setting formula that sets weighting in a predetermined direction in the reproduction space.

[0208] The weight coefficient setting formula includes a weight gain value setting formula and a weight delay value setting formula, and the shape coefficient setting formula includes a shape gain value setting formula and a shape delay value setting formula.

[0209] The weighting coefficient setting formulas include a front-to-back coefficient setting formula for setting weighting in the front-to-back direction of the reproduction space, a left-to-right coefficient setting formula for setting weighting in the left-to-right direction of the reproduction space, and a up-to-down coefficient setting formula for setting weighting in the up-to-down direction of the reproduction space.

[0210] The coefficient setting formula for the shape includes coefficient setting formulas for the left and right directions of the reproduction space.

[0211] The coefficient setting formula for the gain value for weight is, for example, a linear function that combines the gain value of the set weight value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the gain value is to be set (the speaker to be set), and is a formula that determines the gain value in proportion to the difference between the position coordinates of the speaker to be set and the minimum value of the position coordinates.

[0212] The coefficient setting formula for the delay amount for the weight is, for example, a linear function that combines the delay amount of the set weight value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the delay amount is to be set (the speaker to be set), and is a formula that determines the delay amount in proportion to the difference between the position coordinate of the speaker to be set and the minimum value of the position coordinates.

[0213] The coefficient setting formula for the gain value for shape is, for example, a linear function that combines the gain value of the set shape value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the gain value is to be set (the speaker to be set), and is a formula that determines the gain value in proportion to the difference between the position coordinates of the speaker to be set and the minimum value of the position coordinates.

[0214] The coefficient setting formula for the delay amount for shape is, for example, a linear function that combines the delay amount of the set shape value, the maximum and minimum values ​​of the extracted position coordinates, and the position coordinates of the speaker for which the delay amount is to be set (the speaker to be set), and is a formula that determines the delay amount in proportion to the difference between the position coordinates of the speaker to be set and the minimum value of the position coordinates.

[0215] The gain delay setting unit 901 calculates the gain value and delay amount for each speaker to be set using the set gain value and delay amount (acoustic parameters), the maximum and minimum values ​​of the extracted position coordinates, and the coefficient setting formula (S324).

[0216] By using such processing, the gain delay setting unit 901 can automatically calculate and set the gain values ​​and delay amounts of multiple speakers SP1-SP64 arranged in the reproduction space using a coefficient setting formula, without having to set them individually manually.

[0217] The gain delay setting unit 901 outputs the gain values ​​set for each of the plurality of speakers SP1 to SP64 to the plurality of gain control units 9101 to 9164. The gain delay setting unit 901 outputs the delay amounts set for each of the plurality of speakers SP1 to SP64 to the plurality of delay control units 9201 to 9264.

[0218] The plurality of gain control sections 9101-9164 receive the speaker signals Sat1-Sat64 corresponding to the plurality of speakers SP1-SP64 from the adder 80, respectively.

[0219] The plurality of gain control sections 9101-9164 use the gain values ​​set therein to control the signal levels of the speaker signals Sat1-Sat64 and output the signals to the plurality of delay control sections 9201-9264. For example, the gain control section 9101 uses the gain value set therein to control the signal level of the speaker signal Sat1 and output the signals to the delay control section 9201. Similarly, the gain control sections 9102-9164 use the gain values ​​set therein to control the signal levels of the speaker signals Sat2-Sat64 and output the signals to the delay control sections 9202-9164.

[0220] The plurality of delay control sections 9201-9164 use the delay amounts set therein to control the signal levels of the signals input from the plurality of gain control sections 9101-9164, and output the signals to the plurality of speakers SP1-SP64. For example, the delay control section 9201 uses the delay amount set therein to control the signal level of the signal input from the gain control section 9101, and output the signals to the speaker SP1. Similarly, the delay control sections 9202-9164 use the delay amounts set therein to control the signal levels of the signals input from the gain control sections 9102-9164, and output the signals to the speakers SP1-SP64, respectively.

[0221] With this configuration, the sound signal processing device 10 can easily realize a desired sound field corresponding to the set acoustic parameters by using the early reflection sound control signal and the reverberation sound control signal, without forcing the user to perform cumbersome individual settings for multiple speakers. As a result, for example, the sound signal processing device 10 can easily realize a sound field that can obtain the Haas effect at a predetermined position in the reproduction space.

[0222] (Example of sound field realization by output control) Figures 29(A) and 29(B) are diagrams showing setting examples for weighting sound toward the rear of the reproduction space. Figure 29(A) is a diagram showing an example of gain value and delay amount settings, and Figure 29(B) is a diagram showing an image of sound weighting using the settings in Figure 25(A). Note that Figures 29(A) and 29(B) show a case where 14 speakers SP1-SP14 are arranged to simplify the explanation and make it easier to understand.

[0223] 29(A) and 29(B), for example, the gain value and delay amount of the rear end are set as acoustic parameters. The gain delay setting unit 901 sets the gain value and delay amount of the front end to values ​​with the opposite signs to the gain value and delay amount of the rear end. The gain delay setting unit 901 calculates the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14.

[0224] The gain delay setting unit 901 calculates the gain values ​​of the 14 speakers SP1-SP14 using the gain values ​​of the rear end and front end, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula (for setting gain values) for the front-to-rear direction that sets the weighting of the reproduction space in the front-to-rear direction.

[0225] In addition, the gain delay setting unit 901 calculates the delay amounts of the 14 speakers SP1-SP14 using the delay amounts of the rear and front ends, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula (for setting the delay amount) for the front-to-rear direction that sets the weighting of the reproduction space in the front-to-rear direction.

[0226] By this processing, the sound signal processing device 10 can automatically and easily set acoustic parameters such that the gain value and delay amount are larger for speakers further back in the reproduction space and the gain value and delay amount are smaller for speakers further forward, as shown in Fig. 29(A). This enables the sound signal processing device 10 to easily realize a sound field (see Fig. 29(B)) that is spread out toward the rear of the reproduction space and in which reverberation is localized.

[0227] Although an example of the front-rear direction has been shown in this explanation, the sound signal processing device 10 can also realize a weighted sound field in the left-right direction and height direction (up-down direction) in the same way.

[0228] Figures 30(A) and 30(B) are diagrams showing example settings for spreading sound horizontally in the reproduction space. Figure 30(A) is a diagram showing an example of gain value and delay amount settings, and Figure 30(B) is a diagram showing an image of the spread of sound based on the settings in Figure 30(A). Note that Figures 30(A) and 30(B) show a case where 14 speakers SP1-SP14 are arranged to simplify the explanation and make it easier to understand.

[0229] 30(A) and 30(B), for example, a value (spread setting value) that quantifies the spread of sound is set as the acoustic parameter. The gain delay setting unit 901 calculates the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1 to SP14.

[0230] The gain delay setting unit 901 calculates the gain values ​​of the 14 speakers SP1-SP14 using the spread setting value, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula for shape (for setting gain values).

[0231] In addition, the gain delay setting unit 901 calculates the delay amounts of the 14 speakers SP1-SP14 using the delay amounts of the rear and front ends, the maximum and minimum values ​​of the position coordinates of the 14 speakers SP1-SP14, and a coefficient setting formula for shape (for setting the delay amount).

[0232] By this processing, the sound signal processing device 10 can automatically and easily set acoustic parameters such that the closer to the ends of the horizontal direction of the reproduction space the larger the gain value and delay amount, and the closer to the center of the horizontal direction the smaller the gain value and delay amount, as shown in Fig. 30(A). As a result, the sound signal processing device 10 can easily realize a sound field (see Fig. 30(B)) in which the reproduction space is wide in the horizontal direction and reverberation is localized.

[0233] By setting the above-mentioned acoustic parameters, the sound signal processing device 10 can weight and expand the reproduction space not only in the front-rear direction, left-right direction, and horizontal direction, but also in the height direction (up-down direction) of the reproduction space. For example, Fig. 31 is a diagram showing an image of sound expansion when expanding in the height direction.

[0234] The sound signal processing device 10 increases the gain value and delay amount of the speakers SPU on the ceiling side compared to the gain value and delay amount of the speakers SPL and SPR closer to the floor. This enables the sound signal processing device 10 to easily realize a sound field (see FIG. 31) that is wider in the direction of the ceiling in the reproduction space and in which reverberation is localized.

[0235] Furthermore, in the above-described configuration, the output adjustment unit 90 outputs the output signals So1-So64 to the multiple speakers SP1-SP64. However, the sound signal processing device may also perform binaural processing on the output signals So1-So64 and output them.

[0236] Fig. 32 is a functional block diagram showing the configuration of a sound signal processing device with a binaural reproduction function. As shown in Fig. 32, a sound signal processing device 10A with a binaural reproduction function differs from the above-described sound signal processing device 10 in that it includes an output adjustment unit 90A, a reverberation processing unit 97, a selection unit 98, and a binaural processing unit 99.

[0237] The output adjustment unit 90A generates a plurality of output signals So1-So64 from the plurality of speaker signals Sat1-Sat64 output from the adder 80 using the same processing as the output adjustment unit 90 described above.

[0238] The output adjustment unit 90A can select an output target. The selection of the output target is executed, for example, by an operation input from the user using the above-mentioned GUI. More specifically, the GUI displays an operator that can select between speaker output and binaural output, and the output target is selected by operating this operator.

[0239] When speaker output is selected, the output adjustment unit 90A outputs the plurality of output signals So1-So64 to the plurality of speakers SP1-AP64, respectively (the same processing as the output adjustment unit 90). When binaural output is selected, the output adjustment unit 90A outputs the plurality of output signals So1-So64 to the selection unit 98.

[0240] Sound signals S1-S96 of multiple sound sources OBJ1-OBJ96 are input to the reverberation processor 97. The reverberation processor 97 adds an early reflection sound control signal and a reverberation sound control signal to the multiple sound signals S1-S96 and outputs them to the selection unit 98. The early reflection sound control signals for the multiple sound signals S1-S96 are set based on the position coordinates of the multiple sound sources OBJ1-OBJ96. The reverberation processor 97 outputs multiple post-reverberation sound signals S1'-S96' to the selection unit 98.

[0241] The selection unit 98 receives input of the plurality of output signals So1-So64 and the plurality of post-reverberation sound signals S1'-S96'. The selection unit 98 selects the plurality of output signals So1-So64 and the post-reverberation sound signals S1'-S96', for example, in response to an operation input from the user using the above-mentioned GUI. More specifically, the GUI displays controls that allow selection between sound that has been subjected to acoustic processing by the sound signal processing device 10A and sound that has been subjected to virtual acoustic processing based on the position coordinates of the sound sources OBJ1-OBJ96, and the output target is selected by operating the controls.

[0242] When sound that has been subjected to acoustic processing by the sound signal processing device 10A is selected, the selection unit 98 selects multiple output signals So1-So64 and outputs them to the binaural processing unit 99. When sound that has been subjected to virtual acoustic processing based on the position coordinates of the sound sources OBJ1-OBJ96 is selected, the selection unit 98 selects multiple sound signals S1'-S96' after reverberation processing and outputs them to the binaural processing unit 99.

[0243] The binaural processing unit 99 performs binaural processing on the input sound signals. More specifically, when a plurality of output signals So1-So64 are input, the binaural processing unit 99 performs binaural processing on the plurality of output signals So1-So64. When a plurality of reverberation-processed sound signals S1'-S96' are input, the binaural processing unit 99 performs binaural processing on the plurality of reverberation-processed sound signals S1'-S96'.

[0244] Note that the binaural processing uses a head-related transfer function, and the details thereof are known, so a detailed description of the binaural processing will be omitted.

[0245] The binaural processing unit 99 outputs two-channel sound signals that have been subjected to binaural processing.

[0246] This allows the user to listen, through binaural playback, to sounds generated by the sound signal processing device 10A and sounds that have been subjected to virtual reverberation processing based on the position coordinates of the sound sources OBJ1-OBJ96. Therefore, the user can easily check, using headphones or the like, whether the sound processing performed by the sound signal processing device 10A is reproducing the sound of the virtual space, without having to physically construct a reproduction space. Examples of the sound processing performed by the sound signal processing device 10A include the above-mentioned grouping of sound sources, setting of the early reflection sound control signal, setting of the reverberation sound control signal, and setting of the output control. By being able to compare the sounds in this way, the user can adjust the settings of the above-mentioned sound processing so that the sound of the virtual space can be reproduced more faithfully.

[0247] Note that binaural reproduction is not limited to headphones, but may also be performed using stereo speakers or the like.

[0248] The description of the present embodiment is illustrative in all respects and is not restrictive. The scope of the present invention is defined not by the above-described embodiments but by the claims. Furthermore, the scope of the present invention is intended to include all modifications that are equivalent to the claims and fall within the scope thereof. [Explanation of symbols]

[0249] 10, 10A: Sound signal processing device 30: Area setting section 40: Grouping section 41: Sound source position detection unit 42: Area determination section 50: Early reflection sound control signal generator 51: FIR filter circuit 52:LDtap circuit 53: Addition processing unit 60: Mixer 70: Reverberation control signal generator 71:PEQ 72: FIR filter circuit 73: Router 80: Adder 90, 90A: Output adjustment section 91: Gain control section 92: Delay control section 97: Reverberation processing section 98: Selection section 99: Binaural processing section 100, 100A: GUI 400: Matrix mixer 500:Operation unit 501: Tone setting section 502: Imaginary sound source setting section 511-518: FIR filters 521-528:LDtap 700:Operation unit 701: Reverberation area setting section 702: Filter coefficient setting unit 703: Reverberation playback speaker setting section 721-728: FIR filters 900: Operation unit 901: Gain delay setting section 909: Display section 5201: Output speaker setting section 5202: Coefficient setting unit 9101-9164: Gain control section 9201-9264: Delay control section

Claims

1. Acquire position coordinates of a sound receiving point in a reproduction space, position coordinates of a speaker, and information about the shape of a target acoustic space, which is a virtual space having a sound field different from that of the reproduction space; setting a virtual sound source representing a reflected sound reflected by a virtual wall of the acoustic space based on the shape of the acoustic space, the position of the speaker in the acoustic space, and the position of the sound receiving point; a straight line connecting the position of the sound receiving point and the position of the speaker is set, and a predetermined azimuth angle and an elevation / depression angle are set with respect to the straight line, the angle extending toward the speaker with the sound receiving point as a reference; determining whether the virtual sound source is within a region defined by the predetermined azimuth angle and elevation / depression angle; Only when the virtual sound source is within the area, an early reflection sound control signal of the virtual sound source is output and controlled by the speaker. Sound signal processing method.

2. a gain value and a delay amount of the early reflection sound control signal are set based on a positional relationship between the virtual sound source, the sound receiving point, and the speaker; The sound signal processing method according to claim 1 .

3. The virtual sound sources are classified into a first virtual sound source that is located between the speaker and the sound receiving point and indicates a first sound source of a reflected sound in a target acoustic space, and a second virtual sound source that is located outside the speaker and indicates a second sound source of a reflected sound in the target acoustic space, The first virtual sound source and the second virtual sound source are set using different methods for setting the positions of the virtual sound sources.

3. The sound signal processing method according to claim 1.

4. Only when the virtual sound source is the first virtual sound source, the position of the first virtual sound source is moved to a position where the sound can be reproduced using the position of a speaker near the first virtual sound source. The sound signal processing method according to claim 3 .

5. A speaker installed in the playback space; Acquire position coordinates of a sound receiving point in the reproduction space, position coordinates of the speaker, and information about the shape of a target acoustic space, which is a virtual space having a sound field different from that of the reproduction space; setting a virtual sound source representing a reflected sound reflected by a virtual wall of the acoustic space based on the shape of the acoustic space, the position of the speaker in the acoustic space, and the position of the sound receiving point; a straight line connecting the position of the sound receiving point and the position of the speaker is set, and a predetermined azimuth angle and an elevation / depression angle are set with respect to the straight line, the angle extending toward the speaker with the sound receiving point as a reference; determining whether the virtual sound source is within a region defined by the predetermined azimuth angle and elevation / depression angle; Only when the virtual sound source is within the area, an early reflection sound control signal of the virtual sound source is output and controlled by the speaker. an early reflection sound control section; Equipped with Sound signal processing device.

6. The early reflection sound control unit includes: a gain value and a delay amount of the early reflection sound control signal are set based on a positional relationship between the virtual sound source and the speaker; The sound signal processing device according to claim 5 .

7. The early reflection sound control unit includes: The virtual sound sources are classified into a first virtual sound source that is located between the speaker and the sound receiving point and indicates a first sound source of a reflected sound in a target acoustic space, and a second virtual sound source that is located outside the speaker and indicates a second sound source of a reflected sound in the target acoustic space, a setting method of a gain value and a delay amount of the early reflection sound control signal for the first virtual sound source and a setting method of a delay amount of the early reflection sound control signal for the second virtual sound source are different from each other; 7. The sound signal processing device according to claim 5 or 6.

8. The early reflection sound control unit includes: Only when the virtual sound source is the first virtual sound source, the position of the first virtual sound source is moved to a position where the sound can be reproduced using the position of a speaker near the first virtual sound source. The sound signal processing device according to claim 7 .

Images