Sound field support method and sound field support device

Abstract

To compare a sound from a virtual sound source with a simulation reproduced sound emitted in a target space.SOLUTION: A sound field support method selects position information of a sound source set in a virtual space, selects localization information of the sound source when simulating the sound from the sound source with an output sound from a speaker set in a target space by using a simulation reproduced sound signal, compares a sound based on the selected position information with a sound based on the selected localization information, and adjusts a parameter of the simulation reproduced sound signal on the basis of a result of the comparison.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] One embodiment of the present invention relates to a sound field support method and a sound field support device that perform processing for simulating a sound field by a sound source set in a virtual space and a target space where speakers are arranged.

Background Art

[0002] Various techniques have been devised for simulating the sound of a sound source set in a virtual space in the real space.

[0003] For example, a simulation system as shown in Patent Document 1 sets the positions of a plurality of virtual speakers so as to follow while maintaining the relative positional relationship with a viewer on the virtual space as the position of the viewer changes. Further, the simulation system shown in Patent Document 1 sets the volume balance of a plurality of virtual speakers.

[0004] The simulation system shown in Patent Document 1 executes sound processing using a plurality of virtual speakers based on these settings.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, when playing the sound set using a virtual sound source (virtual speaker of Patent Document 1) in the target space, this sound is played by a speaker arranged in the target space and to which the virtual sound source is assigned. That is, the sound played in the target space is a simulation of the sound of the virtual sound source with the sound of the speaker.

[0007] Conventionally, it has been impossible to compare the sound from a virtual sound source with the sound that is simulated and reproduced by a speaker in a target space (simulated reproduction sound). Therefore, viewers have been unable to confirm to what extent the simulated reproduction sound mimics the sound from the virtual sound source and have been unable to easily make adjustments.

[0008] Therefore, an embodiment of this invention aims to enable comparison between the sound of a virtual sound source and the simulated reproduction sound.

Means for Solving the Problem

[0009] The sound field support method selects the position information of a sound source set in a virtual space and selects the localization information of the sound source when simulating the sound of the sound source with a simulated reproduction sound signal using the output sound from speakers set in a target space, compares the sound based on the selected position information and the localization information, and adjusts the parameters of the simulated reproduction sound signal based on the comparison result.

[0010] 。

Advantages of the Invention

[0011] The sound field support method enables a viewer to compare the sound of a virtual sound source with the simulated reproduction sound.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0013] A sound field assistance method and a sound field assistance device according to an embodiment of the present invention will be described with reference to the drawings.

[0014] In the present embodiment, the target space is a space where a viewer actually listens to the sound of a sound source set in a virtual space using a speaker or the like. More specifically, in the sound field assistance method of the present embodiment, the target space does not mean the space where the speakers are actually arranged, but means the space where the speakers are arranged and the viewer is expected to listen to the sound from these speakers. The virtual space is a space where a sound source to be simulated in the target space is set.

[0015] [First Embodiment] FIG. 1 is a functional block diagram showing the configuration of a sound field support system including a sound field support device according to a first embodiment of the present invention. FIG. 2(A) is a diagram showing an example of the positional relationship among a sound source, a viewing point, and a plurality of speakers in the sound field support method according to the first embodiment of the present invention, and FIG. 2(B) is a diagram showing the positional coordinates of the sound source, the positional coordinates of the viewing point, and the positional coordinates of the plurality of speakers in the case of FIG. 2(A). FIG. 3(A) is a diagram showing an image of sound emission from a sound source, and FIG. 3(B) is a diagram showing an image of rendering a sound source to a speaker for sound emission.

[0016] As shown in FIG. 2(A), in a target space 90, a viewing point 900 at which a viewer views and a plurality of speakers SP1 - SP5 are arranged. A virtual space is set in this target space 90. A sound source OBJ is set in the virtual space.

[0017] In the description of this embodiment, there is one sound source, but there may be a plurality of sound sources. When there are a plurality of sound sources, the sound field support method shown below may be applied for each of the plurality of sound sources. Alternatively, the sound field support method shown below may be applied to a plurality of sound sources collectively. In this embodiment, the case of one sound source will be described. Also, in the description of this embodiment, the number of speakers is five, but the number of speakers is not limited to this.

[0018] The coordinate system of the target space 90 and the coordinate system of the virtual space are set such that, for example, the directions of three orthogonal axes and the center points coincide. In this case, the positional coordinates based on the coordinate system in the target space 90 and the positional coordinates based on the coordinate system of the virtual space coincide. Even if the coordinate system of the target space 90 and the coordinate system of the virtual space do not coincide, in this case, a coordinate transformation matrix between the target space 90 and the virtual space may be set.

[0019] As shown in FIG. 1, the sound field assistance system includes a sound field assistance device 10 and headphones 80. The sound field assistance device 10 includes a viewing point setting unit 21, a sound source position setting unit 22, a speaker position setting unit 23, an adjustment operation unit 29, a simulated reproduction sound signal generation unit 30, a selection unit 40, and a binaural processing unit 50. The sound field assistance device 10 is realized by a program that executes each of the above-described functional units, a storage medium that stores this program, and an arithmetic processing device such as a CPU that executes this program.

[0020] The viewing point setting unit 21 sets the position coordinates Pr of the viewing point 900 in the target space 90. The viewing point setting unit 21 outputs the position coordinates Pr of the viewing point 900 to the simulated reproduction sound signal generation unit 30 and the binaural processing unit 50.

[0021] The sound source position setting unit 22 sets the position coordinates Pobj of the sound source OBJ in the virtual space (more specifically, the position coordinates obtained by projecting the sound source in the virtual space onto the target space 90). The sound source position setting unit 22 outputs the position coordinates Pobj of the sound source OBJ to the simulated reproduction sound signal generation unit 30 and the binaural processing unit 50.

[0022] The speaker position setting unit 23 sets the position coordinates Psp1 - Psp5 of the plurality of speakers SP1 - SP5 in the target space 90. The speaker position setting unit 23 outputs the position coordinates Psp1 - Psp5 of the plurality of speakers SP1 - P5 to the simulated reproduction sound signal generation unit 30 and the binaural processing unit 50.

[0023] The adjustment operation unit 29 receives an operation input of adjustment parameters. The adjustment operation unit 29 outputs the adjustment parameters to the simulated reproduction sound signal generation unit 30.

[0024] The simulated reproduction sound signal generation unit 30 generates a simulated reproduction sound signal for output to the speakers SP1 - SP5 in the target space 90 from the object reproduction sound signal.

[0025] Here, the object reproduction sound signal is a sound signal output from the sound source OBJ. The simulated reproduction sound signal is a sound signal for performing sound image localization of the sound source OBJ by speakers that render the sound source OBJ.

[0026] More specifically, the simulated reproduction sound signal generation unit 30 calculates the positional relationship between the positional coordinates Pobj of the sound source OBJ and the positional coordinates Psp1 - Psp5 of the plurality of speakers SP1 - SP5 with the positional coordinates Pr of the viewing point 900 as the reference point. The simulated reproduction sound signal generation unit 30 sets the sound image localization information of the sound source OBJ using this positional relationship. The sound image localization information is information set so that it seems that the sound source OBJ is emitting sound at the viewing point 900 by the sounds output from the plurality of speakers SP1 - SP5, and is information for determining the volume and output timing of the output sounds from the plurality of speakers SP1 - SP5.

[0027] The simulated reproduction sound signal generation unit 30 sets a plurality of speakers that render the sound source OBJ using the sound image localization information of the sound source OBJ (see FIG. 3(B)). The simulated reproduction sound signal generation unit 30 generates a simulated reproduction sound signal to be reproduced by the plurality of speakers on which the sound source OBJ is rendered. The simulated reproduction sound signal generation unit 30 outputs the simulated reproduction sound signal to the selection unit 40.

[0028] The selection unit 40 receives an operation input from a viewer or the like and selects an object reproduction sound signal and a simulated reproduction sound signal. More specifically, when a setting for listening to the sound directly output from the sound source OBJ set in the virtual space (the state in FIG. 3(A)) is selected, the selection unit 40 selects and outputs the object reproduction sound signal. On the other hand, when a setting for listening to the sound from the plurality of rendered speakers (the state in FIG. 3(B)) is selected, the selection unit 40 selects and outputs the simulated reproduction sound signal. In other words, if the position information of the sound source OBJ is selected, the object reproduction sound signal is selected and output, and if the localization information of the sound source OBJ using the speakers is selected, the simulated reproduction sound signal is selected and output.

[0029] The selection unit 40 outputs the selected sound signal to the binaural processing unit 50.

[0030] The binaural processing unit 50 performs binaural processing on the sound signal selected by the selection unit 40. Note that the binaural processing uses a head-related transfer function, the detailed content of which is known, and thus a detailed description of the binaural processing will be omitted.

[0031] More specifically, when the selection unit 40 selects the object reproduction sound signal, the binaural processing unit 50 performs binaural processing on the sound signal of the sound source OBJ using the position coordinates Pobj of the sound source OBJ and the position coordinates Pr of the viewing point 900. When the selection unit 40 selects the simulated reproduction sound signal, the binaural processing unit 50 performs binaural processing on the simulated reproduction sound signal using the position coordinates Psp of the speaker SP on which the sound source OBJ is rendered and the position coordinates Pr of the viewing point 900.

[0032] For example, in the cases of FIGS. 2(A), 2(B), 3(A), and 3(B), when the selection unit 40 selects the object reproduction sound signal, the binaural processing unit 50 performs binaural processing on the object reproduction sound signal using the position coordinates Pobj of the sound source OBJ and the position coordinates Pr of the viewing point 900. When the selection unit 40 selects the simulated reproduction sound signal, the binaural processing unit 50 performs binaural processing on the simulated reproduction sound signal using the position coordinates Psp1 and Psp5 of the speakers SP1 and SP5 on which the sound source OBJ is rendered and the position coordinates Pr of the viewing point 900.

[0033] The binaural processing unit 50 outputs the sound signal (binaural signal) subjected to binaural processing to the headphones 80.

[0034] The headphones 80 reproduce and emit the sound signal by the binaural signal. Note that in the present embodiment, a mode of emitting sound using the headphones 80 is shown, but it is also possible to emit sound using two-channel stereo speakers.

[0035] With such a configuration, when the object reproduction sound signal is selected, the viewer can hear, through the headphones 80, the sound (object reproduction sound) where the sound source is localized at the position of the sound source OBJ. On the other hand, when the simulated reproduction sound signal is selected, the viewer can hear, through the headphones, the sound (simulated reproduction sound) where the sound source is simulatedly localized at the position of the sound source OBJ by the speaker rendered to the sound source OBJ.

[0036] Thus, the viewer can compare and listen to the object reproduction sound and the simulated reproduction sound without actually arranging speakers in the real space. Therefore, the viewer can directly experience the difference between the object reproduction sound and the simulated reproduction sound, and can determine whether the simulated reproduction sound can accurately reproduce (simulate) the object reproduction sound, or whether there is no sense of incongruity between the object reproduction sound and the simulated reproduction sound.

[0037] Also, the viewer can adjust the parameters for adjusting the simulated reproduction sound signal by referring to such an experienced result. And by repeating such adjustment of the parameters, the viewer can accurately reproduce the object reproduction sound with the simulated reproduction sound.

[0038] Here, a mode of adjusting the simulated reproduction sound signal is shown in order to accurately reproduce the sound of the sound source OBJ. However, for example, when it is difficult to change the position of the speaker in the target space 90 or change the parameter setting, and the position setting of the sound source OBJ can be changed, by listening to the above-described binaural processed sound, it is possible for the viewer to change the setting of the sound source OBJ and realize a desired sound field.

[0039] (Sound Field Support Method 1 of the First Embodiment) FIG. 4 is a flowchart showing a first method of a sound field support method according to the first embodiment of the present invention. The sound field support method shown in FIG. 4 is executed until a sound signal subjected to binaural processing is output. Since the detailed description of each process shown in FIG. 4 has been described above, the detailed description below is omitted. Further, below, the case of the arrangement modes shown in FIGS. 2(A), 2(B), 3(A), and 3(B) will be described as an example.

[0040] The sound source position setting unit 22 sets the position of the sound source OBJ in the virtual space (S11). The speaker position setting unit 23 sets the positions of the speakers SP1-SP5 in the target space (S12).

[0041] The simulated reproduction sound signal generation unit 30 renders the sound source OBJ to the speakers SP1 and SP5 using the position coordinates Pobj of the sound source OBJ, the position coordinates Psp1-Psp5 of the speakers SP1-SP5, and the position coordinates Pr of the viewing point 900 (S13). The simulated reproduction sound signal generation unit 30 generates a simulated reproduction sound signal using the rendering result (S14).

[0042] The selection unit 40 selects an object reproduction sound signal and a simulated reproduction sound signal by an operation from a viewer or the like (S15). For example, the sound field support device 10 includes a GUI (Graphical User Interface) or the like. The GUI includes an operator for selecting a sound signal to be reproduced. If the viewer selects the output of the object reproduction sound signal, the selection unit 40 selects the object reproduction sound signal (S150: YES). If the viewer selects the output of the simulated reproduction sound signal, the selection unit 40 selects the simulated reproduction sound signal (S150: NO). Note that the selection between the object reproduction sound signal and the simulated reproduction sound signal can also be set to a switching time and automatically switched according to this time.

[0043] The binaural processing unit 50 performs binaural processing on the selected sound signal to generate a binaural signal. More specifically, if the object reproduction sound signal is selected, the binaural processing unit 50 performs binaural processing on the object reproduction sound signal to generate a binaural signal of the object reproduction sound signal (S161). If the simulated reproduction sound signal is selected, the binaural processing unit 50 performs binaural processing on the simulated reproduction sound signal to generate a binaural signal of the simulated reproduction sound signal (S162).

[0044] The headphones 80 reproduce the binaural signal (S17). More specifically, if the binaural signal of the object reproduction sound signal is input, the headphones 80 reproduce this binaural signal. If the binaural signal of the simulated reproduction sound signal is input, the headphones 80 reproduce this binaural signal.

[0045] By performing such processing, the sound field support method can selectively provide the object reproduction sound and the simulated reproduction sound to the viewer or the like.

[0046] (Sound Field Support Method 2 of the First Embodiment) FIG. 5 is a flowchart showing a second method of the sound field support method according to the first embodiment of the present invention. The sound field support method shown in FIG. 5 is obtained by adding parameter adjustment to the sound field support method shown in FIG. 4. Note that the description of the same processing as that shown in FIG. 4 in the processing shown in FIG. 5 is omitted. Further, hereinafter, the case of the arrangement modes shown in FIGS. 2(A), 2(B), 3(A), and 3(B) will be described as an example.

[0047] The sound field support method shown in FIG. 5 executes the same processing as that of the sound field support method shown in FIG. 4 up to step S17.

[0048] The viewer switches the sound signal to be reproduced by executing the processing from step S15 to step S17. Thereby, the viewer listens to the sound of the binaural signal of the object reproduction sound signal and the sound of the binaural signal of the simulated reproduction sound signal, and compares these sounds.

[0049] If parameter adjustment is not required (S23: NO), that is, if the sound by the binaural signal of the simulated reproduction sound signal can accurately reproduce the sound by the binaural signal of the object reproduction sound signal, the process ends. If parameter adjustment is necessary (S23: YES), the viewer performs parameter adjustment using the adjustment operation unit 29 (S24). The simulated reproduction sound signal generation unit 30 generates a simulated reproduction sound signal using this adjusted parameter (S14).

[0050] The parameters to be adjusted are, for example, the rendering settings between the sound source OBJ and the speakers, the volume level of the simulated reproduction sound signal, the frequency characteristics, etc. FIG. 6 is a diagram showing an example of a GUI for parameter adjustment. As shown in FIG. 6, the GUI 100 includes a positional relationship confirmation window 111, a waveform confirmation window 112, and a plurality of operators 113. Each of the plurality of operators 113 includes a knob 1131 and an adjustment value display window 1132.

[0051] The positional relationship confirmation window 111 displays the sound sources OBJ1 - OBJ3 and the plurality of speakers SP1 - SP5 at the set position coordinates respectively. The setting of the speaker SP assigned to the sound source OBJ can be realized, for example, by selecting the sound source OBJ to be rendered and the speaker SP in the positional relationship confirmation window 111.

[0052] The waveform confirmation window 112 displays the waveform of the simulated reproduction sound signal. The selection of the simulated reproduction sound signal to be displayed can be switched, for example, by selecting the plurality of speakers SP1 - SP5 displayed in the positional relationship confirmation window 111.

[0053] The plurality of operators 113 are operators that receive, for example, Q of the simulated reproduction sound signal, filter processing settings, gain value settings, etc. for each of a plurality of frequency bands (Hi, Mid, Low). The knob 1131 receives operations from the viewer, and the adjustment value display window 1132 displays the numerical values set by the knob 1131. The parameters of the simulated reproduction sound signal are adjusted by the operation input by the plurality of operators 113. Then, the waveform based on the adjusted parameters is displayed in the waveform confirmation window 112.

[0054] The viewer can adjust and set the parameters by operating while viewing this GUI 100.

[0055] Thereafter, the viewer performs parameter adjustment while comparing the sound by the binaural signal of the object reproduction sound signal and the sound by the binaural signal of the simulated reproduction sound signal. As a result, the viewer can adjust so that the sound by the binaural signal of the simulated reproduction sound signal accurately reproduces the sound by the binaural signal of the object reproduction sound signal, that is, the simulated reproduction sound by the speaker can accurately simulate the object reproduction sound of the sound source OBJ. Note that the means for outputting and comparing the object reproduction sound and the simulated reproduction sound, and the adjustment operation unit 29 realize the "adjustment unit" of the present invention.

[0056] Note that the sound field support device 10 and the sound field support method of the present embodiment show an aspect of comparing the object reproduction sound and the simulated reproduction sound by binaural reproduction. However, the sound field support device 10 and the sound field support method of the present embodiment can also perform parameter adjustment by comparing, for example, the waveform and frequency spectrum of the object reproduction sound signal, HOA (higher order ambisonics), with the waveform and frequency spectrum of the simulated reproduction sound signal, HOA (higher order ambisonics).

[0057] [Second Embodiment] The sound field support device and the sound field support method according to the second embodiment of the present invention will be described with reference to the drawings.

[0058] FIG. 7 is a functional block diagram showing the configuration of a sound field support system including a sound field support device according to a second embodiment of the present invention. FIG. 8 is a diagram showing an example of the positional relationship among a sound source, a viewing point, a plurality of speakers, and a virtual space in the sound field support method according to the second embodiment of the present invention.

[0059] As shown in FIG. 7, the sound field support device 10A according to the second embodiment is different from the sound field support device 10 according to the first embodiment in that a reverberation processing unit 60 is added. Other configurations of the sound field support device 10A are the same as those of the sound field support device 10, and descriptions of the same parts are omitted.

[0060] The sound field support device 10A includes a reverberation processing unit 60. An object reproduction sound signal and a simulated reproduction sound signal are input to the reverberation processing unit 60.

[0061] The reverberation processing unit 60 generates an early reflection sound signal and a reverberation sound signal using information on the virtual space 99. The early reflection sound signal is a sound signal that simulates the sound of the sound source OBJ reflected (primary reflection) by the wall of the virtual space and reaching the viewing point. The early reflection sound signal is determined based on the geometric shape of the virtual space, the position of the sound source OBJ in the virtual space, and the position of the viewing point. The reverberation sound signal is a sound signal that simulates the sound that reaches the viewing point after multiple reflections in the virtual space. The reverberation sound signal is determined based on the geometric shape of the virtual space and the position of the viewing point in the virtual space.

[0062] More specifically, the reverberation processing unit 60 generates an early reflection sound signal and a reverberation sound signal for the object reproduction sound signal using the position information of the sound source OBJ, the information on the virtual space 99, and the position information of the viewing point. The reverberation processing unit 60 adds the generated early reflection sound signal and reverberation sound signal to the object reproduction sound signal and outputs the result to the selection unit 40.

[0063] In addition, the reverb processing unit 60 generates an early reflection sound signal and a reverberation sound signal for the simulated reproduction sound signal by using the position information of the sound source OBJ, the position information of the speakers SP1 - SP5, the information of the virtual space 99, and the position information of the viewing point. As a specific example, the reverb processing unit 60 sets a virtual sound source that simulates the generation position of the early reflection sound for this sound source OBJ from the position information of the sound source OBJ and the viewing point and the information of the virtual space 99. The reverb processing unit 60 generates an early reflection sound signal from the positional relationship between this virtual sound source and the speaker SP to which this virtual sound source is assigned. The reverb processing unit 60 generates a reverberation sound signal by using the geometric shape of the virtual space and the position of the viewing point in the virtual space. The reverb processing unit 60 adds the early reflection sound signal and the reverberation sound signal thus generated to the simulated reproduction sound signal and outputs it to the selection unit 40.

[0064] With such a configuration, the sound field support device 10A can add respective reverb components (early reflection sound and reverberation sound) to the object reproduction sound (sound from the sound source OBJ) and the simulated reproduction sound (sound simulated by the speakers), and output them. Thereby, the viewer can judge the accuracy of the reproduction of the object reproduction sound by the simulated reproduction sound in consideration of the reverb components.

[0065] Furthermore, the reverb processing unit 60 can also give a sense of spread and localization to the early reflection sound signal and the reverberation sound signal of the simulated reproduction sound signal. In this case, the viewer can perform adjustments, for example, by using a GUI as shown in FIG. 9. FIG. 9 is a diagram showing an example of a GUI for adjusting the spread and localization of sound. As shown in FIG. 9, the GUI 100A includes a setting display window 111A, an output state display window 115, and a plurality of operators 116. The plurality of operators 116 include a knob 1161 and an adjustment value display window 1162.

[0066] The setting display window 111A displays the virtual sound source SS set for the sound source OBJ, the plurality of speakers SP, the virtual space 99, and the viewing point RP at the position coordinates set for each.

[0067] The plurality of operators 116 are operators for setting a weight volume for setting a weight value, a shape volume for setting a shape value, and the like. The operator 116 for the weight volume includes operators 116 for setting the left and right weights, the front and rear weights, and the up and down weights, respectively, and each includes an operator for setting a gain value and an operator for setting a delay amount. The operator 116 for the shape volume includes an operator for setting the spread, and includes an operator for setting a gain value and an operator for setting a delay amount. The viewer can adjust the spread and the sense of localization of the sound by operating the plurality of operators 116.

[0068] The output state display window 115 graphically and schematically displays the spread and the sense of localization of the sound realized by the weight value and the shape value set by the plurality of operators 116. Thereby, the user can easily recognize the spread and the sense of localization of the sound set by the plurality of operators 116 as an image. Note that the output state display window 115 can also display a combination of an image representing the head and an image representing the spread and the sense of localization of the sound in accordance with the image of the head when listening to the binaural processed sound with the headphones 80.

[0069] Thereby, the viewer can judge the accuracy of the reproduction of the object reproduction sound by the simulated reproduction sound in consideration of the spread and the sense of localization of the sound.

[0070] Note that, for example, by operating the setting display window 111A, the viewer can also adjust the shape of the virtual space 99, the position with respect to the reproduction space, the position of the sound source OBJ, and the positions of the plurality of speakers SP. In this case, the sound field support apparatus generates an object reproduction sound signal and a simulated reproduction sound signal according to the adjusted various contents, and performs the same reverberation processing. Thereby, the viewer can judge the accuracy of the reproduction of the object reproduction sound by the simulated reproduction sound even after this adjustment.

[0071] (Sound Field Support Method of the Second Embodiment) FIG. 10 is a flowchart showing a sound field assistance method according to a second embodiment of the present invention. The sound field assistance method shown in FIG. 10 is obtained by adding a reverberation component addition process to the sound field assistance method shown in FIG. 4. Note that descriptions of the same processes as those shown in FIG. 4 in each process shown in FIG. 10 are omitted.

[0072] The sound field assistance method shown in FIG. 10 executes the same processes as those in the sound field assistance method shown in FIG. 4 up to step S14.

[0073] The reverberation processing unit 60 generates reverberation components (early reflection sound signals and reverberation sound signals) for the object reproduction sound signal and the simulated reproduction sound signal, and adds them to the object reproduction sound signal and the simulated reproduction sound signal (S31).

[0074] The sound field assistance device 10A executes the processes after step S15 using the sound signal with the added reverberation component and the simulated reproduction sound signal with the added reverberation component.

[0075] Thereby, the sound field assistance method of the second embodiment can add respective reverberation components (early reflection sound and reverberation sound) to the object reproduction sound (sound from the sound source OBJ) and the simulated reproduction sound (sound simulated by the speaker) and output them. Thereby, the viewer can judge the accuracy of the reproduction of the object reproduction sound by the simulated reproduction sound in consideration of the reverberation component.

[0076] [Third Embodiment] A sound field assistance device and a sound field assistance method according to a third embodiment of the present invention will be described with reference to the drawings. FIG. 11 is a functional block diagram showing the configuration of a sound field assistance system including the sound field assistance device according to the third embodiment of the present invention.

[0077] As shown in FIG. 11, the sound field assistance device 10B according to the third embodiment is different from the sound field assistance device 10 according to the first embodiment in that a posture detection unit 70 is added. Other configurations of the sound field assistance device 10B are the same as those of the sound field assistance device 10, and descriptions of the same parts are omitted.

[0078] The posture detection unit 70 is attached to the head of the viewer and detects the posture of the viewer's head. For example, the posture detection unit 70 is an orthogonal three-axis posture detection sensor and is attached to the headphones 80. The posture detection unit 70 outputs the detected posture of the viewer's head to the binaural processing unit 50.

[0079] The binaural processing unit 50 performs binaural processing on the object reproduction sound signal and the simulated reproduction sound signal using the posture detection result of the viewer's head, that is, the direction of the viewer's face.

[0080] As a result, the sound field support device 10B can experience the object reproduction sound and the simulated reproduction sound according to the direction of the viewer's face. Therefore, the viewer can compare and listen to the object reproduction sound and the simulated reproduction sound according to the direction of the face while changing the direction of the face within the target space. For this reason, the viewer can directly experience the difference between the object reproduction sound and the simulated reproduction sound in a plurality of directions within the target space, and can more accurately judge whether the simulated reproduction sound can accurately reproduce (simulate) the object reproduction sound or whether there is no sense of incongruity between the object reproduction sound and the simulated reproduction sound. Also, as a result, the viewer can more accurately reproduce the object reproduction sound by the simulated reproduction sound.

[0081] (Sound field support method of the third embodiment) FIG. 12 is a flowchart showing a sound field support method according to the third embodiment of the present invention. The sound field support method shown in FIG. 12 is obtained by adding processing related to head posture detection to the sound field support method shown in FIG. 4. Note that the description of the same processing as that shown in FIG. 4 in each processing shown in FIG. 12 is omitted.

[0082] The sound field support method shown in FIG. 12 performs the same processing as the sound field support method shown in FIG. 4 until step S14.

[0083] The posture detection unit 70 detects the posture of the viewer's head (S41).

[0084] The selection unit 40 selects an object reproduction sound signal and a simulated reproduction sound signal by an operation from a viewer or the like (S15).

[0085] If the object reproduction sound signal is selected (S150: YES), the binaural processing unit 50 performs binaural processing on the object reproduction sound signal using the detected head posture (S461). If the simulated reproduction sound signal is selected (S150: NO), the binaural processing unit 50 performs binaural processing on the simulated reproduction sound signal using the detected head posture (S462).

[0086] The sound field support device 10B executes the process of step S17 using the sound signal subjected to binaural processing.

[0087] Accordingly, the sound field support method of the third embodiment can output the object reproduction sound and the simulated reproduction sound according to the orientation of the viewer's face. Therefore, the viewer can compare and listen to the object reproduction sound and the simulated reproduction sound according to the orientation of the face while changing the orientation of the face within the target space. For this reason, the viewer can directly experience the difference between the object reproduction sound and the simulated reproduction sound in a plurality of orientations within the target space, and can more accurately determine whether the simulated reproduction sound can accurately reproduce (simulate) the object reproduction sound, or whether there is no discomfort between the object reproduction sound and the simulated reproduction sound. Also, as a result, the viewer can more accurately reproduce the object reproduction sound by the simulated reproduction sound.

[0088] Note that the configurations and processes of the above-described embodiments can be combined as appropriate, and the effects corresponding to the respective combinations can be obtained.

[0089] Also, the description of this embodiment is illustrative in all respects and not restrictive. The scope of the present invention is indicated not by the above-described embodiments but by the claims. Furthermore, the scope of the present invention is intended to include all modifications within the meaning and scope equivalent to the claims.

Description of Reference Numerals

[0090] 10, 10A, 10B: Sound field support device 21: Viewing point setting unit 22: Sound source position setting unit 23: Speaker position setting unit 29: Adjustment operation unit 30: Simulated reproduction sound signal generation unit 40: Selection unit 50: Binaural processing unit 60: Reverb processing unit 70: Posture detection unit 80: Headphone 90: Target space 99: Virtual space 100, 100A: GUI 111: Position relationship confirmation window 111A: Setting display window 112: Waveform confirmation window 113, 116: Operator 115: Output state display window 900: Viewing point

Claims

1. Select the location information of the sound source set in the virtual space, selecting localization information of the sound source when simulating the sound of the sound source with a simulated playback sound signal using an output sound from a speaker set in a target space; comparing the selected sounds based on the position information and the localization information, and adjusting parameters of the simulated reproduced sound signal based on the comparison result; Sound field support method.

2. The parameters include position information of the sound source, position information of the speaker, and a volume level or a frequency characteristic of the simulated playback sound signal. The sound field support method according to claim 1 .

3. displaying a graphical user interface for adjusting said parameters; The sound field support method according to claim 1 or 2.

4. displaying the waveform of the simulated reproduced sound signal on a display device; The sound field support method according to claim 3.

5. adding an early reflection sound or a reverberation sound to the sound based on the selected position information and the selected localization information, The sound field support method according to any one of claims 1 to 4.

6. A setting of a spread and a positioning sense for the early reflection sound or the reverberation sound is received. The sound field support method according to claim 5.

7. displaying a simulated image corresponding to the settings of the spread and the sense of localization on a display device; The sound field support method according to claim 6.

8. A viewing position is set in the target space, setting a binaural process based on the position information or the localization information and the listening position; outputting the binaurally processed sound; The sound field support method according to any one of claims 1 to 7.

9. Setting the direction of the viewer's face at the viewing position; setting the binaural processing based on the position information or the localization information, the listening position, and the face direction; The sound field support method according to claim 8.

10. Select the location information of the sound source set in the virtual space, selecting localization information of the sound source when simulating the sound of the sound source with a simulated playback sound signal using output sound from a speaker set in a target space; A selection unit; comparing the selected sounds based on the position information and the localization information, and adjusting parameters of the simulated reproduced sound signal based on the comparison result; An adjustment unit; A sound field support device comprising:

Images