Sound field assistance method and sound field assistance device

Abstract

The present invention relates to a sound field assistance method and a sound field assistance device capable of comparing a sound of a virtual sound source with a simulated sound reproduced in an object space. The sound field assistance method selects any one of position information of a sound source set in a virtual space and positioning information of the sound source when sound of the sound source is simulated by output sound from a speaker set in a target space, and uses the sound based on the selected position information and positioning information. And sound image positioning of the sound source realized by the loudspeaker is adjusted.

Description

[0001] This application is a divisional application of the Chinese national application No. 202210247023.7 filed on March 14, 2022 (Sound field auxiliary method, sound field auxiliary device, and storage medium), the contents of which are incorporated below. TECHNICAL FIELD

[0002] One embodiment of the present application relates to a sound field auxiliary method and a sound field auxiliary device that perform a process for simulating a sound field realized by a sound source set in a virtual space in an object space in which a speaker is arranged. BACKGROUND

[0003] There are various techniques for simulating a sound of a sound source set in a virtual space in an actual space.

[0004] For example, as shown in Patent Literature 1, a simulation system sets positions of a plurality of virtual speakers in such a manner that the positions follow in order to maintain a relative positional relationship with a viewer on a virtual space in association with a change in the position of the viewer. Also, the simulation system shown in Patent Literature 1 sets a volume balance of the plurality of virtual speakers.

[0005] The simulation system shown in Patent Literature 1 performs an audio processing using the plurality of virtual speakers on the basis of the above setting.

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2017-184174

[0007] However, in a case where a sound set using a virtual sound source (virtual speaker of Patent Literature 1) is played in an object space, the sound is played by a speaker arranged in the object space and assigned the virtual sound source. That is, the sound played in the object space is a sound obtained by simulating a sound of the virtual sound source by a sound of the speaker arranged in the object space.

[0008] Also, in the past, a sound from the virtual sound source and a sound (simulated played sound) played by the speaker in the object space by simulation could not be compared. Therefore, a viewer could not confirm to what extent the sound from the virtual sound source could be simulated by the simulated played sound, and could not easily perform adjustment. SUMMARY

[0009] Therefore, an object of one embodiment of the present application is to enable comparison of a sound of a virtual sound source and a simulated played sound.

[0010] The sound field assistance method selects either of position information of a sound source set on a virtual space and positioning information of the sound source when a sound of the sound source is simulated by an output sound from a speaker set on an object space, and adjusts a sound image position of the sound source based on the speaker using a sound based on the selected position information and positioning information.

[0011] Effects of the Invention

[0012] The sound field assistance method enables a viewer to compare a sound of a virtual sound source and a simulated playback sound. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a functional block diagram showing a structure of a sound field assistance system including a sound field assistance apparatus relating to the first embodiment of the present invention.

[0014] Figure 2 is a diagram showing one example of a positional relationship of a sound source, a viewing point, and a plurality of speakers of a sound field assistance method relating to the first embodiment of the present invention, and showing a position coordinate of the sound source, a position coordinate of the viewing point, and position coordinates of the plurality of speakers.

[0015] Figure 3 is a diagram showing an overview of sound emission from a sound source, and showing an overview of sound emission from a sound source reproduced by a speaker.

[0016] Figure 4 is a flowchart showing a first method of a sound field assistance method relating to the first embodiment of the present invention.

[0017] Figure 5 is a flowchart showing a second method of a sound field assistance method relating to the first embodiment of the present invention.

[0018] Figure 6 is a diagram showing one example of a GUI for parameter adjustment.

[0019] Figure 7 is a functional block diagram showing a structure of a sound field assistance system including a sound field assistance apparatus relating to the second embodiment of the present invention.

[0020] Figure 8 is a diagram showing one example of a positional relationship of a sound source, a viewing point, a plurality of speakers, and a virtual space of a sound field assistance method relating to the second embodiment of the present invention.

[0021] Figure 9 is a diagram showing one example of a GUI for adjustment of spread and localization of a sound.

[0022] Figure 10 is a flowchart showing a sound field assistance method relating to the second embodiment of the present invention.

[0023] Figure 11 This is a functional block diagram showing the structure of a sound field assisting system that includes the sound field assisting device according to the third embodiment of the present invention.

[0024] Figure 12 This is a flowchart illustrating the sound field assist method according to the third embodiment of the present invention. Detailed Implementation

[0025] Referring to the accompanying drawings, the sound field assisting method and sound field assisting device according to the embodiments of the present invention will be described.

[0026] In this embodiment, the object space is the space in which a viewer or listener actually hears the sound of a sound source set in a virtual space using a speaker or the like. More specifically, in the sound field assisting method of this embodiment, the object space does not represent the space where a speaker is actually placed, but rather a predetermined space where a viewer or listener hears the sound from that speaker. The virtual space is the space in the object space containing the sound source that is to be simulated.

[0027] [First Embodiment]

[0028] Figure 1 This is a functional block diagram showing the structure of a sound field assisting system that includes the sound field assisting device according to the first embodiment of the present invention. Figure 2 (A) is a diagram illustrating an example of the positional relationship between the sound source, the viewing / listening point, and multiple loudspeakers in the sound field assistance method according to the first embodiment of the present invention. Figure 2 (B) means Figure 2 (A) is a diagram showing the coordinates of the sound source, the audiovisual point, and the multiple loudspeakers. Figure 3 (A) is a diagram showing an overview of sound emission from the sound source. Figure 3 (B) is a diagram showing the general situation of sound source being reproduced and played by loudspeaker.

[0029] like Figure 2 As shown in (A), an audiovisual point 900 for the viewer to watch and listen, and multiple speakers SP1-SP5 are configured in the object space 90. A virtual space is set in the object space 90. A sound source OBJ is set in the virtual space.

[0030] Furthermore, in this embodiment, there is one sound source, but there can be multiple sound sources. When there are multiple sound sources, the sound field assistance method described below can be applied to each of the multiple sound sources separately. Alternatively, the sound field assistance method described below can be applied to multiple sound sources collectively. In this embodiment, the case with one sound source is described. Also, in this embodiment, the number of loudspeakers is 5, but the number of loudspeakers is not limited to this.

[0031] The coordinate system of object space 90 and the coordinate system of virtual space can be set to have the same orthogonal three axes and center point. In this case, the position coordinates based on the coordinate system in object space 90 are the same as the position coordinates based on the coordinate system in virtual space. Furthermore, even if the coordinate system of object space 90 and the coordinate system of virtual space are not the same, in this case, it is only necessary to set the coordinate transformation matrix between object space 90 and virtual space.

[0032] like Figure 1 As shown, the sound field assist system includes a sound field assist device 10 and headphones 80. The sound field assist device 10 includes an audio-visual point setting unit 21, a sound source position setting unit 22, a speaker position setting unit 23, an adjustment operation unit 29, an analog playback sound signal generation unit 30, a selection unit 40, and a binaural processing unit 50. The sound field assist device 10 is implemented by a processing unit such as a CPU that executes the above-mentioned functional units, a storage medium that stores the program, and executes the program.

[0033] The audiovisual point setting unit 21 sets the position coordinates Pr of the audiovisual point 900 in the object space 90. The audiovisual point setting unit 21 outputs the position coordinates Pr of the audiovisual point 900 to the analog playback sound signal generation unit 30 and the binaural processing unit 50.

[0034] The sound source position setting unit 22 sets the position coordinates Pobj of the sound source OBJ in the virtual space (more specifically, the position coordinates of the sound source in the virtual space projected onto the object space 90). The sound source position setting unit 22 outputs the position coordinates Pobj of the sound source OBJ to the analog playback sound signal generation unit 30 and the binaural processing unit 50.

[0035] The speaker position setting unit 23 sets the position coordinates Psp1-Psp5 of the multiple speakers SP1-P5 in the object space 90. The speaker position setting unit 23 outputs the position coordinates Psp1-Psp5 of the multiple speakers SP1-P5 to the analog playback audio signal generation unit 30 and the binaural processing unit 50.

[0036] The adjustment operation unit 29 receives the operation input of adjustment parameters. The adjustment operation unit 29 outputs the adjustment parameters to the analog playback sound signal generation unit 30.

[0037] The analog playback sound signal generation unit 30 generates an analog playback sound signal for output to the speakers SP1-SP5 of the object space 90 based on the object playback sound signal.

[0038] Here, the object playback sound signal refers to the sound signal output from the sound source OBJ. The analog playback sound signal refers to the sound signal used for sound image localization of the sound source OBJ by a speaker that reproduces the sound source OBJ.

[0039] More specifically, the analog playback audio signal generation unit 30 uses the position coordinates Pr of the viewing / audio point 900 as a reference point to calculate the positional relationship between the position coordinates Pobj of the sound source OBJ and the position coordinates Psp1-Psp5 of the multiple speakers SP1-SP5. The analog playback audio signal generation unit 30 uses this positional relationship to set the sound image positioning information of the sound source OBJ. The sound image positioning information refers to the information set for playback of sound from the multiple speakers SP1-SP5 at the viewing / audio point 900 using the sound source OBJ; it is information that determines the volume and timing of the output sound from the multiple speakers SP1-SP5.

[0040] The analog playback audio signal generation unit 30 uses the sound image localization information of the sound source OBJ to set up multiple speakers for reproducing the sound source OBJ (see reference). Figure 3 (B) The analog playback sound signal generation unit 30 generates an analog playback sound signal played by multiple speakers that reproduce the sound source OBJ. The analog playback sound signal generation unit 30 outputs the analog playback sound signal to the selection unit 40.

[0041] The selection unit 40 receives operation input from viewers, listeners, etc., and selects the target playback sound signal or analog playback sound signal. More specifically, if the setting to listen to the sound directly output from the sound source OBJ set in the virtual space is selected ( Figure 3 (A) state), then the selection unit 40 selects and outputs the target playback sound signal. On the other hand, if the setting of listening to the sound from multiple speakers being reproduced is selected ( Figure 3 If the state of (B) is selected, the selection unit 40 selects and outputs an analog playback sound signal. In other words, if the position information of the sound source OBJ is selected, the object playback sound signal is selected and output; if the positioning information of the sound source OBJ using a speaker is selected, the analog playback sound signal is selected and output.

[0042] The selection unit 40 outputs the selected sound signal to the binaural processing unit 50.

[0043] The binaural processing unit 50 performs binaural processing on the sound signal selected by the selection unit 40. Furthermore, the binaural processing uses a head transfer function, the details of which are known and therefore omitted in detail.

[0044] More specifically, when the selection unit 40 selects an object playback audio signal, the binaural processing unit 50 performs binaural processing on the sound signal of the sound source OBJ using the position coordinates Pobj of the sound source OBJ and the position coordinates Pr of the viewing / audio point 900. When the selection unit 40 selects an analog playback audio signal, the binaural processing unit 50 performs binaural processing on the analog playback audio signal using the position coordinates Psp of the speaker SP that reproduces the sound source OBJ and the position coordinates Pr of the viewing / audio point 900.

[0045] For example, if it is Figure 2 (A) Figure 2 (B) Figure 3 (A) Figure 3 In case (B), when the selection unit 40 selects the target playback audio signal, the binaural processing unit 50 uses the position coordinates Pobj of the sound source OBJ and the position coordinates Pr of the viewing / audio point 900 to perform binaural processing on the target playback audio signal. When the selection unit 40 selects an analog playback audio signal, the binaural processing unit 50 uses the position coordinates Psp1 and Psp5 of the speakers SP1 and SP5 that reproduce the sound source OBJ and the position coordinates Pr of the viewing / audio point 900 to perform binaural processing on the analog playback audio signal.

[0046] The binaural processing unit 50 outputs the binaural processed sound signal (binaural signal) to the headphones 80.

[0047] The headphones 80 play back sound by transmitting a sound signal based on a binaural signal. Furthermore, while this embodiment shows the use of headphones 80 for sound playback, a two-channel stereo speaker could also be used.

[0048] With the structure described above, when the object playback sound signal is selected, the viewer can hear the sound (object playback sound) through headphones 80, which locates the sound source at the position of the sound source OBJ. On the other hand, when the analog playback sound signal is selected, the viewer can hear the sound (analog playback sound) through headphones, which is simulated by a speaker that reproduces the sound source OBJ, locating the sound source at the position of the sound source OBJ.

[0049] Therefore, even without actually placing speakers in the actual space, the viewer can compare the sound of the actual playback with the simulated playback. Thus, the viewer can directly perceive the difference between the actual playback and the simulated playback, and can judge whether the simulated playback can reproduce (simulate) the actual playback with high accuracy, and whether there is any discomfort between the actual playback and the simulated playback.

[0050] Further, the listener can adjust the parameters for adjusting the simulated playback sound signal by referring to the above-mentioned perception result. Moreover, by repeatedly adjusting the parameters as mentioned above, the listener can reproduce the target playback sound with high precision through the simulated playback sound.

[0051] Further, here, a manner of adjusting the simulated playback sound signal for reproducing the sound of the sound source OBJ with high precision is shown. However, for example, in a case where the positions of the loudspeakers in the target space 90 are changed, the change of the parameters is difficult, but the position setting of the sound source OBJ can be changed, the above-mentioned sound after the binaural processing can be heard, and the listener changes the setting of the sound source OBJ to realize the desired sound field.

[0052] (Sound field assistance method 1)

[0053] Figure 4 is a flowchart showing the first sound field assistance method according to the first embodiment of the present application. Figure 4 The sound field assistance method shown in the figure performs the flow until the sound signal subjected to the binaural processing is output. Further, Figure 4 Detailed explanations of the respective processes shown in the figure are described above, and thus detailed explanations will be omitted below. Further, below, the case where the configuration manner shown in Figure 2 (A), Figure 2 (B), Figure 3 (A), Figure 3 (B) is explained as an example.

[0054] The sound source position setting section 22 sets the position of the sound source OBJ in the virtual space (S11). The loudspeaker position setting section 23 sets the positions of the loudspeakers SP1-SP5 in the target space (S12).

[0055] The simulated playback sound signal generation section 30 reproduces the sound source OBJ at the loudspeakers SP1, SP5 using the position coordinates Pobj of the sound source OBJ, the position coordinates Psp1-Psp5 of the loudspeakers SP1-SP5, and the position coordinates Pr of the listening point 900 (S13). The simulated playback sound signal generation section 30 generates the simulated playback sound signal using the reproduction result (S14).

[0056] The selection section 40 selects the object playback sound signal, the analog playback sound signal, by operation from a viewer or the like (S15). For example, the sound field assist device 10 has a GUI (Graphical User Interface) or the like. The GUI has an operation member that selects a sound signal of a playback object. If the viewer selects output of the object playback sound signal, the selection section 40 selects the object playback sound signal (S150: YES). If the viewer selects output of the analog playback sound signal, the selection section 40 selects the analog playback sound signal (S150: NO). Further, the selection of the object playback sound signal and the analog playback sound signal can also be to set a switching time, and switching is automatically performed in accordance with the time.

[0057] The binauralization processing section 50 performs binauralization processing on the selected sound signal, and generates a binauralized signal. More specifically, if the object playback sound signal is selected, the binauralization processing section 50 performs binauralization processing on the object playback sound signal, and generates a binauralized signal of the object playback sound signal (S161). If the analog playback sound signal is selected, the binauralization processing section 50 performs binauralization processing on the analog playback sound signal, and generates a binauralized signal of the analog playback sound signal (S162).

[0058] The earphone 80 plays back the binauralized signal (S17). More specifically, the earphone 80 plays back the binauralized signal if the binauralized signal of the object playback sound signal is input. The earphone 80 plays back the binauralized signal if the binauralized signal of the analog playback sound signal is input.

[0059] By performing the processing as described above, the sound field assist method can selectively provide the object playback sound and the analog playback sound to a viewer or the like.

[0060] (Sound field assist method 2)

[0061] Figure 5 is a flowchart showing a second method of the sound field assist method according to the first embodiment of the present application. Figure 5 The sound field assist method shown in Figure 4 The sound field assist method shown in Figure 5 The description of the processing shown in Figure 4 The description of the processing shown in Figure 2 (A), (B), (C), and (D) shown in Figure 2 (A), (B), (C), and (D) shown in Figure 3 (A), (B), (C), and (D) shown in Figure 3 (A), (B), (C), and (D) shown in

[0062] Figure 5 The sound field assist method shown in Figure 4The illustrated sound field auxiliary method performs the same processing up to step S17.

[0063] The listener switches the sound signal played by the processing of steps S15 to S17. As described above, the listener compares the sound of the binauralized signal of the object play sound signal and the sound of the binauralized signal of the simulated play sound signal.

[0064] If parameter adjustment is not needed (S23: NO), that is, the sound based on the binauralized signal of the object play sound signal can be reproduced with high precision from the sound based on the binauralized signal of the simulated play sound signal, the processing ends. If parameter adjustment is needed (S23: YES), the listener performs parameter adjustment using the adjustment operation section 29 (S24). The simulated play sound signal generation section 30 generates a simulated play sound signal using the adjusted parameters (S14).

[0065] Further, the parameters adjusted are, for example, settings of the reproduction of the sound source OBJ and the speaker, the volume level of the simulated play sound signal, the frequency characteristics, and the like. Figure 6 is a diagram showing one example of a GUI for parameter adjustment. As shown in Figure 6 The GUI 100 has a position relationship confirmation window 111, a waveform confirmation window 112, and a plurality of operation sections 113. The plurality of operation sections 113 each have a knob 1131 and an adjustment value display window 1132.

[0066] The position relationship confirmation window 111 displays the sound sources OBJ1 to OBJ3 and the plurality of speakers SP1 to SP5 at respectively set position coordinates. The setting of the speaker SP assigned to the sound source OBJ can be achieved, for example, by selecting the sound source OBJ and the speaker SP to be reproduced in the position relationship confirmation window 111.

[0067] The waveform confirmation window 112 displays the waveform of the simulated play sound signal. The selection of the displayed simulated play sound signal is switched, for example, by selecting the plurality of speakers SP1 to SP5 displayed in the position relationship confirmation window 111.

[0068] The plurality of operation sections 113 are, for example, operation sections that receive the settings of Q, filter processing, and the settings of gain values of the simulated play sound signal for a plurality of frequency bands (Hi, Mid, Low). The knob 1131 receives an operation from the listener, and the adjustment value display window 1132 displays the value set by the knob 1131. The parameters of the simulated play sound signal are adjusted by operation input based on the plurality of operation sections 113. Further, the waveform achieved based on the adjusted parameters is displayed in the waveform confirmation window 112.

[0069] The viewer can adjust and set the parameters while observing the GUI 100.

[0070] Thereafter, the viewer adjusts the parameters while comparing the sound of the binauralized signal based on the object playback sound signal and the sound involved in the binauralized signal based on the analog playback sound signal. As described above, the viewer can adjust so that the sound of the binauralized signal based on the analog playback sound signal can reproduce the sound of the binauralized signal based on the object playback sound signal with high precision, that is, the analog playback sound based on the loudspeaker can analog the object playback sound of the sound source OBJ with high precision. Further, the "adjustment section" of the present application is realized by the unit that outputs the object playback sound and the analog playback sound and compares them, and the adjustment operation section 29.

[0071] Further, the sound field assisting apparatus 10 and the sound field assisting method of the present embodiment show a manner of comparing the object playback sound and the analog playback sound involved in the binauralized playback. However, the sound field assisting apparatus 10 and the sound field assisting method of the present embodiment can also compare the waveform or the frequency spectrum, HOA (High Order Ambisonics) of the object playback sound signal and the waveform or the frequency spectrum, HOA (High Order Ambisonics) of the analog playback sound signal, for example, and adjust the parameters.

[0072] [2nd Embodiment]

[0073] The sound field assisting apparatus and the sound field assisting method involved in the 2nd embodiment of the present application will be described with reference to the drawings.

[0074] Figure 7 is a functional block diagram showing the structure of a sound field assisting system including the sound field assisting apparatus involved in the 2nd embodiment of the present application. Figure 8 is a diagram showing one example of the positional relationship of a sound source, a viewing point, a plurality of loudspeakers, and a virtual space of the sound field assisting method involved in the 2nd embodiment of the present application.

[0075] As shown in Figure 7 , the sound field assisting apparatus 10A involved in the 2nd embodiment differs from the sound field assisting apparatus 10 involved in the 1st embodiment in that a reverb processing section 60 is added. The other structure of the sound field assisting apparatus 10A is the same as that of the sound field assisting apparatus 10, and the description of the same parts is omitted.

[0076] The sound field assisting apparatus 10A has the reverb processing section 60. The object playback sound signal and the analog playback sound signal are input to the reverb processing section 60.

[0077] The reverberation processing unit 60 uses information from the virtual space 99 to generate an initial reflected sound signal and an echo signal. The initial reflected sound signal simulates the sound from the sound source OBJ reflected off the walls of the virtual space (single reflection) and reaching the listening / viewing point. The initial reflected sound signal is determined by the geometry of the virtual space, the position of the sound source OBJ in the virtual space, and the position of the listening / viewing point. The echo signal simulates the sound from the sound that reaches the listening / viewing point after multiple reflections in the virtual space. The echo signal is determined by the geometry of the virtual space and the position of the listening / viewing point in the virtual space.

[0078] More specifically, the reverberation processing unit 60 uses the location information of the sound source OBJ, the information of the virtual space 99, and the location information of the audiovisual point to generate an initial reflected sound signal and an echo sound signal for the target playback sound signal. The reverberation processing unit 60 adds the generated initial reflected sound signal and echo sound signal to the target playback sound signal and outputs them to the selection unit 40.

[0079] Furthermore, the reverberation processing unit 60 uses the position information of the sound source OBJ, the position information of the speakers SP1-SP5, the information of the virtual space 99, and the position information of the viewing / listening point to generate an initial reflected sound signal and an echo signal for the analog playback sound signal. As a specific example, the reverberation processing unit 60 sets a virtual sound source that simulates the location of the initial reflected sound for the sound source OBJ based on the position information of the sound source OBJ, the viewing / listening point, and the information of the virtual space 99. The reverberation processing unit 60 generates the initial reflected sound signal based on the positional relationship between the virtual sound source and the speakers SP to which the virtual sound source is assigned. The reverberation processing unit 60 generates an echo signal using the geometric shape of the virtual space and the position of the viewing / listening point in the virtual space. The reverberation processing unit 60 appends the initial reflected sound signal and the echo signal generated in the above manner to the analog playback sound signal and outputs them to the selection unit 40.

[0080] With the structure described above, the sound field assist device 10A can add and output the respective reverberation components (initial reflection and echo) to the object playback sound (sound from sound source OBJ) and the analog playback sound (sound simulated by a speaker). As described above, the viewer can also consider the reverberation components to judge the accuracy of the reproduction of the object playback sound based on the analog playback sound.

[0081] Furthermore, the reverberation processing unit 60 can also impart a sense of expansion and localization to the initial reflected sound signal and reverberation signal of the analog playback sound signal. In this case, the listener can, for example, use... Figure 9 Adjustments can be made to the GUI shown. Figure 9 This is an example diagram of a GUI used to represent the expansion and adjustment of sound localization. For example... Figure 9As shown, the GUI 100A has a setting display window 111A, an output state display window 115, and a plurality of operation members 116. The plurality of operation members 116 has a knob 1161, and an adjustment value display window 1162.

[0082] The setting display window 111A displays the virtual sound source SS, the plurality of speakers SP, the virtual space 99, and the audio point RP set with respect to the sound source OBJ at the respective set position coordinates.

[0083] The plurality of operation members 116 are operation members for setting a weight value and a shape value, and the like. The weight is a weighting of a sound in a playback space toward a prescribed direction, and the weight value is a value that determines the weighting. The shape indicates an expansion of a sound in a playback space toward a prescribed direction, and the shape value is a value that determines the expansion. The operation members 116 for setting the weight value respectively have operation members for setting the weight of the left and right, the weight of the front and back, and the weight of the upper and lower, and have operation members for setting a gain value and operation members for setting a delay amount. The operation members 116 for setting the shape value have operation members for setting the expansion, and have operation members for setting a gain value and operation members for setting a delay amount. The listener can adjust the expansion and the localization of the sound by operating the plurality of operation members 116.

[0084] The output state display window 115 schematically displays the expansion and the localization of the sound realized by the weight value and the shape value set by the plurality of operation members 116 in a graphical manner. As described above, the listener can easily recognize the expansion and the localization of the sound set by the plurality of operation members 116 as an image. Further, the output state display window 115 can also display an image representing a head and an image representing the expansion and the localization of the sound in combination with the image of the head in a case where the sound after the binauralization processing is heard through the earphone 80.

[0085] As described above, the listener can also judge the accuracy of the reproduction of the object playback sound based on the simulated playback sound, taking into account the expansion and the localization of the sound.

[0086] Further, the listener can also adjust the shape of the virtual space 99, the position with respect to the playback space, the position of the sound source OBJ, and the positions of the plurality of speakers SP, for example, by operating the setting display window 111A. In this case, the sound field assist device generates the object playback sound signal and the simulated playback sound signal in accordance with the adjusted various contents, and implements the same reverberation processing. As described above, the listener can also judge the accuracy of the reproduction of the object playback sound based on the simulated playback sound after the adjustment.

[0087] (Sound field assist method of the second embodiment)

[0088] Figure 10 is a flowchart showing a sound field assist method according to the second embodiment of the present application. Figure 10 The sound field assist method shown in Figure 4 The sound field assist method shown in Figure 10 The same processing as Figure 4 The same processing as

[0089] Figure 10 The sound field assist method shown in Figure 4 The sound field assist method shown in

[0090] The reverb processing section 60 generates reverb components (initial reflection sound signals and reverberation sound signals) for the object playback sound signals and the simulated playback sound signals and adds them to the object playback sound signals and the simulated playback sound signals (S31).

[0091] The sound field assist device 10A uses the object playback sound signals to which the reverb components are added and the simulated playback sound signals to which the reverb components are added and executes the processing from step S15 onward.

[0092] Thus, the sound field assist method according to the second embodiment can output the object playback sound (sound from the sound source OBJ) and the simulated playback sound (sound simulated by the loudspeaker) to which the respective reverb components (initial reflection sound and reverberation sound) are added. Thus, the listener can also consider the reverb components and judge the accuracy of the reproduction of the object playback sound based on the simulated playback sound.

[0093] [Third Embodiment]

[0094] A sound field assist device and a sound field assist method according to a third embodiment of the present application will be described with reference to the drawings. Figure 11 is a functional block diagram showing the structure of a sound field assist system including a sound field assist device according to the third embodiment of the present application.

[0095] As Figure 11 The sound field assist device 10B according to the third embodiment differs from the sound field assist device 10 according to the first embodiment in that the posture detection section 70 is added. The other structure of the sound field assist device 10B is the same as that of the sound field assist device 10, and the description of the same parts is omitted.

[0096] The posture detection section 70 is attached to the listener's head and detects the posture of the listener's head. For example, the posture detection section 70 is a posture detection sensor of orthogonal three axes and is attached to the earphone 80. The posture detection section 70 outputs the detected posture of the listener's head to the binauralization processing section 50.

[0097] The binauralization processing section 50 performs binauralization processing on the object playback sound signal and the simulated playback sound signal using the result of the attitude detection of the head of the listener, i.e., the orientation of the face of the listener.

[0098] Thus, the sound field assist device 10B can perceive the object playback sound and the simulated playback sound corresponding to the orientation of the face of the listener. Therefore, the listener can perceive the difference between the object playback sound and the simulated playback sound directly in the object space while changing the orientation of the face and listening to the object playback sound and the simulated playback sound corresponding to the orientation of the face. Thus, the listener can perceive the difference between the object playback sound and the simulated playback sound directly in the object space while changing the orientation of the face and listening to the object playback sound and the simulated playback sound corresponding to the orientation of the face. Therefore, the listener can more accurately determine whether the simulated playback sound can accurately reproduce (simulate) the object playback sound and whether there is a sense of discomfort between the object playback sound and the simulated playback sound. In addition, as a result, the listener can more accurately reproduce the object playback sound through the simulated playback sound.

[0099] (Sound field assist method of the third embodiment)

[0100] Figure 12 is a flowchart showing a sound field assist method involved in the third embodiment of the present application. Figure 12 The sound field assist method shown in Figure 4 The sound field assist method shown in is supplemented with a flow of processing associated with the attitude detection of the head. Further, the description of Figure 12 The description of the same processing as Figure 4 The description of the same processing as

[0101] Figure 12 The sound field assist method shown in is the same as the sound field assist method shown in Figure 4 The sound field assist method shown in is the same as the sound field assist method shown in

[0102] The attitude detection section 70 detects the attitude of the head of the listener (S41).

[0103] The selection section 40 selects the object playback sound signal and the simulated playback sound signal by operation from the listener or the like (S15).

[0104] If the object playback sound signal is selected (S150: YES), the binauralization processing section 50 performs binauralization processing on the object playback sound signal using the detected attitude of the head (S461). If the simulated playback sound signal is selected (S150: NO), the binauralization processing section 50 performs binauralization processing on the simulated playback sound signal using the detected attitude of the head (S462).

[0105] The sound field assist device 10B performs the processing of step S17 using the sound signal on which the binauralization processing is performed.

[0106] As described above, the sound field assist method of the third embodiment is able to output the object playback sound and the simulated playback sound corresponding to the orientation of the face of the listener. Therefore, the listener is able to listen to the object playback sound and the simulated playback sound corresponding to the orientation of the face of the listener in contrast while changing the orientation of the face within the object space. Therefore, the listener is able to directly perceive the difference between the object playback sound and the simulated playback sound in a plurality of orientations within the object space, and is able to more accurately determine whether the simulated playback sound accurately reproduces (simulates) the object playback sound, and whether there is a sense of discomfort between the object playback sound and the simulated playback sound. In addition, as a result, the listener is able to more accurately reproduce the object playback sound through the simulated playback sound.

[0107] Further, the structure and the process of each of the above-described embodiments can be appropriately combined, and effects corresponding to each combination can be achieved.

[0108] In addition, the description of the present embodiment is illustrative in all aspects, and is not limiting. The scope of the present application is not indicated by the above-described embodiments, but by the claims. Also, the scope of the present application includes all modifications within the equivalent meaning and range of the claims.

[0109] Explanation of Reference Signs

[0110] 10, 10A, 10B: sound field assist apparatus

[0111] 21: listener point setting section

[0112] 22: sound source position setting section

[0113] 23: speaker position setting section

[0114] 29: adjustment operation section

[0115] 30: simulated playback sound signal generation section

[0116] 40: selection section

[0117] 50: binaural processing section

[0118] 60: reverberation processing section

[0119] 70: attitude detection section

[0120] 80: earphone

[0121] 90: object space

[0122] 99: virtual space

[0123] 100, 100A: GUI

[0124] 111: position relationship confirmation window

[0125] 111A: display window setting

[0126] 112: waveform confirmation window

[0127] 113, 116: operation member

[0128] 115: output state display window

[0129] 900: audiovisual point

Claims

1. A sound field assist method, wherein, Select the location information of the sound source set in the virtual space. The location information of the sound source is selected when simulating the sound of the sound source by means of an output sound from a speaker set in the object space to simulate a playback sound signal. The sound based on the selected location information and the positioning information is compared, and the parameters of the simulated playback sound signal are adjusted based on the comparison result.

2. The sound field assist method according to claim 1, wherein, The parameters include the location information of the sound source, the location information of the speaker, and the volume level or frequency characteristics of the analog playback sound signal.

3. The sound field assist method according to claim 1 or 2, wherein, The GUI is used to display the adjustment of the parameters; the GUI is a graphical user interface.

4. The sound field assist method according to claim 3, wherein, The waveform of the simulated audio signal is displayed on the monitor.

5. The sound field assist method according to any one of claims 1 to 4, wherein, An initial reflected sound or an echo is added to the sound based on the selected location information and the positioning information, respectively.

6. The sound field assist method according to claim 5, wherein, Receive settings for the expansion and localization of the initial reflected sound or the echo.

7. The sound field assist method according to claim 6, wherein, A simulated image corresponding to the settings of the expansion and the positioning sense is displayed on the screen.

8. The sound field assist method according to any one of claims 1 to 7, wherein, Set the audiovisual position for the object space. Based on the location information or the positioning information and the audiovisual location, binaural processing is configured. Output the sound after the binaural processing.

9. The sound field assist method according to claim 8, wherein, The orientation of the viewer's face at the aforementioned audiovisual location is set. The binaural processing is set based on the location information or positioning information, the audiovisual position, and the orientation of the face.

10. A sound field assist device, comprising: The selection unit selects the location information of a sound source set in a virtual space, and selects the location information of the sound source when simulating the sound of the sound source by means of an output sound from a speaker set in an object space to simulate a playback sound signal; and The adjustment unit compares the sound based on the selected location information and the positioning information, and adjusts the parameters of the analog playback sound signal based on the comparison result.

Images