Acoustic signal generating device, program and method and acoustic information db generating method in which the direction of sound source is taken into consideration

Abstract

To provide an apparatus capable of generating an acoustic signal suited to a sound source and a listener while taking the direction of a sound source to the listener into consideration.SOLUTION: An acoustic signal generating device is a device for generating an output acoustic signal by using an input acoustic signal which is acquired together with orientation information of a sound source. The device includes acoustic information search means that searches for acoustic information related to the orientation information by using the orientation information from a database for storing acoustic information which relates to the relationship between a sound source for generating a database and a sound receiver in the case of each of a plurality of orientations of the sound source for generating the database to the sound receiver, and acoustic signal generating means that generates an output acoustic signal from the input acoustic signal by using the searched acoustic information. The acoustic information here can be set to an impulse response, a transfer function, or at least one of frequency characteristics, phase characteristics, and amplitude characteristics.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for generating an acoustic signal according to a sound source and a listener. [Background technology]

[0002] In recent years, research and development of sound image localization technology that allows listeners to perceive a sound image at a specific location has been actively conducted. Sound image localization technology is essential for constructing a realistic virtual acoustic space in, for example, the metaverse, and for realizing realistic communication through voice.

[0003] Regarding this sound image localization technology, for example, Non-Patent Document 1 discloses one method of sound image localization experiments. Also, Non-Patent Document 2 discloses a method of measuring the head-related impulse response (HRIR) used in sound image localization. Furthermore, Patent Document 1 discloses a sound image localization control device with small hardware scale using a small sound image localization filter that simulates the head-related transfer function (HRTF).

[0004] Also, currently, attention is being paid to object audio technology, which is introduced in, for example, Non-Patent Document 3. With object audio technology, information such as the position of a sound source is added to the sound material of the sound source as metadata, making it possible to realize a more realistic three-dimensional sound field.

[0005] As an example of the use of this object audio technology, for example, Patent Document 2 discloses an audio signal decoding method in which a downmix signal and object-based parameter information are extracted from an input audio signal, an object audio signal is generated using the downmix signal and the object-based parameter information, and 3D information is applied to the object audio signal to generate an object audio signal that provides a 3D effect. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2000-023299 A [Patent Document 2] Special Publication No. 2009-526467 [Patent Document 3] JP 2007-041164 A [Non-patent literature]

[0007] [Non-Patent Document 1] Daisuke Morikawa, "First Experiment on Sound Image Localization", Journal of the Acoustical Society of Japan, Vol. 74, No. 10, pp. 547-554, 2018, [online], [Retrieved June 14, 2023], Internet<URL: https: / / www.jstage.jst.go.jp / article / jasj / 74 / 10 / 74_547 / _pdf / -char / ja> [Non-Patent Document 2] Tatsuya Hirahara, "Measurement of binaural impulse responses", Journal of the Acoustical Society of Japan, Vol. 76, No. 3, pp. 172-179, 2020, [online], [Retrieved June 14, 2023], Internet<URL: https: / / www.jstage.jst.go.jp / article / jasj / 76 / 3 / 76_172 / _pdf> [Non-Patent Document 3] Toru Shirayanagi, "Object Audio", Journal of the Institute of Image Information and Television Engineers, Vol. 71, No. 6, pp. 846-848, 2017, [online], [Retrieved June 14, 2023], Internet<URL: https: / / www.ite.or.jp / contents / keywords / 1711keyword.pdf> [Non-Patent Document 4] Katsuhiro Maki, "Spatial radiation characteristics and timbre - through instrument sound analysis and speaker design", IPSJ Technical Report, vol.2016-MUS-111, No.13, 2016 Summary of the Invention [Problem to be solved by the invention]

[0008] By using the sound image localization techniques disclosed in the above-mentioned Non-Patent Documents 1 and 2, and Patent Document 1, it is possible to present a desired sound image position to a listener, for example, via headphones. In addition, by using the object audio techniques disclosed in Non-Patent Documents 3 and 2, it is also possible for a listener to perceive a sound image at a desired position, for example, via a plurality of speakers forming a surround audio system.

[0009] Here, as disclosed in Non-Patent Document 4, for example, the loudness and tone of a sound radiated from a sound source (a musical instrument in Non-Patent Document 4) usually change considerably depending on the radiation direction. For example, the loudness and tone of a sound propagating from a sound source moving across in front of the listener generally change considerably as the direction of the sound source changes relative to the listener. Therefore, when localizing the sound image of such a sound source, it is important to generate an acoustic signal by applying an impulse response or transfer function that is in line with the radiation direction.

[0010] However, in the sound image localization technology and object audio technology disclosed in Non-Patent Documents 1 to 3 and Patent Documents 1 and 2, impulse responses and transfer functions in front of the sound source or in a predetermined fixed direction are used as in the past. As a result, depending on the positional relationship between the sound source and the listener, it becomes impossible to generate an appropriate acoustic signal that matches the sound source and the listener.

[0011] In this regard, Patent Document 3 discloses an audio signal processing technology that measures impulse responses corresponding to a plurality of measurement speakers arranged on a first closed surface based on the sound produced by a directional speaker, obtains transfer functions for each, and performs arithmetic processing on an input audio signal using these transfer functions. It also discloses that the reproduction audio signal obtained by this arithmetic processing can include information on the directional direction of the sound output from the above-mentioned directional speaker.

[0012] However, even in this audio signal processing technology, the positions of the directional speaker and microphone are fixed, and the position of the listener where suitable sound image localization can be performed is also limited. Therefore, for example, in a situation where the relative positions of the sound source and the listener change significantly because one or both of the sound source and the listener are moving, it is not possible to perform arithmetic processing that reflects the radiation characteristics of the sound source depending on the relative positions, and it is possible that an audio signal for reproduction that reproduces the original tone color cannot be obtained.

[0013] Therefore, an object of the present invention is to provide an acoustic signal generating device, program, and method capable of generating an acoustic signal suited to a sound source and a listener, taking into account the direction of the sound source with respect to the listener, and to provide a method for generating an acoustic information database (DB) that can be used when generating such an acoustic signal. [Means for solving the problem]

[0014] According to the present invention, there is provided an apparatus for generating an output audio signal using an input audio signal, the apparatus comprising: The input sound signal is obtained together with direction information of a sound source related to the sound signal; an acoustic information retrieval means for retrieving acoustic information relating to a direction information from a database storing acoustic information relating to a relationship between a sound source and a sound receiver for generating a database, the acoustic information being for each of a plurality of directions in which the direction of the sound source for generating the database with respect to the sound receiver is different; an audio signal generating means for generating an output audio signal from the input audio signal by using the searched audio information; An apparatus for generating an acoustic signal is provided, comprising:

[0015] In one embodiment of the audio signal generating device according to the present invention, the input audio signal is obtained together with position information of a sound source related to the audio signal, The database stores acoustic information for each of a plurality of directions of a sound source for generating the database relative to the sound receiver, and a plurality of degrees of separation between a position of the sound source for generating the database and a position of the sound receiver, It is also preferable that the acoustic information retrieval means retrieves acoustic information related to the orientation information and the position information from the database using the orientation information and the position information.

[0016] In the above embodiment, the acoustic signal generating device further includes a listener information acquiring unit that acquires listener orientation information related to an orientation of a listener to whom the output acoustic signal is to be provided, The database stores acoustic information for each of a plurality of directions of the sound source for generating the database relative to the sound receiver, a plurality of ways of separation between the position of the sound source for generating the database and the position of the sound receiver, and a plurality of directions of the sound receiver relative to the sound source for generating the database; It is also preferable that the acoustic information retrieval means retrieves acoustic information relating to the orientation information, the position information and the listener orientation information from the database using the orientation information, the position information and the listener orientation information.

[0017] Furthermore, as another embodiment of the audio signal generating device according to the present invention, the output audio signal is input to headphones or earphones worn by a listener who is a recipient of the output audio signal, The database stores acoustic information relating to the right ear and acoustic information relating to the left ear, It is also preferable that the acoustic signal generating means generates the output acoustic signal to be input to the headphones or earphones from the input acoustic signal using the acoustic information relating to the right ear and the acoustic information relating to the left ear as the searched acoustic information.

[0018] In still another embodiment of the audio signal generating device according to the present invention, the output audio signal is input to a plurality of installed speakers, It is also preferable that the acoustic signal generating means generates the output acoustic signal to be input to the speaker from the input acoustic signal using the searched acoustic information, installation position information of the speaker, and position information of the sound source related to the input acoustic signal.

[0019] Here, it is also preferable that the input acoustic signal is provided with direction information and position information of a sound source related to the acoustic signal. It is also preferable that the direction information and position information of the sound source related to the input acoustic signal are information obtained by measuring the direction and position of the sound source. Furthermore, it is also preferable that the acoustic information is at least one of an impulse response, a transfer function, or a frequency characteristic, a phase characteristic, and an amplitude characteristic.

[0020] According to the present invention, there is also provided an apparatus for generating an output sound signal relating to sound output from a sound source, the apparatus comprising: A sound source information acquiring means for acquiring direction information and position information of the sound source; an acoustic information retrieval means for retrieving acoustic signal information relating to the direction information and the position information from a database storing acoustic signal information relating to the relationship between the sound source and the sound receiver, the direction of the sound source relative to the sound receiver being each of a plurality of directions, and the distance between the position of the sound source and the position of the sound receiver being each of a plurality of distances, using the direction information and the position information; an audio signal generating means for generating the output audio signal from the retrieved audio signal information; An apparatus for generating an acoustic signal is provided, comprising:

[0021] According to the present invention, there is further provided a method for generating a database storing acoustic information relating to a relationship between a sound source and a sound receiver, comprising the steps of: acquiring a measurement result of the acoustic information in each of a plurality of cases where the orientation of the sound source with respect to the sound receiver is each of a plurality of orientations and the position of the sound source and the position of the sound receiver are each of a plurality of distances; storing the acquired acoustic information as a measurement result in a searchable form using the information related to the direction and the information related to the degree of separation as search parameters; A computer-implemented method for generating a database is provided, comprising:

[0022] According to the present invention, there is further provided a method for generating a database storing acoustic information relating to a relationship between a sound source and a sound receiver, comprising the steps of: acquiring information related to a plurality of sounds generated by a plurality of microphones arranged around the sound source; a step of calculating, by an acoustic simulation, acoustic information for each of a plurality of directions of the sound source relative to the sound receiver and a plurality of degrees of separation between the position of the sound source and the position of the sound receiver, using the acquired information related to the plurality of sounds as a simulation input; storing the calculated acoustic information in a searchable form using the information relating to the direction and the information relating to the degree of separation as search parameters; A computer-implemented method for generating a database is provided, comprising:

[0023] In the database generating method according to the present invention described above, it is also preferable that the plurality of orientations are set individually for each of the plurality of separation degrees, and the interval between adjacent orientations among the plurality of orientations is set to a larger value as the corresponding value of the separation degree increases. In this case, it is also preferable that the orientation of the sound source relative to the sound receiver is each of a plurality of orientations, the separation degree between the position of the sound source and the position of the sound receiver is each of a plurality of separation degrees, and the orientation of the sound receiver relative to the sound source is each of a plurality of orientations.

[0024] According to the present invention, there is also provided a program for generating an output audio signal using an input audio signal, the program comprising: The input sound signal is obtained together with direction information of a sound source related to the sound signal; an acoustic information retrieval means for retrieving acoustic information relating to a direction information from a database storing acoustic information relating to a relationship between a sound source and a sound receiver for generating a database, the acoustic information being for each of a plurality of directions in which the direction of the sound source for generating the database with respect to the sound receiver is different; an audio signal generating means for generating an output audio signal from the input audio signal by using the searched audio information; An audio signal generating program is provided that causes a computer to function as:

[0025] According to a further aspect of the present invention there is provided a method for generating an output audio signal using an input audio signal, the method comprising the steps of: The input sound signal is obtained together with direction information of a sound source related to the sound signal; A step of searching for acoustic information related to a relationship between a sound source and a sound receiver for database generation, the acoustic information being related to the orientation information from a database that stores acoustic information for each of a plurality of orientations of the sound source for database generation relative to the sound receiver, using the orientation information; generating the output acoustic signal from the input acoustic signal using the retrieved acoustic information; A computer-implemented method for generating an acoustic signal is provided, comprising: Effect of the Invention

[0026] According to the acoustic signal generating device, program, and method of the present invention, it is possible to generate an acoustic signal suited to a sound source and a listener, taking into consideration the direction of the sound source with respect to the listener. Also, according to the acoustic information DB generating method of the present invention, it is possible to generate an acoustic information DB that can be used when generating such an acoustic signal. [Brief description of the drawings]

[0027] [Figure 1]1 is a functional block diagram showing a functional configuration of an embodiment of an acoustic signal generating device according to the present invention, and a functional block diagram showing a functional configuration of an embodiment of an acoustic information database (DB) generating device according to the present invention; [Diagram 2] FIG. 2 is a schematic diagram for explaining a specific example of a method for generating acoustic information for generating an acoustic information DB according to the present invention. [Diagram 3] FIG. 11 is a schematic diagram for explaining another specific example of the acoustic information generating method for generating an acoustic information DB according to the present invention. [Figure 4] 3 is a schematic diagram for explaining a specific example of a method for generating acoustic information for DB generation, which is carried out by an acoustic information acquisition means according to the present invention. FIG. [Diagram 5] FIG. 2 is a schematic diagram for explaining one method of generating input information for an acoustic simulation. [Figure 6] 11 is a schematic diagram for explaining another specific example of a method for generating acoustic information for DB generation, which is carried out by the acoustic information acquiring means according to the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0029] [Acoustic signal generation device, acoustic information DB generation device] FIG. 1 is a functional block diagram showing the functional configuration of an embodiment of an acoustic signal generating device according to the present invention, and a functional block diagram showing the functional configuration of an embodiment of an acoustic information database (DB) generating device according to the present invention.

[0030] In this embodiment, the acoustic signal generating device 1 according to the present invention shown in FIG. 1 takes in the following acoustic signal (a) or (b) as a sound source acoustic signal related to a sound source, and generates an output acoustic signal using this sound source acoustic signal and “acoustic information” (e.g., impulse response) searched for in an acoustic information database (DB) 201. (a) Acoustic signals measured and generated using a microphone 3 in a sound collection space, such as an anechoic chamber, and which are acoustic signals for each sound source (a speaker or a drone in FIG. 1) related to sounds emitted from the sound source. (b) Acoustic signals generated for each sound source (speaker and drone in Figure 1) set in the designed virtual acoustic space.

[0031] In this embodiment, the output acoustic signal thus generated is (c) a plurality of speakers 51 installed in a reproduction space in which a listener is present; or (d) Left and right ear speakers of headphones 52 worn by a listener watching a display showing a virtual acoustic space. The sound output from the speaker to which this output acoustic signal is input enables the listener to perceive (localize) the sound image of the sound source (e.g., a speaker or a drone) at a position corresponding to the sound source.

[0032] Generally, the loudness and tone of a sound emitted from a sound source (e.g., a speaker or a drone) changes considerably depending on the direction of emission. For example, the loudness and tone of a sound propagating from a sound source moving across the room will generally change considerably as the direction of the sound source changes relative to the listener. Therefore, when localizing the sound image of such a sound source, it is important to generate an output sound signal by applying "acoustic information" (e.g., impulse response) that matches the direction of emission.

[0033] In this way, the acoustic signal generating device 1 generates an output acoustic signal suited to the sound source and the listener, taking into account the direction of the sound source with respect to the listener. (A) an acoustic information search unit 113 that searches an acoustic information DB 201 for "acoustic information" (e.g., an impulse response) related to the "sound source direction information" that is information on the direction of the sound source related to the sound source acoustic signal, which is acquired together with the input sound source acoustic signal; (B) an acoustic signal generating unit 114 that generates an output acoustic signal from a sound source acoustic signal using the searched “acoustic information”; It has.

[0034] This allows the acoustic signal generating device 1 to generate an output acoustic signal suited to the sound source and the listener, taking into account the direction of the sound source relative to the listener. In addition, in this embodiment, it is possible to realize suitable sound image localization that reflects the radiation characteristics (distribution across the radiation directions of the sound) of the sound emitted from the sound source (e.g., a speaker or a drone). For example, even when the sound source (a speaker or a drone) moves, it is possible to perform sound image localization in which the perceived position and timbre of the sound image change appropriately according to the movement of the sound source by searching for "acoustic information" (corresponding to "sound source direction information") that reflects the radiation characteristics of the sound source at each time point.

[0035] Regarding the "sound source direction information" in (A), it is also preferable that the input sound source acoustic signal is provided with this "sound source direction information". Furthermore, this "sound source direction information" can be information obtained by measuring the direction of the sound source using, for example, a position / direction sensor 41 installed in the sound collection space. Alternatively, it can be information on the "direction" set for the sound source generated when designing the virtual acoustic space.

[0036] The acoustic information DB201 of (A) is a database that stores "acoustic information" relating to the relationship between a sound source for generating a DB and a sound receiver, in each of a plurality of directions of the sound source for generating a DB relative to the sound receiver. Here, the sound receiver can be a sound collection device for generating a DB (for example, a dummy head, described later) that is installed assuming a listener. A method for generating the acoustic information DB201 and a method for searching the acoustic information DB201 for "acoustic information" relating to "sound source direction information" will be described in detail later.

[0037] Furthermore, in this embodiment, the "acoustic information" can be an impulse response, but it may also be a transfer function, and further, it can also be at least one of frequency characteristics, phase characteristics, and amplitude characteristics.

[0038] Furthermore, in the above (B), the generation of the output audio signal from the input sound source audio signal can be performed by known techniques such as amplitude panning and object audio in this embodiment. Furthermore, when there are multiple sound sources and a sound source audio signal and "sound source direction information" are acquired for each sound source, it is possible to generate an output audio signal by adding up the (output) audio signals generated for each sound source. Naturally, the output audio signal may be a monaural signal that is output to, for example, one speaker 51 installed in the reproduction space.

[0039] [Functional configuration of an acoustic signal generating device, and an acoustic signal generating program and method] The functional configuration of the acoustic signal generating device 1 according to an embodiment of the present invention will be described in more detail below. In the functional block diagram of FIG. 1, the acoustic signal generating device 1 includes a communication interface (IF) 101 and a processing unit including an IC (Integrated Circuit) such as a processor and various memories. The processing unit has a computer (arithmetic processing) function, and in this embodiment functions as a sound source information acquiring unit 111, a listener information acquiring unit 112, an acoustic information searching unit 113, and an acoustic signal generating unit 114. These functional components may be realized by executing an embodiment of an acoustic signal generating program according to the present invention stored in the processing unit. At least a part of these functional components may be realized by wired logic. The process flow shown by connecting the functional components of the acoustic signal generating device 1 in the functional block diagram of FIG. 1 with arrows can also be understood as an embodiment of an acoustic signal generating method according to the present invention.

[0040] Here, the acoustic signal generating device 1 may be a device dedicated to acoustic signal generation processing, or may be a general-purpose cloud server or non-cloud server equipped with an embodiment of the acoustic signal generating program according to the present invention, or may be a personal computer (PC), a notebook or tablet computer, a smartphone, or even a wearable device such as a head mounted display (HMD).Furthermore, it may be an acoustic signal reproducing device equipped with an embodiment of the acoustic signal generating program according to the present invention, such as a speaker 51 or headphones 52.

[0041] The acoustic signal generating device 1 may internally hold the acoustic information DB 201. As will be described later, the acoustic information DB generating device 2 may also be incorporated into the device.

[0042] <Sound source information acquisition means> Similarly, in the functional block diagram of FIG. 1, the sound source information acquisition unit 111 of this embodiment includes: (a) "sound source direction information" relating to the direction of the sound source and "sound source position information" relating to the position of the sound source; (b) "Source acoustic signal" which is an acoustic signal related to the sound emitted from the sound source; via the communication interface 101. Here, the "sound source direction information" and "sound source position information" in (a) above can be information generated by a position / direction sensor 41 installed in a sound collection space where the sound source exists, and acquired together with (linked to) the "sound source acoustic signal."

[0043] The position / orientation sensor 41 can be, for example, an RGB-D camera, which generates an RGB image and a distance image of the sound source and analyzes them to (c) Generate and output "sound source direction information" (for example, the spherical coordinate (θ, φ) value of the vector representing the front direction of the sound source) from the generated 3D shape information of the sound source, (d) Generate and output "sound source position information" (e.g., the three-dimensional position coordinates (x, y, z) of the sound source) within the sound collection space. It is possible.

[0044] The "sound source direction information" and "sound source position information" in (a) above may be information on the "direction" and "position" set for the sound source generated when designing the virtual acoustic space. In this case, the "sound source acoustic signal" in (b) above inputted via the communication interface 101 is provided with the "sound source direction information" and "sound source position information."

[0045] Furthermore, the "sound source acoustic signal" in (b) above can be, for example, a monaural signal generated by a microphone 3 such as a shotgun microphone, a condenser microphone, or a lavalier microphone (pin microphone) attached to a sound source (for example, a speaker). This makes it possible to obtain a "sound source acoustic signal" related to the sound of only this sound source. Alternatively, the "sound source acoustic signal" may be, for example, a monaural signal generated by a microphone 3 installed at a predetermined position in the collection space. When there is only one sound source, or when there is only one sound source near this predetermined position, the generated "sound source acoustic signal" is an acoustic signal related to this sound source. Furthermore, the "sound source acoustic signal" may be an acoustic signal set for the sound source generated when designing the virtual acoustic space.

[0046] When the "sound source acoustic signal" input via the communication interface 101 is an analog acoustic signal, it is also preferable to provide an A / D converter between the communication interface 101 and the sound source information acquisition unit 111. In this case, the sound source information acquisition unit 111 takes in the "sound source acoustic signal" digitized by this A / D converter.

[0047] <Means of acquiring listener information> 1, in the case of performing sound image localization in the headphones 52, the listener information acquisition unit 112 of this embodiment acquires "listener orientation information" and "listener position information" that are set in advance in the virtual acoustic space and the sound collection space via the communication interface 101. As a modification, the "listener orientation information" can be information generated by a position / orientation sensor 43, such as an RGB-D camera, installed at a location where it is possible to capture an image of the listener.

[0048] Furthermore, when performing sound image localization in the reproduction space, the listener information acquisition unit 112 of this embodiment acquires "listener position information" relating to the positions of listeners present in the reproduction space via the communication interface 101. This "listener position information" can be information generated by a position / orientation sensor 42, such as an RGB-D camera, installed in the reproduction space.

[0049] <Audio information retrieval means> Similarly, in the functional block diagram of FIG. 1, the acoustic information retrieval unit 113 of this embodiment performs the following when performing sound image localization in the headphones 52: (a) "sound source direction information" and "sound source position information" received from the sound source information acquisition unit 111, and (b) “Listener orientation information” and “listener position information” received from the listener information acquisition unit 112 Using these, acoustic information DB 201 is searched for acoustic information related to “sound source direction information”, “sound source position information”, “listener direction information”, and “listener position information” (in this embodiment, left ear impulse response and right ear impulse response).

[0050] The acoustic information DB 201 will be described in detail later, including a method for generating the same. In this embodiment, (a) A "sound source relative orientation" which is the orientation of a DB generation sound source with respect to a sound receiver is each of a plurality of sound source relative orientations, (b) The "distance" which is the distance between the position of the DB generating sound source and the position of the sound receiver is each of a plurality of distances, (c) The "relative orientation of the sound receiver" which is the orientation of the sound receiver with respect to the sound source for generating the DB is each of a plurality of relative orientations of the sound receiver. It is a database that stores the left and right ear impulse responses (acoustic information) for each set frequency band for each case.

[0051] In order to search such an acoustic information DB 201, the acoustic information search unit 113 (a) determining a "sound source relative orientation" as a search parameter using "sound source orientation information" relating to the absolute orientation of the sound source in the sound collection space or virtual acoustic space and the direction of the listener as seen from the sound source determined from the "sound source position information" and the "listener position information"; (b) From the "sound source location information" and the "listener location information," determine the "distance" as a search parameter; (c) A "sound receiver relative orientation" as a search parameter is determined using "listener orientation information" relating to the absolute orientation of the listener in the virtual acoustic space and the direction of the sound source as seen by the listener, which is determined from the "sound source position information" and the "listener position information."

[0052] Next, the acoustic information search unit 113 searches the acoustic information DB 201 for left-ear and right-ear impulse responses (acoustic information) in a case where the sound source is in a situation related to the "sound source direction information" and "sound source position information" and the listener is in a situation related to the "listener direction information" and "listener position information" using the search parameters determined in (a) to (c) above. In this way, the acoustic information search unit 113 can acquire left-ear and right-ear impulse responses (acoustic information) taking into account the direction of the sound source and the direction of the listener.

[0053] As a modified embodiment, the acoustic information DB 201 may be a database in which only the "relative sound source orientation" and "distance" are used as search parameters, as will be described in detail later. In this case, the acoustic information search unit 113 does not need "listener orientation information" and can perform a search using the "relative sound source orientation" and the above-mentioned (b) "distance" as search parameters. Furthermore, in order to further reduce the amount of stored data, the acoustic information DB 201 may be a database in which only the "relative sound source orientation" is used as a search parameter. In this case, the acoustic information search unit 113 performs a search using the "relative sound source orientation" as a search parameter.

[0054] The above describes acoustic information retrieval when performing sound image localization with the headphones 52, but below we will explain acoustic information retrieval when performing sound image localization with multiple speakers 51. Incidentally, the reproduction space in which the multiple speakers 51 are installed is a space that has a corresponding relationship with the sound collection space or virtual acoustic space in terms of intra-space positions.

[0055] In this case, the acoustic information search unit 113 (a) "sound source direction information" and "sound source position information" received from the sound source information acquisition unit 111, and (b) “Listener orientation information” and “listener position information” received from the listener information acquisition unit 112 Using these, acoustic information DB 201 is searched for acoustic information (impulse response in this embodiment) relating to "sound source direction information", "sound source position information" and "listener position information".

[0056] In this case, the acoustic information DB 201 is (a) A "sound source relative orientation" which is the orientation of a DB generation sound source with respect to a sound receiver is each of a plurality of sound source relative orientations, (b) The "distance" between the position of the DB generating sound source and the position of the sound receiver is each of a plurality of distances. It is a database that stores the impulse response (acoustic information) in each set frequency band for each case.

[0057] In order to search such an acoustic information DB 201, the acoustic information search unit 113 (a) determining a "sound source relative orientation" as a search parameter using "sound source orientation information" relating to the absolute orientation of the sound source in the sound collection space or virtual acoustic space and the direction of the listener as seen from the sound source determined from the "sound source position information" and the "listener position information"; (b) "Distance" is determined as a search parameter from the "sound source position information" and the "listener position information."

[0058] Next, the acoustic information search unit 113 searches the acoustic information DB 201 for impulse responses (acoustic information) in a case where the sound source is in a situation related to the "sound source direction information" and the "sound source position information" and the listener is in a situation related to the "listener position information" using the search parameters determined in (a) and (b) above. In this way, the acoustic information search unit 113 can obtain impulse responses (acoustic information) that also take into account the direction of the sound source.

[0059] The acoustic information search process for performing sound image localization using a plurality of speakers 51 and headphones 52 has been described above. In the acoustic information DB 201 of this embodiment, the search parameters "sound source relative orientation" and "distance" (and further "sound receiver relative orientation") have predetermined values. For example, the "sound source relative orientation" is a discrete value having a predetermined interval such as 0°, 5°, 10°, . . . as will be described later, which is merely an example. In contrast, when the acoustic information search unit 113 wishes to perform a search using a value between these, for example 8°, it may determine the values ​​immediately before and after that, for example 5° and 10°, as the "sound source relative orientation" as a search parameter and perform a search.

[0060] In this case, the acoustic information search unit 113 searches to obtain an impulse response h_5° when the "relative sound source orientation" is 5°, and an impulse response h_10° when the "relative sound source orientation" is 10°. Here, the acoustic information search unit 113 may obtain a desired impulse response h_8° when the "relative sound source orientation" is 8° by linear two-point interpolation, for example, using the following formula: (1) h_8°=(1-r)×h_5°+r×h_10° In the above formula (1), r (0≦r≦1) is the internal division ratio, and in this case, r = (8-5) / (10-5) = 0.6.

[0061] In this way, the acoustic information search unit 113 can determine search parameter values ​​immediately before or after a desired parameter value for the search parameters "relative sound source orientation", "distance", and even "relative sound receiver orientation", and calculate and determine an impulse response (acoustic information) corresponding to the desired parameter value from the search results by linear interpolation (e.g., three-dimensional two-point interpolation, etc.). As a modification, the acoustic information search unit 113 can determine a search parameter value closest to the desired parameter value, and use the search results as they are as the impulse response (acoustic information) to be used for generating an acoustic signal.

[0062] <Acoustic signal generation means> Also in the functional block diagram of FIG. 1, the acoustic signal generation unit 114 of this embodiment uses acoustic information (impulse response in this embodiment) received as a search result from the acoustic information search unit 113 to generate output acoustic signals from the sound source acoustic signal received from the sound source information acquisition unit 111 to be output to multiple speakers 51 installed in the reproduction space and to left and right speakers of the headphones 52.

[0063] (Generation of output sound signal for speaker 51) Specifically, a case where output sound signals for a plurality of speakers 51 installed in a reproduction space are generated will be described. First, a window function such as a Hanning window is applied to the sound source sound signal related to the acquired sound source s1 to generate N sound source sound signals x1_n, x1_n+1, . . . , x1_n+N-1 from time point (sampling number) t_n to time point t_n+N-1, and Fast Fourier transform (FFT) processing is performed on these N sound source sound signals to generate sound signal x1_n(ω) in the angular frequency ω domain. In addition, FFT processing is performed on the impulse response h1(t) related to the sound source s1 as the acquired search result to generate a transfer function H1(ω). Here, N is the sampling length of the impulse response, and can be set to 512, 1024, 2048, 4096, etc. when the sampling frequency fs is 48 kHz or 44.1 kHz, for example.

[0064] Next, the generated acoustic signal x1_n(ω) is convolved with a transfer function H1(ω), and the convolution result (x1_n(ω) * H1(ω)) is subjected to inverse FFT processing to generate first output acoustic signals y1_n, y1_n+1, ... related to the sound source s1.

[0065] Next, the above window function is applied to the acquired sound source acoustic signal related to the sound source s1 with a time shift of Δt to generate N sound source acoustic signals x1_n+Δt, x1_n+Δt+1, ..., x1_n+Δt+N-1 from time t_n+Δt to time t_n+Δt+N-1, and a second output acoustic signal y1_n+Δt, y1_n+Δt+1, ... related to the sound source s1 is generated in the same manner as above. Furthermore, the above window function is applied with a further time shift of Δt to generate a third output acoustic signal y1_n+2Δt, y1_n+2Δt+1, ... related to the sound source s1, and the above processes are sequentially performed to generate a fourth output acoustic signal, a fifth output acoustic signal, ...

[0066] Finally, the generated first output acoustic signal, second output acoustic signal, third output acoustic signal, etc. are sequentially shifted by Δt, that is, data at the same time point are added together to generate an output acoustic signal y1 related to the sound source s1. Incidentally, the method of generating the output acoustic signal y1 described above is a known method known as an overlap-add method. However, in this embodiment, the first output acoustic signal, the second output acoustic signal, the third output acoustic signal, etc. are calculated using an impulse response h_1 that is adapted to the sound source direction, distance, and listener direction at the time points t_n, t_n+Δt, t_n+2Δt, etc. As a result, in this embodiment, the output acoustic signal y1 of the sound source s1 is generated taking into consideration the moment-to-moment direction of the sound source s1 relative to the listener and the moment-to-moment direction of the listener relative to the sound source s1, and is more adapted to the actual situation and design contents related to the sound.

[0067] Next, a method for generating output sound signals for a plurality of speakers 51 installed in a reproduction space from an output sound signal y1 related to the generated sound source s1 will be described. In this embodiment, first, based on the acquired position information of the listener in the reproduction space, the acquired position information of the sound source s1 (information of the position 51' where the sound image is to be localized), and the installation position information of each speaker 51, three speakers 51a, 51b, and 51c (FIG. 1) between which the sound image position will be located as seen by the listener are selected. Incidentally, if the position of the listener in the reproduction space is determined in advance, there is no need to acquire the position information of the listener from the listener information acquisition unit 112.

[0068] Next, the output sound signal y1 related to the sound source s1 is distributed to each speaker (51a, 51b, 51c) by weighting factors ga, gb, and gc. In this embodiment, the weighting factors ga, gb, and gc are determined using a VBAP (Vector Based Amplitude Panning) method, which is a representative three-dimensional sound panning method, and output sound signals y1_a, yl_b, and yl_c to be output to the three speakers 51a, 51b, and 51c, respectively, are generated.

[0069] Specifically, the listener's position is set as the origin, and a unit vector pointing from the origin to the sound image position 51' is defined as p=(p1, p2, p3). T The three unit vectors pointing from the origin to the three speakers 51a, 51b, and 51c are defined as q1=(q11, q12, q13). T , q2=(q21, q22, q23) T , and q3=(q31, q32, q33) T These unit vectors can be calculated and determined from the information obtained above.

[0070] The weight coefficient vector with the weight coefficients ga, gb, and gc as components is g(=(ga, gb, gc) T ), the following equation is obtained: (2) p=Qg Here, Q is a 3×3 square matrix with q1, q2, and q3 as the first, second, and third rows, respectively. Therefore, the following equation (3) g=Q -1 p The weighting coefficients ga, gb, and gc can be calculated by using the above. The VBAP method is described in detail in, for example, non-patent document: Ando Akio, "High Reality Sound Technology," Journal of the Institute of Image Information and Television Engineers, Vol. 66, No. 8, pp. 671-677, 2012.

[0071] In this embodiment, the generation of output acoustic signals for the multiple speakers 51 described above is performed at each point in the signal generation process, including searching for an impulse response (acoustic information) to be used and selecting a speaker 51. As a result, even when the sound source s1 is moving, for example, it is possible to generate suitable output acoustic signals (y1_a, yl_b, yl_c) that take into account the radiation characteristics of the sound source s1. Also, even when multiple sound sources (s1, s2, ...) exist, the sound source acoustic signals of each sound source are used to generate an output acoustic signal for the speaker 51 by the method described above, and the output acoustic signals generated for each sound source in each speaker 51 are added to generate a final output acoustic signal.

[0072] (Generation of output audio signal for headphones 52) Next, a case where an output acoustic signal for the headphones 52 worn by a listener is generated will be described. In this case, the acoustic information stored in the acoustic information DB 201 is a left ear impulse response and a right ear impulse response. In this embodiment, the acoustic information search unit 113 searches the acoustic information DB 201 for the corresponding left ear impulse response h1_l and right ear impulse response h1_r using the "relative direction of the sound source", "distance" and "relative direction of the sound receiver" as search parameters. Next, the acoustic signal generation unit 114 uses the searched left ear impulse response h1_l and right ear impulse response h1_r to generate an output acoustic signal y1_l related to the sound source s1 for the left ear speaker of the headphones 52 and an output acoustic signal y1_r related to the sound source s1 for the right ear speaker by the same method as above (such as overlap-add method).

[0073] In this manner, in this embodiment, the generation of the output acoustic signal for the headphones 52 is performed at each point in the signal generation process, including the search for the left-ear and right-ear impulse responses (acoustic information) to be used. As a result, even when the sound source s1 is moving, for example, it is possible to generate a suitable output acoustic signal (y1_l, yl_r) that takes into account the radiation characteristics of the sound source s1. Also, even when there are multiple sound sources (s1, s2, ...), the sound source acoustic signals of each sound source are used to generate output acoustic signals for the left-ear speaker and the right-ear speaker of the headphones 52 by the above-described method, and the output acoustic signals generated for each sound source in each of the left-ear speaker and the right-ear speaker are added to generate a final output acoustic signal.

[0074] The above has described the generation process of the output acoustic signal in the acoustic signal generation unit 114, but the output acoustic signal generated here and output (transmitted) to the speaker 51 and the headphones 52 is a digital acoustic signal in this embodiment. In this case, for example, the speaker 51 and the headphones 52 are provided with a D / A converter, and an analog output acoustic signal is reproduced. Incidentally, as a modification, the output acoustic signal generated in the acoustic signal generation unit 114 may be converted into an analog acoustic signal by a D / A converter provided before the communication interface 101, and this analog output acoustic signal may be output (transmitted) to the speaker 51 and the headphones 52 (for example, it may be transmitted via a wire for analog signals).

[0075] [Functional configuration of an acoustic information DB generating device, and an acoustic signal generating program and method] Next, the functional configuration of the acoustic information DB generating device 2 according to an embodiment of the present invention will be described in detail. In the functional block diagram of FIG. 1, the acoustic information DB generating device 2 includes an acoustic information DB 201 and a processing unit including an IC such as a processor and various memories. The processing unit has a computer (arithmetic processing) function, and functions as an acoustic information acquiring unit 211 and a DB generating unit 212 in this embodiment. These functional configuration units may be realized by executing an embodiment of an acoustic information DB generating program according to the present invention stored in the processing unit. At least a part of these functional configuration units may be realized by wired logic. The process flow shown by connecting the functional configuration units of the acoustic information DB generating device 2 in the functional block diagram of FIG. 1 with arrows can also be understood as an embodiment of an acoustic information DB generating method according to the present invention.

[0076] Here, the acoustic information DB generating device 2 may be a device dedicated to acoustic information DB generation processing, or it may be a general-purpose cloud server or non-cloud type server equipped with an embodiment of the acoustic information DB generating program according to the present invention, or it may be a personal computer (PC), a notebook type or a tablet type computer, etc.

[0077] As another embodiment, the acoustic information DB generating device 2 may be incorporated into the acoustic signal generating device 1. In this case, the acoustic signal generating device 1 further includes an acoustic information acquiring unit 211, a DB generating unit 212, and an acoustic information DB 201, and is also provided with an acoustic information DB generating function.

[0078] The acoustic information DB 201 of the present embodiment described below includes: (a) The "sound source relative orientation" which is the orientation of the sound source for DB generation with respect to the sound receiver is each of a plurality of orientations, (b) The "distance" between the position of the sound source for generating the DB and the position of the sound receiver is each of a plurality of distances, (c) The "sound receiver relative orientation," which is the orientation of the sound receiver relative to the sound source used for DB generation, is one of multiple orientations. In this embodiment, the database stores the acoustic information for each of the cases, which is the impulse response in this embodiment. In other words, the database is constructed so that it can be searched using three search parameters: "relative direction of sound source," "distance," and "relative direction of sound receiver."

[0079] However, as a modified embodiment, the acoustic information DB 201 may store acoustic information (impulse responses) for each of the cases where (a) the "sound source relative orientation," which is the orientation of the sound source used to generate the DB with respect to the sound receiver, is each of a plurality of orientations, and (b) the "distance," which is the degree of separation between the position of the sound source used to generate the DB and the position of the sound receiver, is each of a plurality of degrees of separation (distances), and may be a database constructed so as to be searchable using two search parameters, the "sound source relative orientation" and the "distance."

[0080] As a further modification, the acoustic information DB201 (a) can store acoustic information (impulse responses) for each of a plurality of "sound source relative orientations," which are the orientations of the sound source used to generate the DB with respect to the sound receiver, and can be a database constructed so as to be searchable by a single search parameter, "sound source relative orientation." In any case, the acoustic information DB201 can provide more suitable acoustic information (impulse responses) that reflects the orientation of the sound source with respect to the listener as a search result.

[0081] <Acoustic information acquisition means> 1 also acquires, as one embodiment, measurement results of acoustic information (impulse response in this embodiment) in each of the cases where (a) the "sound source relative orientation" is each of a plurality of orientations, (b) the "distance" is each of a plurality of distances, and (c) the "sound receiver relative orientation" is each of a plurality of orientations. A method for generating the acoustic information (impulse response) acquired by the acoustic information acquisition unit 211 will be described below.

[0082] (Method of generating acoustic information for DB generation: Acoustic information is measured using a dummy head) FIG. 2 is a schematic diagram for explaining a specific example of a method for generating acoustic information for generating an acoustic information DB according to the present invention.

[0083] In carrying out this specific example, first, as shown in Figures 2(A1) and (A2), a DB generation speaker 6 as a sound source and a dummy head 7 as a sound receiver are attached to "mounting stands with adjustable elevation angles provided on a rotatable and movable turntable with casters" and placed in an anechoic chamber (DB generation space). The dummy head 7 is a recording device that has a shape that imitates a human head and has microphones at the positions of the left and right ears.

[0084] Here, the relative orientation of the DB generation speaker 6 with respect to the dummy head 7 in the horizontal plane, i.e., the angle (relative orientation in the horizontal plane) between the horizontal component of the DB generation speaker 6 facing forward and the horizontal component of the direction from the DB generation speaker 6 to the dummy head 7, is defined as θ1. Furthermore, the angle (relative orientation in the vertical plane) between the vertical component of the DB generation speaker 6 facing forward and the vertical component of the direction from the DB generation speaker 6 to the dummy head 7 is defined as θ3. Moreover, the distance between the DB generation speaker 6 and the dummy head 7 is defined as d.

[0085] Furthermore, the relative orientation of the dummy head 7 with respect to the DB generation speaker 6 in the horizontal plane, that is, the angle (relative orientation in the horizontal plane) between the horizontal component of the forward facing of the dummy head 7 and the horizontal component of the direction from the dummy head 7 to the DB generation speaker 6, is defined as θ2. Also, the angle (relative orientation in the vertical plane) between the vertical component of the forward facing of the dummy head 7 and the vertical component of the direction from the dummy head 7 to the DB generation speaker 6 is defined as θ4. In this specific example, the DB generation speaker 6 and the dummy head 7 are initially placed in a state in which θ1, θ2, θ3, and θ4 are zero, and the distance d is the minimum listening distance d0 (set in the DB generation speaker 6). Also, the DB generation speaker 6 and the dummy head 7 can be placed in the same horizontal plane.

[0086] In this example, under the above-mentioned settings, as shown in Figs. 2(B1) and (B2), (a) The relative direction θ1 of the DB generation speaker 6 is defined as {θ s1 (0° in this example), θ s1 +Δθ1, θ s1 +2Δθ1, θ s1 +3Δθ1, . . . , θ e1} in sequence, (b) The relative direction θ3 of the DB generation speaker 6 is defined as {θ s3 (0° in this example), θ s3 +Δθ3, θ s3 +2Δθ3, θ s3 +3Δθ3, . . . , θ e3} in sequence, (c) The relative orientation θ2 of the dummy head 7 is expressed as {θ s2 (0° in this example), θ s2 +Δθ2, θ s2 +2Δθ2, θ s2 +3Δθ2,...,θ e2} in sequence, (d) The relative orientation θ4 of the dummy head 7 is expressed as {θ s4 (0° in this example), θ s4 +Δθ4, θ s4 +2Δθ4, θ s4 +3Δθ4,...,θ e4} in sequence, (e) Let the distance d be {d0, d0+Δd, d0+2Δd, ···, d e} For all installation conditions obtained for the case, i.e., for each of the orientation-distance conditions represented by all pairs of (θ1, θ2, θ3, θ4, d), a left ear impulse response and a right ear impulse response are generated.

[0087] Here, Δθ1, Δθ2, Δθ3, and Δθ4 are, for example, 1°, 5°, or 10°, and θ e1 , θ e2 , θ e3 and θ e4 can be, for example, 359°, 355°, or 350°, and Δd can be, for example, 0.5 m or 1 m.

[0088] Furthermore, the left ear (right ear) impulse response in the direction-distance state for each set of (θ1, θ2, θ3, θ4, d) can be generated by the following methods (A) to (C). Incidentally, the sampling length N of the generated impulse response may be set to 512, 1024, 2048, 4096, etc., when the sampling frequency fs is 48 kHz or 44.1 kHz, for example. (A) A TSP (Time Stretched Pulse) signal is reproduced from the DB generating speaker 6 in the corresponding direction and distance state, (B) A TSP response of the reproduced sound is generated by a microphone at the left ear (right ear) position of the dummy head 7 in the corresponding direction and distance state that receives the reproduced sound in (A) above, (c) The left ear (right ear) impulse response is generated by performing convolution processing of the inverse TSP signal on the generated TSP response.

[0089] The TSP signal is a signal in which an impulse signal determined by the definition of the impulse response is stretched in the time direction, and is a well-known signal for measuring impulse responses. As a modification, it is also possible to generate left ear (right ear) impulse responses for each direction and distance state by using an M-sequence (Maximum length sequence) signal.

[0090] A specific example of a method for generating acoustic information for generating an acoustic information DB has been described above. The acoustic information DB201 (FIG. 1) constructed using the acoustic information (left-ear and right-ear impulse responses) generated in this way becomes a database used, for example, when localizing a sound image in the headphones 52. As a modification, the relative orientation θ3 of the DB generation speaker 6 and the relative orientation θ4 of the dummy head 7 may be fixed to a predetermined value, preferably zero, without being changed, and acoustic information (left-ear and right-ear impulse responses) for each remaining set of changing {θ1, θ2, d} may be generated. In this case, the acoustic information DB201 related to the direction and distance in a two-dimensional plane (horizontal plane) is generated.

[0091] As a further modification, the relative orientations θ2 and θ4 of the dummy head 7 can be fixed to a predetermined value, preferably zero, without being changed, and acoustic information (left and right ear impulse responses) for each remaining set of {θ1, θ3, d} that changes can be generated. It is also possible to fix θ3 (to zero) and generate acoustic information (left and right ear impulse responses) for each remaining set of {θ1, d} that changes, or to fix d (for example, to the minimum listening distance d0) and generate acoustic information (left and right ear impulse responses) for each value of the only changing θ1. In any case, an acoustic information DB 201 with appropriate search parameters is generated.

[0092] (Method of generating acoustic information for DB generation: Acoustic information is measured using an omnidirectional microphone) FIG. 3 is a schematic diagram for explaining another specific example of the acoustic information generating method for generating an acoustic information DB according to the present invention.

[0093] In this specific example, as shown in Figures 3(A1) and (A2), a DB generation speaker 6 as a sound source is attached to a "mounting stand with adjustable elevation angle provided on a rotatable and movable turntable with casters," while an omnidirectional microphone 8 as a sound receiver is attached to a "rotatable and movable turntable with casters" and placed within the DB generation space.

[0094] Also, the relative orientation of the DB generation speaker 6 with respect to the omnidirectional microphone 8 in the horizontal plane, i.e., the angle (relative orientation in the horizontal plane) between the horizontal component of the DB generation speaker 6 facing forward and the horizontal component of the direction from the DB generation speaker 6 to the omnidirectional microphone 8, is defined as θ1. Furthermore, the angle (relative orientation in the vertical plane) between the vertical component of the DB generation speaker 6 facing forward and the vertical component of the direction from the DB generation speaker 6 to the omnidirectional microphone 8 is defined as θ3. Also, the distance between the DB generation speaker 6 and the omnidirectional microphone 8 is defined as d.

[0095] In this specific example, the DB generation speaker 6 and the omnidirectional microphone 8 are initially placed in a state in which θ1 and θ3 are zero and the distance d is the minimum listening distance d0 (set in the DB generation speaker 6). Furthermore, the DB generation speaker 6 and the omnidirectional microphone 8 can be placed in the same horizontal plane.

[0096] In this example, under the above-mentioned settings, as shown in Figs. 3(B1) and (B2), (a) The relative direction θ1 of the DB generation speaker 6 is defined as {θ s1 (0° in this example), θ s1 +Δθ1, θ s1 +2Δθ1, θ s1 +3Δθ1, . . . , θ e1} in sequence, (b) The relative direction θ3 of the DB generation speaker 6 is defined as {θ s3 (0° in this example), θ s3 +Δθ3, θ s3 +2Δθ3, θ s3 +3Δθ3, . . . , θ e3} in sequence, (c) Let the distance d be {d0, d0+Δd, d0+2Δd, . . . , d e} For all installation conditions obtained for the case, i.e., for all orientation-distance conditions represented by the pairs (θ1, θ3, d), an impulse response is generated. Here, Δθ1 and Δθ3 are, for example, 1°, 5°, and 10°, and θ e1 and θ e3 can be, for example, 359°, 355°, or 350°, and Δd can be, for example, 0.5 m or 1 m.

[0097] Furthermore, the generation of impulse responses in the direction and distance states for each pair of (θ1, θ3, d) can be performed in the same manner as the above-mentioned methods (A) to (C) described in the specific example of Fig. 2. As a modification, it is also possible to generate impulse responses in each direction and distance state by using an M-sequence signal.

[0098] The acoustic information DB201 (FIG. 1) constructed using the acoustic information (impulse response) generated by the method described above is a database used, for example, when localizing a sound image using a plurality of speakers 51. As a modification, the relative orientation θ3 of the DB generation speaker 6 may be fixed to a predetermined value, preferably zero, without being changed, and acoustic information (impulse response) may be generated for each pair of the remaining changing {θ1, d}. In this case, an acoustic information DB201 relating to orientation and distance in a two-dimensional plane (horizontal plane) is generated. It is also possible to fix d (for example, to the minimum listening distance d0) and generate acoustic information (impulse response) for each value of the only changing θ1. In either case, an acoustic information DB201 with appropriate search parameters is generated.

[0099] A specific example of a method for generating acoustic information (impulse response) acquired by the acoustic information acquisition unit 211 (FIG. 1) has been described above. Below, an embodiment will be described in which, unlike the above, the acoustic information acquisition unit 211 generates acoustic information (impulse response) for DB generation using acoustic simulation.

[0100] (Method of generating acoustic information for DB generation: Calculate acoustic information in dummy head voxels by acoustic simulation) Fig. 4 is a schematic diagram for explaining a specific example of a method for generating acoustic information for DB generation, which is implemented by the acoustic information acquisition unit 211. Fig. 5 is a schematic diagram for explaining a method for generating input information for acoustic simulation.

[0101] In this specific example, the acoustic information acquisition unit 211 generates an impulse response (acoustic information) for DB generation by acoustic simulation using the CE-FDTD (Finite Difference Time Domain) method. Incidentally, acoustic (sound field) simulation using the CE-FDTD method is described in detail in, for example, non-patent document: Takao Tsuchiya, "Sound Field Simulation and Sound Space Rendering," Institute of Electronics, Information and Communication Engineers, Fundamentals Review, Vol. 10, No. 3, pp. 206-218, 2017.

[0102] In this specific example, the acoustic information acquisition unit 211 first acquires a plurality of impulse responses (acoustic information) generated by a plurality (a large number) of sound-collecting microphones 9 capable of collecting sounds emitted from a DB generation speaker 6 installed in an anechoic chamber (DB generation space). Here, the plurality of sound-collecting microphones 9 are arranged around the DB generation speaker 6, receive the reproduced sound of the TSP signal emitted from the DB generation speaker 6, and output the TSP responses.

[0103] The acoustic information acquisition unit 211 may acquire an impulse response generated by performing a convolution process of the inverse TSP signal on this TSP response, or may generate an impulse response by performing a convolution process of the inverse TSP signal on the acquired TSP response. The sound-collecting microphones 9 may be arranged at the vertices of a polyhedron (with a circumscribed sphere radius of, for example, 0.5 m) centered on the DB generation speaker 6, as shown in Fig. 5. Of course, the sound-collecting microphones 9 may be arranged in various other shapes as long as they surround the DB generation speaker 6 evenly.

[0104] Next, the acoustic information acquisition unit 211 sets a DB generation speaker grid 6' corresponding to the position of the DB generation speaker 6 and a dummy head voxel 7' corresponding to the dummy head 7 within the acoustic simulation space.

[0105] Here, the multiple impulse responses generated by the multiple sound-collecting microphones 9 are treated as impulse responses (simulation inputs) in multiple input sound source grids (FIG. 4(B)) corresponding to the positions of the multiple sound-collecting microphones 9, respectively. In this specific example, when calculating the impulse response (sound pressure) of each grid, an Interpolated Wide Band (IWB) method that refers to the coordinate axis direction, diagonal direction, and solid diagonal direction can be adopted. Furthermore, it is also preferable that the boundary conditions of the simulation are free sound field conditions that do not take into account reflections at the boundaries.

[0106] Under the above-mentioned settings, the acoustic information acquisition unit 211 (a) The "sound source relative orientation" which is the orientation of the DB generation speaker 6 (as a sound source) with respect to the dummy head 7 (identified as a sound receiver) is each of a plurality of orientations, (b) The "distance" between the position of the DB generating speaker 6 and the position of the dummy head 7 is each of a plurality of distances, (c) The "sound receiver orientation" which is the orientation of the dummy head 7 relative to the DB generation speaker 6 is each of a plurality of orientations. The left ear impulse response and right ear impulse response (acoustic information) for each case are calculated and generated by acoustic simulation.

[0107] 4(A1) and (A2), the relative orientation of the DB generation speaker 6 with respect to the dummy head voxel 7' in the horizontal plane, i.e., the angle (relative orientation in the horizontal plane) between the horizontal component of the DB generation speaker 6 facing forward and the horizontal component of the direction from the DB generation speaker grid 6' to the dummy head voxel 7' is defined as θ1. Furthermore, the angle (relative orientation in the vertical plane) between the vertical component of the DB generation speaker 6 facing forward and the vertical component of the direction from the DB generation speaker grid 6' to the dummy head voxel 7' is defined as θ3. Moreover, the distance between the DB generation speaker grid 6' and the dummy head voxel 7' is defined as d.

[0108] Furthermore, the relative orientation of the dummy head voxel 7' in the horizontal plane with respect to the DB generation speaker grid 6', i.e., the angle (relative orientation in the horizontal plane) between the horizontal component of the forward facing dummy head voxel 7' and the horizontal component of the direction from the dummy head voxel 7' to the DB generation speaker 6 is defined as θ2. Also, the angle (relative orientation in the vertical plane) between the vertical component of the forward facing dummy head voxel 7' and the vertical component of the direction of the DB generation speaker 6 as seen from the dummy head voxel 7' is defined as θ4. In this specific example, θ1, θ2, θ3, and θ4 are initially zero, and the distance d is set to the minimum listening distance d0 (set in the DB generation speaker 6).

[0109] Under the above-mentioned settings, the acoustic information acquisition unit 211 (a) As shown in FIG. 4B, the relative direction θ1 of the DB generation speaker 6 is set as {θ s1 (0° in this example), θ s1 +Δθ1, θ s1 +2Δθ1, θ s1 +3Δθ1, . . . , θ e1} in sequence, (b) As shown in Fig. 4(B), the distance d is expressed as {d0, d0+Δd, d0+2Δd, . . . , d e} in sequence, (c) As shown in FIG. 4(C), the relative direction θ3 is expressed as {θ s3 (0° in this example), θ s3 +Δθ3, θ s3 +2Δθ3, θ s3 +3Δθ3, . . . , θ e3} in sequence, (d) As shown in FIG. 4(C), the relative direction θ4 is expressed as {θ s4 (0° in this example), θ s4 +Δθ4, θ s4 +2Δθ4, θ s4 +3Δθ4,...,θ e4} in sequence, (e) As shown in FIG. 4(D), the relative direction θ2 is expressed as {θ s2(0° in this example), θ s2 +Δθ2, θ s2 +2Δθ2, θ s2 +3Δθ2,...,θ e2} For all the configurations obtained for the case, i.e., for all the orientation-distance configurations represented by the pairs (θ1, θ2, θ3, θ4, d), calculate the sound pressure for each grid.

[0110] Here, Δθ1, Δθ2, Δθ3, and Δθ4 are, for example, 1°, 5°, or 10°, and θ e1 , θ e2 , θ e3 and θ e4 can be, for example, 359°, 355°, or 350°, and Δd can be, for example, 0.5 m or 1 m.

[0111] Finally, the acoustic information acquisition unit 211 treats the calculation results of the sound pressure in the grid corresponding to the left ear of the dummy head voxel 7' and the calculation results of the sound pressure in the grid corresponding to the right ear as the left ear impulse response and the right ear impulse response for the corresponding set of (θ1, θ2, θ3, θ4, d) (corresponding orientation / distance setting state), respectively.

[0112] A specific example of the method for generating acoustic information for generating an acoustic information DB has been described above. The acoustic information DB 201 (FIG. 1) constructed using the acoustic information (left-ear and right-ear impulse responses) generated in this way becomes a database used, for example, when localizing a sound image in the headphones 52.

[0113] As a modification, the relative orientations θ3 and θ4 may be fixed to a predetermined value, preferably zero, without being changed, and sound pressures (left ear and right ear impulse responses) for each set of the remaining changing {θ1, θ2, d} may be calculated. In this case, an acoustic information DB201 relating to orientation and distance in a two-dimensional plane (horizontal plane) is generated. Incidentally, in this case, the sound-collecting microphone 9 may be 360 ​​speakers that surround the DB generation speaker 6 in the horizontal plane, with the deflection angles differing by 1°, for example. The acoustic simulation is also performed in a two-dimensional grid space, and the input sound source grid may be 360 ​​grids that surround the DB generation speaker grid 6', with the deflection angles differing by 1°, for example.

[0114] As a further modification, the relative orientations θ2 and θ4 can be fixed to a predetermined value, preferably zero, without being changed, and the sound pressure (left ear and right ear impulse response) for each remaining set of {θ1, θ3, d} that changes can be calculated. It is also possible to fix θ3 (to zero) and calculate the sound pressure (left ear and right ear impulse response) for each remaining set of {θ1, d} that changes, or to fix d (for example, to the minimum listening distance d0) and calculate the sound pressure (left ear and right ear impulse response) for each value of the only change in θ1. In any case, an acoustic information DB 201 with appropriate search parameters is generated.

[0115] (Method of generating acoustic information for DB generation: Calculate acoustic information at microphone grid by acoustic simulation) FIG. 6 is a schematic diagram for explaining another specific example of the method for generating acoustic information for DB generation, which is carried out by the acoustic information acquisition unit 211. In FIG.

[0116] In this specific example, the acoustic information acquisition unit 211 generates an impulse response (acoustic information) for DB generation by acoustic simulation using the CE-FDTD method, similar to the above specific example shown in Fig. 4. First, the acoustic information acquisition unit 211 acquires multiple impulse responses (acoustic information) generated by multiple sound pickup microphones 9 capable of collecting sounds emitted from a DB generation speaker 6 installed in an anechoic chamber (DB generation space), similar to the above specific example shown in Fig. 4.

[0117] Next, the acoustic information acquisition unit 211 sets a DB generation speaker grid 6' corresponding to the position of the DB generation speaker 6 and an omnidirectional microphone grid 8' corresponding to the assumed position of the omnidirectional microphone 8 within the acoustic simulation space.

[0118] Here, the multiple impulse responses generated by the multiple sound-collecting microphones 9 are treated as impulse responses (simulation inputs) in multiple input sound source grids (FIG. 6(B)) corresponding to the positions of the multiple sound-collecting microphones 9, respectively. Also in this specific example, when calculating the impulse response (sound pressure) of each grid, an IWB (Interpolated Wide Band) method that refers to the coordinate axis direction, diagonal direction, and solid diagonal direction can be adopted. Furthermore, it is also preferable that the boundary conditions of the simulation are free sound field conditions that do not take into account reflections at the boundaries.

[0119] Under the above-mentioned settings, the acoustic information acquisition unit 211 (a) The "sound source relative orientation" which is the orientation of the DB generation speaker 6 (as a sound source) with respect to the omnidirectional microphone 8 (assumed to be a sound receiver) is each of a plurality of orientations, (b) The "distance" which is the distance between the position of the DB generation speaker 6 and the position of the omnidirectional microphone 8 is each of a plurality of distances, (c) The "sound receiver direction", which is the direction of the omnidirectional microphone 8 relative to the DB generation speaker 6, is one of a plurality of directions. The sound pressure (impulse response) for each case is calculated and generated by acoustic simulation.

[0120] Here, as shown in Fig. 6 (A1) and (A2), the relative orientation of the DB generation speaker 6 in the horizontal plane with respect to the non-directional microphone grid 8', that is, the angle (relative orientation in the horizontal plane) between the horizontal component of the DB generation speaker 6 facing forward and the horizontal component of the direction from the DB generation speaker grid 6' to the non-directional microphone grid 8' is θ1. Furthermore, the angle (relative orientation in the vertical plane) between the vertical component of the DB generation speaker 6 facing forward and the vertical component of the direction from the DB generation speaker grid 6' to the non-directional microphone grid 8' is θ3. Moreover, the distance between the DB generation speaker grid 6' and the non-directional microphone grid 8' is d. Note that in this specific example as well, θ1 and θ3 are initially zero, and the distance d is set to the minimum listening distance d0 (set in the DB generation speaker 6).

[0121] Under the above-mentioned settings, the acoustic information acquisition unit 211 (a) As shown in FIG. 6B, the relative direction θ1 of the DB generation speaker 6 is set as {θ s1 (0° in this example), θ s1 +Δθ1, θ s1 +2Δθ1, θ s1 +3Δθ1, . . . , θ e1} in sequence, (b) As shown in Fig. 6(B), the distance d is expressed as {d0, d0+Δd, d0+2Δd, . . . , d e} in sequence, (c) As shown in FIG. 6(C), the relative direction θ3 is expressed as {θ s3 (0° in this example), θ s3 +Δθ3, θ s3 +2Δθ3, θ s3 +3Δθ3, . . . , θ e3} For all the configurations obtained for the case, i.e., for all the orientation-distance configurations represented by the (θ1, θ3, d) pairs, the sound pressure of each grid is calculated.

[0122] Here, Δθ1 and Δθ3 are, for example, 1°, 5°, or 10°, and θ e1 and θ e3 can be, for example, 359°, 355°, or 350°, and Δd can be, for example, 0.5 m or 1 m.

[0123] Finally, the acoustic information acquisition unit 211 determines the calculation result of the sound pressure in the non-directional microphone grid 8' as the impulse response for the corresponding set of (θ1, θ3, d) (corresponding setting state of direction and distance).

[0124] Another specific example of the method for generating acoustic information for generating an acoustic information DB has been described above. The acoustic information DB 201 (FIG. 1) constructed using the acoustic information (impulse response) generated in this way becomes a database used, for example, when localizing sound images in a plurality of speakers 51.

[0125] As a modification, the relative orientation θ3 may be fixed to a predetermined value, preferably zero, without being changed, and the sound pressure (impulse response) for each remaining set of {θ1, d} that changes may be calculated. In this case, an acoustic information DB201 relating to orientation and distance in a two-dimensional plane (horizontal plane) is generated. Incidentally, in this case, the sound-collecting microphone 9 may be 360 ​​speakers that surround the DB generation speaker 6 in a horizontal plane, for example, with a deviation angle of 1° each. The acoustic simulation is also performed in a two-dimensional grid space, and the input sound source grid may also be 360 ​​grids that surround the DB generation speaker grid 6', for example, with a deviation angle of 1° each. As a further modification, the distance d may also be fixed (for example, to the minimum listening distance d0), and the sound pressure (impulse response) for each value of the only variable θ1 may be calculated. In any case, an acoustic information DB201 with appropriate search parameters is generated.

[0126] 2 to 6, specific examples have been described in which a DB generation speaker 6 having a predetermined radiation characteristic is used to generate acoustic information (impulse response) for DB generation that also takes into account the radiation characteristic of the sound source. In any of the specific examples, a dummy head 6'' with a speaker placed at the mouth can be used as the sound source instead of the DB generation speaker 6. This makes it possible to obtain acoustic information (impulse response) for DB generation when the sound source is a human (speaker) and generate acoustic information DB201 (FIG. 1) specifically for cases when the sound source is a human.

[0127] (Reducing the amount of data in the acoustic information database) A method of generating (calculating) acoustic information that can reduce the amount of data in the acoustic information DB 201 will be described below.

[0128] Specifically, the change (increment) in the relative direction θ1 of the DB generation speaker 6 is Δθ1, and the maximum and minimum values ​​that Δθ1 can take are Δθ1_max and Δθ1_min, respectively. Also, r_* is an element of the set of divisors of 360 {r_1, r_2, . . . , r_24}. Here, the following equation (4) Δθ1'=(Δθ1_max-Δθ1_min)×(d-d0) / (d e -d0)+Δθ1_min The Δθ1′ calculated by the following formula (5) r_n≦Δθ1'<(r_n+r_n+1) / 2 If the above equation is satisfied, Δθ1 is set to r_n (Δθ1=r_n). On the other hand, if Δθ1' satisfies the following equation, (6)(r_n+r_n+1) / 2≦Δθ1' <r_n+1 If this is satisfied, Δθ1 is set to r_n+1 (Δθ1=r_n+1).

[0129] Using the determined Δθ1, the relative direction θ1 when the distance is d is calculated as {θ s1 , θ s1 +Δθ1, θ s1 +2Δθ1, θ s1 +3Δθ1, . . . , θe1 The values are sequentially changed for each of {}, and acoustic information (impulse response) is generated (calculated). As a result, the farther the position from the sound source, the larger the interval Δθ1, the smaller the number of acoustic information (impulse response) generated (calculated), and since the distance d is large, the number of data for which high accuracy is not required for the relative orientation θ1 is reduced, and it becomes possible to reduce the data amount of the acoustic information DB201. Also, the burden (time required for measurement and calculation, etc.) for generating (calculating) the acoustic information for DB generation can be reduced.

[0130] As a modification, instead of the above formula (4), the following formula (4’) Δθ1'={(Δθ1_max - Δθ1_min)^(1 / (d e - d0))}^(d - d0)+Δθ1_min may be used to calculate Δθ1'. Here, the arithmetic symbol ^ represents exponentiation. In any case, (a) a plurality of possible directions of the relative orientation θ1 are individually set for each of the plurality of possible distances of the distance (degree of separation) d, (b) the interval Δθ1 between adjacent directions among the plurality of possible directions of the relative orientation θ1 is set to a larger value as the corresponding distance d value is larger then, various other ways of taking the value of the relative orientation θ1 are possible.

[0131] Furthermore, for the relative orientation θ3 of the DB generation speaker 6 as well, the number of generated DB generation acoustic information can be reduced using the method described above (that is, by replacing θ1 with θ3). For example, the larger the distance d, the smaller the number of data for the relative orientations θ1 and θ3, and thus it is also possible to generate the acoustic information DB201 with a much larger reduction in the data amount.

[0132] <DB generation means> Returning to the functional block diagram of FIG. 1, the DB generation unit 212 uses the acoustic information (impulse response) acquired (generated / calculated) by the acoustic information acquisition unit 211 as (a) the “sound source relative orientation” (θ1, θ3) which is the orientation with respect to the sound receiver of the sound source, and (b) "Distance" (d), which is the distance between the position of the sound source and the position of the sound receiver, (c) The "sound receiver relative orientation" (θ2, θ4) which is the orientation of the sound receiver with respect to the sound source, are stored in a database storage in a searchable form as search parameters, generating the acoustic information DB 201. For example, each piece of acoustic information (impulse response) may be tagged with a corresponding set of (θ1, θ3, θ2, θ4, d), and the tagged acoustic information (impulse response) may be stored and managed. Also, each piece of acoustic information (impulse response) may be stored in an organized form in a classification folder with a predetermined hierarchy.

[0133] Here, the acoustic information DB201 using the above (a) to (c) as search parameters is generated from acoustic information (impulse response) acquired, for example, by the specific examples shown in Figures 2 and 4. On the other hand, the acoustic information DB201 generated from acoustic information (impulse response) acquired, for example, by the specific examples shown in Figures 3 and 6 is a database using the above (a) and (b) as search parameters.

[0134] 2 and 4, in which a dummy set (voxel) is installed (set) as a sound receiver, the acoustic information acquired is a left ear impulse response and a right ear impulse response. In this case, the DB generating unit 212 (d) "Ear left / right designation information" indicating whether the information pertains to the left ear or the right ear Alternatively, by searching with (θ1, θ3, θ2, θ4, d), it is possible to generate an acoustic information DB 201 in which both left-ear and right-ear impulse responses are searched.

[0135] In the above-described embodiment and specific example, the acoustic information stored in the acoustic information DB 201 (FIG. 1) is an impulse response (for the left and right ears), but it is not limited to this. For example, it can be a transfer function (as a result of performing FFT processing on the impulse response).

[0136] The acoustic information stored in the acoustic information DB 201 may be at least one of frequency characteristics, phase characteristics, and amplitude characteristics. Here, the frequency characteristics, phase characteristics, and amplitude characteristics may each be a change (e.g., dB, ratio, increment, etc.) in a value in a state of (θ1, θ3, θ2, θ4, d) relative to a value in a reference state (e.g., a state in which θ1, θ3, θ2, and θ4 are zero and d=d0). For example, if "when a speaker turns away (θ1=180°), the amplitude (at 8 kHz) is 6 dB lower (with a Q value of 0.5) compared to when the speaker faces forward," the amplitude characteristic (at 8 kHz) at θ1=180° may be -6 dB. When such acoustic information is stored in a database, the acoustic signal generating unit 114 (FIG. 1) can generate a biquadratic filter (in each set frequency band) corresponding to, for example, a state of (θ1, θ3, θ2, θ4, d) by using at least one of the retrieved frequency characteristics, phase characteristics, and amplitude characteristics and the value in the reference state. Next, the generated biquadratic filter is applied to the sound source acoustic signal received from the sound source information acquiring unit 111 to generate an output acoustic signal.

[0137] <Another embodiment of the acoustic signal generating device and acoustic information DB>

[0138] Below, we will explain an embodiment in which the acoustic information DB 201 (Figure 1) stores the "acoustic signal" (acoustic signal information) itself as acoustic information, and the acoustic signal generating device 1 searches for this "acoustic signal" (acoustic signal information) as acoustic information to generate an output acoustic signal.

[0139] In this embodiment, an "acoustic signal" for generating such an acoustic information DB 201 is generated in the same manner as the specific examples using acoustic simulation shown in Fig. 4 and Fig. 6. Specifically, first, using the recording system shown in Fig. 5, sounds radiated from a sound source (e.g., a musical instrument) to the surroundings are collected by multiple (multiple) sound-collecting microphones 9.

[0140] Here, the acoustic information acquisition unit 211 (Fig. 1) performs acoustic simulation using the CE-FDTD method with the acoustic signal related to the collected sound as the acoustic signal (simulation input) in the input sound source grid shown in Fig. 4 or 6, and calculates (a) the "acoustic signal" (sound pressure) at the left ear position and the "acoustic signal" (sound pressure) at the right ear position in the dummy head voxel 7' (Fig. 4), or (b) the "acoustic signal" (sound pressure) in the omnidirectional microphone grid 8' (Fig. 6).

[0141] Next, the DB generating unit 212 (FIG. 1) uses the "acoustic signal" calculated in this manner to construct an acoustic information DB 201 with the "relative sound source direction" and "distance" (and further "relative sound receiver direction") as search parameters. The acoustic signal generating device 1 searches this acoustic information DB 201 for an "acoustic signal" corresponding to the acquired "sound source direction information" and the like, and can (a) use the searched "acoustic signals" at the left and right ear positions as output acoustic signals for the headphones 52, or (b) generate output acoustic signals for the multiple speakers 51 from the searched "acoustic signals".

[0142] In this manner, in this embodiment, it is possible to generate an output audio signal that takes into consideration the radiation characteristics of a target sound source, even if the target sound source is not capable of emitting a sound for impulse response measurement, such as the DB generation speaker 6. For example, it is possible to generate an output audio signal that is suitable for a sound (voice) emitted to the surroundings from a musical instrument such as a violin or guitar, or from a singer or actor as a vocalizer.

[0143] In the acoustic information DB 201 of this embodiment, the "relative sound source orientation" is, for example, a discrete value having a predetermined interval such as 0°, 5°, 10°, .... In response to this, when a search is to be performed using a value between these values, for example 8°, the acoustic information search unit 113 (FIG. 1) may determine the values ​​immediately before and after that, for example 5° and 10°, as the "relative sound source orientation" as a search parameter and perform a search.

[0144] In this case, the acoustic information search unit 113 searches to obtain an acoustic signal y(t)_5° when the "relative sound source orientation" is 5°, and an acoustic signal y(t)_10° when the "relative sound source orientation" is 10°. Here, the acoustic information search unit 113 may obtain a desired acoustic signal y(t)_8° when the "relative sound source orientation" is 8° by linear two-point interpolation, for example, using the following formula: (7) y(t)_8°=(1-r)×y(t)_5°+r×y(t)_10° In the above formula (7), r (0≦r≦1) is the internal division ratio, and in this case, r = (8-5) / (10-5) = 0.6.

[0145] In this way, the acoustic information search unit 113 can determine search parameter values ​​immediately before or after a desired parameter value for the search parameters "sound source relative orientation", "distance", and even "sound receiver relative orientation", and calculate and determine an "acoustic signal" (acoustic signal information) corresponding to the desired parameter value from the search results by linear interpolation (e.g., three-dimensional two-point interpolation, etc.). As a modification, the acoustic information search unit 113 can determine a search parameter value closest to the desired parameter value, and use the search results as is as the acoustic signal for generating an output acoustic signal.

[0146] As described above in detail, according to the present invention, it is possible to generate an output acoustic signal suitable for a sound source and a listener by taking into consideration the direction of the sound source with respect to the listener, using an acoustic information DB. For example, as one application form, it is possible to realize a suitable sound image localization that reflects the radiation characteristics of the sound emitted from the sound source.

[0147] Also, as an application example, even when the sound source moves, it is possible to perform sound image localization such that the perceived position of the sound image changes appropriately according to the movement of the sound source while changing the relative direction by searching for "sound information" that takes into account the relative direction of the sound source at each time. Furthermore, according to the present invention, it is possible to generate an acoustic information DB that can be used when generating such an output acoustic signal.

[0148] Furthermore, for example, children and students not only in urban areas but also in rural areas can enjoy the playback of multimedia teaching materials in a way that they can perceive sound images with less discomfort or in an appropriate position using, for example, a tablet terminal or a smartphone that is an audio signal generating device according to the present invention. This also makes it possible to realize higher quality online classes. In other words, the present invention can contribute to Goal 4 of the United Nations' Sustainable Development Goals (SDGs), which is to "Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all."

[0149] Furthermore, adults in urban areas as well as in rural areas can enjoy the playback of multimedia digital materials in a manner that allows them to perceive sound images with less discomfort or at an appropriate position using, for example, a tablet terminal or a smartphone that is an audio signal generating device according to the present invention. This also makes it possible to make online business meetings more efficient and of higher quality. In other words, the present invention can contribute to Goal 8 of the United Nations-led SDGs, "Promote inclusive and sustainable economic growth, employment and decent work for all."

[0150] Regarding the various embodiments of the present invention described above, various changes, modifications, and omissions within the scope of the technical idea and perspective of the present invention can be easily made by those skilled in the art. The above description is merely an example and is not intended to be restrictive in any way. The present invention is limited only by the scope of the claims and their equivalents. [Explanation of symbols]

[0151] 1 Acoustic signal generation device 101 Communication Interface (IF) 111 Sound source information acquisition section 112 Listener information acquisition unit 113 Acoustic Information Retrieval Department 114 Acoustic signal generation section 2. Acoustic information database (DB) generator 201 Acoustic information DB 211 Acoustic information acquisition section 212 DB generation section 3 Microphone 41, 42, 43 Position and orientation sensor (RGB-D camera) 51 Speaker 52 Headphones 6 Speaker for DB generation 6' Speaker grid for DB generation 7 Dummy Head 7' Dummy Head Voxel 8 Omnidirectional microphones 8' Omnidirectional Microphone Grid 9. Sound pickup microphone

Claims

1. An apparatus for generating an output acoustic signal using an input acoustic signal, comprising: The input sound signal is obtained together with direction information of a sound source related to the sound signal; an acoustic information retrieval means for retrieving acoustic information relating to a direction information from a database storing acoustic information relating to a relationship between a sound source and a sound receiver for generating a database, the acoustic information being for each of a plurality of directions in which the direction of the sound source for generating the database with respect to the sound receiver is different; an audio signal generating means for generating an output audio signal from the input audio signal by using the searched audio information; An acoustic signal generating device comprising:

2. The input sound signal is acquired together with position information of a sound source related to the sound signal, The database stores acoustic information for each of a plurality of directions of a sound source for generating the database relative to the sound receiver, and a plurality of degrees of separation between a position of the sound source for generating the database and a position of the sound receiver, The acoustic information retrieval means retrieves acoustic information related to the orientation information and the position information from the database using the orientation information and the position information.

2. The acoustic signal generating device according to claim 1 .

3. The output sound signal is transmitted to a listener via a listener information acquisition unit that acquires information about a direction of the listener, the listener information being transmitted to the listener information acquisition unit. The database stores acoustic information for each of a plurality of directions of the sound source for generating the database relative to the sound receiver, a plurality of ways of separation between the position of the sound source for generating the database and the position of the sound receiver, and a plurality of directions of the sound receiver relative to the sound source for generating the database; The acoustic information retrieval means retrieves acoustic information related to the orientation information, the position information, and the listener orientation information from the database using the orientation information, the position information, and the listener orientation information.

3. The acoustic signal generating device according to claim 2.

4. The output acoustic signal is input to headphones or earphones worn by a listener who is to receive the output acoustic signal; The database stores acoustic information relating to the right ear and acoustic information relating to the left ear, The acoustic signal generating means generates the output acoustic signal to be input to the headphones or earphones from the input acoustic signal by using the acoustic information related to the right ear and the acoustic information related to the left ear as the retrieved acoustic information.

4. The acoustic signal generating device according to claim 1, wherein the acoustic signal generating device is a first type.

5. The output acoustic signal is input to a plurality of installed speakers, The acoustic signal generating means generates the output acoustic signal to be input to the speaker from the input acoustic signal by using the searched acoustic information, the installation position information of the speaker, and the position information of the sound source related to the input acoustic signal.

3. The acoustic signal generating device according to claim 2.

6. 4. The acoustic signal generating device according to claim 2, wherein the input acoustic signal is provided with direction information and position information of a sound source related to the acoustic signal.

7. 4. The acoustic signal generating device according to claim 2, wherein the direction information and position information of the sound source related to the input acoustic signal are information obtained by measuring the direction and position of the sound source.

8. 4. The acoustic signal generating device according to claim 1, wherein the acoustic information is at least one of an impulse response, a transfer function, or a frequency characteristic, a phase characteristic, and an amplitude characteristic.

9. An apparatus for generating an output sound signal relating to sound output from a sound source, comprising: A sound source information acquiring means for acquiring direction information and position information of the sound source; an acoustic information retrieval means for retrieving acoustic signal information relating to the direction information and the position information from a database storing acoustic signal information relating to the relationship between the sound source and the sound receiver, the direction of the sound source relative to the sound receiver being each of a plurality of directions, and the distance between the position of the sound source and the position of the sound receiver being each of a plurality of distances, using the direction information and the position information; an audio signal generating means for generating the output audio signal from the retrieved audio signal information; An acoustic signal generating device comprising:

10. A method for generating a database that stores acoustic information relating to a relationship between a sound source and a sound receiver, comprising the steps of: acquiring a measurement result of the acoustic information in each of a plurality of cases where the orientation of the sound source with respect to the sound receiver is each of a plurality of orientations and the position of the sound source and the position of the sound receiver are each of a plurality of distances; storing the acquired acoustic information as a measurement result in a searchable form using the information related to the direction and the information related to the degree of separation as search parameters; 13. A computer-implemented method for generating a database, comprising:

11. A method for generating a database that stores acoustic information relating to a relationship between a sound source and a sound receiver, comprising the steps of: acquiring information related to a plurality of sounds generated by a plurality of microphones arranged around the sound source; a step of calculating, by an acoustic simulation, acoustic information for each of a plurality of directions of the sound source relative to the sound receiver and a plurality of degrees of separation between the position of the sound source and the position of the sound receiver, using the acquired information related to the plurality of sounds as a simulation input; storing the calculated acoustic information in a searchable form using the information relating to the direction and the information relating to the degree of separation as search parameters; 13. A computer-implemented method for generating a database, comprising:

12. The database generation method according to claim 10 or 11, characterized in that the multiple orientations are set individually for each of the multiple degrees of separation, and the interval between adjacent orientations in the multiple orientations is set to a larger value as the value of the corresponding degree of separation increases.

13. The database generation method according to claim 10 or 11, characterized in that in this case, the orientation of the sound source relative to the sound receiver is one of a plurality of orientations, the distance between the position of the sound source and the position of the sound receiver is one of a plurality of distances, and the orientation of the sound receiver relative to the sound source is one of a plurality of orientations.

14. A program for generating an output audio signal using an input audio signal, The input sound signal is obtained together with direction information of a sound source related to the sound signal; an acoustic information retrieval means for retrieving acoustic information relating to a direction information from a database storing acoustic information relating to a relationship between a sound source and a sound receiver for generating a database, the acoustic information being for each of a plurality of directions in which the direction of the sound source for generating the database with respect to the sound receiver is different; an audio signal generating means for generating an output audio signal from the input audio signal by using the searched audio information; 2. An acoustic signal generating program for causing a computer to function as:

15. 1. A method for generating an output audio signal using an input audio signal, comprising: The input sound signal is obtained together with direction information of a sound source related to the sound signal; A step of searching for acoustic information related to a relationship between a sound source and a sound receiver for database generation, the acoustic information being related to the orientation information from a database that stores acoustic information for each of a plurality of orientations of the sound source for database generation relative to the sound receiver, using the orientation information; generating the output acoustic signal from the input acoustic signal using the retrieved acoustic information; 16. A computer-implemented method for generating an acoustic signal, comprising:

Images