Acoustic sonification device and acoustic sonification program

Abstract

To obtain a sound auralization apparatus and a sound auralization program capable of reproducing sound that is listened to by a listener in the case that the listener moves virtually.SOLUTION: A sound auralization apparatus 10 comprises: an acquisition unit 11A for acquiring shape information showing the shape of a target room, sound related information related to the sound characteristic of the interior of the room, and sound source information showing sound generated from a target sound source; an identification unit 11G for identifying the position of a sound-receiving point, a spot at which a listener who confirms a sound design effect in the room exists in the case of movement of the sound-receiving point; a derivation unit 11C that uses the shape information, the sound related information, and the sound source information acquired by the acquisition unit 11A and the position of the sound-receiving point identified by the identification unit 11G to derive sound auralization information showing how sound generated from the sound source is heard at the sound-receiving point; and a reproduction control unit 11D for performing control of making a reproduction unit reproduce sound shown by the sound auralization information derived by the derivation unit 11C.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a sound sonification device and a sound sonification program. [Background technology]

[0002] In the acoustic design of buildings, reverberation time and sound pressure distribution obtained from statistical acoustics theory and numerical calculations are evaluated to determine the specifications of components, plans, etc. However, a certain level of experience and knowledge is required to properly evaluate the results.

[0003] On the other hand, there is a technology that uses the results of numerical calculations and model experiments to reproduce the sounds inside a building under design so that even those with little experience or knowledge can make an intuitive evaluation. This technology is called acoustic auralization technology.

[0004] Conventionally, the following techniques have been available as techniques relating to acoustic auralization technology.

[0005] Patent Document 1 discloses a spatial sound generating device that aims to generate a sound field accompanied by a three-dimensional acoustic wavefront that gives a sense of realism.

[0006] This spatial sound generating device is equipped with a memory unit and a control unit connected to multiple speakers, and the control unit applies an inverse system while changing transfer characteristics over time in accordance with the movement of the sound generating body based on information indicating a moving sound generating body, to calculate multiple input signals to each of the speakers from a sound source signal indicating the sound emitted by the sound generating body. Furthermore, the inverse system outputs the input signals in accordance with the transfer characteristics in the space in which the multiple speakers are arranged, so as to cause the speakers to form a three-dimensional acoustic wavefront based on the input signals in boundary sound field control. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Republished Publication No. 2018-070487 Summary of the Invention [Problem to be solved by the invention]

[0008] However, the technology disclosed in Patent Document 1 does not take into consideration the movement of the sound receiving point, and has the problem that it is not possible to reproduce the sound heard by a listener when the listener virtually moves in the target room.

[0009] The present disclosure has been made in consideration of the above circumstances, and aims to provide an acoustic auralization device and an acoustic auralization program that can reproduce the sound heard by a listener when the listener virtually moves. [Means for solving the problem]

[0010] The sound auralization device according to the present invention as set forth in claim 1 comprises an acquisition unit that acquires shape information that indicates the shape of a target room, acoustic-related information related to the acoustic characteristics of the interior of the room, and sound source information that indicates sound emitted from a target sound source; an identification unit that identifies the position of a sound receiving point, which is a position where a listener who confirms the acoustic design effect of the room is located, as the sound receiving point moves; a derivation unit that derives acoustic auralization information that indicates how the sound emitted from the sound source will be heard at the sound receiving point, using the shape information, the acoustic-related information, and the sound source information acquired by the acquisition unit and the position of the sound receiving point identified by the identification unit; and a reproduction control unit that controls a reproduction unit to reproduce the sound indicated by the acoustic auralization information derived by the derivation unit, wherein the derivation unit derives the acoustic auralization information using a transfer function. At the same time, the sound receiving point and the sound source are positioned at the same position. and deriving the initial height of the sound receiving point from the listener. ear The height.

[0011] According to the acoustic auralization device of the present invention as set forth in claim 1, shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from the target sound source are acquired, the position of a sound receiving point, which is the location of a listener who checks the acoustic design effects of the room, is identified as the point moves, and acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point is derived using the acquired shape information, acoustic-related information, and sound source information, as well as the identified position of the sound receiving point.By controlling the playback unit to play the sound indicated by the derived acoustic auralization information, it is possible to reproduce the sound that would be heard by the listener when the listener virtually moves.

[0013] Claim 1 According to the acoustic auralization device of the present invention described above, by deriving acoustic auralization information using a transfer function, it is possible to reproduce the sound emitted from the sound source more easily than when a transfer function is not used.

[0014] Claim 2 The acoustic sonification device according to the present invention is 1 The acoustic sonification device described in the above item further includes a generation unit that uses the shape information to generate video information showing an image of the room that the listener can see in accordance with the listener's movement, and a display control unit that controls a display unit to virtually display the image shown by the video information generated by the generation unit.

[0015] Claim 2 According to the acoustic sonification device of the present invention described above, the shape information is used to generate video information showing an image of the room as seen by the listener as the listener moves, and the image shown by the generated video information is virtually displayed, thereby making it possible to reproduce the sound emitted from the sound source with a more realistic feel than when the image is not displayed.

[0016] Claim 3 The acoustic sonification device according to the present invention is 2In the acoustic auralization device described in the above, the derivation of the acoustic auralization information by the derivation unit and the generation of the video information by the generation unit are performed by a single computer.

[0017] Claim 3 According to the acoustic auralization device of the present invention described above, the derivation of acoustic auralization information and the generation of video information are performed by a single computer, which makes it easier to synchronize the playback of sound indicated by the acoustic auralization information with the display of video indicated by the video information, compared to when the derivation of acoustic auralization information and the generation of video information are performed by different computers.

[0018] Claim 4 The sound sonification device according to the present invention is as follows: 3 10. The sound auralization device according to claim 1, wherein the derivation unit derives the sound auralization information as information that reflects the influence of the Doppler effect caused by movement of the sound source.

[0019] Claim 4 According to the acoustic sonification device of the present invention described above, by making the acoustic sonification information information that reflects the influence of the Doppler effect due to the movement of the sound source, it is possible to reproduce the sound emitted from the sound source with higher accuracy than when the influence of the Doppler effect is not reflected.

[0020] The sound auralization program according to the present invention as set forth in claim 5 is a process of acquiring shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating a sound emitted from a target sound source, identifying a position of a sound receiving point, which is a position where a listener who confirms the acoustic design effect of the room is located, as the sound receiving point moves, deriving acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point using the acquired shape information, acoustic-related information, and sound source information, and the identified position of the sound receiving point, and controlling a playback unit to play back the sound indicated by the derived acoustic auralization information, and At the same time, the sound receiving point and the sound source are positioned at the same position. and deriving the initial height of the sound receiving point from the listener. ear The computer executes the process, determining the height.

[0021] Claim 5 According to the acoustic auralization program of the present invention described in the above, shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from the target sound source are acquired, the position of a sound receiving point, which is a location where a listener who checks the acoustic design effect of the room is located, is identified as the sound receiving point moves, and acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point is derived using the acquired shape information, acoustic-related information, and sound source information, and the identified position of the sound receiving point.By controlling the playback unit to play the sound indicated by the derived acoustic auralization information, it is possible to reproduce the sound heard by the listener when the listener moves virtually. [Effects of the Invention]

[0022] As described above, according to the present invention, it is possible to reproduce sounds heard by a listener when the listener virtually moves. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a block diagram showing an example of a hardware configuration of a sound auralization device according to an embodiment. [Figure 2] 1 is a block diagram showing an example of a functional configuration of a sound auralization device according to an embodiment. [Figure 3] FIG. 2 is a front view showing an example of the configuration of a controller according to the embodiment. [Figure 4] FIG. 2 is a schematic diagram illustrating an example of a configuration of a building-related information database according to the embodiment. [Figure 5] FIG. 2 is a schematic diagram illustrating an example of a configuration of a transfer function related information database according to the embodiment. [Figure 6] FIG. 10 is a perspective view showing an example of a setting target for calculating a transfer function according to the embodiment. [Figure 7]FIG. 10 is a side view showing an example of a setting target for calculating a transfer function when wave properties are not taken into consideration according to the embodiment. [Figure 8] FIG. 10 is a side view showing an example of a setting target for calculating a transfer function when wave properties are taken into consideration according to the embodiment. [Figure 9] 10 is a flowchart illustrating an example of a setting process according to the embodiment. [Figure 10] FIG. 10 is a front view showing an example of a configuration of a setting target input screen according to the embodiment. [Figure 11] FIG. 10 is a front view showing an example of the configuration of a setting information input screen according to the embodiment. [Figure 12] 10 is a flowchart illustrating an example of a sound auralization process according to the embodiment. [Figure 13] FIG. 10 is a front view showing an example of a configuration of a processing target input screen according to the embodiment. [Figure 14] FIG. 2 is a front view showing an example of the configuration of a video screen according to the embodiment. [Figure 15] FIG. 10 is a front view showing another example of the configuration of the video screen according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0024] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0025] First, the configuration of the sound auralization device 10 according to this embodiment will be described with reference to Figs. 1 to 3. Fig. 1 is a block diagram showing an example of the hardware configuration of the sound auralization device 10 according to this embodiment. Fig. 2 is a block diagram showing an example of the functional configuration of the sound auralization device 10 according to this embodiment. Fig. 3 is a front view showing an example of the configuration of a controller 21 according to this embodiment. Examples of the sound auralization device 10 include information processing devices such as a personal computer and a server computer.

[0026] 1, an acoustic sonification device 10 according to this embodiment includes a CPU (Central Processing Unit) 11, a memory 12 as a temporary storage area, a non-volatile storage unit 13, an input unit 14 such as a keyboard and a mouse, a display unit 15 such as a liquid crystal display, a medium read / write device (R / W) 16, and a communication interface (I / F) unit 18. The CPU 11, memory 12, storage unit 13, input unit 14, display unit 15, medium read / write device 16, and communication interface (I / F) unit 18 are connected to one another via a bus B. The medium read / write device 16 reads information written in a recording medium 17 and writes information to the recording medium 17.

[0027] The storage unit 13 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. A setting program 13A and an acoustic auralization program 13B are stored in the storage unit 13 as a storage medium. The setting program 13A is stored in the storage unit 13 when a recording medium 17 on which the setting program 13A is written is set in the medium reading and writing device 16 and the medium reading and writing device 16 reads the setting program 13A from the recording medium 17. The acoustic auralization program 13B is also stored in the storage unit 13 when the recording medium 17 on which the acoustic auralization program 13B is written is set in the medium reading and writing device 16 and the medium reading and writing device 16 reads the acoustic auralization program 13B from the recording medium 17. The CPU 11 reads each of the setting program 13A and the acoustic auralization program 13B from the storage unit 13 as appropriate, expands them in the memory 12, and sequentially executes the processes of each program.

[0028] Furthermore, a building-related information database 13C and a transfer function-related information database 13D are stored in the storage unit 13. The building-related information database 13C and the transfer function-related information database 13D will be described in detail later.

[0029] Furthermore, as shown in FIG. 1, a microphone (hereinafter referred to as "mic") 20, a controller 21, and a speaker 22 are connected to the communication I / F unit 18 according to this embodiment.

[0030] The microphone 20 according to this embodiment is for collecting voices emitted by a user of the sound auralization device 10 (corresponding to the "listener" of the present invention) and sound source information indicating sounds emitted from a sound source, which will be described later. In this embodiment, a unidirectional condenser microphone is used as the microphone 20, but this is not limiting. For example, a bidirectional or omnidirectional dynamic microphone may also be used as the microphone 20.

[0031] The controller 21 according to this embodiment is for inputting various information in response to user operations. As an example, as shown in Fig. 3, in this embodiment, a general-purpose controller for a game console is used as the controller 21, but this is not limiting, and a controller dedicated to the sound auralization device 10 may also be used as the controller 21.

[0032] 3, controller 21 according to this embodiment includes a pair of gripping portions 21A and 21B that are held by the user's left and right hands, respectively. Grip portions 21A and 21B are connected by an intermediate portion 21C, and are provided with buttons 21A1 and 21B1, respectively, that can be operated with the thumb of the gripping hand.

[0033] 3 of the intermediate portion 21C and in the vicinity of the grip portion 21A and the grip portion 21B, respectively. The controller 21 according to this embodiment is provided with a stick portion 21C1 and a stick portion 21C2 that can be operated with the thumb of the hand holding the corresponding grip portion. In the sound auralization device 10 according to this embodiment, the stick portion 21C1 receives an instruction input for moving a sound receiving point, which will be described later, and the stick portion 21C2 receives an instruction input for the orientation of the sound receiving point.

[0034] Furthermore, the speaker 22 according to this embodiment is for reproducing the sound generated by the sound auralization device 10. In this embodiment, a dynamic and stereo speaker is used as the speaker 22, but this is not limiting. For example, a capacitor type, a piezoelectric type, or a monaural type speaker may also be used as the speaker 22.

[0035] Next, the functional configuration of the sound auralization device 10 according to this embodiment will be described with reference to FIG.

[0036] 2, the sound auralization device 10 according to this embodiment includes an acquisition unit 11A, a detection unit 11B, a derivation unit 11C, a playback control unit 11D, a generation unit 11E, a display control unit 11F, and an identification unit 11G. The CPU 11 of the sound auralization device 10 executes the setting program 13A and the sound auralization program 13B, thereby functioning as the acquisition unit 11A, the detection unit 11B, the derivation unit 11C, the playback control unit 11D, the generation unit 11E, the display control unit 11F, and the identification unit 11G.

[0037] The acquisition unit 11A according to this embodiment acquires shape information indicating the shape of a target room (hereinafter also referred to as a "target room") and acoustic-related information relating to the acoustic characteristics of the interior of the room.

[0038] In this embodiment, information based on 3D Computer Aided Design (CAD) information, the details of which will be described later, is applied as the shape information, but this is not limiting. For example, information based on Building Information Modeling (BIM) information may also be applied as the shape information. Furthermore, in this embodiment, information including the sound absorption coefficient, scattering coefficient, and transmittance of surfaces such as wall surfaces, ceiling surfaces, and floor surfaces (hereinafter referred to as "component surfaces") in the target room is applied as the acoustic-related information, but this is not limiting. For example, information including any one or a combination of two of the sound absorption coefficient, scattering coefficient, and transmittance may also be applied as the acoustic-related information.

[0039] The sound absorption coefficient is an index showing how much sound a material absorbs; at its maximum value, all incident sound is absorbed, and at its minimum value (usually 0 (zero)), the incident sound is reflected without being absorbed. The scattering coefficient is an index showing the proportion of sound incident on the material that is not reflected specularly; when the scattering coefficient is high, the sound is reflected randomly in all directions, and when the scattering coefficient is low, the sound is reflected specularly like a mirror. The transmittance is an index showing how much sound passes through the material; at its maximum value, all incident sound is transmitted in the opposite direction to the incident direction, and at its minimum value (usually 0 (zero)), the incident sound is completely blocked without being transmitted.

[0040] Furthermore, the detection unit 11B according to this embodiment detects sound source information indicating at least one of a sound (hereinafter also referred to as a "sound source sound") emitted from a target sound source (hereinafter also referred to as a "target sound source") and a sound at a sound receiving point (hereinafter also referred to as a "sound receiving point sound"), which is a location where a listener who checks the acoustic design effect in the target room is located.

[0041] In this embodiment, a case where the detection unit 11B detects sound source information indicating both the sound source sound and the sound receiving point sound will be described, but the present invention is not limited thereto. For example, the detection unit 11B may detect only the sound source information indicating the sound receiving point sound, or may detect only the sound source information indicating the sound source sound. In addition, in this embodiment, a case where the detection unit 11B detects the sound source information indicating the sound source sound indirectly by detecting a sound emitted from a target sound source using the microphone 20 and storing the sound source information in advance in the storage unit 13, and then reading the sound source information from the storage unit 13 will be described. In this case, the sound source information is acquired by reading it from the storage unit 13. In other words, the sound source information is acquired by the acquisition unit 11A. However, the present invention is not limited to this embodiment. For example, the detection or acquisition of the sound source information indicating the sound source sound may be performed directly via the microphone 20 without going through the storage unit 13.

[0042] In addition, the derivation unit 11C according to this embodiment derives acoustic aurization information indicating how a sound emitted from at least one of the target sound source and the sound receiving point sounds at the sound receiving point, using the shape information and acoustic-related information acquired by the acquisition unit 11A and the sound source information detected by the detection unit 11B.

[0043] In this embodiment, a case will be described in which the derivation unit 11C derives acoustic aurization information corresponding to both the sound source sound and the sound receiving point sound, but the present invention is not limited to this. For example, the derivation unit 11C may derive acoustic aurization information corresponding only to the sound receiving point sound, or the derivation unit 11C may derive acoustic aurization information corresponding only to the sound source sound.

[0044] In this embodiment, the derivation unit 11C derives the acoustic auralization information using a transfer function.

[0045] Then, the reproduction control unit 11D according to this embodiment controls the reproduction unit (the speaker 22 in this embodiment) to reproduce the sound indicated by the acoustic auralization information derived by the derivation unit 11C.

[0046] On the other hand, the generation unit 11E according to this embodiment uses the shape information to generate video information showing an image of the target room that the listener can see as the listener moves. Then, the display control unit 11F according to this embodiment controls the display unit 15 to virtually display the image shown by the video information generated by the generation unit 11E.

[0047] Furthermore, the determination unit 11G according to this embodiment determines the position of the sound receiving point as the sound receiving point moves. Then, the derivation unit 11C according to this embodiment further derives the sound auralization information using the position of the sound receiving point determined by the determination unit 11G.

[0048] In this embodiment, the derivation of the acoustic auralization information by the derivation unit 11C and the generation of the video information by the generation unit 11E are performed by a single computer (CPU 11 in this embodiment). This makes it easier to synchronize the playback of the sound indicated by the acoustic auralization information with the display of the video indicated by the video information. However, this is not limiting, and for example, the processing by the derivation unit 11C and the processing by the generation unit 11E may be performed by different computers. In this embodiment, the processing load by the derivation unit 11C and the processing load by the generation unit 11E can be distributed, allowing for more accurate and faster playback of the sound and display of the video.

[0049] Next, the building-related information database 13C according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing an example of the configuration of the building-related information database 13C according to this embodiment. The building-related information database 13C is a database that stores information about buildings that are the subject of the sound auralization device 10 according to this embodiment.

[0050] As shown in Figure 4, the building-related information database 13C of this embodiment stores, in association with each other, the building name, 3D CAD information, and sound source-related information for each building that the acoustic auralization device 10 handles.

[0051] The building name is information indicating the name of the corresponding building, and the 3D CAD information is information indicating a model (hereinafter referred to as a "building-related model") that includes building shape information indicating the shape of the corresponding building and specific information for identifying each room in the building.

[0052] In this embodiment, the building-related model is created using predetermined 3D CAD software. In this embodiment, Rhinoceros (registered trademark) is used as the 3D CAD software, but the 3D CAD software is not limited to this. For example, other software such as Revit (registered trademark) may also be used as the 3D CAD software.

[0053] The sound source-related information is information related to the target sound source described above, and includes room information, sound source information, position information, and directivity information. The room information is information for identifying the corresponding room (target room) of the corresponding building, and the above-mentioned identification information is applied in this embodiment. As described above, the sound source information is information indicating the sound itself emitted from the corresponding target sound source and acquired by the microphone 20. The position information is information indicating the three-dimensional position of the corresponding target sound source, and the directivity information is information indicating the directivity of the corresponding target sound source.

[0054] For example, if the target room is a school classroom, a specific example of the target sound source is the teacher's voice. In this case, the position of the target sound source is applied as a position in front of the blackboard or on the podium where the teacher is likely to stand, the directionality is applied as the directionality of a human voice, and the sound source information is applied as time-series information indicating the voice of the teacher when teaching a lesson.

[0055] Next, the transfer function-related information database 13D according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a schematic diagram showing an example of the configuration of the transfer function-related information database 13D according to this embodiment. The transfer function-related information database 13D is a database that stores information related to the calculation of transfer functions, the details of which will be described later, generated by the acoustic auralization device 10 according to this embodiment.

[0056] As shown in FIG. 5, the transfer function related information database 13D according to this embodiment stores, for each building that the sound auralization device 10 handles, the building name and each piece of transfer function related information in association with each other.

[0057] The building name is the same information as the building name in the building-related information database 13C, and the transfer function-related information is information related to transfer functions (to be described later) corresponding to each room in the corresponding building, and includes room information and transfer function information. The room information is the same information as the room information in the building-related information database 13C, and the transfer function information is information related to the calculation of a transfer function between the position of a target sound source and the position of a sound receiving point, which is set for the corresponding room by a setting process (to be described later).

[0058] Here, settings related to the calculation of the transfer function applied in this embodiment will be described with reference to Fig. 6 to Fig. 8. Fig. 6 is a perspective view showing an example of a setting target related to the calculation of the transfer function according to this embodiment. Fig. 7 is a side view showing an example of a setting target related to the calculation of the transfer function when the wave property according to this embodiment is not taken into account. Fig. 8 is a side view showing an example of a setting target related to the calculation of the transfer function when the wave property according to this embodiment is taken into account.

[0059] As an example, in this embodiment, a transfer function is calculated based on the ray tracing method, which does not take into account the wave properties of sound, as shown in Fig. 6. In the ray tracing method, results vary depending on the number of sound rays generated from one target sound source, the maximum number of reflections, etc., so appropriate values ​​are set taking into account the calculation load on the CPU 11, etc.

[0060] As shown in Fig. 7 as an example, when wave properties are not taken into consideration, sound flows in a straight line, so if there is a soundproof wall or the like between the target sound source and the sound receiving point, the sound will not reach the sound receiving point. On the other hand, as shown in Fig. 8 as an example, when wave properties are taken into consideration, sound from the target sound source will go around the soundproof wall or the like and reach the sound receiving point as diffracted sound. In this embodiment, taking into consideration the processing capabilities of current computers, the sound ray method that does not take wave properties into consideration is applied as the calculation means for the sound ray method, but it goes without saying that the present invention is not limited to this and a calculation method that takes wave properties into consideration may also be applied.

[0061] Next, the operation of the sound auralization device 10 according to this embodiment will be described with reference to Figs. 9 to 15. First, the operation of the sound auralization device 10 when executing a setting process for setting information related to the calculation of a transfer function will be described with reference to Figs. 9 to 11. When a user inputs an instruction to start the execution of the setting process via the input unit 14, the CPU 11 of the sound auralization device 10 executes the setting program 13A, thereby executing the setting process shown in Fig. 9. Fig. 9 is a flowchart showing an example of the setting process according to this embodiment. Note that, in order to avoid confusion, the case where a building-related information database 13C has already been constructed will be described here.

[0062] In step 100 of FIG. 9, the CPU 11 controls the display unit 15 to display a setting object input screen having a predetermined configuration, and in step 102, the CPU 11 waits until predetermined information is input.

[0063] An example of a setting target input screen according to this embodiment is shown in Fig. 10. As shown in Fig. 10, the setting target input screen according to this embodiment displays a message prompting the user to input a room (hereinafter referred to as "setting target room") that is to be set as the setting target for information regarding the calculation of the transfer function. The setting target input screen according to this embodiment also displays an input area 15A for inputting the building in which the setting target room is located and each piece of information about the setting target room.

[0064] 10 is displayed on the display unit 15, the user inputs the corresponding information into the corresponding input area 15A via the input unit 14, and then presses the end button 15D. In response to this, the determination in step 102 is affirmative, and the process proceeds to step 104.

[0065] In step 104, the CPU 11 reads out from the building-related information database 13C the 3D CAD information corresponding to the building entered on the setting target input screen and the sound source-related information corresponding to the setting target room entered on the setting target input screen (hereinafter, this information will be collectively referred to as "building-related information").

[0066] In step 106, the CPU 11 controls the display unit 15 to display a setting information input screen having a predetermined configuration, and in step 108, the CPU 11 waits until predetermined information is input.

[0067] An example of a setting information input screen according to this embodiment is shown in Fig. 11. As shown in Fig. 11, the setting information input screen according to this embodiment displays a message prompting the user to input information about the acoustic characteristics of the target room. The setting information input screen according to this embodiment also displays an input area 15B for inputting values ​​of acoustic-related information such as the sound absorption coefficient, scattering coefficient, and transmittance of the constituent surfaces of the target room.

[0068] As shown in Fig. 11, in this embodiment, a straight line of a predetermined length extending in the left-right direction is displayed as the input area 15B for each corresponding piece of sound-related information. In this embodiment, one end (the left end in this embodiment) of the line is set to the minimum value and the other end is set to the maximum value, and the value of the corresponding sound-related information is input by specifying the position of a marker using the input unit 14, but this is not limited to this. For example, as with the input area 15A shown in Fig. 10, a rectangular input area for directly inputting a corresponding value may be used as the input area for inputting the value of the sound-related information.

[0069] In this embodiment, in order to avoid confusion, all of the constituent surfaces in the target room are made of the same material, and values ​​of the acoustic-related information corresponding only to that material are input, but this is not limiting. For example, each constituent surface in the target room, such as the wall, ceiling, and floor, may be made of a different material, and values ​​of the acoustic-related information corresponding to each type of material may be input.

[0070] 11 is displayed on the display unit 15, the user designates the position of the marker to be a position corresponding to the value of the corresponding sound-related information via the input unit 14, and then designates the end button 15D. In response to this, the determination in step 108 is affirmative, and the process proceeds to step 110.

[0071] In step 110, the CPU 11 uses the acoustic-related information input on the setting information input screen to perform settings related to the calculation of the transfer function between the position of the target sound source in the setting target room indicated by the position information in the read building-related information and the position of the sound receiving point (i.e., the position of the listener) by applying the above-mentioned sound ray method.

[0072] In step 112, the CPU 11 stores (registers) information indicating the setting result related to the calculation of the transfer function in the setting target room obtained by the above processing as transfer function information in the transfer function related information database 13D together with the building name indicating the target building and the room information indicating the setting target room. Then, when the storage is completed, the CPU 11 ends this setting processing.

[0073] Next, the operation of the sound auralization device 10 when executing the sound auralization process will be described with reference to Fig. 12 to Fig. 15. When a user inputs an instruction to start the execution of the sound auralization process via the input unit 14, the CPU 11 of the sound auralization device 10 executes the sound auralization program 13B, thereby executing the sound auralization process shown in Fig. 12. Fig. 12 is a flowchart showing an example of the sound auralization process according to this embodiment. Note that, in order to avoid confusion, the case will be described where information on the target room to be subjected to the sound auralization process has already been registered in the transfer function related information database 13D.

[0074] In step 200 of FIG. 12, the CPU 11 controls the display unit 15 to display a processing object input screen having a predetermined configuration, and in step 202, the CPU 11 waits until predetermined information is input.

[0075] An example of a processing target input screen according to this embodiment is shown in Fig. 13. As shown in Fig. 13, the processing target input screen according to this embodiment displays a message prompting the user to input a room to be subjected to sound auralization processing (hereinafter referred to as "sound auralization target room"). The processing target input screen according to this embodiment also displays an input area 15C for inputting the building in which the sound auralization target room is located and various pieces of information about the sound auralization target room.

[0076] 13 is displayed on the display unit 15, the user inputs the corresponding information into the corresponding input area 15C via the input unit 14, and then presses the end button 15D. In response to this, the determination in step 202 is affirmative, and the process proceeds to step 204.

[0077] In step 204, the CPU 11 reads building-related information corresponding to the building input on the processing target input screen from the building-related information database 13C. Also in step 204, the CPU 11 reads transfer function information corresponding to the acoustic auralization target room input on the processing target input screen from the transfer function-related information database 13D.

[0078] In step 206, the CPU 11 uses the read building-related information and transfer function information to calculate how the direct sound and reflected sound arrive from the target sound source to the sound receiving point using the sound ray method described above, and calculates the transfer function by Fourier transforming the direct sound and reflected sound group.

[0079] In step 208, CPU 11 uses the calculation result of the transfer function to start generating acoustic auralization information that indicates how sound will be heard when looking in a predetermined direction from the sound receiving point in the acoustic auralization target room. Also, in step 208, CPU 11 uses information about the acoustic auralization target room in the read 3D CAD information to start generating image information that indicates a 3D image of the acoustic auralization target room when looking in the predetermined direction from the sound receiving point.

[0080] In this embodiment, the initial position of the sound receiving point is set to a predetermined height (1.6 m in this embodiment) at the entrance of the room to be auralized, but this is not limiting. For example, the predetermined height may be set to a height corresponding to the actual height of the user, or the center (center of gravity) of the room to be auralized may be set to the initial position of the sound receiving point.

[0081] In this embodiment, the initial predetermined direction is the direction from the entrance of the target room for sound auralization facing the center of the target room for sound auralization, but this is not limited to this. For example, a direction preset by a user or the like depending on the purpose of the sound auralization device 10 may be applied as the initial predetermined direction.

[0082] In step 210, the CPU 11 starts reproducing sound from the speaker 22 using the generated acoustic auralization information, and starts displaying a video screen indicated by the generated video information on the display unit 15. After this, the CPU 11 synchronizes the reproduction of the sound with the display of the video screen.

[0083] Fig. 14 shows an example of a video screen according to the embodiment. As shown in Fig. 14, the video screen according to the embodiment displays a message indicating that sound and video are being played, and also displays a three-dimensional image of the sound auralization target room when viewed from the sound receiving point in the predetermined direction. The video screen according to the embodiment also displays a method for moving the position of the sound receiving point and a method for changing the facing direction (hereinafter simply referred to as "orientation") using the controller 21. Therefore, by referring to the video screen, the user can virtually view the sound auralization target room from the line of sight that the user would have if the user were located at the sound receiving point, and can virtually hear from the speaker 22 the sound that would be heard if the user were facing the line of sight.

[0084] When the video screen is displayed, if the user wishes to change at least one of the position and orientation of the sound receiving point, the user uses the controller 21 to change the target of change in the manner displayed on the video screen. At this time, if the user wishes to make their own voice the target of acoustic auralization, the user speaks their own voice toward the microphone 20. Then, if the user wishes to end the display of the video screen and the playback of sound, the user uses the input unit 14 to designate the end button 15D.

[0085] Therefore, in step 212, the CPU 11 determines whether or not the user has performed an operation to change at least one of the position and orientation of the sound receiving point, and if the determination is negative, the CPU 11 proceeds to step 218, whereas if the determination is positive, the CPU 11 proceeds to step 214.

[0086] In step 214, the CPU 11 identifies the position and orientation of the sound receiving point according to the user's operation on the controller 21. In step 216, the CPU 11 recalculates the transfer function between the target sound source and the sound receiving point using the identified position and orientation of the sound receiving point, and executes sound / image update processing to update the sound being reproduced by the speaker 22 and the image being displayed on the display unit 15.

[0087] In the audio / video update process according to this embodiment, when acoustic information indicating a user's voice is input via the microphone 20, the sound indicated by the acoustic information is assumed to be emanating from the sound-receiving point, and is reflected in the sound auralization information. That is, in this case, the audio / video update process convolves the sound collected by the microphone 20 with a transfer function obtained by performing a calculation assuming that the sound-receiving point and the sound source are at the same position. After executing the audio / video update process, the CPU 11 proceeds to step 218. This audio / video update process updates the sound being reproduced by the speaker 22 and the video screen being displayed on the display unit 15 in real time to a state corresponding to the user's operation on the controller 21.

[0088] In step 218, the CPU 11 determines whether the end button 15D has been selected by the user, thereby determining whether the timing for ending the acoustic auralization process has arrived. If the determination is negative, the process returns to step 212. If the determination is positive, the process proceeds to step 220.

[0089] Fig. 15 shows an example of a video screen when the processing of steps 212 to 218 is repeatedly executed. Note that the example shown in Fig. 15 shows an example when the direction of the sound receiving point has moved leftward with respect to the video screen shown in Fig. 14.

[0090] In step 220, the CPU 11 stops the sound playback and video display that started in step 210, and in step 222, the CPU 11 stops the generation of the acoustic auralization information and video information that started in step 208, and then ends this acoustic auralization process.

[0091] As described above, this embodiment includes an acquisition unit 11A that acquires shape information indicating the shape of a target room, acoustic-related information related to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from a target sound source, an identification unit 11G that identifies the position of a sound receiving point, which is a position where a listener who checks the acoustic design effect of the room is located, as the sound receiving point moves, a derivation unit 11C that derives acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point using the shape information, acoustic-related information, and sound source information acquired by the acquisition unit 11A and the position of the sound receiving point identified by the identification unit 11G, and a playback control unit 11D that controls the playback unit to play the sound indicated by the acoustic auralization information derived by the derivation unit 11C. Thus, it is possible to reproduce the sound heard by the listener when the listener virtually moves.

[0092] Furthermore, according to this embodiment, the acoustic auralization information is derived using a transfer function, which makes it possible to reproduce the sound emitted from the sound source more easily than when a transfer function is not used.

[0093] Furthermore, according to this embodiment, the shape information is used to generate video information showing an image of the room as seen by the listener as the listener moves, and the video shown by the generated video information is virtually displayed. Therefore, the sound emitted from the sound source can be reproduced with a more realistic feeling than when the video is not displayed.

[0094] Furthermore, according to this embodiment, the derivation of the acoustic auralization information and the generation of the video information are performed by a single computer, which makes it easier to synchronize the playback of the sound indicated by the acoustic auralization information with the display of the video indicated by the video information, compared to when the derivation of the acoustic auralization information and the generation of the video information are performed by different computers.

[0095] In the above embodiment, a case where there is only one sound source for one target room has been described, but this is not limiting. For example, there may be multiple sound sources in each target room. In this case, calculations related to the transfer function are performed for each combination of each sound source and sound receiving point.

[0096] Furthermore, in the above embodiment, a case has been described in which the sound source does not move and only the sound receiving point moves, but this is not limiting. For example, a configuration in which only the sound source moves, or both the sound source and the sound receiving point may move. In this case, the frequency of the sound emitted from the sound source is corrected using the Doppler effect based on the moving speed of the sound source and whether the sound source is moving toward or away from the sound receiving point, and then a transfer function is convolved with the sound. By correcting the frequency of the sound emitted by the sound source in this way, the accuracy of reproduction of the sound can be improved. Furthermore, by convolving this transfer function, it is possible to reproduce a sound with reverberation in the reproduced sound field.

[0097] An example of a moving sound source is highway traffic noise when the target room is a room in a building adjacent to a highway. In this case, the location of the sound source is assumed to be the path of cars on the highway, the directivity is assumed to be a point sound source (omnidirectional) or the directivity when the car is moving, and the sound source information is assumed to be time-series information showing the sound when the car is moving.

[0098] In the above embodiment, the head-related transfer function at the sound receiving point is not taken into consideration, but the present invention is not limited to this. For example, a head-related transfer function according to the direction of the sound receiving point may be further convoluted.

[0099] In this case, head-related transfer function data for each direction, such as when the person is facing forward or to the side, is prepared in advance, and the head-related transfer function data to be convolved is selectively switched depending on the direction of the sound receiving point. By applying this head-related transfer function, the direction of the sound source can be accurately reproduced.

[0100] In the above embodiment, the controller 21 is used to change the position and orientation of the sound receiving point, but the present invention is not limited to this. For example, the position and orientation of the sound receiving point may be changed using the input unit 14 according to the present embodiment. Also, for example, the sound auralization device 10 may be provided with a voice recognition function, and the user may specify the position and orientation of the sound receiving point by voice.

[0101] Furthermore, in the above embodiment, a case has been described in which the image obtained by the acoustic auralization process is displayed on the display unit 15 provided in the acoustic auralization device 10, and the sound obtained by the acoustic auralization process is reproduced by the speaker 22 connected to the acoustic auralization device 10, but this is not limiting. For example, the image and sound obtained by the acoustic auralization device 10 may be displayed and reproduced by a head-mounted display equipped with headphones. Although not mentioned in the above embodiment, when the sound is reproduced by the speaker 22, it is preferable to prevent feedback caused by the closed loop between the microphone 20 and the speaker 22 by using a feedback canceller, for example.

[0102] Furthermore, the configurations of the various databases applied in the above embodiment are merely examples, and it goes without saying that the present invention is not limited to the examples.

[0103] Furthermore, in the above embodiment, for example, the following various processors can be used as the hardware structure of the processing units that execute the processes of the acquisition unit 11A, detection unit 11B, derivation unit 11C, playback control unit 11D, generation unit 11E, display control unit 11F, and identification unit 11G. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as a processing unit, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to execute specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field-Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0104] The processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA).The processing unit may also be configured with a single processor.

[0105] Examples of configuring a processing unit with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as the processing unit, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of the entire system, including the processing unit, on a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, the processing unit is configured using one or more of the above-mentioned various processors as a hardware structure.

[0106] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0107] 10 Acoustic Sonification Device 11 CPU 11A Acquisition Department 11B Detector 11C Derivation part 11D Playback control section 11E Generation part 11F Display control unit 11G Specific part 12 Memory 13 Storage section 13A Setting Program 13B Acoustic Auralization Program 13C Building-related information database 13D Transfer Function Related Information Database 14 Input section 15 Display section 15A input range 15B Input area 15C Input Area 15D Exit button 16 Media reading and writing device 17 Recording Media 18 Communication I / F section 20. Mike 21 Controller 22 Speaker

Claims

1. an acquisition unit that acquires shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating sounds emitted from a target sound source; an identification unit that identifies the position of a sound receiving point, which is a position where a listener who checks the acoustic design effect in the room is located, as the sound receiving point moves; a derivation unit that derives acoustic aurization information indicating how the sound emitted from the sound source is heard at the sound receiving point, using the shape information, the acoustic-related information, and the sound source information acquired by the acquisition unit and the position of the sound receiving point identified by the identification unit; and a playback control unit that controls a playback unit to play back the sound indicated by the acoustic auralization information derived by the derivation unit; Equipped with the derivation unit derives the acoustic auralization information by using a transfer function and assuming that the sound receiving point and the sound source are at the same position; The initial height of the sound receiving point is set to the height of the listener's ears. Acoustic sonification device.

2. a generating unit that generates video information showing a video of the room that can be seen by the listener according to the listener's movement using the shape information; a display control unit that controls a display unit to virtually display the image indicated by the image information generated by the generation unit; The acoustic sonification device of claim 1 further comprising:

3. The derivation of the acoustic auralization information by the derivation unit and the generation of the video information by the generation unit are performed by a single computer.

3. The acoustic sonification device of claim 2.

4. the derivation unit derives the acoustic auralization information as information that reflects the influence of the Doppler effect caused by movement of the sound source. The acoustic sonification device according to any one of claims 1 to 3.

5. Acquire shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from the target sound source; Identifying the position of a sound receiving point, which is a position where a listener who will confirm the acoustic design effect in the room is located, as the sound receiving point moves; deriving acoustic auralization information indicating how the sound emitted from the sound source is heard at the sound receiving point using the acquired shape information, the acoustic-related information, and the sound source information, and the identified position of the sound receiving point; a process of controlling a playback unit to play back a sound indicated by the derived acoustic auralization information; deriving the acoustic auralization information using a transfer function and assuming that the sound receiving point and the sound source are at the same position; The initial height of the sound receiving point is set to the height of the listener's ears. An acoustic sonification program that allows a computer to carry out the process.

Images