Acoustic visualization device and acoustic visualization program

Abstract

To obtain a sound auralization apparatus and a sound auralization program capable of immediately reproducing sound generated from a sound source.SOLUTION: A sound auralization apparatus 10 comprises: an acquisition unit 11A for acquiring shape information showing the shape of a target room and sound related information related to the sound characteristic of the interior of the room; a detection unit 11B for detecting sound source information showing at least one of sound generated from a target sound source and sound at a sound-receiving point, a spot at which a listener who confirms a sound design effect in the room exists; a derivation unit 11C that uses the shape information and the sound related information acquired by the acquisition unit 11A and the sound source information detected by the detection unit 11B to derive sound auralization information showing how sound generated from at least one of the sound source and the sound-receiving point is heard at the sound-receiving point; and a reproduction control unit 11D for performing control of making a reproduction unit reproduce sound shown by the sound auralization information derived by the derivation unit 11C.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an acoustic visualization device and an acoustic visualization program.

Background Art

[0002] In acoustic design of buildings, reverberation time and sound pressure distribution obtained from statistical acoustic theory and numerical calculations are evaluated in order to determine specifications such as component members and plans. However, certain experience and knowledge are required to appropriately evaluate the results.

[0003] On the other hand, even when lacking experience and knowledge, there is a technique to reproduce the sound in a building under design based on the results of numerical calculations and model experiments so as to enable intuitive evaluation. This technique is called acoustic visualization technology.

[0004] Conventionally, the following techniques have existed as techniques related to acoustic visualization technology.

[0005] Patent Document 1 discloses a method for perceiving a noise environment aimed at enabling reliable evaluation in a form including individual sensibilities.

[0006] This perception method records the sound source waveform of environmental noise, predicts and calculates the attenuation amount for each propagation path into the receiving room, convolves the impulse response waveform obtained from the attenuation amount with the recorded sound source waveform of environmental noise to create an evaluation sound, and is a method for perceiving a noise environment that individually or comprehensively visualizes the evaluation sound, and the propagation path into the adjacent room is composed of at least a direct transmission path through the door partition wall, a detour propagation path from the opening, and a solid propagation path through the side wall.

[0007] Also, Patent Document 2 discloses an acoustic simulation device aimed at performing high-precision sound field prediction including phase characteristics.

[0008] This acoustic simulation device includes an omnidirectional impulse response data storage means for storing omnidirectional impulse response data of a sound source, and a sound field analysis means for analyzing a sound field formed by the sound source using the omnidirectional impulse response data of the sound source.

Prior Art Documents

Patent Documents

[0009]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0010] However, in the technologies disclosed in Patent Document 1 and Patent Document 2, there is a problem that sounds cannot be detected and sounds emitted by the listener himself or sounds emitted from some sound source other than the listener cannot be reproduced immediately.

[0011] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide an acoustic visualization device and an acoustic visualization program capable of immediately reproducing sounds emitted from a sound source.

Means for Solving the Problems

[0012] The acoustic visualization device according to the present invention described in claim 1 includes an acquisition unit that acquires shape information indicating the shape of a target room and acoustic-related information related to the acoustic characteristics of the interior of the room, a detection unit that detects sound source information indicating at least one of the sound emitted from a target sound source and the sound at a sound reception point which is the point where a listener who confirms the acoustic design effect in the room is located, a derivation unit that derives acoustic visualization information indicating how the sound emitted from at least one of the sound source and the sound reception point is heard at the sound reception point, using the shape information and the acoustic-related information acquired by the acquisition unit and the sound source information detected by the detection unit, and a reproduction control unit that performs control to reproduce the sound indicated by the acoustic visualization information derived by the derivation unit by a reproduction unit. The derivation unit derives the acoustic audible information with the sound receiving point and the sound source being in the same position by using a transfer function. 。

[0013] According to the acoustic visualization device according to the present invention described in claim 1, shape information indicating the shape of a target room and acoustic-related information related to the acoustic characteristics of the interior of the room are acquired, sound source information indicating at least one of the sound emitted from a target sound source and the sound at a sound reception point which is the point where a listener who confirms the acoustic design effect in the room is located is detected, and acoustic visualization information indicating how the sound emitted from at least one of the sound source and the sound reception point is heard at the sound reception point is derived using the acquired shape information and acoustic-related information and the detected sound source information, and by performing control to reproduce the sound indicated by the derived acoustic visualization information by a reproduction unit, the sound emitted from the sound source can be reproduced immediately.

[0015] Claim 1 According to the acoustic visualization device according to the present invention described in, by deriving the acoustic visualization information using a transfer function, the sound emitted from the sound source can be reproduced more simply compared to the case where no transfer function is used.

[0016] Claim 2 The acoustic visualization device according to the present invention described in claim is to 1 The described acoustic visualization device further includes a generation unit that uses the shape information to generate video information showing a video of the room visible to the listener according to the movement of the listener, and a display control unit that performs control to virtually display the video shown by the video information generated by the generation unit on a display unit.

[0017] Claim 2 According to the acoustic visualization device according to the present invention described in claim, by using the above shape information to generate video information showing a video of the room visible to the listener according to the movement of the listener, and virtually displaying the video shown by the generated video information, compared with the case where the video is not displayed, the sound emitted from the sound source can be reproduced with a stronger sense of presence.

[0018] Claim 3 The acoustic visualization device according to the present invention described in claim is the acoustic visualization device described in claim 2 wherein the derivation of the acoustic visualization information by the derivation unit and the generation of the video information by the generation unit are performed by a single computer.

[0019] Claim 3 According to the acoustic visualization device according to the present invention described in claim, by performing the derivation of the acoustic visualization information and the generation of the video information by a single computer, compared with the case where the derivation of the acoustic visualization information and the generation of the video information are performed by different computers, it is possible to more easily synchronize the reproduction of the sound shown by the acoustic visualization information and the display of the video shown by the video information.

[0020] Claim 4 The acoustic visualization device according to the present invention described in claim is the acoustic visualization device described in any one of claims 1 to claim 3 wherein the derivation unit derives the acoustic visualization information as information reflecting the influence of the Doppler effect due to the movement of the sound source.

[0021] Claim 4According to the acoustic visualization device according to the present invention described in , by using acoustic visualization information that reflects the influence of the Doppler effect due to the movement of the sound source, the sound emitted from the sound source can be reproduced with higher accuracy compared to the case where the influence of the Doppler effect is not reflected.

[0022] Claim 5 The acoustic visualization device according to the present invention described in is an acoustic visualization device according to any one of Claims 1 to 4 , further comprising a specifying unit that specifies the position of the sound reception point as the sound reception point moves, and the deriving unit derives the acoustic visualization information by further using the position of the sound reception point specified by the specifying unit. 4

[0023] Claim 5 According to the acoustic visualization device according to the present invention described in , the position of the sound reception point as the sound reception point moves is specified, and the acoustic visualization information is further derived by using the specified position of the sound reception point, so that it is possible to correspond to the movement of the sound reception point.

[0024] Claim 6 The acoustic visualization program according to the present invention described in acquires shape information indicating the shape of the target room and acoustic-related information related to the acoustic characteristics of the interior of the room, detects sound source information indicating at least one of the sound emitted from the target sound source and the sound at the sound reception point, which is the point where the listener who confirms the acoustic design effect in the room is located, and uses the acquired shape information and acoustic-related information and the detected sound source information to derive acoustic visualization information indicating how the sound emitted from at least one of the sound source and the sound reception point sounds at the sound reception point, and controls the reproduction unit to reproduce the sound indicated by the derived acoustic visualization information. processing, and derives the acoustic audible information with the sound receiving point and the sound source being in the same position by using a transfer function. Execute the process on the computer.

[0025] Claim 6According to the acoustic visualization program related to the present invention described in [reference], shape information indicating the shape of the target room and acoustic-related information related to the acoustic characteristics of the interior of the room are acquired, and at least one of the sound emitted from the target sound source and the sound at the sound receiving point, which is the point where the listener who confirms the acoustic design effect in the above room is located, is detected. Using the acquired shape information and acoustic-related information and the detected sound source information, acoustic visualization information indicating the way of hearing at the sound receiving point of the sound emitted from at least one of the sound source and the sound receiving point is derived, and by performing control to reproduce the sound indicated by the derived acoustic visualization information by the reproduction unit, the sound emitted from the sound source can be reproduced immediately.

Effect of the Invention

[0026] As described above, according to the present invention, the sound emitted from the sound source can be reproduced immediately.

Brief Description of the Drawings

[0027]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0028] Hereinafter, with reference to the drawings, an example of a mode for carrying out the present invention will be described in detail.

[0029] First, with reference to FIGS. 1 to 3, the configuration of the acoustic visualization device 10 according to the present embodiment will be described. FIG. 1 is a block diagram showing an example of the hardware configuration of the acoustic visualization device 10 according to the present embodiment. Further, FIG. 2 is a block diagram showing an example of the functional configuration of the acoustic visualization device 10 according to the present embodiment. Furthermore, FIG. 3 is a front view showing an example of the configuration of the controller 21 according to the present embodiment. Note that examples of the acoustic visualization device 10 include information processing devices such as personal computers and server computers.

[0030] As shown in FIG. 1, the acoustic visualization device 10 according to the present embodiment includes a CPU (Central Processing Unit) 11, a memory 12 as a temporary storage area, a non-volatile storage unit 13, an input unit 14 such as a keyboard and a mouse, a display unit 15 such as a liquid crystal display, a medium reading / writing device (R / W) 16, and a communication interface (I / F) unit 18. The CPU 11, the memory 12, the storage unit 13, the input unit 14, the display unit 15, the medium reading / writing device 16, and the communication I / F unit 18 are connected to each other via a bus B. The medium reading / writing device 16 reads the information written in the recording medium 17 and writes the information to the recording medium 17.

[0031] The storage unit 13 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. In the storage unit 13 as a storage medium, a setting program 13A and an acoustic visualization program 13B are stored. The setting program 13A is stored in the storage unit 13 when the recording medium 17 in which the setting program 13A is written is set in the medium reading / writing device 16 and the medium reading / writing device 16 reads the setting program 13A from the recording medium 17. Also, the acoustic visualization program 13B is stored in the storage unit 13 when the recording medium 17 in which the acoustic visualization program 13B is written is set in the medium reading / writing device 16 and the medium reading / writing device 16 reads the acoustic visualization program 13B from the recording medium 17. The CPU 11 appropriately reads the programs of the setting program 13A and the acoustic visualization program 13B from the storage unit 13, expands them in the memory 12, and sequentially executes the processes included in the respective programs.

[0032] Further, a building-related information database 13C and a transfer function-related information database 13D are stored in the storage unit 13. Details of the building-related information database 13C and the transfer function-related information database 13D will be described later.

[0033] Furthermore, as shown in FIG. 1, a microphone (hereinafter referred to as "mic") 20, a controller 21, and a speaker 22 are connected to the communication I / F unit 18 according to the present embodiment.

[0034] The mic 20 according to the present embodiment is for collecting sound source information indicating voices emitted by the user of the acoustic visualization device 10 (corresponding to the "listener" of the present invention) and sounds emitted from a sound source described later. In the present embodiment, a single-directional and condenser-type mic is applied as the mic 20, but it is not limited thereto. For example, a bidirectional or omnidirectional mic, or a dynamic-type mic may be applied as the mic 20.

[0035] In addition, the controller 21 according to the present embodiment is for inputting various information according to operations by the user. As an example, as shown in FIG. 3, in the present embodiment, a general-purpose controller for a game machine is applied as the controller 21, but it is not limited thereto, and a dedicated controller for the acoustic visualization device 10 may be applied as the controller 21.

[0036] As shown in FIG. 3, the controller 21 according to the present embodiment includes a pair of gripping portions 21A and 21B that are respectively gripped by the user's left and right hands. The gripping portions 21A and 21B are connected by an intermediate portion 21C, and buttons 21A1 and 21B1 that can be operated by the thumb of the gripping hand are respectively provided.

[0037] In addition, the controller 21 according to the present embodiment is provided with a stick portion 21C1 and a stick portion 21C2 that can be operated by the thumb of the hand gripping the corresponding gripping portion at the lower end of the intermediate portion 21C in FIG. 3 and in the vicinity of each of the gripping portions 21A and 21B. In the acoustic visualization device 10 according to the present embodiment, the stick portion 21C1 receives an instruction input for moving a sound reception point described later, and the stick portion 21C2 receives an instruction input for the direction of the sound reception point.

[0038] Furthermore, the speaker 22 according to the present embodiment is for reproducing the sound generated by the acoustic visualization device 10. In the present embodiment, a dynamic and stereo speaker 22 is applied, but the present invention is not limited to this. For example, a condenser type, a piezoelectric type, or the like, and a monaural type may be applied as the speaker 22.

[0039] Next, with reference to FIG. 2, the functional configuration of the acoustic visualization device 10 according to the present embodiment will be described.

[0040] As shown in FIG. 2, the acoustic visualization device 10 according to the present embodiment includes an acquisition unit 11A, a detection unit 11B, a derivation unit 11C, a reproduction control unit 11D, a generation unit 11E, a display control unit 11F, and a specification unit 11G. By executing the setting program 13A and the acoustic visualization program 13B, the CPU 11 of the acoustic visualization device 10 functions as the acquisition unit 11A, the detection unit 11B, the derivation unit 11C, the reproduction control unit 11D, the generation unit 11E, the display control unit 11F, and the specification unit 11G.

[0041] The acquisition unit 11A according to the present embodiment acquires shape information indicating the shape of a target room (hereinafter also referred to as the "target room") and acoustic-related information related to the acoustic characteristics of the interior of the room.

[0042] In the present embodiment, as the above shape information, information based on 3D CAD (Computer Aided Design) information, the details of which will be described later, is applied, but the present invention is not limited to this. For example, information based on BIM (Building Information Modeling) information may be applied as the above shape information. Further, in the present embodiment, as the above acoustic-related information, information including the sound absorption rate, scattering coefficient, and transmission rate of surfaces such as wall surfaces, ceiling surfaces, and floor surfaces (hereinafter referred to as "constituent surfaces") in the target room is applied, but the present invention is not limited to this. For example, information including any one or a combination of two of the sound absorption rate, scattering coefficient, and transmission rate may be applied as the above acoustic-related information.

[0043] Note that the above sound absorption rate is an index indicating how much sound the target material absorbs. If it is the maximum value, all the incident sound is absorbed. If it is the minimum value (usually 0 (zero)), it means that the incident sound is reflected without being absorbed. Also, the above scattering coefficient is an index indicating the ratio of the sound incident on the target material that is reflected in a direction other than specular reflection. When the scattering coefficient is high, the sound is randomly reflected in all directions. When the scattering coefficient is low, the sound is specularly reflected like a mirror. Furthermore, the above transmittance is an index indicating how much sound the target material transmits. If it is the maximum value, all the incident sound is transmitted in the direction opposite to the incident direction. If it is the minimum value (usually 0 (zero)), it means that the incident sound is completely blocked without being transmitted.

[0044] In addition, the detection unit 11B according to the present embodiment detects sound source information indicating at least one of sound (hereinafter also referred to as "sound source sound") emitted from a target sound source (hereinafter also referred to as "target sound source") and sound (hereinafter also referred to as "sound at the sound reception point") at a sound reception point which is a point where a listener who confirms the acoustic design effect in the target room is present.

[0045] Note that in the present embodiment, a case where the detection unit 11B detects sound source information indicating both the sound source sound and the sound at the sound reception point will be described, but the present invention is not limited thereto. For example, the detection unit 11B may detect only the sound source information indicating the sound at the sound reception point, or the detection unit 11B may detect only the sound source information indicating the sound source sound. Also, in the present embodiment, a case where the detection of the sound source information indicating the sound source sound by the detection unit 11B is indirectly performed by detecting the sound emitted from the target sound source by the microphone 20, storing it in the storage unit 13 in advance as the sound source information, and reading out the sound source information from the storage unit 13 will be described. In this case, since the sound source information is acquired by reading it out from the storage unit 13, in other words, it can be said that the sound source information is acquired by the acquisition unit 11A. However, the present invention is not limited to this form. For example, the detection or acquisition of the sound source information indicating the sound source sound may be directly detected or acquired via the microphone 20 without passing through the storage unit 13.

[0046] In addition, the derivation unit 11C according to the present embodiment derives acoustic visualization information indicating how a sound emitted from at least one of the target sound source and the sound receiving point is heard at the sound receiving point, using the shape information and the sound-related information acquired by the acquisition unit 11A and the sound source information detected by the detection unit 11B.

[0047] Note that, in the present embodiment, a case where the derivation unit 11C derives acoustic visualization information corresponding to both the sound source sound and the sound receiving point sound will be described, but the present invention is not limited thereto. For example, the derivation unit 11C may derive acoustic visualization information corresponding only to the sound receiving point sound, or the derivation unit 11C may derive acoustic visualization information corresponding only to the sound source sound.

[0048] In addition, in the present embodiment, the derivation unit 11C derives acoustic visualization information using a transfer function.

[0049] Then, the playback control unit 11D according to the present embodiment performs control to cause the playback unit (the speaker 22 in the present embodiment) to play the sound indicated by the acoustic visualization information derived by the derivation unit 11C.

[0050] On the other hand, the generation unit 11E according to the present embodiment generates video information indicating a video of the target room visible to the listener according to the movement of the listener, using the above-described shape information. Then, the display control unit 11F according to the present embodiment performs control to virtually display the video indicated by the video information generated by the generation unit 11E on the display unit 15.

[0051] In addition, the specifying unit 11G according to the present embodiment specifies the position of the sound receiving point as the sound receiving point moves. Then, the derivation unit 11C according to the present embodiment derives acoustic visualization information using, further, the position of the sound receiving point specified by the specifying unit 11G.

[0052] In the present embodiment, the derivation of the acoustic visualization information by the derivation unit 11C and the generation of the video information by the generation unit 11E are performed by a single computer (in this embodiment, the CPU 11). Thereby, synchronization between the reproduction of the sound indicated by the acoustic visualization information and the display of the video indicated by the video information can be more easily achieved. However, the present invention is not limited to this. For example, the processing by the derivation unit 11C and the processing by the generation unit 11E may be performed by different computers. In this form, the load of the processing by the derivation unit 11C and the load of the processing by the generation unit 11E can be dispersed, and the sound reproduction and the video display can be performed with higher precision and at higher speed.

[0053] Next, with reference to FIG. 4, the building-related information database 13C according to the present embodiment will be described. FIG. 4 is a schematic diagram showing an example of the configuration of the building-related information database 13C according to the present embodiment. The building-related information database 13C is a database in which information related to the building that the acoustic visualization device 10 according to the present embodiment deals with is stored.

[0054] As shown in FIG. 4, in the building-related information database 13C according to the present embodiment, for each building that the acoustic visualization device 10 deals with, information such as the building name, 3D CAD information, and sound source-related information is stored in an associated manner.

[0055] The above building name is information indicating the name of the corresponding building, and the above 3D CAD information is information indicating a model (hereinafter referred to as a "building-related model") including building shape information indicating the shape of the corresponding building and specific information for specifying each room in the building.

[0056] In this embodiment, the building-related model is created using predetermined 3D CAD software. In this embodiment, Rhinoceros (registered trademark) is applied as the above-mentioned 3D CAD software, but it is not limited thereto. For example, other software such as Revit (registered trademark) may be applied as the above-mentioned 3D CAD software.

[0057] Also, the above sound source-related information is information regarding the target sound source described above, and includes each of room information, sound source information, position information, and directivity information. The above room information is information for specifying the corresponding room (target room) of the corresponding building, and the above-mentioned specific information is applied in this embodiment. Also, as described above, the above sound source information is information indicating the sound itself emitted from the corresponding target sound source and obtained by the microphone 20. Also, the above position information is information indicating the three-dimensional position of the corresponding target sound source, and the above directivity information is information indicating the directivity of the corresponding target sound source.

[0058] For example, when the target room is a classroom in a school, a specific example of the target sound source is the voice of a teacher. In this case, as the position of the target sound source, positions such as the front side of the blackboard or the podium where the teacher is considered to stand are applied, as the directivity, the directivity of a human voice is applied, and as the sound source information, time-series information indicating the voice emitted by the teacher during the class is applied.

[0059] Next, with reference to FIG. 5, the transfer function-related information database 13D according to this embodiment will be described. FIG. 5 is a schematic diagram showing an example of the configuration of the transfer function-related information database 13D according to this embodiment. The transfer function-related information database 13D is a database that stores information regarding the calculation of a transfer function, which will be described in detail later, and is generated by the acoustic visualization device 10 according to this embodiment.

[0060] As shown in FIG. 5, the transfer function related information database 13D according to the present embodiment stores, for each building that the acoustic visualization device 10 targets, the building name and each piece of information related to the transfer function related information associated with each other.

[0061] The above building name is the same information as the building name in the building related information database 13C. The above transfer function related information is information regarding the transfer function described later, corresponding to each room in the corresponding building, and includes each piece of information of room information and transfer function information. The above room information is the same information as the room information in the building related information database 13C. The above transfer function information is information regarding the calculation of the transfer function between the position of the target sound source and the position of the sound receiving point, which is set for the corresponding room by the setting process described later.

[0062] Here, referring to FIGS. 6 to 8, the setting regarding the calculation of the transfer function applied in the present embodiment will be described. FIG. 6 is a perspective view showing an example of a setting target regarding the calculation of the transfer function according to the present embodiment. Further, FIG. 7 is a side view showing an example of a setting target regarding the calculation of the transfer function when wave properties are not considered according to the present embodiment. Furthermore, FIG. 8 is a side view showing an example of a setting target regarding the calculation of the transfer function when wave properties are considered according to the present embodiment.

[0063] As an example, as shown in FIG. 6, in the present embodiment, the calculation of the transfer function based on the sound ray method that does not consider the wave properties of sound is performed. In the sound ray method, the result changes depending on the number of sound rays generated from one target sound source, the maximum number of reflections, etc. Therefore, appropriate values are set while considering the calculation load by the CPU 11 and the like.

[0064] As an example, as shown in FIG. 7, when wave characteristics are not considered, the sound flow advances linearly, so if there is a soundproof wall or the like between the target sound source and the sound receiving point, the sound does not reach the sound receiving point. On the other hand, as an example, as shown in FIG. 8, when wave characteristics are considered, the sound from the target sound source wraps around the soundproof wall or the like and reaches the sound receiving point as diffracted sound. In the present embodiment, in consideration of the current processing capabilities of computers, a ray method that does not consider wave characteristics is applied as the ray method for the calculation means, but it is needless to say that the present invention is not limited thereto, and a calculation method that considers wave characteristics may also be applied.

[0065] Next, with reference to FIGS. 9 to 15, the operation of the acoustic visualization device 10 according to the present embodiment will be described. First, with reference to FIGS. 9 to 11, the operation of the acoustic visualization device 10 when executing a setting process for setting information related to the calculation of the transfer function will be described. When an instruction input to start the execution of the setting process is input by the user via the input unit 14, the CPU 11 of the acoustic visualization device 10 executes the setting program 13A, and the setting process shown in FIG. 9 is executed. FIG. 9 is a flowchart showing an example of the setting process according to the present embodiment. Here, in order to avoid complication, the case where the building-related information database 13C has been constructed will be described.

[0066] In step 100 of FIG. 9, the CPU 11 controls the display unit 15 to display a setting target input screen having a predetermined configuration, and in step 102, the CPU 11 waits until predetermined information is input.

[0067] FIG. 10 shows an example of the setting target input screen according to the present embodiment. As shown in FIG. 10, in the setting target input screen according to the present embodiment, a message prompting the input of the room (hereinafter referred to as the "setting target room") for which information related to the calculation of the transfer function is to be set is displayed. Further, in the setting target input screen according to the present embodiment, the building in which the setting target room is provided and an input area 15A for inputting each piece of information of the setting target room are displayed.

[0068] As an example, when the setting target input screen shown in FIG. 10 is displayed on the display unit 15, the user inputs corresponding information into the corresponding input area 15A via the input unit 14, and then designates the end button 15D. In response to this, step 102 becomes an affirmative determination, and the process proceeds to step 104.

[0069] In step 104, the CPU 11 reads out the 3D CAD information corresponding to the building input on the setting target input screen and the sound source related information corresponding to the setting target room input on the setting target input screen (hereinafter, these information are collectively referred to as "building related information") from the building related information database 13C.

[0070] In step 106, the CPU 11 controls the display unit 15 to display a setting information input screen having a predetermined configuration, and in step 108, the CPU 11 waits until predetermined information is input.

[0071] FIG. 11 shows an example of the setting information input screen according to the present embodiment. As shown in FIG. 11, in the setting information input screen according to the present embodiment, a message prompting input of information regarding the acoustic characteristics of the setting target room is displayed. Further, in the setting information input screen according to the present embodiment, an input area 15B for inputting each value of acoustic related information such as the sound absorption rate, scattering coefficient, and transmittance of the constituent surfaces in the setting target room is displayed.

[0072] As shown in FIG. 11, in the present embodiment, as the input area 15B, a straight line having a predetermined length extending in the left - right direction is displayed for each corresponding acoustic related information. Then, in the present embodiment, with one end (the left end in the present embodiment) of the straight line as the minimum value and the other end as the maximum value, the value of the corresponding acoustic related information is input by designating the position of the marker using the input unit 14, but it is not limited to this. For example, similar to the input area 15A shown in FIG. 10 as an example, a rectangular input area or the like for directly inputting the corresponding value may be applied as the input area for inputting the value of the acoustic related information.

[0073] In addition, in the present embodiment, in order to avoid complication, all the constituent surfaces in the room to be set are made of the same material, and the value of the acoustic-related information corresponding only to the material is input. However, the present invention is not limited to this. For example, each constituent surface in the room to be set may be made of different materials for each part such as a wall surface, a ceiling surface, and a floor surface, and the value of the acoustic-related information corresponding to each type of material may be input.

[0074] As an example, when the setting information input screen shown in FIG. 11 is displayed on the display unit 15, the user designates the position of the marker to be the position corresponding to the value of the acoustic-related information via the input unit 14, and then designates the end button 15D. In response to this, step 108 becomes an affirmative determination, and the process proceeds to step 110.

[0075] In step 110, the CPU 11 performs setting related to the calculation of the transfer function between the position of the target sound source indicated by the position information in the read building-related information in the room to be set and the position of the sound receiving point (i.e., the position of the listener) using the acoustic-related information input on the setting information input screen, by applying the ray method described above.

[0076] In step 112, the CPU 11 stores (registers) the information indicating the setting result related to the calculation of the transfer function in the room to be set obtained by the above processing as transfer function information, together with the building name indicating the target building and the room information indicating the room to be set, in the transfer function-related information database 13D. When the storage is completed, the CPU 11 ends the present setting process.

[0077] Next, with reference to FIGS. 12 to 15, the operation of the acoustic visualization device 10 when performing acoustic visualization processing will be described. When an instruction input to start the execution of the acoustic visualization processing is input by the user via the input unit 14, the CPU 11 of the acoustic visualization device 10 executes the acoustic visualization program 13B, and the acoustic visualization processing shown in FIG. 12 is executed. FIG. 12 is a flowchart showing an example of the acoustic visualization processing according to the present embodiment. Here, in order to avoid complication, the case where information regarding the target room to be subjected to the acoustic visualization processing is already registered in the transfer function related information database 13D will be described.

[0078] In step 200 of FIG. 12, the CPU 11 controls the display unit 15 to display a processing target input screen having a predetermined configuration, and in step 202, the CPU 11 waits until predetermined information is input.

[0079] FIG. 13 shows an example of the processing target input screen according to the present embodiment. As shown in FIG. 13, in the processing target input screen according to the present embodiment, a message prompting the input of the room to be subjected to the acoustic visualization processing (hereinafter referred to as the "acoustic visualization target room") is displayed. Further, in the processing target input screen according to the present embodiment, the building in which the acoustic visualization target room is provided and an input area 15C for inputting each piece of information of the acoustic visualization target room are displayed.

[0080] As an example, when the processing target input screen shown in FIG. 13 is displayed on the display unit 15, the user inputs the corresponding information to the corresponding input area 15C via the input unit 14 and then designates the end button 15D. In response to this, step 202 becomes an affirmative determination and the process proceeds to step 204.

[0081] In step 204, the CPU 11 reads out the building related information corresponding to the building input on the processing target input screen from the building related information database 13C. Also, in step 204, the CPU 11 reads out the transfer function information corresponding to the acoustic visualization target room input on the processing target input screen from the transfer function related information database 13D.

[0082] In step 206, the CPU 11 uses the read building-related information and transfer function information to calculate how the direct sound and the reflected sound arrive from the target sound source to the sound receiving point by the above-described ray tracing method, and calculates the transfer function by performing Fourier transform on the direct sound and the reflected sound group.

[0083] In step 208, the CPU 11 starts generating acoustic visualization information indicating how the sound heard when looking in a predetermined direction from the sound receiving point in the acoustic visualization target room sounds, using the calculation result of the transfer function. Also in step 208, the CPU 11 starts generating video information indicating a three-dimensional video of the acoustic visualization target room when looking in the predetermined direction from the sound receiving point, using the information regarding the acoustic visualization target room in the read three-dimensional CAD information.

[0084] Note that in this embodiment, the position of the sound receiving point at the initial stage at this time is applied with a predetermined height (1.6 m in this embodiment) at the position of the entrance of the acoustic visualization target room, but it is not limited thereto. For example, as the predetermined height, a form in which a height corresponding to the actual height of the user is applied as the height of the sound receiving point at the initial stage, or a form in which the center point (center of gravity point) of the acoustic visualization target room is applied as the position of the sound receiving point at the initial stage may be used.

[0085] Also in this embodiment, as the predetermined direction at the initial stage at this time, the direction facing the center of the interior of the acoustic visualization target room from the entrance of the acoustic visualization target room is used, but it is not limited thereto. For example, a form in which a direction preset according to the use of the acoustic visualization device 10 or the like by the user or the like is applied as the predetermined direction at the initial stage may be used.

[0086] In step 210, the CPU 11 starts playing the sound by the speaker 22 using the generated acoustic visualization information, and starts displaying the video screen indicated by the generated video information on the display unit 15. Thereafter, the CPU 11 causes the sound playback and the display of the video screen to be synchronized.

[0087] FIG. 14 shows an example of a video screen according to an embodiment. As shown in FIG. 14, in the video screen according to this embodiment, it is displayed that sound and video are being played, and a three-dimensional image of the acoustic visualization target room is displayed when viewed from the sound reception point in the predetermined direction. Further, in the video screen according to this embodiment, the method of moving the position of the sound reception point by the controller 21 and the method of changing the direction (hereinafter simply referred to as "orientation") are displayed. Therefore, by referring to the video screen, the user can virtually view the acoustic visualization target room with the line of sight when the user is at the position of the sound reception point, and can virtually listen to the sound heard when the user is facing the direction of the line of sight from the speaker 22.

[0088] When the video screen is displayed, when the user wants to change at least one of the position and orientation of the sound reception point, the user uses the controller 21 to change the object to be changed by the method displayed on the video screen. At this time, when the user wants to make his or her own voice the target of acoustic visualization, the user emits his or her own voice toward the microphone 20. Then, when the user wants to end the display of the video screen and the reproduction of the sound, the user designates the end button 15D using the input unit 14.

[0089] Therefore, in step 212, the CPU 11 determines whether or not an operation has been performed by the user to change at least one of the position and orientation of the sound reception point. If the determination is negative, the process proceeds to step 218, while if the determination is positive, the process proceeds to step 214.

[0090] In step 214, the CPU 11 specifies the position and orientation of the sound reception point according to the operation on the controller 21 by the user. In step 216, the CPU 11 re-calculates the transfer function between the target sound source and the sound reception point using the specified position and orientation of the sound reception point, and executes a sound / video update process for updating the sound being played by the speaker 22 and the video being displayed by the display unit 15.

[0091] In the sound / video update process according to this embodiment, when acoustic information indicating the voice of the user is input via the microphone 20, the sound indicated by the acoustic information is assumed to be emitted from the sound reception point and reflected in the acoustic visualization information. That is, in this case, in the sound / video update process, the transfer function obtained by performing an operation with the sound reception point and the sound source at the same position is convolved with the sound collected by the microphone 20. Then, the CPU 11 proceeds to step 218 after executing the sound / video update process. By this sound / video update process, the sound reproduced by the speaker 22 and the video screen displayed by the display unit 15 are updated in real time to the state corresponding to the operation of the user on the controller 21.

[0092] In step 218, the CPU 11 determines whether the end button 15D has been specified by the user, thereby determining whether the end timing of this acoustic visualization process has arrived. If the determination is negative, the process returns to step 212. If the determination is positive, the process proceeds to step 220.

[0093] FIG. 15 shows an example of a video screen when the processes of steps 212 to 218 are repeatedly executed. Note that the example shown in FIG. 15 is an example in which the direction of the sound reception point has moved to the left with respect to the video screen shown in FIG. 14.

[0094] In step 220, the CPU 11 stops the reproduction of the sound and the display of the video started in step 210. In step 222, the CPU 11 stops the generation of the acoustic visualization information and the video information started in step 208, and then ends this acoustic visualization process.

[0095] As described above, according to the present embodiment, an acquisition unit 11A that acquires shape information indicating the shape of a target room and acoustic-related information related to the acoustic characteristics of the interior of the room, a detection unit 11B that detects sound emitted from a target sound source and sound source information indicating at least one of the sound at a sound reception point that is a point where a listener who confirms the acoustic design effect in the above room is located, and the shape information and acoustic-related information acquired by the acquisition unit 11A, and the sound source information detected by the detection unit 11B, are used to derive acoustic visualization information indicating how the sound emitted from at least one of the sound source and the sound reception point is heard at the sound reception point, and a reproduction control unit 11D that controls the reproduction unit (speaker 22) to reproduce the sound indicated by the acoustic visualization information derived by the derivation unit 11C. Therefore, the sound emitted from the sound source can be reproduced immediately.

[0096] Also, according to the present embodiment, the acoustic visualization information is derived using a transfer function. Therefore, compared with the case where a transfer function is not used, the sound emitted from the sound source can be reproduced more simply.

[0097] Also, according to the present embodiment, using the above shape information, video information indicating the video of the above room that the listener can see according to the movement of the listener is generated, and the video indicated by the generated video information is virtually displayed. Therefore, compared with the case where the video is not displayed, the sound emitted from the sound source can be reproduced with a greater sense of presence.

[0098] Also, according to the present embodiment, the derivation of the acoustic visualization information and the generation of the video information are performed by a single computer. Therefore, compared with the case where the derivation of the acoustic visualization information and the generation of the video information are performed by different computers, the synchronization between the reproduction of the sound indicated by the acoustic visualization information and the display of the video indicated by the video information can be achieved more easily.

[0099] Furthermore, according to the present embodiment, the position of the sound reception point accompanying the movement of the sound reception point is specified, and the acoustic visualization information is further derived using the specified position of the sound reception point. Therefore, it is possible to correspond to the movement of the sound reception point.

[0100] In the above embodiment, the case where there is only one sound source for one target room has been described, but the present invention is not limited thereto. For example, a form in which there are a plurality of sound sources in each target room may also be used. In this case, calculations regarding the transfer function are performed for each combination of each sound source and the sound receiving point.

[0101] Also, in the above embodiment, the case where the sound source does not move and only the sound receiving point moves has been described, but the present invention is not limited thereto. For example, a form in which only the sound source, or both the sound source and the sound receiving point move may also be used. In this case, based on the moving speed of the sound source and whether the sound source is approaching or moving away from the sound receiving point, after correcting the frequency of the sound emitted from the sound source due to the Doppler effect, the transfer function is convolved with the sound. By correcting the frequency of the sound by this sound source, the reproduction accuracy of the sound can be improved. Also, by performing this convolution of the transfer function, a sound with reverberation in the reproduced sound field can be reproduced.

[0102] As an example of a form in which the sound source moves, for example, the traffic noise on a highway in the case where the target room is a room in a building adjacent to the highway can be cited. In this case, as the position of the sound source, the track of the vehicle on the highway is applied, as the directivity, a point sound source (omnidirectional) or the directivity when the vehicle is running is applied, and as the sound source information, time-series information indicating the sound when the vehicle is running is applied.

[0103] Also, in the above embodiment, the case where the head-related transfer function at the sound receiving point is not considered has been described, but the present invention is not limited thereto. For example, a form in which the head-related transfer function corresponding to the direction of the sound receiving point is further convolved may also be used.

[0104] In this case, data on the head-related transfer function for each direction such as when a person is facing forward or sideways is prepared in advance, and the data on the head-related transfer function to be convolved is selectively switched according to the direction of the sound receiving point. By applying this head-related transfer function, the direction of the sound source can be accurately reproduced.

[0105] In the above-described embodiment, the case where the controller 21 is used to change the position and orientation of the sound reception point has been described, but the present invention is not limited thereto. For example, the position and orientation of the sound reception point may be changed using the input unit 14 according to the present embodiment. Further, for example, the acoustic visualization device 10 may be provided with a voice recognition function, and the user may indicate the position and orientation of the sound reception point by voice.

[0106] In the above-described embodiment, the case where the video obtained by the acoustic visualization process is displayed by the display unit 15 provided in the acoustic visualization device 10 and the sound obtained by the acoustic visualization process is reproduced by the speaker 22 connected to the acoustic visualization device 10 has been described, but the present invention is not limited thereto. For example, the video and sound obtained by the acoustic visualization device 10 may be displayed and reproduced by a head-mounted display provided with headphones. Although not mentioned in the above-described embodiment, when the sound is reproduced by the speaker 22, it is preferable to use a howling canceller or the like to prevent howling caused by the closed loop between the microphone 20 and the speaker 22.

[0107] The configuration of each of the various databases applied in the above-described embodiment is an example, and it goes without saying that the present invention is not limited to the illustrated ones.

[0108] Also, in the above-described embodiment, for example, as the hardware structure of a processing unit that executes each process of the acquisition unit 11A, the detection unit 11B, the derivation unit 11C, the reproduction control unit 11D, the generation unit 11E, the display control unit 11F, and the specification unit 11G, the following various processors can be used. As described above, in addition to the CPU, which is a general-purpose processor that executes software (program) and functions as a processing unit, the above various processors include a programmable logic device (PLD), such as an FPGA (Field-Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing, and an application-specific electric circuit, which is a processor having a circuit configuration specifically designed to execute a specific process, such as an ASIC (Application Specific Integrated Circuit).

[0109] The processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the processing unit may be composed of one processor.

[0110] As an example of configuring the processing unit with one processor, first, as represented by computers such as clients and servers, there is a form in which one processor is configured by a combination of one or more CPUs and software, and this processor functions as the processing unit. Second, as represented by a system on chip (SoC), there is a form in which a processor that realizes the functions of the entire system including the processing unit with one IC (Integrated Circuit) chip is used. Thus, the processing unit is configured using one or more of the above various processors as its hardware structure.

[0111] Furthermore, as the hardware structure of these various processors, more specifically, circuitry combining circuit elements such as semiconductor elements can be used.

Explanation of Signs

[0112] 10 Acoustic visualization device 11 CPU 11A Acquisition unit 11B Detection unit 11C Derivation unit 11D Reproduction control unit 11E Generation unit 11F Display control unit 11G Identification unit 12 Memory 13 Storage unit 13A Setting program 13B Acoustic visualization program 13C Building-related information database 13D Transfer function-related information database 14 Input unit 15 Display unit 15A Input area 15B Input area 15C Input area 15D End button 16 Medium reading / writing device 17 Recording medium 18 Communication I / F unit 20 Microphone 21 Controller 22 Speaker

Claims

1. An acquisition unit that acquires shape information indicating the shape of a target room and acoustic-related information related to the acoustic characteristics of the interior of the room; A detection unit that detects at least one of the sound emitted from a target sound source and the sound at a sound reception point, which is the location where a listener who confirms the acoustic design effect in the room is present, indicating sound source information; Using the shape information and the acoustic-related information acquired by the acquisition unit and the sound source information detected by the detection unit, a derivation unit that derives acoustic visualization information indicating how the sound emitted from at least one of the sound source and the sound reception point is heard at the sound reception point; A reproduction control unit that performs control to cause a reproduction unit to reproduce the sound indicated by the acoustic visualization information derived by the derivation unit; Comprising: The derivation unit derives the acoustic visualization information with the sound reception point and the sound source being in the same position using a transfer function. An acoustic visualization device.

2. A generation unit that generates video information indicating a video of the room visible to the listener according to the movement of the listener using the shape information; A display control unit that performs control to virtually display the video indicated by the video information generated by the generation unit on a display unit; The acoustic visualization device according to claim 1, further comprising:

3. The derivation of the acoustic visualization information by the derivation unit and the generation of the video information by the generation unit are performed by a single computer. The acoustic visualization device according to claim 2.

4. The derivation unit derives the acoustic visualization information as information reflecting the influence of the Doppler effect due to the movement of the sound source. The acoustic visualization device according to any one of claims 1 to 3.

5. Further comprising an identification unit that identifies the position of the sound reception point as the sound reception point moves; The derivation unit further derives the acoustic visualization information using the position of the sound reception point identified by the identification unit. The acoustic visualization device according to any one of claims 1 to 4.

6. Acquire shape information indicating the shape of a target room and acoustic-related information related to the acoustic characteristics of the interior of the room; Detect at least one of the sound emitted from a target sound source and the sound at a sound reception point, which is the location where a listener who confirms the acoustic design effect in the room is present, indicating sound source information; Using the obtained shape information, the acoustic-related information, and the detected sound source information, derive acoustic visualization information indicating how the sound emitted from at least one of the sound source and the sound reception point is heard at the sound reception point. This is a process of performing control to reproduce the sound indicated by the derived acoustic visualization information by a reproduction unit. Derive the acoustic visualization information with the sound reception point and the sound source being in the same position using a transfer function. An acoustic visualization program for causing a computer to execute the process.

Images