Acoustic computation method, acoustic computation system, and acoustic computation program

Abstract

An acoustic computation method according to one embodiment of the present disclosure comprises: an acquisition step in which a computer acquires, during video content production, an information item regarding a video device that displays a background video, an information item regarding a space in which the video device is installed and which is subjected to an acoustic simulation, and an information item regarding a sound reception point and a sound emission point in said space; an analysis step in which the computer executes the acoustic simulation in said space on the basis of the acquired information items; and a display control step in which the computer visualizes the result of the acoustic simulation to display the result on a user interface.

Description

Acoustic calculation method, acoustic calculation system, and acoustic calculation program

[0001] The present disclosure relates to an acoustic calculation method, an acoustic calculation system, and an acoustic calculation program.

[0002] In the production of video content such as TVs, movies, and CM (Commercial Message), shooting techniques known as virtual production or in-camera VFX (Visual Effects) have begun to be used. These are technologies that use a huge LED display (Light Emitting Diode Display, also called "LED Wall") that displays video and CG (Computer Graphics) in real time on the background of the set, and can shoot in a space that is almost indistinguishable from a real set or an outdoor location.

[0003] In the production of such content, it is useful to simulate the acoustics of the space used for shooting in advance and know what kind of sound can be obtained. For example, regarding video content such as the Meta-verse and games, there is a known technology for reproducing an acoustic space using a learned model that has learned the relationship between the structural data constituting the virtual space and the acoustic signal at the sound reception point.

[0004] International Publication No. 2023 / 182024

[0005] According to the prior art, by using a transfer function, a realistic acoustic space can be reproduced without performing high-load calculation processing such as using the wave equation.

[0006] On the other hand, virtual production, unlike virtual spaces, presents a unique problem: during video recording, the performer's voice resonates excessively within the LED wall, resulting in recordings with excessive reverberation on the microphones. Naturally, the performer themselves will hear this excessive reverberation, making acting and singing difficult and potentially hindering content creation. Therefore, virtual production requires not only simulating the acoustics of the space, but also using the results of the simulation to ensure smooth filming and sound recording in the actual location.

[0007] Therefore, this disclosure proposes an acoustic calculation method, an acoustic calculation system, and an acoustic calculation program that can facilitate content creation in virtual production.

[0008] To solve the above problems, one form of acoustic calculation method according to the present disclosure includes an acquisition step in which a computer acquires information about a video device that displays background images when creating video content, information about the space in which the video device is installed and which is the subject of acoustic simulation processing, and information about sound receiving points and sound generating points in the space; an analysis step in which an acoustic simulation is performed in the space based on the acquired information; and a display control step in which the results of the acoustic simulation are visualized and displayed on a user interface.

[0009] This figure shows an overview of the acoustic calculation system according to the embodiment. This figure shows an example of acoustic simulation results in virtual production. This is Figure (1) showing an example of simulation results according to the embodiment. This is Figure (2) showing an example of simulation results according to the embodiment. This figure shows an overview of the simulation tool according to the embodiment. This is Figure (1) showing an example of the procedure for performing acoustic calculation processing. This is Figure (2) showing an example of the procedure for performing acoustic calculation processing. This is a flowchart (1) showing the procedure for acoustic calculation processing. This is a flowchart (2) showing the procedure for acoustic calculation processing. This is a flowchart (3) showing the procedure for acoustic calculation processing. This is a flowchart (4) showing the procedure for acoustic calculation processing. This figure shows an example of the display of the user interface. This figure shows an example of the configuration of the acoustic calculation device and the sound collection device. This is a hardware configuration diagram showing an example of a computer that realizes the functions of the acoustic calculation device.

[0010] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same parts will be denoted by the same reference numerals to avoid redundant descriptions.

[0011] The one or more embodiments (including examples and modifications) described below can each be implemented independently. On the other hand, at least some of the embodiments described below may be implemented in appropriate combination with at least some of the other embodiments. These embodiments may include novel features that differ from each other. Therefore, these embodiments may contribute to solving different objectives or problems and may produce different effects.

[0012] Furthermore, this disclosure will be described in the following order of items: 1. Embodiments 1-1. Overview of the acoustic calculation processing according to the embodiment 1-2. Procedure of the acoustic calculation processing according to the embodiment 1-3. Configuration of the acoustic calculation system according to the embodiment 2. Modifications 2-1. Example of the configuration of the acoustic calculation system 2-2. Example of simulation 3. Other embodiments 4. Summary of the acoustic calculation method according to this disclosure 5. Example of hardware configuration

[0013] (1. Embodiments) (1-1. Overview of the acoustic calculation processing according to the embodiment) First, an overview of the acoustic calculation processing according to the embodiment will be described using Figure 1. Figure 1 is a diagram showing an overview of the acoustic calculation system 1 according to the embodiment.

[0014] As shown in Figure 1, the acoustic calculation system 1 includes an acoustic calculation device 100 and an LED wall 400 installed in the shooting space 300.

[0015] The various information processing operations according to the embodiment are performed by the acoustic computing device 100 shown in Figure 1. The acoustic computing device 100 is an example of an acoustic computing device according to this disclosure and is an information processing terminal used by a creator 200 who produces video content using virtual production. For example, the acoustic computing device 100 is a PC (Personal Computer), a server device, a tablet terminal, etc.

[0016] The acoustic calculation device 100 has output units such as a display and speakers, and outputs various information to the creator 200. For example, the acoustic calculation device 100 displays the GUI (Graphical User Interface) of software related to acoustic simulation on the display. The acoustic calculation device 100 also outputs the generated acoustic signals from the speakers according to the operations instructed by the creator 200 on the GUI.

[0017] In this embodiment, the acoustic calculation device 100 calculates how the sound output from the sound source point will be reproduced as a sound at the receiving point in the virtual production, and then reproduces the calculated sound. That is, the acoustic calculation device 100 performs acoustic simulation in the virtual production to predict in advance the sounds that will be emitted at the virtual production shooting site, and to perform processing to realize the sound desired by the creator 200.

[0018] The shooting space 300 is a three-dimensional area where the creator 200 intends to design the sound, and represents the site where filming for the video content takes place. The shooting space 300 is, for example, an area that includes the LED wall 400 in the field of view during filming, and also a predetermined area within which the performers appearing in the video content can move while acting. In other words, the performers perform in the shooting space 300 in front of a virtual background displayed on the LED wall 400. The cameraman then films the performers together with the virtual background displayed on the LED wall 400, and creates the content based on the filmed footage.

[0019] Incidentally, as mentioned above, virtual productions have unique acoustic problems, such as the recording of audio with excessive reverberation due to the performer's voice echoing excessively within the LED wall 400 during filming, which can interfere with the performer's performance. Such acoustic problems often occur because the LED wall 400 acts as an acoustic reflective surface, and are particularly noticeable when the LED wall 400 has a curved shape that surrounds the performer.

[0020] This point will be illustrated using Figure 2. Figure 2 shows an example of the sound simulation results in a virtual production. In the following explanation, the side of the LED wall 400 closer to the center of the recess will be referred to as the "back side," and the side of the LED wall 400 closer to the opening will be referred to as the "front side."

[0021] As shown in Figure 2, in the shooting space 300, the performer 410 moves around the entire area surrounded by the LED wall 400 while performing. The performer 410's voice is captured using microphones 420, such as a shotgun microphone installed in the shooting space 300 or provided by the sound engineer, or a lavalier microphone attached to the performer 410's clothing.

[0022] At this time, the simulation results 302 and 306 in Figure 2 show how the sound emitted by performer 410 is observed in the shooting space 300.

[0023] Simulation result 302 shows the simulation result using sound ray tracing when the performer 410 stands near the center of the recess in the LED wall 400, i.e., towards the back, and emits sound. Sound ray tracing is a method that estimates the amount of sound energy at a predetermined location by emitting multiple sound rays from a sound source and tracking those sound rays.

[0024] In simulation result 302, the performer 410 is used as the sound source, and the sound emitted from the performer 410 is simulated as a line. In simulation result 302, it can be observed that in region 304, the sound line extends from the back to the front along the curved surface of the LED wall 400. This phenomenon is called the "Whispering Gallery," and it is a phenomenon in which even relatively quiet sounds, such as whispers, can be heard clearly from a distance because the sound propagates along the wall. It is mainly observed in arch-shaped structures.

[0025] Furthermore, simulation result 306 shows the results of a sound ray simulation when the performer 410 stands near the center of the space surrounded by the LED wall 400, where the two walls of the LED wall 400 are facing each other and approximately parallel, and emits sound.

[0026] In simulation result 306, it can be observed that in region 308, sound is reflected multiple times between the two opposing walls of the LED wall 400. In addition, in region 310, it can be observed that sound rays are concentrated near the performer 410. The phenomenon of sound being reflected multiple times between opposing walls is called "flutter echo," and it is a phenomenon in which excessive reverberation is produced by the repeated and continuous reflection of sound. It is mainly observed between walls or between the floor and ceiling. The phenomenon of sound rays being concentrated at a specific location is called "focus of sound," and it refers to a phenomenon in which spatial non-uniformity of sound is produced by the concentration of sound at a specific location.

[0027] The phenomena described above are generally considered acoustic problems because they impair sound clarity or cause excessive reverberation. While sound recording sometimes involves using two types of microphones 420, including a shotgun microphone and a lavalier microphone attached to the performer's chest, measurements have shown that even a lavalier microphone placed quite close to the performer's mouth can produce excessive reverberation if acoustic problems are present. For example, measuring the impulse response of multiple microphones with various characteristics in a virtual production studio reveals that periodic peaks (flutter echoes) in the late reflections of all microphones.

[0028] Incidentally, when excessive reverberation makes simultaneous recording of video and audio difficult, Automated Dialogue Replacement (ADR), also known as after-recording, is sometimes employed, where audio is recorded separately. However, after-recording requires securing actors on separate dates, increases costs, and extends the production period, so simultaneous recording on-site is generally preferred. For this reason, in shooting locations utilizing virtual production, after-recording is either unavoidably implemented, or measures such as placing numerous sound-absorbing materials are taken to reduce reverberation. Acoustic measurements in virtual production studios have observed that reverberation is indeed reduced by sound-absorbing materials. Therefore, currently, the shooting staff brings large quantities of sound-absorbing materials into the virtual production studio as a countermeasure, but this presents problems such as the hassle of transporting the materials and the extension of shooting time due to the placement of the sound-absorbing materials. Furthermore, because the placement of sound-absorbing materials is determined through trial and error, the effectiveness of these materials is not maximized, and numerous problems have become apparent in acoustic measures for virtual production.

[0029] Furthermore, while each virtual production studio is equipped with different video equipment such as LED walls, it has been measured that the sound field characteristics of each studio differ depending on the video equipment used. For example, the reverberation time, frequency characteristics related to reverberation, and the state of flutter echo generation vary from one virtual production studio to another. Thus, even though they are all called virtual production studios, the sound field characteristics differ from facility to facility.

[0030] To address these problems, it is conceivable to conduct indoor acoustic simulations in the field of architectural acoustics (geometric acoustic analysis (such as the sound ray method and the virtual image method (Image Source Method)) and wave acoustic analysis (such as FDTD (Finite-Difference Time-Domain Method), FEM (Finite Element Method), and BEM (Boundary Element Method)). For example, the filming staff can use such simulations to predict the sound field of a space in advance and explore appropriate placement of sound-absorbing materials. However, existing simulation tools in architectural acoustics are intended for use by architectural acoustics specialists, making it difficult for non-specialists (such as video content creators) to utilize the tools, interpret the simulation results, and determine the appropriate placement of sound-absorbing materials. Furthermore, it is known that general existing simulation tools take several minutes to tens of minutes for geometric acoustic analysis and several hours for wave acoustic analysis. Therefore, in the time-limited shooting flow of virtual production, it is not practical for the filming staff or video creators to add simulation work to the shooting process.

[0031] Therefore, as a method to solve this problem, the acoustic calculation device 100 executes the acoustic calculation method according to the embodiment. Specifically, the acoustic calculation device 100 acquires information about the LED wall 400, which is a video device that displays background images when creating video content, information about the shooting space 300, which is a three-dimensional space including the LED wall 400, and information about the sound receiving point (microphone 420) and sound output point (performer 410) in the shooting space 300. Subsequently, the acoustic calculation device 100 performs an acoustic simulation in the shooting space 300 based on the acquired information. Furthermore, the acoustic calculation device 100 visualizes the results of the acoustic simulation and displays them on the user interface.

[0032] For example, the acoustic calculation device 100 enables high-speed calculations by using different simulation methods for each piece of information to be visualized, and further provides information that can be easily understood even by non-experts by displaying the results in an easy-to-understand visual format. Specifically, the acoustic calculation device 100 is an acoustic simulation tool for creators 200 who utilize virtual production, and provides software that interactively executes simulations according to the requests of the creators 200. For example, the acoustic calculation device 100 displays on the tool's user interface a position within the LED wall 400 that allows performers 410 to perform easily and enables sound collection with reduced reverberation. Alternatively, regarding the placement of sound-absorbing materials, the acoustic calculation device 100 displays on the user interface the placement position that provides the highest sound absorption effect. Furthermore, the acoustic calculation device 100 performs a simulation including the directivity of the sound receiving point and then controls the directivity to propose sound collection that avoids reverberation. Based on the simulation results, the acoustic calculation device 100 makes the sound that is expected to be ultimately recorded audible and evaluates the echo before actual shooting. These features enable the sound processing device 100 to facilitate content creation in virtual production.

[0033] An example of the process according to the above embodiment will be explained using Figures 3 to 5. Figure 3 is Figure (1) showing an example of the simulation results according to the embodiment.

[0034] Figure 3 shows the tool screen 320, which is a user interface displayed by the acoustic calculation device 100 and viewable by the creator 200. The tool screen 320 displays the LED wall 400, a grid 321, and an echo evaluation 322, all represented as objects.

[0035] The grid 321 divides the space surrounded by the LED wall 400 into predetermined ranges and visualizes the echo evaluation values ​​at each location. The echo evaluation 322 is a color-coded display of echo strength so that the creator 200 can understand the echo situation at a glance. For example, the darker the color of the echo evaluation 322 (closer to the bottom of the echo evaluation 322 in the example of Figure 3), the stronger the echo, meaning that the reverberation time is longer. Note that the echo evaluation is not necessarily limited to reverberation time; it could also be the strength (energy amount) of the reverberation, the reverberation exhibiting frequency characteristics that are unpleasant to humans, or any other evaluation index.

[0036] In the example of grid 321 shown in Figure 3, darker grids are scattered around the rows extending from the two opposing walls of the LED wall 400 toward the center. This suggests that flutter echo is generated by the two walls, resulting in a strong reverberation, and therefore the echo evaluation is low in this area. On the other hand, near the recess of the LED wall 400, lighter grids are scattered compared to other areas. This indicates that the reverberation is relatively weak near the recess of the LED wall 400, and the echo evaluation is high in this area, meaning it is a place where performers 410 can perform easily and sound recording is easy.

[0037] Although the tool screen 320 in Figure 3 is represented as a two-dimensional plan view, such a view may also be a so-called contour map, which is a two-dimensional representation of three-dimensional information. In other words, the tool screen 320 may be a user interface that includes information about sound lines in three-dimensional space, including the height direction, and represents the grid 321 in two dimensions for visual clarity.

[0038] Figure 4 is also shown as Figure (2) in an example of the simulation results according to the embodiment. The tool screen 323 in Figure 4 displays the LED wall 400 and performer 410, which are represented as three-dimensional objects, and the virtual sound-absorbing material 324. The tool screen 323 may also display lines that simulate the reflection of sound emitted from the performer 410, which is the sound-producing point. Furthermore, the tool screen 323 may also include a field of view display 325 that shows the range (field of view) captured by the camera.

[0039] The virtual sound-absorbing material 324 is a sound-absorbing material that, based on the simulation results, is positioned at the location of the performer 410 or at the sound-receiving point where the microphone 420 is installed, in a location that is assumed to effectively reduce the effect of echo. For example, based on the simulation results, the acoustic calculation device 100 displays the virtual sound-absorbing material 324, which is an object representing the sound-absorbing material, on the tool screen 323 at a position that maximizes the sound absorption effect at the sound-receiving point. The acoustic calculation device 100 may display not just one, but multiple virtual sound-absorbing materials 324. This allows the creator 200 to quickly understand where to place the sound-absorbing material when the performer 410 performs at a predetermined location in an actual setting. The acoustic calculation device 100 may also delete or change the position of the virtual sound-absorbing material 324 if it enters the field of view display 325, so that the virtual sound-absorbing material 324 is not included in the field of view display 325.

[0040] As described above, the acoustic calculation device 100 provides the creator 200 with various acoustic information through tools related to acoustic simulation. Figure 5 shows an overview diagram of the main functions of the tools provided by the acoustic calculation device 100. Figure 5 is a diagram showing an overview of the simulation tool according to the embodiment.

[0041] Figure 5 shows examples of screen displays when the main functions of the tool according to the embodiment—"visualization of echo evaluation," "suggestion of sound-absorbing material placement," and "audibility of sound"—are performed on the tool.

[0042] Tool screen 330 corresponds to tool screen 320 shown in Figure 3, and visualizes the echo evaluation by changing the floor color for each echo evaluation in the shooting space 300, which includes the LED wall 400. When tool screen 330 is displayed, the acoustic calculation device 100 performs visualization based on the simulation results calculated by the sound ray method. Tool screen 331 represents the sound emitted from the sound source point as a line, and is displayed together with tool screen 330, for example. This allows the creator 200 to check the echo intensity at each position before and during shooting, taking into account the shape of the LED wall 400 and the influence of the shooting set placed in the shooting space 300.

[0043] Tool screen 332 corresponds to tool screen 323 shown in Figure 4, and is a display for suggesting the placement of sound-absorbing material in the shooting space 300, including the LED wall 400, at locations where reflection effects are expected to be strong. When tool screen 332 is displayed, the acoustic calculation device 100 calculates the energy and reflection position of sound emitted from a virtually placed sound-producing point (which simulates reflected sound) using the virtual image method. The virtual image method is a technique for obtaining a virtual image of the sound source as reflected sound (for example, a hypothetical sound source placed behind the wall) by assuming specular reflection (incident angle and exit angle are the same) on the wall surface, etc. Based on these simulation results, the acoustic calculation device 100 suggests to the creator 200 the placement of sound-absorbing material by displaying an object indicating the sound-absorbing material at a location where reflected sound can be effectively attenuated.

[0044] The tool screen 333 is a display that enlarges the vicinity of the performer 410 on the tool screen 332. By visually recognizing the sound-absorbing material object 334 displayed on such a screen, the creator 200 can confirm the position where the sound-absorbing material is to be arranged at the actual site. That is, the acoustic calculation device 100 can propose to the creator 200 the optimal arrangement of the sound-absorbing material that reduces unnecessary echoes according to the recording environment.

[0045] The tool screen 335 is a display for the creator 200 to confirm how the sound actually emitted from the performer 410 sounds through the microphone. On the tool screen 335, for example, characteristics such as the position of the performer 410, the position of the sound-absorbing material, the position and directivity of the virtual microphone 420, etc., tentatively determined on the tool screen 330 or the tool screen 332 are input. Then, the acoustic calculation device 100 outputs the acoustic characteristics (such as IR (Impulse Response)) in the shooting space 300 by simulation calculated by the ray method, and based on the output result, generates the sound observed at the position of the microphone 420 which is the sound receiving point. When the acoustic characteristics in the shooting space is obtained by prior measurement, etc., the pre-measured acoustic characteristics may be input on the tool screen 335 and the simulation using the input information may be executed.

[0046] The tool screen 336 is a display showing the sound observed at the position of the microphone 4 as a waveform. By listening to the sound of the waveform displayed on the tool screen 336 or visually recognizing the displayed waveform, the creator 200 can confirm the degree of acoustic interference prior to the actual recording.

[0047] In this way, the acoustic calculation device 100 provides the creator 200 with tools having functions such as visualization of echoes, proposal of arrangement of sound-absorbing materials, and audibilization of sounds. Also, in the simulations used to realize the respective functions, the acoustic calculation device 100 appropriately uses different methods such as the ray acoustics method and the virtual image method according to the respective functions. In this way, the acoustic calculation device 100 significantly reduces the simulation time by using an appropriate acoustic simulation method for each function. That is, the acoustic calculation device 100 provides a tool that can easily and quickly check and adjust the acoustics of a space even when used by a video creator or the like other than an acoustic expert. Thereby, the acoustic calculation device 100 can smoothly perform content production in virtual production.

[0048] (Procedure of Acoustic Calculation Processing According to Embodiment) Next, the procedure of the acoustic calculation processing according to the embodiment will be described with reference to FIGS. 6 to 10. FIG. 6 is a diagram (1) showing an example of the implementation procedure of the acoustic calculation processing.

[0049] The acoustic calculation processing according to the embodiment is mainly used in the pre-production (Pre-Production), which is the stage of concept planning and production preparation, and the production (Production), which is the actual shooting and recording stage, during the content production process. Specifically, the acoustic calculation device 100 receives as input information used in the shooting workflow (such as the LED wall 400 in the virtual production facility where shooting is performed, the 3D model of the shooting set, the position information of the camera, etc.). Then, the acoustic calculation device 100 executes a simulation based on the input information and interactively presents the information desired by the creator (such as the positions of the performer 410, the microphone, the sound-absorbing material, etc.). Note that the processing shown in FIGS. 6 and 7 is an example, and the acoustic calculation processing according to the embodiment does not necessarily have to be used only in the pre-production and production stages, and may be used at various stages.

[0050] Figure 6 shows an example of how the acoustic computation processing according to the embodiment is used, primarily in pre-production. In pre-production, prior to actual filming, it is decided what kind of set to use, what kind of scenes to film, and how to film them.

[0051] The arrangement example 340 shown in Figure 6 shows the state in which the creator 200 has input the shape of the LED wall 400 and the position of the shooting set within the virtual production facility. The acoustic calculation device 100 performs a simulation based on the input 3D models of the LED wall 400 and the shooting set, the position information of the shooting set, etc. The acoustic calculation device 100 then outputs a color map (echo map) 342 that visually displays where there are strong echoes or long reverberations within the space surrounded by the LED wall 400.

[0052] For example, creator 200 checks the color map 342 and infers that when performer 410 actually performs, the area near position 346 further back from the LED wall 400 will be easier to perform in, or that it will be possible to capture sound with reduced reverberation in that area. Based on the inferred information, creator 200 plans the performer 410's position and camera angles during the production stage.

[0053] In this case, the sound calculation device 100 may display a color map 344 in parallel with the color map 342, which has a different grid size and color gradation. This allows the creator 200 to arbitrarily choose whether to consider the position of the performer 410 using more detailed information or to consider it using a simpler display.

[0054] Next, using Figure 7, we will show an example of how the acoustic calculation processing according to this embodiment is used, mainly in the production stage. Figure 7 is Figure (2) showing an example of the procedure for performing acoustic calculation processing.

[0055] Creator 200 inputs into the tool the positions of performers 410, microphones 420, cameras used for filming, and sets used for filming, which were determined during the pre-production stage by referring to color maps 342 and 344. Arrangement example 350 shows the state after Creator 200 has inputted the positions of performers 410, microphones 420, and cameras 430 into the tool. Note that Creator 200 is not necessarily required to determine the positions of performers 410, etc., according to color map 342, etc., and may set performers 410, etc., in any position.

[0056] Next, the acoustic calculation device 100 performs a simulation and proposes the optimal placement of sound-absorbing materials to reduce echoes during filming and shorten the reverberation time in the studio. At this time, the acoustic calculation device 100 may propose multiple placement examples according to the placement requests of the creator 200.

[0057] For example, the acoustic calculation device 100 proposes placing sound-absorbing material at the position that best reduces the effect of reflected sound at the receiving point, when the performer 410 is the sound-emitting point, as shown in arrangement example 352. In other words, it proposes placing sound-absorbing material at the position of the reflection point where the energy of the reflected sound is estimated to be strongest. Alternatively, the acoustic calculation device 100 may propose placing sound-absorbing material at a position directly facing the performer 410 and the microphone 420, as shown in arrangement example 354. Alternatively, the acoustic calculation device 100 may propose placing sound-absorbing material so as to surround the camera from behind, as shown in arrangement example 356.

[0058] Furthermore, the acoustic calculation device 100 may display the echo evaluation 360 when sound-absorbing materials are arranged as in arrangement example 352 to arrangement example 356 as a simulation result compared with the echo evaluation 358 when no sound-absorbing materials are arranged. That is, the acoustic calculation device 100 presents the creator 200 with the simulation result of an echo map that has been further input with the 3D model of the sound-absorbing materials, including their position and size, in addition to the information that has been input so far. This allows the creator 200 to grasp at a glance the change in echo evaluation when sound-absorbing materials are arranged.

[0059] Furthermore, the acoustic processing unit 100 may accept the characteristics of the microphone 420 used for recording as input. For example, the acoustic processing unit 100 may pre-store microphone information such as the directivity of the microphone 420 in a database or the like as input to the tool. Then, when the creator 200 selects the microphone 420 to be used for recording, the acoustic processing unit 100 performs a simulation using its directivity and frequency characteristics.

[0060] For example, the acoustic processing unit 100 receives input from the creator 200 to place a highly sensitive directional (cardioid) microphone 420 in front of the microphone, as in microphone placement example 362. Alternatively, the acoustic processing unit 100 may accept input for a bidirectional microphone 420, as in microphone placement example 364, or input for a beamforming microphone 420 that exhibits extremely strong directivity (beamforming) to one side, as in microphone placement example 366.

[0061] Then, once the creator 200 has determined the characteristics of the microphone 420, the sound calculation device 100 performs a simulation and reproduces the sound recorded by the determined microphone 420. Specifically, the sound calculation device 100 simulates what kind of echo and reverberation the sound emitted from the performer 410 (sound-emitting point) will have at the microphone 420 (sound-receiving point) when the performer 410 and microphone 420 are positioned as in arrangement example 368. In this way, the creator 200 can smoothly proceed with production by listening to the sound made audible by the tool and moving the performer 410 to the appropriate position, etc.

[0062] Next, the processing steps for the shooting and recording process shown in Figures 6 and 7 will be specifically explained using flowcharts shown in Figures 8 to 11. Figure 8 is flowchart (1) showing the procedure for sound calculation processing.

[0063] First, the acoustic calculation device 100 imports 3D models of the virtual production and shooting set to be used in the simulation, in accordance with the input operations of the creator 200 (step S101). Import here means taking in data for use in information processing from a database or external device. Specifically, the acoustic calculation device 100 imports 3D models of the room shape of the virtual production studio where shooting takes place, the LED wall 400, and the shooting set. At this time, the acoustic calculation device 100 may accept pre-created 3D models, or it may accept 3D models measured by the creator 200 on site using sensors such as LiDAR (Light Detection and Ranging). Furthermore, regarding the material information of the 3D models, the acoustic calculation device 100 may use pre-set values ​​such as sound absorption coefficient, use values ​​estimated from the material, or accept individual input from the creator 200. By adding material information, the acoustic calculation device 100 can perform a more accurate simulation. Furthermore, if the acoustic computing device 100 experiences simulation delays due to an increase in the amount of information, such as the number of meshes in the 3D model, it may perform processing such as reducing the number of meshes using existing technologies (such as Edge Collapse).

[0064] Next, the sound calculation device 100 determines the shooting location according to the creator 200's specifications (step S102). Here, the shooting location refers to the position of the performer 410, for example, as shown in Figure 6. Figure 9 shows the details of the process in step S102. Figure 9 is a flowchart (2) showing the procedure for sound calculation processing.

[0065] First, the acoustic calculation device 100 accepts inputs such as sound sources and receiving points, in addition to 3D models of the virtual production and shooting set, in order to perform a simulation using the sound ray method (step S201). Then, the acoustic calculation device 100 performs a simulation based on the input information (step S202). Note that in step S202, as described later, the grid is generated and the sound-producing and receiving points placed on the grid are automatically generated, so the creator 200 may omit inputting sound sources and receiving points in step S201.

[0066] During the simulation, the acoustic calculation device 100 automatically calculates a grid from the space to be processed based on the planar shape of the studio and the LED wall 400. This grid is, for example, a rectangle divided into predetermined sizes, specifically areas of 1 meter square or 3 meters square.

[0067] Next, the acoustic calculation device 100 automatically positions the sound source and the receiving point near the center of the grid. Here, the sound source is assumed to be the performer 410, and the receiving point is assumed to be a lavalier microphone worn on the performer 410's chest. Specifically, the acoustic calculation device 100 positions the sound source at the center of the grid and positions the receiving point several centimeters to several tens of centimeters away from the sound source position in the X-axis or Y-axis direction.

[0068] The acoustic calculation device 100 performs the sound ray method for each combination of sound source and receiving point within each grid. For example, the acoustic calculation device 100 radiates sound rays onto a two-dimensional plane, calculates the initial reflections (approximately fifth-order reflections), and counts the number of sound rays at the corresponding receiving point positions. The counted sound rays correspond to the integral value of the sound energy at the receiving point. Using this method, the acoustic calculation device 100 efficiently and quickly predicts the reflected sound from the LED wall 400 causing the acoustic disturbance.

[0069] Thus, unlike simulation methods that generally assume that the sound source and the receiving point are far apart, the acoustic calculation device 100 can obtain data that focuses on "how much acoustic interference occurs at the performer's (410) position" by simulating a sound source and receiving point that are located within the same grid.

[0070] Next, the acoustic calculation device 100 presents the simulation results to the user (creator 200, filming staff, etc.) (step S203). For example, the acoustic calculation device 100 displays the result of plotting the number of sound lines at each receiving point in a color map, as shown in the color map 342 in Figure 6. The color map may be one in which the colors change continuously, or it may display different colors in stages using predetermined thresholds. For example, the acoustic calculation device 100 can present the echo evaluation to the creator 200 in an easy-to-understand manner by displaying each grid in red (strong acoustic damage), green (weak acoustic damage), and blue (almost no acoustic damage).

[0071] Furthermore, by performing the sound ray method, the reverberation time can also be determined based on the energy of the obtained sound rays. Therefore, the acoustic calculation device 100 can also use a three-stage color map to display the reverberation time. As an example, the acoustic calculation device 100 uses the reverberation time as a threshold and displays a color map in which reverberation time of 1.5 seconds or more is red, reverberation time of 1 second or more but less than 1.5 seconds is green, and reverberation time of less than 1 second is blue.

[0072] Furthermore, the acoustic calculation device 100 can accept changes to the arrangement of the shooting set on the user interface, allowing it to perform further simulations with the relocated shooting set and reflect the results. This enables the creator 200 to obtain simulation results that reflect the effects of the shooting set moved on the tool in real time. Moreover, the acoustic calculation device 100 can further improve prediction accuracy by performing simulations that radiate sound rays not only in two dimensions but also in three dimensions. Even when radiating sound rays in three dimensions, the acoustic calculation device 100 calculates a grid, similar to the two-dimensional simulation, and then places the sound source and receiving point at the center of the grid. In this case, since the receiving point is assumed to be a lavalier microphone attached to the chest of the performer 410, in addition to the X and Y coordinate shifts, the Z coordinate (height), which is the shift from the mouth of the performer 410 (the sound source position), is set to be several tens of centimeters away in the negative direction. Furthermore, by applying the simulation shown in Figure 9, the acoustic calculation device 100 can also determine the shape and arrangement of the LED wall 400, the interior shape of the virtual production studio, etc., through optimization problems before the construction of the studio.

[0073] Incidentally, as mentioned above, speed is required in field simulations. Therefore, as a useful technique for reducing computational costs, temporal and spatial interpolation can be applied to the energy histogram and impulse response obtained using the sound ray method.

[0074] First, regarding temporal interpolation of the impulse response, time-domain methods such as the FDTD method and the ray method generally increase computational cost as the analysis time increases. Therefore, computational cost can be reduced by calculating only the initial reflections and calculating the later reflections using a different algorithm. For example, the later reflections can be estimated using existing methods related to indoor building acoustics. However, these methods are not capable of accurately simulating the characteristics of a real sound field, and may not be able to reproduce acoustic problems of later reflections such as flutter echoes. For this reason, the acoustic computing device 100 may also employ methods such as using a ray method with a reduced number of sound lines to obtain the entire time impulse response while adding diffused sound (missing pulses). Furthermore, the acoustic computing device 100 may also employ methods to obtain the energy histogram and the overall waveform of the impulse response while suppressing actual computation time by predicting the later reflections from the initial reflections in the energy histogram and impulse response. For example, if the shape of the virtual production is known, the acoustic computing device 100 prepares a training set of energy histograms and impulse responses obtained when the number of sound lines is reduced and when the number of sound lines is increased, and constructs a machine learning model. Then, by using the energy histograms and impulse responses when the number of sound lines is reduced as input to the machine learning model, the acoustic computing device 100 can predict energy histograms and impulse responses with high accuracy.

[0075] Furthermore, as spatial interpolation of the impulse response, the acoustic computing device 100 may employ a method to reduce computational cost by arranging the sound receiving points so that they are not densely packed in the space to be processed, and predicting the impulse response at the points between them. Alternatively, if it is possible to measure the actual sound field in advance, the acoustic computing device 100 may provide the measured data.

[0076] Returning to Figure 8, the explanation continues. After the shooting location is determined, the sound calculation device 100 determines the positions of the performer 410, microphone 420, camera 430, etc. (step S103). For example, the sound calculation device 100 determines these positions according to the input operations of the creator 200 who refer to the color map. Alternatively, the sound calculation device 100 may automatically select the optimal positions for the performer 410, etc., based on the color map, without any operation by the creator 200. In some cases, the position of the performer 410, etc., may be uniquely determined due to shooting constraints; in this case, the sound calculation device 100 sets the performer 410, etc., in that constrained position. Also in step S103, the sound calculation device 100 may present information to help the creator 200 visualize the actual shooting scene, such as displaying the camera's field of view on the user interface, as shown in the tool screen 323 of Figure 4.

[0077] Next, the acoustic calculation device 100 determines the position of the sound-absorbing material (step S104). The process of step S104 will be explained in detail using Figure 10. Figure 10 is a flowchart (3) showing the procedure of the acoustic calculation process.

[0078] First, the acoustic calculation device 100 accepts input information such as a 3D model to be used for processing, similar to step S201 (step S301). Subsequently, the acoustic calculation device 100 calculates the position of the sound-absorbing material to be proposed to the creator 200 by processing, for example, multiple patterns of methods shown below in parallel.

[0079] As the first method, the acoustic calculation device 100 calculates the placement of sound-absorbing material based on the simulation results using the virtual image method. This method corresponds to the method shown in the placement example 352 of Figure 7. That is, the acoustic calculation device 100 calculates the reflection path and reflection points of the first or second reflected sound between the performer 410 and the microphone 420 using the virtual image method (step S302). Then, the acoustic calculation device 100 places the sound-absorbing material at the calculated reflection point locations (step S303). After that, the acoustic calculation device 100 determines whether the placed sound-absorbing material is visible within the field of view of the camera 430, and if the sound-absorbing material is visible, it deletes it (step S304).

[0080] As a second method, the sound calculation device 100 determines the placement of sound-absorbing material according to the position and orientation of the performer 410. This method corresponds to the method shown in the placement example 354 of Figure 7. That is, the sound calculation device 100 calculates the position opposite the performer 410 in the three-dimensional space to be processed and places the sound-absorbing material at the calculated position (step S305). After that, the sound calculation device 100 removes any sound-absorbing material that is visible in the field of view, similar to step S304 (step S306).

[0081] As a third method, the acoustic calculation device 100 determines the placement of sound-absorbing material according to the position and orientation of the microphone 420. This method also corresponds to the method shown in the placement example 354 of Figure 7. That is, the acoustic calculation device 100 calculates the position opposite the microphone 420 in the three-dimensional space to be processed and places the sound-absorbing material at the calculated position (step S307). After that, the acoustic calculation device 100 deletes any sound-absorbing material that is visible in the field of view, similar to step S306 (step S308).

[0082] As a fourth method, the acoustic calculation device 100 determines the placement of sound-absorbing material based on the position and orientation of the camera 430. This method corresponds to the method shown in the placement example 356 of Figure 7. For example, the acoustic calculation device 100 places the sound-absorbing material in a position that surrounds the camera 430 in the three-dimensional space to be processed (step S309).

[0083] Subsequently, the acoustic calculation device 100 draws sound-absorbing material at the positions calculated for each method in the simulation results in the shooting space 300 (step S310). At this time, the acoustic calculation device 100 may choose not to draw the results for all methods, but to display only the results for the method desired by the creator 200.

[0084] The sound calculation device 100 determines whether any dynamic changes occur during the performance, such as the performer 410 moving (step S311). If the position changes, the simulation may be repeated to match the changed position (step S311; Yes). The sound calculation device 100 continues to draw the sound-absorbing material in accordance with the change in the performer 410's position, so that instead of recommending the position of the sound-absorbing material at each point, it can present a predetermined range as a recommendation area to the creator 200. If the position of the performer 410, etc., does not change (step S311; No), the sound calculation device 100 terminates the simulation related to the sound-absorbing material.

[0085] Furthermore, when the acoustic calculation device 100 proposes sound-absorbing materials at reflection point locations obtained by the virtual image method, it can also prioritize proposing reflection point locations that are close to areas where echoes are expected to be strong or reflection point locations with high reflection energy. For example, if the creator 200 specifies the size of the sound-absorbing material, the acoustic calculation device 100 can optimize the position of the sound-absorbing material using information on its size, reflection point location, and reflection energy at that location. This allows the acoustic calculation device 100 to propose an arrangement of sound-absorbing material that greatly contributes to echo reduction, even in situations where the number of sound-absorbing materials available at the shooting location is limited or only of a certain size are available. For example, the acoustic calculation device 100 may accept input of the number and size of sound-absorbing materials that can be prepared in the studio in advance and propose an arrangement of a number of sound-absorbing materials corresponding to that number.

[0086] Returning to Figure 8, we continue the explanation. After the position of the sound-absorbing material is determined, the acoustic calculation device 100 displays the effect of the sound-absorbing material in a simulation space where the sound-absorbing material is assumed to be placed (step S105). The details of the process in step S105 are shown in Figure 11. Figure 11 is a flowchart (4) showing the procedure of the acoustic calculation process.

[0087] First, the acoustic calculation device 100 receives input of the position of the sound-absorbing material along with information such as the 3D model used for processing (step S401). Next, the acoustic calculation device 100 performs a simulation using the sound ray method in the shooting space 300 including the sound-absorbing material (step S402). When a sound ray collides with the sound-absorbing material, the acoustic calculation device 100 reflects the sound absorption in the simulation result by multiplying the corresponding sound ray by the reflectance calculated from a pre-set sound absorption coefficient. As an example, the acoustic calculation device 100 performs the simulation by giving the sound absorption coefficient at the position where the sound-absorbing material is placed a characteristic of "1" (reflectance "0"). Also, the acoustic calculation device 100 may apply the sound ray method in three dimensions instead of two dimensions, as in the example shown in Figure 9, and display the results. Furthermore, the acoustic calculation device 100 also performs a simulation using the sound ray method in the shooting space 300 without the sound-absorbing material (step S403).

[0088] The acoustic calculation device 100 then presents the simulation results to the user in a manner that allows for comparison between the case with sound-absorbing material and the case without sound-absorbing material (step S404). An example of such a display is shown in Figure 12. Figure 12 is a diagram showing an example of the user interface display.

[0089] The tool screen 380 shown in Figure 12 is an example of a user interface displaying a color map using the sound ray method when no sound-absorbing material is placed. The tool screen 381 is another example of a user interface displaying the simulation space corresponding to the tool screen 380 in three dimensions.

[0090] On the other hand, tool screen 382 is an example of a user interface that displays a color map using the sound ray method when sound-absorbing material is placed. Tool screen 383 is an example of a user interface that displays the simulation space corresponding to tool screen 382 in three dimensions. The example shown in tool screen 383 shows that sound-absorbing material 384 and sound-absorbing material 385 have been placed on a part of the LED wall 400. Furthermore, a part of the color map 386 in tool screen 383 shows that the color is lighter compared to the same position in the color map shown in tool screen 380. This indicates that the flutter echo generated by the LED wall 400 has been reduced due to the placement of sound-absorbing material 384 and sound-absorbing material 385.

[0091] The creator 200 may change the positions of the sound-absorbing materials 384 and 385 on the tool screen 383. The acoustic calculation device 100 reflects the simulation results, which have changed according to the positions of the sound-absorbing materials 384 and 385, on the tool screens 382 and 383. This allows the creator 200 to interactively check the effect of the sound-absorbing materials.

[0092] Let's return to Figure 8 and continue the explanation. Next, the acoustic calculation device 100 determines the microphone directivity in the simulation (step S106). In the example of the color map display described above, an omnidirectional lavalier microphone capable of recording sound from all directions was assumed, with the receiving point attached to the chest of the performer 410. However, in actual situations, microphones 420 with different characteristics may be used, such as a shotgun microphone with extremely strong unidirectional directivity that is pointed towards the mouth of the performer 410. For example, in actual recording, multiple microphones 420 such as lavalier microphones and shotgun microphones may be used in combination, and as a result, audio data with less unwanted noise or less excessive acoustic interference, or a mixture of both sounds, is used in the post-production stage.

[0093] Therefore, the creator 200 desires to verify the sound recorded by microphones 420 with various characteristics during the simulation. Specifically, the creator 200 can select a microphone 420 from several characteristics, such as omnidirectional, unidirectional, and narrow-directional, which are pre-registered in the database, on the tool's user interface. Alternatively, if the creator 200 knows the directional information of the microphone to be used on-site, that directional information may be entered into the tool.

[0094] Once the directivity is determined, the acoustic processing unit 100 can obtain an energy histogram for the microphone 420 with that directivity by weighting the directivity corresponding to the angle of incidence for each sound ray at each receiving point. The conversion from the energy histogram to the impulse response can be performed using various known techniques.

[0095] Furthermore, the microphone 420 placed in the simulation is not limited to one; for example, a so-called microphone array, in which multiple microphone elements are arranged linearly, circumferentially, or spherically, can also be applied. By arranging a microphone array, beamforming technology, which can arbitrarily create directivity, can be applied in the simulation. In this case, the input to the simulation includes a 3D model of the shooting space 300 and the position of the performer 410, as well as the shape of the microphone array (linear, circular, or spherical, etc.), the number of microphone elements, and positional information of the microphone array.

[0096] In this case, the acoustic calculation device 100 may, similar to step S104, calculate the coordinates and directions of the first or second reflection points by using the virtual image method on the position of the performer 410 and the position of the microphone array, and pre-calculate filter coefficients that enable null beamforming, such as capturing sound from the direction of the performer 410 without capturing reflected sound. By applying these filter coefficients to the microphone array used in the actual shooting location, the acoustic calculation device 100 can further reduce reflected sound during sound capture, in addition to the effect of the sound-absorbing material. Alternatively, the acoustic calculation device 100 may capture the performer 410's voice with the microphone array during the shooting rehearsal stage, estimate the direction of arrival of the reflected sound in advance, and form null beamforming in the direction of arrival of the reflected sound during the actual take. This allows the acoustic calculation device 100 to capture sound with reduced reverberation without requiring prior simulation.

[0097] Next, the acoustic calculation device 100 performs audibility in the simulation and evaluation of the sound reproduced in the simulation (step S107). The simulation method in step S107 may be not only the ray method, but also a hybrid method of the ray method and the virtual image method, or the radiosity method. The radiosity method is a surface-based geometric acoustic method in which the geometry to be simulated is determined, that geometry is meshed, and the contribution from that mesh is determined for combinations of sound sources and receiving points, making it suitable for real-time calculations. In this way, the acoustic calculation device 100 speeds up processing by using different methods, such as the ray method in step S102, the virtual image method in step S104, the ray method in step S105, and multiple methods including the ray method in step S107, by using the optimal method for each flow.

[0098] First, let's discuss audibility. When a simulation is performed to radiate sound rays two-dimensionally in order to increase speed and ensure interactivity, it is difficult for the acoustic computing device 100 to directly obtain the impulse response. For this reason, the acoustic computing device 100 may audibly make the sound of the imaging space 300 audible by using an energy histogram obtained from a general geometric acoustic analysis method (such as the sound ray method, virtual image method, or radiosity method) to weight a filtered random sequence sample by sample. In this case, in order to increase processing speed, the acoustic computing device 100 may employ temporal interpolation, such as calculating only the initial reflections and interpolating the later reflections, as described above. In this case, the acoustic computing device 100 may use known techniques to simulate reverberation with flutter echo by adding flutter echo to the later reverberation.

[0099] Next, let's discuss evaluation. Here, evaluation refers to determining whether the sound reproduced in the simulation is of good quality for content creation, as it represents the quality of audio recorded on-site. Whether or not something is of good quality for content creation is often measured by the workload involved in the post-production stage of sound. In other words, audio that contains a lot of unwanted reverberation or noise generally results in a high workload in post-processing and is therefore not judged to be of good quality for content creation. Furthermore, if the recorded sound includes flutter echoes due to multiple reflections from walls or extremely large early reflections, it is considered difficult to correct them through post-processing. For this reason, the sound calculation device 100 evaluates both reverberation and flutter echoes.

[0100] First, reverberation is generally evaluated using reverberation time. Reverberation time is defined as the time elapsed until the slope of the decay curve, calculated by squaring the impulse response and then performing a Schröder integral, decreases by 60 decibels. If it is difficult to observe a 60-decibel decay, for convenience, a value obtained by doubling the time elapsed for decay from -5 decibels to -35 decibels (referred to as "T30") may be used.

[0101] Various known evaluation metrics can be used for flutter echoes. For example, evaluations may include: (1) % disturbance (acoustic disturbance), which is the percentage of the echo's presence that causes interference, (2) the energy of the early reflections based on the ray method, (3) the coefficient of variation of the attenuation-removed impulse response, (4) the energy ratio of the diffuse and non-diffuse components of the late reflections, and (5) evaluations based on the opposing area and angle between the sound source receiving point position and the geometry.

[0102] Of these, (2) above is a method that determines the amount of energy (number of sound lines) of the initial reflected sound at each receiving point based on the sound line method, and evaluates the flutter echo using its absolute value, which is the same method as the algorithm in step S105. If on-site measurements are possible in advance, it is also possible to measure the impulse response at points where flutter echo occurs and points where it does not occur, and set the number of sound lines at points where flutter echo does not occur as a threshold, thereby simulating the presence or absence of flutter echo from the amount of energy (number of sound lines) of the initial reflected sound.

[0103] Furthermore, since (3) above is an index for evaluating flutter echo based on the measured impulse response, when applied to the processing according to the embodiment, the acoustic computing device 100 may add processing to simulate the measured impulse response, such as by adding diffused sound of late reflections. In addition, in this method, the probability of flutter echo occurrence is evaluated from the coefficient of variation of the attenuation-removed impulse response, but in the embodiment, in order to evaluate whether high-quality sound can be recorded in content production, the probability of flutter echo occurrence may be evaluated by conducting a subjective evaluation experiment with creators 200 or the like as subjects.

[0104] The acoustic calculation device 100 may output a three-stage result, such as no acoustic damage, almost no acoustic damage, or acoustic damage, based on a predetermined threshold, by using, for example, reverberation time or one of the evaluation indices (1) to (5) above, or a combination thereof, as an evaluation result of reverberation and flutter echo. This allows the creator 200 to check in advance how the sound will be captured, that is, how much reverberation and echo will be captured, using an audible sound and a display evaluated in three stages. For example, the evaluation indices calculated in step S107 may be presented to the creator 200 in a manner similar to the color map in "Determination of Shooting Location" in step S102.

[0105] Furthermore, the acoustic computing device 100 may, in relation to the evaluation in the methods (1) to (5) above, have the creator 200 listen to audio including flutter echo in advance and have them respond whether or not the flutter echo is acceptable. For example, the acoustic computing device 100 may present several test sound sources to the creator 200 and have them respond whether each test sound source falls under one of the following categories: "no acoustic defects," "almost no acoustic defects," or "acoustic defects present." Then, based on the evaluation values ​​of the test sound sources and the evaluation by the creator 200, the acoustic computing device 100 associates "what kind of evaluation the creator 200 will give" at what level of reverberation or flutter echo occurs. In this way, the acoustic computing device 100 can set thresholds for reverberation and flutter echo evaluation not only to the specified values ​​of the evaluation method, but also as desired by the creator 200 who actually uses the tool.

[0106] (1-3. Configuration of the Acoustic Calculation System According to the Embodiment) Next, the configuration of the acoustic calculation device 100 and the configuration of the sound collection device 500, which is a device for performing adjustments related to sound collection in the acoustic calculation system 1, will be described using Figure 13. Figure 13 is a diagram showing an example of the configuration of the acoustic calculation device 100 and the sound collection device 500.

[0107] As shown in Figure 13, the acoustic calculation device 100 includes a communication unit 110, a storage unit 120, a control unit 130, and an output unit 140. The acoustic calculation device 100 may also have input means (for example, a touch panel, a keyboard, a pointing device such as a mouse, a microphone for voice input, a camera for image input (gaze input, gesture input)) to acquire various operation inputs from a creator 200 or the like who operates the acoustic calculation device 100.

[0108] The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to a network N (cloud, internet, local area network, NFC (Near Field Communication), Bluetooth®, etc.) by wire or wireless connection and transmits and receives information (data) with other information devices via the network N.

[0109] The memory unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. The memory unit 120 stores various data, such as a database of the characteristics of the microphone 420, recorded audio data, shape data and 3D models of the LED wall 400, and preset sound absorption coefficient settings.

[0110] The output unit 140 outputs various types of information. As shown in Figure 13, the output unit 140 includes a display 150 and a speaker 160. The display 150, under the control of the display control unit 133, displays the shooting space 300 to be processed, or displays a GUI for receiving operation input from the creator 200. The speaker 160, under the control of the analysis unit 132, outputs audio signals and the like that made audible from the calculation results.

[0111] The control unit 130 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program (for example, the acoustic calculation program according to this disclosure) stored inside the acoustic calculation device 100 using RAM or the like as a working area. The control unit 130 is also a controller and may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0112] As shown in Figure 13, the control unit 130 includes an acquisition unit 131, an analysis unit 132, and a display control unit 133, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Figure 13, and other configurations are also acceptable as long as they perform the information processing described later.

[0113] The acquisition unit 131 acquires various types of information used in the information processing according to the embodiment. For example, the acquisition unit 131 acquires information about a video device (LED wall 400, etc.) that displays background images when creating video content, information about the shooting space 300 which is the space in which the video device is installed and which is the subject of sound simulation processing, and information about the sound receiving point and sound output point in the space. The sound receiving point is, for example, a microphone 420. The sound output point (sound source) is, for example, a performer 410.

[0114] Specifically, the acquisition unit 131 acquires information regarding the shape and 3D model of the LED wall 400, the shooting space 300, and the shooting set. The acquisition unit 131 also acquires information regarding the position and orientation of the performer 410, the position and orientation of the microphone 420, and its characteristics (directivity, frequency characteristics, etc.).

[0115] The analysis unit 132 performs an acoustic simulation in the shooting space 300 based on the information acquired by the acquisition unit 131. As described above, the analysis unit 132 obtains information on the degree of acoustic interference and reflected sound in the space through acoustic simulation. At this time, the analysis unit 132 performs the simulation using a method suitable for the information to be obtained, such as the sound ray method or the virtual image method.

[0116] For example, the analysis unit 132 calculates the echo evaluation (i.e., the degree of acoustic interference) for each region in which the shooting space 300 is divided into predetermined ranges, as shown by the grid, as a result of the simulation. At this time, the analysis unit 132 calculates the echo evaluation assuming that a performer 410 and a microphone 420 are present in each grid. In this way, the analysis unit 132 can perform a simulation that is adapted to the phenomenon unique to shooting, where the sound source and the receiving point are extremely close together.

[0117] Furthermore, the analysis unit 132 may calculate information regarding reflections, such as reflection points in the shooting space 300 of sound emitted from the sound-producing point, as a result of a simulation using the virtual image method. In this case, the analysis unit 132 may also calculate information regarding reflections corresponding to dynamic changes in the position of the performer 410 and the microphone 420. Based on the calculated results, the analysis unit 132 can determine the positions in the shooting space 300 where sound-absorbing material should be placed. In this case, the analysis unit 132 can calculate locations where stronger reflected sound is expected to occur, and then suggest to the creator 200 that sound-absorbing material be preferentially placed in those locations.

[0118] Furthermore, the analysis unit 132 may calculate for each grid the degree of acoustic damage assuming that sound-absorbing material is placed, and the degree of acoustic damage assuming that sound-absorbing material is not placed.

[0119] Furthermore, when calculating the degree of acoustic damage, the analysis unit 132 may analyze the results according to the directivity of the microphone 420. In this case, the analysis unit 132 may calculate the degree of acoustic damage in stages using the sound evaluation index described above. The analysis unit 132 may also use a standard that is subjective to the creator 200 as the sound evaluation index.

[0120] The display control unit 133 visualizes the simulation results from the analysis unit 132 and displays them on the user interface.

[0121] For example, the display control unit 133 visualizes the echo evaluation, which is the degree of acoustic interference for each grid, in stages and displays it on the user interface as a color map. At this time, the display control unit 133 may also display, as recommendations, areas that are presumed to be suitable for sound recording in video content production, or areas that are presumed to be easy for the performer 410 to act in, along with the degree of acoustic interference.

[0122] Furthermore, the display control unit 133 displays suggestions on the user interface indicating the positions where sound-absorbing materials should be placed in the shooting space 300, based on the simulation results regarding sound-absorbing materials from the analysis unit 132. As described above, the display control unit 133 can display suggestions according to various situations, such as not displaying recommendations for sound-absorbing materials in positions included in the camera's field of view, or displaying a number of recommendations corresponding to the number of sound-absorbing materials available.

[0123] Furthermore, the display control unit 133 visualizes and displays the sound evaluation results in the shooting space 300 based on the simulation results after information such as the camera position, sound-absorbing material, and microphone 420 characteristics has been input. For example, the display control unit 133 can accurately present the creator 200 with reverberation and flutter echo that are expected to occur when sound is recorded in an actual location by displaying the sound evaluation results, i.e., the degree of acoustic interference, as a color map.

[0124] Next, the sound pickup device 500 will be described. The sound pickup device 500 is used in the acoustic calculation system 1 to reflect the microphone directivity set in the acoustic calculation device 100 to the microphone 420. For example, the sound pickup device 500 is a device that combines the microphone 420 with a control unit that can freely switch the characteristics of the microphone 420 during sound pickup. The sound pickup device 500 is used, for example, when performing sound pickup according to reflected sound, such as the null beamforming described above.

[0125] The communication unit 510 corresponds to the communication unit 110 and controls communication with the acoustic processing unit 100, etc. The control unit 530 corresponds to the control unit 130 and controls various information processing. The acoustic-to-electrical conversion unit 531 converts the sound input to the microphone element into an electrical signal. The ADC (Analog-to-Digital Converter) 532 converts the analog audio signal received from the acoustic-to-electrical conversion unit 531 into a digital format.

[0126] The directional control unit 540 controls the directionality of the microphone 420, such as switching its directionality. The directional signal processing unit 533 processes the audio signal according to the directionality set by the directional control unit 540. The DAC (Digital-to-Analog Converter) 534 converts the digital signal processed by the directional signal processing unit 533 into an analog signal and outputs the analog signal as needed. The audio communication unit 535 controls the output of the wireless signal when the processed audio signal is output as a wireless signal.

[0127] (2. Modified Examples) (2-1. Example Configuration of an Acoustic Calculation System) In the above embodiment, an example was shown in which the acoustic calculation method according to the present disclosure is executed by the acoustic calculation device 100. However, the acoustic calculation method may be executed using multiple devices. That is, the acoustic calculation system 1 according to the present disclosure may be composed of multiple devices, including the acoustic calculation device 100.

[0128] For example, the sound processing device 100 may consist of an interface device for direct operation by the creator 200 and a simulation device that primarily performs calculation processing such as simulations. In this case, the simulation device may be a server installed in a studio or the like, or a cloud server operating on the cloud. In this case, the sound pickup device 500 shown in Figure 13, the interface device, and the simulation device may be connected to each other. This allows, for example, the microphone directivity calculated by the simulation device to be reflected in the sound pickup device 500, enabling the sound pickup device 500 to perform sound pickup with reduced reverberation using a microphone 420 with controlled directivity.

[0129] (2-2. Example of Simulation) In the above embodiment, the process described was illustrated using only one performer 410 as the sound-producing point, but there may be multiple sound-producing points and receiving points. For example, if there are multiple sound-producing points, the sound calculation device 100 may calculate the sound lines from each sound-producing point to the receiving point and present the creator 200 with simulation results based on the calculation results.

[0130] Furthermore, in the above embodiment, an LED wall 400 was given as an example of a video device, and an example of the simulation tool being used in a studio where the LED wall 400 is installed was shown. However, the examples of the use of the simulation tool of this disclosure are not limited to this. For example, the acoustic calculation device 100 may perform acoustic simulation in a so-called immersive experience facility called LBE (Location Based Entertainment). Specifically, by performing a simulation that includes sets placed in the LBE facility, the acoustic calculation device 100 can show the user that echo reduction is possible by changing some of the sets to sets with high sound absorption coefficients.

[0131] In this case, the video device may be a display that shows information used in LBE, and there may be more than one video device arranged in the space. Furthermore, the video device may not only be installed on the floor surface, such as the LED wall 400, but may also be placed on the wall surface or ceiling surface, or it may be arranged in a manner that it protrudes from the ceiling or wall surface and is supported by a support material. In this case, the background video may include game characters or the virtual space in the game. In other words, the background video in this disclosure may not necessarily be just a background for creating video content, but may include various contents that are displayed on the video device in the space in which the simulation is performed.

[0132] Furthermore, the simulation tool disclosed herein can be used in facilities such as general photography studios where video equipment such as the LED wall 400 is not installed. For example, the acoustic calculation device 100 can present the user with an echo map of the room by taking models of the walls and ceiling that make up the studio as input.

[0133] (3. Other Embodiments) The processes according to each of the embodiments described above may be carried out in various other forms besides those described above.

[0134] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and data including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various data shown in each figure are not limited to the data shown.

[0135] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0136] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0137] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0138] (4. Summary of the acoustic calculation method relating to this disclosure) As described above, the acoustic calculation method relating to this disclosure includes an acquisition step, an analysis step, and a display control step, all of which are performed by a computer (acoustic calculation device 100 in this embodiment). The acquisition step acquires information about a video device (e.g., an LED wall 400) that displays background images when creating video content, information about the space in which the video device is installed and which is the subject of acoustic simulation processing, and information about the sound receiving point and sound output point in that space. The analysis step performs an acoustic simulation in the space based on the acquired information. The display control step visualizes the results of the acoustic simulation and displays them on a user interface.

[0139] As described above, the acoustic calculation method described herein performs a simulation of a space including equipment specific to virtual production, such as an LED wall 400, and presents the results to the creator 200. Furthermore, the acoustic calculation method provides an interactive user interface that accepts input from the creator 200, so it is possible to immediately receive on-site information and execute the simulation. Through these processes, the acoustic calculation method provides the creator 200 with useful information related to content creation, enabling smooth content creation in virtual production.

[0140] Furthermore, the analysis process calculates the degree of acoustic interference in a space as a result of the acoustic simulation. The display control process visualizes the calculated degree of acoustic interference and displays it on the user interface. For example, the analysis process calculates the degree of acoustic interference for each region into which the space is divided into predetermined ranges as a result of the acoustic simulation. The display control process visualizes the degree of acoustic interference for each region in stages and displays it on the user interface. More specifically, the analysis process calculates the degree of acoustic interference for each region, assuming that a sound receiving point and a sound generating point exist for each region, as a result of the acoustic simulation.

[0141] Thus, the acoustic calculation method allows the creator 200 to understand the shooting environment in advance by presenting the degree of acoustic interference, such as excessive reverberation or flutter echo, in a grid within a predetermined area of ​​the space where shooting is planned. Furthermore, the acoustic calculation method performs a simulation that is in line with the shooting environment, assuming that the receiving point and the sound-producing point exist in the same predetermined area, that is, assuming a performer 410 wearing a lavalier microphone. As a result, the acoustic calculation method can perform a highly accurate simulation that is very close to the actual situation.

[0142] Furthermore, the display control process displays the degree of acoustic interference, along with a portion of the area or space that is presumed to be suitable for sound recording in video content production.

[0143] Thus, the acoustic calculation method allows creators 200 to be presented with areas where acoustic interference is minimal and where performers 410 are expected to perform easily during filming. In other words, the acoustic calculation method allows creators 200 to avoid situations during filming where reverberation makes it difficult for performers 410 to act, or where unwanted echoes are included in the sound recording.

[0144] Furthermore, the analysis process calculates information about the reflection of sound emitted from the sound source in space as a result of acoustic simulation. The display control process displays suggestions on the user interface indicating the positions in space where sound-absorbing materials should be placed, based on the reflection information.

[0145] Thus, the acoustic calculation method simulates the location where high-energy reflected sound occurs and suggests placing sound-absorbing materials at that location, essentially providing a recommendation display. In other words, the acoustic calculation method allows even inexperienced filming staff to perform tasks that would normally require the expertise of an experienced acoustics professional, such as placing sound-absorbing materials in the appropriate locations.

[0146] Furthermore, the display control step displays a suggestion on the user interface indicating the placement of sound-absorbing material in space, based on information regarding the position and orientation of the sound-producing point, or the position, orientation, and characteristics of the sound-receiving point. In addition, the acquisition step may acquire information regarding changes in the positions of the sound-receiving point and the sound-producing point that are anticipated during video content production. The analysis step calculates information regarding the reflection of sound emitted from the sound-producing point in space, based on the information regarding changes in position. The display control step displays a suggestion on the user interface indicating the range in which sound-absorbing material should be placed in space, based on the information regarding reflection.

[0147] Thus, in the acoustic calculation method, recommendations for sound-absorbing materials may be displayed according to the positions of the performer 410 and the microphone 420. This allows the creator 200 to capture sound while suppressing the influence of sound directly reflected by the performer 410 and the microphone 420.

[0148] Furthermore, the acquisition process acquires information about the shooting equipment (e.g., camera 430) used in the production of video content. The display control process displays suggestions on the user interface indicating the positions in which sound-absorbing materials should be placed in the space, based on the position and orientation of the shooting equipment.

[0149] Thus, the acoustic calculation method may also include a recommendation display that places sound-absorbing material around the camera 430. This allows the creator 200 to place the sound-absorbing material in a way that does not affect the captured video.

[0150] Furthermore, the acquisition process acquires information regarding the field of view, which is the range in which the camera captures images. The display control process removes the proposed sound-absorbing material from the display if its position is included within the field of view in the space.

[0151] Thus, in this acoustic calculation method, by performing simulations using information including the field of view, even if it is desirable to place sound-absorbing material in a certain location, if that location is included in the field of view, it can be excluded from the recommendation display. This allows the creator 200 to place sound-absorbing material in a way that does not affect the captured video.

[0152] Furthermore, the analysis process calculates the degree of acoustic damage assuming that sound-absorbing material is placed in the acoustic simulation. The display control process can display both the result of the degree of acoustic damage assuming that sound-absorbing material is placed and the result of the degree of acoustic damage assuming that the sound-absorbing material is not placed on the user interface.

[0153] Thus, the acoustic calculation method can show the creator 200 how the acoustics in the space would change if sound-absorbing material were placed, making it easy for the creator 200 to decide where and how much sound-absorbing material to place.

[0154] Furthermore, the acquisition process obtains information regarding the number or size of sound-absorbing materials available for use in video content production. The analysis process calculates the impact of each sound-absorbing material on the degree of acoustic damage, assuming that multiple sound-absorbing materials are placed in the acoustic simulation. The display control process prioritizes displaying proposals for sound-absorbing materials that meet the conditions for the number or size of sound-absorbing materials available for use in video content production and are presumed to have a significant impact on the degree of acoustic damage.

[0155] Thus, the acoustic calculation method performs simulations that correspond to the number of sound-absorbing materials available on-site, providing creator 200 with useful information that does not deviate from the actual on-site conditions.

[0156] Furthermore, the acquisition process acquires the position and directivity of the receiving point. The analysis process analyzes the results of the acoustic simulation, in which the sound emitted from the sound source is measured at the receiving point according to the directivity. The display control process visualizes the results of the sound emitted from the sound source is measured at the receiving point according to the directivity and displays them on the user interface.

[0157] Thus, the acoustic calculation method allows for the execution of a simulation that mimics the actual situation when performers 410 and microphones 420 are positioned and sound recording is performed on-site, and the results can be presented to the creator 200. This allows the creator 200 to check what kind of sounds can be captured prior to the actual recording and make adjustments accordingly, thereby facilitating the smooth progress of content production.

[0158] Furthermore, the display control process makes the results of measuring the sound emitted from the sound-emitting point at the sound-receiving point according to the directionality audible, and outputs the audible sound via the user interface.

[0159] In this way, the acoustic calculation method can simulate and reproduce the sound that is expected to be actually captured. This allows the creator 200 to make various decisions before shooting, such as changing the microphone 420 used on site, adjusting its directivity, or changing the position of the performer 410.

[0160] Furthermore, the analysis process evaluates the results of measuring the sound emitted from the point of origin at the receiving point according to its directivity, using an index that indicates acoustic damage. The display control process visualizes the evaluation results and displays them on the user interface. For example, the analysis process evaluates acoustic damage using evaluation criteria that have been previously received from the user.

[0161] Thus, the acoustic calculation method presents the creator 200 with the results of evaluating the effects of reverberation and flutter echo in a form similar to a color map, making it easy for the creator 200 to understand the sound recording situation. Furthermore, by adopting the creator 200's subjective evaluation as the evaluation criterion, the creator 200 can appropriately determine whether or not the acoustic damage is within their acceptable range.

[0162] Furthermore, the analysis process may employ different methods for acoustic simulation related to the degree of acoustic damage and acoustic simulation related to reflection. For example, the analysis process may use the ray method for acoustic simulation related to the degree of acoustic damage and the virtual image method for acoustic simulation related to reflection.

[0163] Thus, because the acoustic calculation method uses different calculation techniques depending on the type of simulation, the calculation process can be sped up. In other words, the acoustic calculation method can improve the usability of the tool in the field.

[0164] (5. Hardware Configuration Examples) The information processing devices that constitute the acoustic calculation system 1 according to each embodiment described above are realized by a computer 1000 having a configuration such as that shown in Figure 14. Hereinafter, the acoustic calculation device 100 will be used as an example. Figure 14 is a hardware configuration diagram showing an example of a computer that realizes the functions of the acoustic calculation device 100. The computer 1000 has a processing circuitry 1100, RAM 1200, ROM 1300, secondary storage device 1400, communication interface 1500, input / output interface 1600, display unit 1700, camera unit 1800, microphone 1900, and speaker 2000. The parts of the computer 1000 are connected by a bus 1050.

[0165] The processing circuit 1100 operates based on a program stored in the ROM 1300 or secondary storage device 1400, and controls each part. For example, the processing circuit 1100 loads the program stored in the ROM 1300 or secondary storage device 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0166] ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that are executed by the processing circuit 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0167] The secondary storage device 1400 is a computer-readable recording medium that non-temporarily stores programs executed by the processing circuit 1100 and data used by such programs. Specifically, the secondary storage device 1400 is a recording medium that stores programs for each process of the acoustic processing device 100 according to each embodiment, which are examples of program data 1450.

[0168] The communication interface 1500 is an interface for the computer 1000 to connect to the external network 1550. The communication interface 1500 corresponds to the communication unit 110 of the acoustic processing unit 100. For example, the processing circuit 1100 receives data from other devices or transmits data generated by the processing circuit 1100 to other devices via the communication interface 1500.

[0169] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the processing circuit 1100 receives data from input devices such as a microphone 1900 or a touch panel via the input / output interface 1600. The processing circuit 1100 also transmits data to output devices such as a display unit 1700 or a speaker 2000 via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0170] The display unit 1700 is an interface for displaying information processed by the computer 1000. The display unit 1700 is, for example, a liquid crystal display or an organic electroluminescent display (Organic Electro Luminescence Display). Alternatively, the display unit 1700 may be a touch panel display device or an image projection device.

[0171] The camera unit 1800 is an interface for the computer 1000 to capture images. The microphone 1900 is an interface for the computer 1000 to capture sound. The speaker 2000 is an interface for the computer 1000 to output the processed sound. Each interface does not necessarily have to be located inside the computer 1000, and may be located outside the computer 1000 via a network or the like. Furthermore, each part of the computer 1000 may be controlled by a circuit different from the processing circuit 1100. For example, the display unit 1700 may be controlled not by the processing circuit 1100, but by a circuit dedicated to display processing that is provided within the display unit 1700.

[0172] For example, when the computer 1000 functions as an acoustic calculation device 100 according to each embodiment, the processing circuit 1100 of the computer 1000 functions as a control unit 130 by executing a program loaded onto the RAM 1200. The secondary storage device 1400 stores the acoustic calculation program according to this disclosure and various data stored in the storage unit 120. The processing circuit 1100 reads and executes the program data 1450 from the secondary storage device 1400, but as another example, these programs may be obtained from other devices via an external network 1550. In other words, the secondary storage device 1400 is not limited to being inside the computer 1000, but may be located outside the computer 1000. The processing circuit 1100 is an example of an integrated circuit, and CPU, MPU, GPU, APU, ASIC, and FPGA can all be considered integrated circuits.

[0173] Furthermore, the effects described herein are merely illustrative and not limiting, and other effects may also occur.

[0174] Furthermore, this technology can also be configured as follows: (1) An acoustic calculation method comprising: an acquisition step in which a computer acquires information relating to a video device that displays background images when creating video content, information relating to the space in which the video device is installed and which is subject to acoustic simulation processing, and information relating to the sound receiving point and sound output point in the space; an analysis step in which an acoustic simulation is performed in the space based on the acquired information; and a display control step in which the results of the acoustic simulation are visualized and displayed on a user interface. (2) The acoustic calculation method according to (1), wherein the analysis step calculates the degree of acoustic damage in the space as a result of the acoustic simulation, and the display control step visualizes the calculated degree of acoustic damage and displays it on the user interface. (3) The acoustic calculation method according to (2), wherein the analysis step calculates the degree of acoustic damage for each region into predetermined ranges of the space as a result of the acoustic simulation, and the display control step visualizes the degree of acoustic damage for each region in stages and displays it on the user interface. (4) The acoustic calculation method according to (3), wherein the analysis step calculates the degree of acoustic interference for each region, assuming that the sound receiving point and sound source point exist for each region as a result of the acoustic simulation. (5) The acoustic calculation method according to (3) or (4), wherein the display control step displays the degree of acoustic interference along with a portion of the region or space that is presumed to be suitable for sound collection in the video content production. (6) The acoustic calculation method according to any one of (2) to (5), wherein the analysis step calculates information regarding the reflection of sound emitted from the sound source point in the space as a result of the acoustic simulation, and the display control step displays a suggestion on the user interface indicating the position in which sound-absorbing material should be placed in the space based on the information regarding the reflection.(7) The acoustic calculation method according to any one of (2) to (6), wherein the display control step displays on the user interface a suggestion indicating the location of sound-absorbing material in the space based on information regarding the position and orientation of the sound-producing point, or the position, orientation and characteristics of the sound-receiving point. (8) The acoustic calculation method according to (7), wherein the acquisition step acquires information regarding changes in the positions of the sound-receiving point and the sound-producing point that are expected when producing the video content; the analysis step calculates information regarding the reflection of sound emitted from the sound-producing point in the space based on the information regarding changes in position; and the display control step displays on the user interface a suggestion indicating the range in which sound-absorbing material is to be placed in the space based on the information regarding reflection. (9) The acoustic calculation method according to any one of (2) to (8), wherein the acquisition step acquires information regarding a shooting device used when producing the video content; and the display control step displays on the user interface a suggestion indicating the location of sound-absorbing material in the space based on the position and orientation of the shooting device. (10) The acoustic calculation method according to (9), wherein the acquisition step acquires information relating to the field of view, which is the range in which the shooting device shoots images, and the display control step removes the proposed sound-absorbing material from the display if the position of the proposed sound-absorbing material in the space is included in the field of view. (11) The acoustic calculation method according to any one of (6) to (10), wherein the analysis step calculates the degree of acoustic damage assuming that the sound-absorbing material is placed in the acoustic simulation, and the display control step displays both the result of the degree of acoustic damage assuming that the sound-absorbing material is placed and the result of the degree of acoustic damage assuming that the sound-absorbing material is not placed on the user interface.(12) The acoustic calculation method according to (11), wherein the acquisition step acquires information regarding the number or size of sound-absorbing materials available for use in producing the video content; the analysis step calculates the effect of each sound-absorbing material on the degree of acoustic damage, assuming that a plurality of sound-absorbing materials are arranged in the acoustic simulation; and the display control step preferentially displays suggestions for sound-absorbing materials that satisfy the conditions regarding the number or size of sound-absorbing materials available for use in producing the video content and are presumed to have a large effect on the degree of acoustic damage. (13) The acoustic calculation method according to any one of (2) to (12), wherein the acquisition step acquires the position and directivity of the sound receiving point; the analysis step analyzes the results of measuring the sound emitted from the sound-emitting point at the sound receiving point according to the directivity as a result of the acoustic simulation; and the display control step visualizes the results of measuring the sound emitted from the sound-emitting point at the sound receiving point according to the directivity and displays them on the user interface. (14) The acoustic calculation method according to (13), wherein the display control step makes audible the result of measuring the sound emitted from the sound-emitting point at the sound-receiving point according to the directivity, and outputs the audible sound via the user interface. (15) The acoustic calculation method according to (13) or (14), wherein the analysis step evaluates the result of measuring the sound emitted from the sound-emitting point at the sound-receiving point according to the directivity using an index indicating acoustic damage, and the display control step visualizes the result of the evaluation and displays it on the user interface. (16) The acoustic calculation method according to (15), wherein the analysis step evaluates the acoustic damage using evaluation criteria received in advance from the user, and the display control step visualizes the result of the evaluation and displays it on the user interface. (17) The acoustic calculation method according to (6), wherein the analysis step uses different methods for acoustic simulation regarding the degree of acoustic damage and acoustic simulation regarding reflection.(18) The acoustic calculation method according to (17), wherein the analysis step uses the sound ray method in the acoustic simulation relating to the degree of acoustic damage and the virtual image method in the acoustic simulation relating to reflection. (19) An acoustic calculation system comprising: an acquisition unit that acquires information relating to a video device that displays background images when producing video content; information relating to a space in which the video device is installed and which is subject to acoustic simulation processing; and information relating to a sound receiving point and a sound output point in the space; an analysis unit that performs an acoustic simulation in the space based on the acquired information; and a display control unit that visualizes the results of the acoustic simulation and displays them on a user interface. (20) An acoustic calculation program that causes a computer to function as an acoustic calculation system comprising: an acquisition unit that acquires information relating to a video device that displays background images when producing video content; information relating to a space in which the video device is installed and which is subject to acoustic simulation processing; and information relating to a sound receiving point and a sound output point in the space; an analysis unit that performs an acoustic simulation in the space based on the acquired information; and a display control unit that visualizes the results of the acoustic simulation and displays them on a user interface.

[0175] 100 Acoustic calculation device 110 Communication unit 120 Storage unit 130 Control unit 131 Acquisition unit 132 Analysis unit 133 Display control unit 140 Output unit 200 Creator 300 Shooting space 400 LED wall 500 Sound collection device

Claims

1. An acoustic calculation method comprising: an acquisition step in which a computer acquires information relating to a video device that displays background images when creating video content, information relating to the space in which the video device is installed and which is subject to acoustic simulation processing, and information relating to sound receiving points and sound generating points in the space; an analysis step in which an acoustic simulation is performed in the space based on the acquired information; and a display control step in which the results of the acoustic simulation are visualized and displayed on a user interface.

2. The acoustic calculation method according to claim 1, wherein the analysis step calculates the degree of acoustic interference in the space as a result of the acoustic simulation, and the display control step visualizes the calculated degree of acoustic interference and displays it on the user interface.

3. The acoustic calculation method according to claim 2, wherein the analysis step calculates the degree of acoustic damage for each region obtained by dividing the space into predetermined ranges as a result of the acoustic simulation, and the display control step visualizes the degree of acoustic damage for each region in stages and displays it on the user interface.

4. The acoustic calculation method according to claim 3, wherein the analysis step calculates the degree of acoustic interference for each region, assuming that the sound receiving point and sound generating point exist for each region as a result of the acoustic simulation.

5. The acoustic calculation method according to claim 3, wherein the display control step displays the degree of acoustic interference along with a portion of the area or space that is presumed to be suitable for sound collection in the production of the video content.

6. The acoustic calculation method according to claim 2, wherein the analysis step calculates information regarding the reflection of sound emitted from the sound-emitting point in the space as a result of the acoustic simulation, and the display control step displays on the user interface a suggestion indicating the position in which sound-absorbing material should be placed in the space, based on the information regarding the reflection.

7. The acoustic calculation method according to claim 2, wherein the display control step displays on the user interface a suggestion indicating the position of sound-absorbing material in the space based on information regarding the position and orientation of the sound-producing point, or the position, orientation and characteristics of the sound-receiving point.

8. The acoustic calculation method according to claim 7, wherein the acquisition step acquires information regarding changes in the positions of the sound receiving point and the sound generating point that are expected to occur during the production of the video content; the analysis step calculates information regarding the reflection of sound emitted from the sound generating point in the space based on the information regarding the changes in positions; and the display control step displays on the user interface a proposal indicating the range in which sound-absorbing material should be placed in the space based on the information regarding the reflection.

9. The acoustic calculation method according to claim 2, wherein the acquisition step acquires information about a shooting device used in the production of the video content, and the display control step displays on the user interface a suggestion indicating the position in which sound-absorbing material is to be placed in the space, based on the position and orientation of the shooting device.

10. The acoustic calculation method according to claim 9, wherein the acquisition step acquires information regarding the field of view, which is the range in which the shooting device captures images, and the display control step removes the proposed sound-absorbing material from the display if the position of the proposed sound-absorbing material in the space is included in the field of view.

11. The acoustic calculation method according to claim 6, wherein the analysis step calculates the degree of acoustic impairment assuming that the sound-absorbing material is placed in the acoustic simulation, and the display control step displays both the result of the degree of acoustic impairment assuming that the sound-absorbing material is placed and the result of the degree of acoustic impairment assuming that the sound-absorbing material is not placed on the user interface.

12. The acoustic calculation method according to claim 11, wherein the acquisition step acquires information regarding the number or size of sound-absorbing materials available for use in producing the video content; the analysis step calculates the influence of each sound-absorbing material on the degree of acoustic damage, assuming that a plurality of sound-absorbing materials are arranged in the acoustic simulation; and the display control step preferentially displays suggestions for sound-absorbing materials that satisfy the conditions regarding the number or size of sound-absorbing materials available for use in producing the video content and are presumed to have a significant influence on the degree of acoustic damage.

13. The acoustic calculation method according to claim 2, wherein the acquisition step acquires the position and directivity of the sound receiving point; the analysis step analyzes the results of measuring the sound emitted from the sound source point at the sound receiving point according to the directivity as a result of the acoustic simulation; and the display control step visualizes the results of measuring the sound emitted from the sound source point at the sound receiving point according to the directivity and displays them on the user interface.

14. The acoustic calculation method according to claim 13, wherein the display control step makes audible the result of measuring the sound emitted from the sound-emitting point at the sound-receiving point according to the directivity, and outputs the audible sound via the user interface.

15. The acoustic calculation method according to claim 13, wherein the analysis step evaluates the results of measuring the sound emitted from the sound-emitting point at the sound-receiving point according to the directivity using an index indicating acoustic damage, and the display control step visualizes the results of the evaluation and displays them on the user interface.

16. The acoustic calculation method according to claim 15, wherein the analysis step evaluates the acoustic impairment using evaluation criteria received in advance from the user, and the display control step visualizes the results of the evaluation and displays them on the user interface.

17. The acoustic calculation method according to claim 6, wherein the analysis step uses different methods for acoustic simulation regarding the degree of acoustic interference and acoustic simulation regarding reflection.

18. The acoustic calculation method according to claim 17, wherein the analysis step uses the ray method in the acoustic simulation relating to the degree of acoustic damage and the virtual image method in the acoustic simulation relating to reflection.

19. An acoustic calculation system comprising: an acquisition unit that acquires information relating to a video device that displays background images when producing video content; information relating to the space in which the video device is installed and which is subject to acoustic simulation processing; and information relating to the sound receiving point and sound output point in the space; an analysis unit that performs acoustic simulation in the space based on the acquired information; and a display control unit that visualizes the results of the acoustic simulation and displays them on a user interface.

20. An acoustic calculation program that causes a computer to function as an acoustic calculation system comprising: an acquisition unit that acquires information relating to a video device that displays background images when creating video content; information relating to the space in which the video device is installed and which is the subject of acoustic simulation processing; and information relating to the sound receiving point and sound output point in the space; an analysis unit that performs acoustic simulation in the space based on the acquired information; and a display control unit that visualizes the results of the acoustic simulation and displays them on a user interface.

Images