Information display method, information display device, and information display program

Abstract

This information display device receives conditions including room information and information relating to the positions of a speaker and a listener in the room information, calculates an acoustic characteristic at the position of the listener on the basis of the received conditions, determines the necessity of a sound amplification system on the basis of the calculation result of the acoustic characteristic and target information of a sound pressure characteristic, and displays result information relating to the determination result.

Description

Information display method, information display device, and information display program

[0001] One embodiment of the present invention relates to an information display method, an information display device, and an information display program.

[0002] Patent Document 1 discloses a sound processing device in which a level bar indicating the state of amplified sound in three levels is displayed on a screen of a display device. The sound processing device of Patent Document 1 displays a message such as "Sound quality is good. You can further increase the volume." when the sound quality of the amplified sound is high.

[0003] U.S. Patent No. 11336999 Specification

[0004] The technology of Patent Document 1 cannot recognize the necessity of sound amplification.

[0005] An object of one embodiment of the present invention is to provide an information display method that allows easy understanding of the necessity of sound amplification.

[0006] An information display device according to one embodiment of the present invention receives conditions including room information and information on positions of a speaker and a listener in the room information, calculates acoustic characteristics at the position of the listener based on the received conditions, determines the necessity of a sound amplification system based on the calculation result of the acoustic characteristics and target information of acoustic characteristics, and displays result information related to the determination result.

[0007] According to an information processing method according to one embodiment of the present invention, a user can easily understand the necessity of sound amplification.

[0008] This is a block diagram showing the configuration of the information display device 1. This is a flowchart showing the operation of the information display method executed by the processor 12. This is a diagram showing an example of the GUI. This is a diagram showing an example of the GUI. This is a diagram showing an example of the GUI. This is a diagram showing an example of the output screen for result information. This is a diagram showing an example of the display of equipment information for the microphone (MIC) and speaker (SP). This is a diagram showing an example of the display related to Modification 1. This is a diagram showing an example of the display related to Modification 3. This is a diagram showing an example of the display related to Modification 4. This is a diagram showing an example of the display related to other examples. This is a diagram showing an example of the display of the communication score when a loudspeaker system is not introduced. This is a diagram showing an example of the display of the communication score when a loudspeaker system is introduced. This is an example of the display of the sound pressure distribution when a loudspeaker system is not introduced. This is an example of the display of the sound pressure distribution when a loudspeaker system is introduced.

[0009] Figure 1 is a block diagram showing the configuration of the information display device 1. The information display device 1 is implemented using an information processing device such as a PC (personal computer), smartphone, set-top box, or audio receiver.

[0010] The information display device 1 includes a communication unit 11, a processor 12, RAM 13, flash memory 14, a display unit 15, and a user interface 16.

[0011] The communication unit 11 has wireless communication functions such as Bluetooth® or Wi-Fi®, or wired communication functions such as USB or LAN.

[0012] The display unit 15 consists of an LCD or OLED, etc. The display unit 15 displays the video output by the processor 12. Note that the display unit 15 does not necessarily need to be built into the information display device 1.

[0013] The user interface 16 is an example of an operating unit. The user interface 16 consists of a mouse, keyboard, or touch panel, etc. The user interface 16 receives user input. The touch panel may be stacked on the display unit 15.

[0014] The processor 12 consists of a CPU, DSP, or SoC (System on a Chip), etc. The processor 12 performs various operations by reading a program from the flash memory 14, which is a storage medium, and temporarily storing it in RAM 13. Note that the program does not need to be stored in the flash memory 14. The processor 12 may, for example, download a program from another device such as a server when necessary and temporarily store it in RAM 13.

[0015] Figure 2 is a flowchart illustrating the operation of the information display method executed by the processor 12. The processor 12 executes the information display method shown in Figure 2 using the application program read from the flash memory 14.

[0016] First, the processor 12 receives input information, including room information and information regarding the speaker's position and the listener's position within that room (S11). Figure 3 shows an example of an application program screen (GUI). The processor 12 receives room information settings from the user via an information input screen as shown in Figure 3.

[0017] The GUI shown in Figure 3 includes an acoustic space interface 101 and an acoustic space setting box 102. Users configure room information via the acoustic space interface 101 or the acoustic space setting box 102. For example, a user configures room information by entering the room's width, depth, and ceiling height into the acoustic space setting box 102. Alternatively, a user may modify the room's width, depth, and ceiling height by moving the mouse cursor on the GUI to a certain position on the acoustic space interface 101, clicking, and dragging.

[0018] Alternatively, if the information display device 1 is equipped with a camera or a sensor such as LiDAR (Light Detection and Ranging), the processor 12 may scan the shape of the room via the sensor and accept the width, depth, and ceiling height. If there is a light-transmitting material such as glass, the processor 12 can estimate the shape of the room by assuming the shape of the room is, for example, a rectangular parallelepiped, and supplementing the light-transmitting parts.

[0019] Furthermore, the room information may also include information about the material of the walls (board, glass, concrete, tiles, carpet, rock wool sound-absorbing panels, etc.). If the information display device 1 is equipped with a camera, the processor 12 may estimate the material of the walls by image recognition processing based on the image acquired by the camera. In addition to room information, the information processing device 1 may also accept conditions such as the arrangement of chairs, the arrangement of desks, usage patterns, whether or not the use of the public address system is desired, and objectives.

[0020] In this embodiment, an example of setting a three-dimensional acoustic space is shown, but two-dimensional room information viewed from a planar perspective may also be set, or one-dimensional room information along a certain direction may be set. The user may set a straight line or a curve as the one-dimensional room information. The user may set a two-dimensional plane formed by straight lines, a two-dimensional plane formed by curves, or a composite two-dimensional plane formed by straight lines and curves as the two-dimensional room information. The user may set a polyhedron formed by polygons, a prism or cone shape including curved surfaces, or a sphere as the three-dimensional room information.

[0021] Furthermore, the processor 12 displays a GUI screen as shown in Figure 4 on the display unit 15 and receives information about the position of speaker S1 in the room information and the positions of listeners L1 and L2 in the room. Speaker S1 may also receive orientation information. Alternatively, if the information display device 1 is equipped with a camera or a sensor such as LiDAR, the processor 12 may receive the position of speaker S1 and the positions of listeners L1 and L2 via the sensor. Although Figure 4 shows an example of receiving the positions of speaker S1 and listeners, the processor 12 may, for example, receive the speaker's area as information about the speaker's position, or receive the listener's area as information about the listener's position.

[0022] Furthermore, the processor 12 may receive information on the position of objects that act as acoustic obstacles in the GUI (e.g., desks, partitions, etc.), or the position and directionality of objects that act as noise sources (e.g., ventilation fans, air conditioners, or projectors, etc.). If the information display device 1 is equipped with a camera or a sensor such as LiDAR, the processor 12 may receive information on the position and orientation of objects via the sensor. For example, the processor 12 may receive information on the placement of a chair object and accept the position of the chair as the listener's position.

[0023] Note that the room information settings shown in Figures 3 and 4 are just examples, and the application program may accept acoustic space information through any interface. For example, the user may input the name of a real hall, and the application program may accept the room information settings based on the 3D CAD data corresponding to the input hall.

[0024] Next, the processor 12 calculates the sound pressure characteristics at the listener's position based on the accepted conditions (S12) and displays the calculation result of the sound pressure characteristics (S13). The processor 12 may also virtually set candidate placements of the microphone and speaker based on the accepted conditions and calculate the sound pressure characteristics or other acoustic characteristics at the listener's position. The calculation result in S13 does not necessarily need to be displayed and may be used only for internal processing. The sound pressure characteristics at the listener's position are obtained by simulation by inputting information such as the transfer function H1 of the direct transmission path from the speaker S1 position to the listeners L1 and L2 positions, the sound pressure of the speaker S1, the position of the speaker S1, the positions of the listeners L1 and L2, the position of the room walls, and the reflectivity of the walls.

[0025] Figure 5 shows an example of how the calculation results are displayed. As shown in Figure 5, the processor 12 displays the sound pressure distribution on the display unit 15 as an example of the sound pressure distribution of speaker S1, as an example of how the sound pressure characteristics are displayed. As an example, the processor 12 displays the distribution up to 40 dB of sound pressure from speaker S1 in the room. This allows the user to easily understand how much sound pressure speaker S1's voice reaches listeners L1 and L2, and to easily understand the need for amplification. For example, in the example in Figure 5, since the sound pressure of speaker S1 has fallen below 50 dB for both listeners L1 and L2, it can be understood that amplification is necessary.

[0026] The processor 12 then receives target information for sound pressure characteristics and determines the necessity of a sound reinforcement system based on the calculation results and the target information (S14). The processor 12 displays the target sound pressure input interface 103 on the display screen for the sound pressure characteristics calculation results, as shown in Figure 6, for example, and receives target information for sound pressure characteristics. In this example, the target sound pressure refers to the average sound pressure at listeners L1 and L2. In the example in Figure 6, the user inputs a value of 50 dB as the target sound pressure. Although this example shows an example where the sound pressure value is received from the user as target information, the target information does not necessarily have to be received from the user. For example, the processor 12 may acquire information on acoustic systems built in the past and obtain the target information from that past information. The determination of necessity does not simply mean whether or not a sound reinforcement system should be introduced, but is a concept that includes the determination of the effectiveness of obtaining the desired effect by introducing a sound reinforcement system. That is, the determination of necessity in this specification may include the determination of whether or not it is desirable to perform sound reinforcement, and the determination of whether or not the target acoustic characteristics can be achieved if sound reinforcement is performed. The processor 12 may determine the necessity and effectiveness of the public address system based on the calculation results and target information (for example, an indicator of the effectiveness of the public address system).

[0027] If the processor 12 determines that a public address system is necessary, it requests information about the equipment that constitutes the public address system, including the microphone and speaker. For example, in the example shown in Figure 6, the user has entered a value of 50 dB as the target sound pressure, but the sound pressure at listeners L1 and L2 is lower than 50 dB. Therefore, the processor 12 determines that a public address system is necessary.

[0028] Processor 12 uses a mathematical model or a trained model to obtain equipment information, including microphones and speakers, that constitute the public address system. Processor 12 uses a mathematical model or a trained model to obtain the placement distribution of microphones and speakers corresponding to the received sound pressure specificity target information. The trained model is a model that has been trained using any machine training algorithm such as DNN (Deep Neural Network), CNN (Convolutional Neural Network), or RNN (Recurrent Neural Network) to determine the relationship between the sound pressure distribution and equipment information, including microphones and speakers.

[0029] A computer that generates a trained model (e.g., an audio signal processing unit 20 or a server) acquires a large number of data sets of equipment information, room information, and sound pressure when audio equipment such as microphones and speakers are installed in a room, as part of the training phase. The server uses the acquired large number of data sets to train a predetermined model using a predetermined algorithm to establish the relationship between equipment information, room information, and sound pressure.

[0030] In the execution phase, the processor 12 receives room information and target sound pressure characteristics information from the trained model and obtains corresponding equipment information, including microphones and speakers.

[0031] The processor 12 then displays the calculated microphone and speaker device information on the display unit 15 as result information related to the judgment result (S15). After step S15, if the processor 12 determines that the public address system is necessary and effective, it may transmit the device information and sound processing parameter information to the audio equipment. The audio equipment may adjust various parameters based on the received information.

[0032] Figure 7 shows an example of the display of equipment information for a microphone (MIC) and a speaker (SP). The processor 12 displays the calculated placement of the microphone and speaker within the acoustic space set by the acoustic space interface 101. Thus, the equipment information includes information on the placement of the microphone and speaker.

[0033] Therefore, users can easily understand the need for public address systems and information about the necessary acoustic equipment. The information display device 1 does not need to determine the need for a public address system. The information display device 1 may simply accept room information and information regarding the positions of speakers and listeners in the room, calculate the acoustic characteristics at the listener's position based on the accepted conditions, and display the calculation results as shown in Figure 5. In this case, the information display device 1 outputs the calculation results as shown in Figure 5 as result information. Users may decide for themselves whether a public address system is necessary by referring to the displayed result information without inputting target information. Alternatively, users may decide for themselves what equipment is needed for a public address system by referring to the displayed result information. Or, without deciding the need for a public address system, users may instruct the information display device 1 to obtain equipment information, including microphones and speakers, that would constitute a public address system if one were implemented. In this case, the information display device 1 may obtain equipment information for a public address system based on the user's instructions and display the acoustic characteristic distribution when public address is implemented. In that case, the information display device 1 may determine, based on the obtained acoustic characteristics, whether amplification is necessary or whether sufficient amplification is possible. The present invention may be configured to simply display the calculation results and allow the user to determine the necessity, or the user may not input target information and may select equipment from the displayed results, or the user may request the presentation of equipment information and the determination may be made based on the acoustic characteristics when amplification is performed.

[0034] (Modification 1) Figure 8 shows an example of a display according to Modification 1. The processor 12 of Modification 1 calculates the appropriate equipment settings for amplification from the received room information (room acoustic conditions) and the obtained equipment information, and displays the acoustic characteristic distribution in the amplification system. For example, in the example of Figure 8, the processor 12 displays the sound pressure distribution, which integrates the direct transmission path and the amplification path, on the display 15 as an example of the acoustic characteristic distribution for speaker S1. As an example, the processor 12 displays the distribution up to 40 dB of sound pressure for speaker S1 in the room.

[0035] This allows users to easily understand the sound pressure at which speaker S1's voice reaches listeners L1 and L2 when a loudspeaker system is installed. For example, in the example in Figure 8, it can be seen that with the installation of the loudspeaker system, the sound pressure of speaker S1 reaches both listeners L1 and L2 at 50 dB or more.

[0036] (Modification 2) In Modification 2, the processor 12 obtains sound processing parameters for the public address system and sets them in the public address system. The processor 12 (or a server of other equipment, etc.) acquires a large dataset of parameters for public address systems built in the past during the training phase and trains a predetermined model. In the execution phase, the processor 12 uses the trained model to refer to the equipment information shown in Figure 7 as input and obtains parameters to be set for each piece of equipment. The processor 12 sets the obtained parameters for each piece of sound equipment.

[0037] This allows users to automate parameter settings when installing the public address system. Therefore, even users without knowledge of public address system construction can properly configure the system.

[0038] (Modification 3) Figures 9 and 10 show an example of the display according to Modification 3. In Modification 3, multiple speaker positions are set. In the example in Figures 9 and 10, a first speaker S1 and a second speaker S2 are set. The processor 12 displays the acoustic characteristic distribution (sound pressure distribution as an example) in the loudspeaker system for each speaker as a determination result.

[0039] The user selects either the first speaker S1 or the second speaker S2 on the GUI. If the user selects the first speaker S1, the processor 12 displays the sound pressure distribution for speaker S1, integrating the direct transmission path and the amplification path, on the display 15, as shown in Figure 9. If the user selects the second speaker S2, the processor 12 displays the sound pressure distribution for speaker S2, integrating the direct transmission path and the amplification path, on the display 15, as shown in Figure 10. The processor 12 may also display the first sound pressure distribution from the direct transmission path and the second sound pressure distribution integrating the direct transmission path and the amplification path simultaneously. In that case, the processor 12 may change the display method, such as changing the color, for the first and second sound pressure distributions.

[0040] This allows users to easily understand the sound pressure at which each speaker's voice reaches listeners L1 and L2. Alternatively, users can understand the effects of using a public address system compared to not using one, and to what extent the sound pressure reaches listeners L1 and L2.

[0041] (Modification 4) Figure 11 is a diagram showing an example of the display according to Modification 4. In Modification 4, the processor 12 displays an integrated acoustic characteristic distribution (for example, an integrated sound pressure distribution) which is an integrated acoustic characteristic distribution for each of the multiple speakers. In the example in Figure 11, the processor 12 displays a distribution in which the sound pressure of speaker S1 and speaker S2 is 50 dB or more. Alternatively, the processor 12 may display a distribution in which the sound pressure of speaker S1 and speaker S2 is at its minimum value (for example, 40 dB).

[0042] This allows users to easily understand how much sound pressure is reaching listeners L1 and L2 when a loudspeaker system is installed in a situation with multiple speakers.

[0043] (Modification 5) The sound signal processing device 20 according to Modification 5 obtains and displays a communication score as an example of acoustic characteristic distribution. The sound signal processing device 20 defines, for a speaker i, the sound pressure level at which the speech of the speaker i is transmitted to a listener j as LTiLj, and defines the sound pressure level at which the speech of the speaker j is transmitted to a listener i as LTjLi. For the speaker i, when both LTiLj and LTjLi exceed a threshold value Lth, the sound signal processing device 20 determines that the speaker i can mutually communicate with the listener j, and assigns a score qij=1. Alternatively, qij is not limited to 1, and for example, a value greater than 1 or a value less than 1 may be assigned according to the sound pressure level. When either LTiLj or LTjLi is below the threshold value Lth, the sound signal processing device 20 determines that it is difficult for the speaker i to mutually communicate with the listener j, and sets qij=0. Alternatively, qij is not limited to 0, and for example, a value less than 1 may be assigned according to the sound pressure level. Then, the sound signal processing device 20 obtains a communication score Sci indicating the extent to which mutual communication is possible in the whole or a partial area of a room (for example, a cover area specified by a user) according to the following equation (1).

[0044] As shown in FIGS. 14 and 15, for example, the sound signal processing device 20 displays the distribution of the communication score Sci as an example of acoustic characteristic distribution. FIG. 14 is a diagram showing a display example of a communication score when a public address system is not installed. FIG. 15 is a diagram showing a display example of a communication score when a public address system is installed. As shown in FIG. 15, a user can determine that, due to the installation of the public address system, mutual communication is possible in most regions within the cover area surrounded by the solid-line rectangle. The user can easily understand whether speech from a certain speaker reaches a certain cover area (the whole or a partial area of a room) at a sound pressure level that enables mutual communication, and can also easily understand the necessity of public address.

[0045] (Other Examples) FIGS. 12 and 13 are diagrams showing display examples according to other examples. As shown in FIG. 12, the processor 12 further displays a list of device information (list information including models, characteristics and the like of acoustic devices) on a display screen of acoustic characteristic distribution. Accordingly, the processor 12 proposes necessary acoustic devices in accordance with received room information and target information of sound pressure characteristics.

[0046] In addition, as shown in FIG. 13, the processor 12 displays a wiring diagram of a complete set of devices necessary for the entire acoustic system as a block diagram. The processor 12 may switch between the sound pressure distribution screen and the block diagram screen by accepting selection of a tab displayed at an upper part of the sound pressure distribution display screen and the block diagram display screen. In addition, the processor 12 may output a quotation for a purchase price of a complete set of devices necessary for the entire system.

[0047] In addition, the processor 12 may accept an instruction to add or delete an acoustic device from a user on the sound pressure distribution screen or the block diagram screen. In this case, in accordance with the instruction to add or delete an acoustic device, the processor 12 may change a list of part numbers and quantities of the complete set of devices necessary for the entire public address system, the block diagram, an arrangement diagram of microphones and speakers in the acoustic space, a sound pressure distribution diagram, and a quotation for the purchase price of the complete set of devices necessary for the entire system, and display information of changed results in real time.

[0048] In addition, in the present embodiment, the processor 12 illustrates sound pressure distribution as an example of acoustic characteristic distribution, but the processor 12 may also display other physical acoustic characteristics or psychological acoustic characteristics such as clarity, for example. Based on the obtained acoustic characteristics, the processor 12 may evaluate information indicating an index of whether sound can be appropriately amplified or the degree of sound amplification, and display an evaluation value.

[0049] The information display device 1 may transmit the result information to other devices. In this case, the information display device 1 does not need to display the result information on its own display 15. The receiving device may display the type and placement information of the device based on the received result information (device information). The user may configure the device based on this displayed information. Alternatively, the receiving device may use the received results (device information or acoustic characteristics results) as parameters for automatic adjustment to configure the device. That is, the receiving device may use the result information to configure the device automatically or based on user instructions. The device configuration may include the mapping (logical connection) between logical channels and physical outputs. The receiving device may also display, based on the received result information, the sound pressure distribution of the direct transmission path as shown in Figure 5, or the sound pressure distribution integrating the direct transmission path and amplification path as shown in Figure 8. Alternatively, the receiving device may perform the processing shown in the various modifications described above according to the received result information. The receiving device may measure the impulse response and estimate the distance and direction between the speaker and the microphone. The receiving device may compare the estimated distance and direction with the distance and direction obtained from the received result information, and if there is a significant difference, it may output a warning indicating a discrepancy between the design and construction. Furthermore, the receiving device may automatically determine which speaker corresponds to which channel based on the impulse response measurement results and automatically perform the logical wiring of the channels. In addition, the receiving device may determine whether the determined equipment layout matches the actually constructed equipment layout, and display a warning if there is a mismatch.

[0050] The information display device 1 or other device may accept an on / off switching operation for the public address system and switch and display the sound pressure distribution accordingly. Figure 16 is an example of the sound pressure distribution display when the public address system is not installed, and Figure 17 is an example of the sound pressure distribution display when the public address system is installed. The information display device 1 or other device displays an on / off switch button 151 for the public address system on the sound pressure distribution display screen. When the user selects "off" with the switch button 151, the information display device 1 or other device displays only the sound pressure distribution by the direct transmission path shown in Figure 16. When the user selects "on" with the switch button 151, the information display device 1 or other device displays the sound pressure distribution that integrates the direct transmission path and the public address path shown in Figure 17.

[0051] The above description of embodiments should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not by the above embodiments. Furthermore, the scope of the present invention is intended to include all modifications within the meaning and scope equivalent to the claims.

[0052] 1: Information display device, 11: Communication unit, 12: Processor, 13: RAM, 14: Flash memory, 15: Display unit, 16: User interface

Claims

1. An information display method that receives conditions such as room information and information regarding the positions of the speaker and listener in the room; calculates the acoustic characteristics at the listener's position based on the received conditions; determines the necessity of a public address system based on the calculated acoustic characteristics and target acoustic characteristics information; and displays result information related to the determination result.

2. The information display method according to claim 1, wherein, when it is determined that the loudspeaker system is necessary, the method obtains and displays equipment information including the microphone and speaker that constitute the loudspeaker system.

3. The information display method according to claim 2, wherein the equipment information includes information on the arrangement of the microphone and the speaker.

4. The information display method according to claim 1 or claim 2, which displays the acoustic characteristic distribution in the loudspeaker system.

5. The information display method according to claim 2, wherein sound processing parameters in the loudspeaker system are determined from the room information and the obtained equipment information, and a first acoustic characteristic distribution in the loudspeaker system, or a second acoustic characteristic distribution combining the transmitted sound of the loudspeaker system and the direct transmitted sound from the speaker to the listener is displayed.

6. The information display method according to claim 2, wherein sound processing parameters for the sound reinforcement system are obtained from the room information and the obtained equipment information, and set in the sound reinforcement system.

7. The information display method according to claim 1 or 2, wherein multiple speaker positions are set, and the acoustic characteristic distribution in the sound amplification system for each speaker is displayed.

8. The information display method according to claim 6, which displays an integrated acoustic characteristic distribution obtained by integrating the acoustic characteristic distributions for each of the multiple speakers.

9. The information display method according to claim 1 or 2, wherein, as an acoustic characteristic distribution, a distribution is determined that allows for mutual communication between the speaker and the listener, based on the sound pressure level transmitted from the speaker's voice to the listener and the sound pressure level transmitted from the listener to the speaker.

10. An information display device comprising: a display unit; a processor that receives conditions for room information and information regarding the positions of the speaker and listener in the room; calculates the acoustic characteristics at the listener's position based on the received conditions; determines the necessity of a public address system based on the calculated acoustic characteristics and target acoustic characteristics information; and displays result information related to the determination result.

11. An information display program that causes an information display device equipped with a display and a processor to receive conditions such as room information and information regarding the positions of speakers and listeners in the room; calculate the acoustic characteristics at the listener's position based on the received conditions; determine the necessity of a public address system based on the calculated acoustic characteristics and target acoustic characteristics information; and display result information related to the determination result.

Images