Volume display program, volume display method, information processing device

Abstract

The present invention provides a volume display program, a volume display method, and an information processing device that can visualize which speaker is outputting how much volume from multi-channel audio data. [Solution] The volume display program causes the information processing device 3 to function as a display unit 15 that displays a plurality of speakers SP representing the sound-emitting position, which are arranged in a predetermined array based on the sound-receiving position in a virtual three-dimensional space W based on audio data corresponding to multiple channels, and a volume meter M representing the volume of the sound emitted from each of the plurality of speakers SP between the sound-receiving position and each of the plurality of speakers SP.

Description

Technical Field

[0001] The present invention relates to a volume display program, a volume display method, and an information processing apparatus.

Background Art

[0002] Conventionally, as a meter for measuring the level of sound included in various video contents and the like, a meter for measuring a loudness value, which is a sound magnitude closer to human perception, has been proposed. As such a meter, for example, Patent Document 1 describes a loudness measurement apparatus capable of measuring the loudness value of a multi-channel audio signal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the loudness measurement apparatus of the above prior art, although it is described that the measured loudness value is displayed on a display unit, the display mode is not particularly described. For example, it is conceivable to use a two-dimensional meter display with the loudness value on the vertical axis and the speaker arrangement on the horizontal axis. On the other hand, in recent years, the number of channels of multi-channel audio data has been increasing in order to improve the sense of presence. In the above display, as the number of speakers increases, speakers are added to the horizontal axis, so there is a problem that it is not intuitive to understand which meter represents the loudness value of which speaker.

[0005] The present invention has been made in view of such problems, and an object thereof is to provide a volume display program, a volume display method, and an information processing apparatus capable of visualizing how much volume is output from which speaker for multi-channel audio data. [Means for solving the problem]

[0006] To achieve the above objective, the volume display program of the present invention causes the information processing device to function as a display unit that displays a plurality of first objects representing the position from which the sound is emitted, arranged in a predetermined array based on the position from which the sound is received in a virtual three-dimensional space, based on audio data corresponding to multiple channels, and second objects representing the volume of the sound emitted from each of the plurality of first objects, between the position from which the sound is received and each of the plurality of first objects.

[0007] Furthermore, in order to achieve the above objective, the volume display method of the present invention is a volume display method executed by an information processing device, and includes the steps of displaying, based on audio data corresponding to multiple channels, a plurality of first objects arranged in a predetermined arrangement with reference to the position where the sound is received in a virtual three-dimensional space, and representing the position where the sound is emitted, and second objects representing the volume of the sound emitted from each of the plurality of first objects between the position where the sound is received and each of the plurality of first objects.

[0008] Furthermore, in order to achieve the above objective, the information processing device of the present invention has a display unit that displays, based on audio data corresponding to multiple channels, a plurality of first objects arranged in a predetermined array with reference to the position where the sound is received in a virtual three-dimensional space, representing the position where the sound is emitted, and second objects representing the volume of the sound emitted from each of the plurality of first objects, between the position where the sound is received and each of the plurality of first objects. [Effects of the Invention]

[0009] According to the volume display program of the present invention, it is possible to visualize which speaker is outputting what volume level in multi-channel audio data. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram shows an example of the overall configuration of the game system according to the embodiment. [Figure 2] This is a block diagram illustrating an example of the functional configuration of an information processing device. [Figure 3] This diagram illustrates an example of how an object is displayed by the display unit. [Figure 4] This diagram illustrates an example of how the volume meters change in response to the movement of the sound source. [Figure 5] This diagram illustrates an example of how the volume meters change in response to the listener's movement. [Figure 6] This diagram illustrates an example of object display by the display unit when there are multiple sound sources. [Figure 7] This flowchart illustrates an example of a processing procedure performed by an information processing device. [Figure 8] This figure shows an example of object display by the display unit in a modified version in which the color of an object changes according to the volume. [Figure 9] This figure shows an example of object display by the display unit in a modified version in which the color of an object changes according to the frequency. [Figure 10] This figure shows an example of object display by the display unit in a modified version where the listener is not displayed. [Figure 11] This is a block diagram showing an example of the hardware configuration of an information processing device. [Modes for carrying out the invention]

[0011] Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

[0012] <1. Overall structure of the game system> An example of the overall configuration of the game system 1 according to this embodiment will be described using Figure 1. As shown in Figure 1, the game system 1 includes an information processing device 3, a server device 5, and a terminal device 7. The information processing device 3 and the server device 5 are connected in a communicative manner. The server device 5 and the terminal device 7 are connected in a communicative manner via a network NW such as the Internet, a mobile phone network, or a LAN. The network NW may be wireless or wired. In Figure 1, the server device 5 is configured as a single computer, but it may be configured as multiple computers.

[0013] The information processing device 3 receives multi-channel audio data used in games provided by the server device 5 and visualizes the audio data. The visualized audio data may be displayed on the display 3a of the information processing device 3, or on another display device. The information processing device 3 may perform debugging, for example, to check whether the sound of each channel is coming out of the speaker as expected, and correct the audio data if there are any problems. Alternatively, the information processing device 3 may generate a sound library related to the game provided by the server device 5. The information processing device 3 is, for example, a computer of a business that develops or provides games.

[0014] Server device 5 accepts access from player terminal devices 7 via the network NW and provides the game to each terminal device 7 via the network NW. Server device 5 is, for example, a server computer of a company that develops or provides games.

[0015] An input device 9 and a display device 11 are communicably connected to the terminal device 7. The player performs various operation inputs using the input device 9. The terminal device 7 downloads game data including multi-channel audio data from the server device 5. The terminal device 7 executes the game while outputting audio based on the audio data. The terminal device 7 is, for example, a stationary game machine. However, it is not limited thereto, and for example, a portable game machine having an integrated input unit, display unit, etc. may also be used. Further, in addition to game machines, for example, computer devices such as server computers, desktop computers, notebook computers, tablet computers, etc., and devices having a telephone function such as smartphones, mobile phones, phablets, etc. may also be used.

[0016] Note that the game may be provided to the terminal device 7 by the terminal device 7 reading a recording medium on which game data including audio data corresponding to multiple channels is recorded. The recording medium is, for example, an optical disk such as a CD-ROM, DVD, etc., a semiconductor memory, etc.

[0017] <2. Functional Configuration of Information Processing Device> Next, an example of the functional configuration of the information processing device 3 will be described using FIGS. 2 and FIGS. 3 to 6.

[0018] As shown in Figure 2, the information processing device 3 has an input unit 13 and a display unit 15. The input unit 13 receives audio data corresponding to multiple channels used in the game provided by the server device 5. The audio data includes the volume of each speaker, etc. The input unit 13 also acquires game setting information. The setting information includes the position information of each speaker corresponding to multiple channels, etc. The game setting information is, for example, the setting information of the game's sound library. The input unit 13 also receives game data of the game provided by the server device 5. The game data includes, for example, the position information of sound sources within the game. The input unit 13 may also receive audio data and game data, for example, frame by frame. A frame is a still image representing the smallest time unit that makes up a video of a game. For example, if the frame rate is 60 FPS (Frames Per Second), 60 still images are switched per second to render the video.

[0019] The display unit 15 displays objects for displaying volume in a virtual three-dimensional space based on audio data corresponding to multiple channels input by the input unit 13. Specifically, the display unit 15 displays multiple speakers (an example of a first object), which are objects representing the position from which sound is emitted, arranged in a predetermined array based on the position from which sound is received, and volume meters (an example of a second object), which are objects representing the volume of sound emitted from each of the multiple speakers, between the position from which sound is received and each of the multiple speakers.

[0020] The number of channels in "multiple channels" is not particularly limited as long as it is 3 or more. For example, it may be a three-dimensional arrangement such as 8.4.4 channels or 7.1.2 channels, or a two-dimensional arrangement such as 7.1 channels or 5.1 channels. The "predetermined arrangement" is an arrangement corresponding to the number of channels of the audio data input by the input unit 13. For example, if the number of channels is 8.4.4 channels, the speakers will be arranged in a three-dimensional configuration with 8 speakers on the same plane as the listener, 4 speakers on the upper surface of the listener, and 4 speakers on the lower surface of the listener. Also, if the number of channels is, for example, 7.1.2 channels, the speakers will be arranged in a three-dimensional configuration with 7 speakers and 1 woofer on the same plane as the listener, and 2 speakers on the upper surface of the listener. Also, if the number of channels is, for example, 7.1 channels, the speakers will be arranged in a two-dimensional configuration with 7 speakers and 1 woofer on the same plane as the listener. Also, if the number of channels is, for example, 5.1 channels, the speakers will be arranged in a two-dimensional configuration with 5 speakers and 1 woofer on the same plane as the listener. The position of each speaker in each arrangement may be defined, for example, by coordinate data in a Cartesian coordinate system consisting of the X, Y, and Z axes in a virtual three-dimensional space. Note that the above number of channels and arrangement are just examples, and the number of channels and arrangement may be different from those described above. For example, "a predetermined arrangement" is not limited to an arrangement corresponding to the number of channels in the audio data, but may also include arrangements where the speakers are arranged irregularly. Note that "multiple speakers" means three or more.

[0021] The display unit 15 displays the volume meter as an object that extends in the direction connecting the sound receiving position and the speaker, and has a length corresponding to the volume of the sound emitted from the speaker. That is, the volume meter expands and contracts according to the volume along the direction connecting the sound receiving position and the speaker. The display unit 15 also displays the volume meter as an object that extends from the sound receiving position toward the speaker. Furthermore, the display unit 15 displays the volume meter as an object that gradually narrows toward the sound receiving position from the speaker side. The unit of volume represented by the length of the volume meter is not particularly limited, but could be decibels, for example.

[0022] Furthermore, the display unit 15 displays a listener (an example of a third object), which is an object representing the position from which sound is received. The listener corresponds to, for example, the player character controlled by the player in a game. Therefore, it can also be said that the volume meter represents the volume that the player character receives from each speaker. The listener may also be a virtual microphone or camera placed in the game space. In addition, the listener does not necessarily have to be displayed; only the speakers and volume meter may be displayed.

[0023] Furthermore, the display unit 15 displays a sound source (an example of a fourth object), which is an object representing the location of the sound source, so that it can move relative to the sound receiving position and the speaker. The "sound source" is an object that produces sound in the game space. For example, if a character in the game speaks or takes an action that produces sound, that character is the sound source. Also, if a machine, device, item, etc., in the game produces sound, that machine, device, item, etc. Also, if an attack or skill activated by a character produces sound effects, that attack or skill, etc. The display unit 15 changes the volume meter based on the relative position of the sound source to the sound receiving position and the speaker. The display unit 15 also changes the volume meter based on the relative position of the sound receiving position to the sound source. The "change" of the volume meter is not particularly limited in form as long as the change in volume can be visually represented. For example, the size, length, color, shape, etc., of the volume meter may be changed.

[0024] Figure 3 shows an example of object display by the display unit 15 when the number of channels is 8.4.4. As shown in Figure 3, a listener L (an example of a third object), which is an object representing the position where sound is received, is displayed at approximately the center of the virtual three-dimensional space W. The shape of the listener L is not particularly limited, but in this embodiment, it is displayed as a spherical object, for example. However, it may also be an object other than the above, such as an object representing a human body. The positional information of each object in the three-dimensional space W is defined by coordinate data in a Cartesian coordinate system consisting of the X, Y, and Z axes with the listener L as the origin, for example. However, a predetermined position other than the listener L may be defined as the origin. In this embodiment, the Cartesian coordinate system is set, for example, with the X axis in the direction extending horizontally from the listener L, the Y axis in the direction extending horizontally from the listener L and perpendicular to the X axis, and the Z axis in the direction perpendicular to the X and Y axes, i.e., the direction extending in the height direction from the listener L.

[0025] Furthermore, in the virtual three-dimensional space W, 16 speakers SP1 to S16 (an example of the first object), which are objects representing the sound-emitting positions, are displayed in a predetermined arrangement corresponding to 8.4.4 channels with respect to the listener L. In the example shown in Figure 3, eight speakers SP1 to SP8 are arranged in a roughly rectangular shape surrounding the listener L in the same horizontal plane (XY plane) as the listener L. Four speakers SP9 to SP12 are arranged in a roughly rectangular shape in the horizontal plane above the listener L (positive Z-axis side). Four speakers SP13 to SP16 are arranged in a roughly rectangular shape in the horizontal plane below the listener L (negative Z-axis side). The shape of each speaker SP1 to SP16 is not particularly limited, but in this embodiment, they are displayed as, for example, rectangular parallelepiped objects. Other shapes, such as objects representing speakers, may also be used. Each speaker SP1 to SP16 is positioned so that its front faces the listener L. In this embodiment, when no distinction is made between the speakers, they are referred to as "speaker SP".

[0026] Furthermore, in the virtual 3D space W, volume meters M1 to M16 (an example of a second object) are displayed, which are objects representing the volume of sound emitted from each of the speakers SP1 to SP16. In the example shown in Figure 3, volume meter M1 displays the volume of speaker SP1, volume meter M2 displays the volume of speaker SP2, volume meter M3 displays the volume of speaker SP3, and so on, with volume meters M1 to M16 each displaying the volume of the corresponding speaker SP1 to SP16. Volume meters M1 to M16 are positioned between the listener L and each of the speakers SP1 to SP16.

[0027] The shape of the volume meters M1 to M16 is not particularly limited, but in this embodiment, they are displayed as objects that extend in the direction connecting the listener L and each speaker SP1 to SP16, and have a length corresponding to the volume of the sound emitted from each speaker SP1 to SP16. Furthermore, the volume meters M1 to M16 are displayed as objects that extend from the listener L toward each speaker SP1 to SP16. Also, the volume meters M1 to M16 are displayed as objects that gradually narrow from the speaker SP1 to SP16 side toward the listener L side. In the example shown in Figure 3, the volume meters M1 to M16 are displayed as roughly conical objects that extend radially from the listener L toward the corresponding speakers SP1 to SP16. Note that the shape of the volume meters M1 to M16 is not limited to cones; they may also be pyramidal. Additionally, the volume meter may not be displayed for speaker SP with a volume of 0, or it may be displayed with a length corresponding to a volume of 0. In this embodiment, if the individual volume meters are not distinguished, they will be referred to as "Volume Meter M" as appropriate.

[0028] Furthermore, in the virtual 3D space W, a sound source SS (an example of a fourth object) is displayed, representing the location of the sound source that generates sound from speakers SP1 to SP16. As mentioned above, the sound source SS is, for example, the character in the game when the character speaks or takes an action that produces sound. Also, if the machine, device, item, etc., in the game produces sound, the sound source SS is the machine, device, item, etc. Also, if the attack or skill activated by the character produces sound effects, the sound source SS is the attack or skill, etc. Based on the sound source location information in the acquired game data, the sound source SS is displayed in any position and direction in the 3D space W, and can be moved.

[0029] In the example shown in Figure 3, the sound source SS is located approximately at the same position as speaker SP1. In this case, the volume of speaker SP1, which is oriented in the same direction as the sound source SS, is the loudest, so in Figure 3, the length of volume meter M1 is displayed as the longest. Also, the volumes of speakers SP2, SP8, SP9, and SP13 surrounding the sound source SS are at a moderate level corresponding to their respective positions, so volume meters M2, M8, M9, and M13 are displayed as being of a moderate length. Furthermore, the volumes of speakers SP located further away from the sound source SS are at a low level corresponding to their respective positions, so the volume meters M for these speakers are displayed as being of a short length. Note that the sound source SS may be located at a different position from speaker SP.

[0030] If we consider each volume meter M1 to M16 as a vector with direction and length, the direction of the sound source SS relative to the listener L will approximately coincide with the direction of the composite vector obtained by combining all the vectors corresponding to volume meters M1 to M16. Furthermore, the position of the sound source SS will approximately coincide with the position (sound image) of the sound that the player would perceive as coming from a specific direction and distance, assuming the player is located at the position of the listener L. Note that, for example, in the case of non-directional sounds such as background music, the volume meters M1 to M16 of all speakers SP1 to SP16 will be displayed as having a uniform length.

[0031] The sound source SS moves in 3D space W based on the sound source's position information in the acquired game data. The length of each volume meter M1 to M16 changes based on the relative position of the sound source SS with respect to the listener L and speaker SP. In other words, the length of each volume meter M1 to M16 changes in accordance with the movement of the sound source SS. Basically, the closer the direction of speaker SP (direction relative to listener L) is to the direction of sound source SS, the louder the volume of speaker SP becomes, and the longer the volume meter M becomes. Also, the closer the sound source SS gets to listener L, the louder the volume (total volume of speakers SP1 to SP16) becomes, and the longer the volume meter M for each speaker SP becomes.

[0032] Figure 4 shows an example of how volume meters M1 to M16 change in response to the movement of the sound source SS. In the example shown in Figure 4, the sound source SS moves a predetermined distance from the position of speaker SP1, for example, toward the negative X-axis (arrow AR1), to a position midway between speakers SP2 and SP3. As a result, the volume of speakers SP2 and SP3, which are closer to the sound source SS, increases, so the lengths of volume meters M2 and M3 change to increase. Also, the volume of speakers SP4, SP10, SP14, etc., surrounding the sound source SS becomes a moderate volume according to their respective positions, so volume meters M4, M10, M14, etc. change to a moderate length. Furthermore, the volume of speakers SP that are further away from the sound source SS becomes a low volume according to their respective positions, so the lengths of the other volume meters M change to shorter lengths. In Figure 4, the sound source SS is shown as an example of movement within the horizontal plane (XY plane). However, in addition to or instead of movement within the horizontal plane, movement in the height direction (Z-axis direction) is also possible.

[0033] Listener L moves in 3D space W based on the position information of the player character, etc., in the acquired game data. At this time, speakers SP1 to SP16 move in accordance with the movement of listener L, maintaining their arrangement, while sound source SS does not follow the movement of listener L. Therefore, when listener L moves, the positions of listener L and speakers SP1 to SP16 on the display screen do not change, and sound source SS moves relatively in the opposite direction to the direction of listener L's movement. As a result, the length of each volume meter M1 to M16 changes based on the relative position of listener L with respect to sound source SS. In other words, the length of each volume meter M1 to M16 changes in accordance with the movement of listener L (the relative movement of sound source SS accompanying the movement of listener L).

[0034] Figure 5 shows an example of how volume meters M1 to M16 change in response to the movement of listener L. In the example shown in Figure 5, from the state shown in Figure 3, listener L moves a predetermined distance in the negative Y-axis direction (arrow AR2), causing the sound source SS to move the same distance in the opposite positive Y-axis direction (arrow AR3), moving to a position midway between speakers SP7 and SP8. As a result, the volume of speakers SP7 and SP8, which are closer to sound source SS, increases, so the lengths of volume meters M7 and M8 change to increase. Also, the volume of speakers SP6, SP12, SP16, etc., surrounding sound source SS becomes a moderate volume according to the position of each speaker, so the lengths of volume meters M6, M12, M16, etc., change to a moderate length. Furthermore, the volume of speakers SP that are further away from sound source SS becomes a low volume according to the position of each speaker, so the lengths of the other volume meters M change to a shorter length. In Figure 5, the listener L is shown as an example of movement within the horizontal plane (XY plane), but in addition to or instead of movement within the horizontal plane, it may also move in the height direction (Z-axis direction).

[0035] Only one listener L is displayed, but two or more sound sources SS may be displayed. For example, if multiple characters in the game speak or perform actions that produce sound simultaneously, multiple sound sources SS will be displayed. Figure 6 shows an example of object display by the display unit 15 when there are multiple sound sources SS. In the example shown in Figure 6, two sound sources SS1 and SS2 are displayed. Sound source SS1 is located approximately at the same position as speaker SP1, for example, and sound source SS2 is located midway between speaker SP4 and speaker SP5. In this case, each speaker SP1 to SP16 outputs a sound that is a composite of the sounds of both sound sources SS1 and SS2, but the closer a speaker SP is to sound source SS1, the larger the proportion of sound from sound source SS1, and the closer a speaker SP is to sound source SS2, the larger the proportion of sound from sound source SS2. In the example shown in Figure 6, speakers SP1, SP2, SP8, SP9, SP13, etc. are close to sound source SS1, so the lengths of volume meters M1, M2, M8, M9, M13 are relatively long, and the proportion of sound from sound source SS1 at those volumes is relatively large. Similarly, speakers SP4, SP5, SP11, SP15, etc. are close to sound source SS2, so the lengths of volume meters M4, M5, M11, M15 are relatively long, and the proportion of sound from sound source SS2 at those volumes is relatively large. On the other hand, speakers SP other than those mentioned above are located midway between sound sources SS1 and SS2, so the length of volume meter M is relatively short compared to speakers SP, and the proportion of sound from sound sources SS1 and SS2 at those volumes is closer to equal compared to speakers SP.

[0036] Furthermore, the processing in each processing unit described above is not limited to these examples of processing division, and may, for example, be processed by further subdivided processing units. In addition, the functions of each processing unit described above are implemented by the volume display program executed by the CPU 301 (see Figure 10 below), but for example, a part of it may be implemented by actual devices such as dedicated integrated circuits like ASICs (Application Specific Integrated Circuits) or FPGAs (Field Programmable Gate Arrays), or other electrical circuits.

[0037] <3. Processing Procedure by Information Processing Device> Next, an example of a processing procedure performed by the information processing device 3 will be explained using Figure 7. Note that the processing procedure shown in Figure 7 is an example of a volume display method performed by the information processing device.

[0038] In step S10, the information processing device 3 acquires game setting information provided by the server device 5 via the input unit 13.

[0039] In step S20, the information processing device 3 uses the input unit 13 to obtain coordinate data for each speaker SP corresponding to multiple channels from the game setting information acquired in step S10. The coordinate data for each speaker SP is coordinate data representing the relative position from the listener L, for example, coordinate data with the listener L as the origin.

[0040] In step S30, the information processing device 3 uses the display unit 15 to draw the listener L and multiple speakers SP in a virtual three-dimensional space W based on the coordinate data acquired in step S20.

[0041] In step S40, the information processing device 3 receives the game data of the frame to be processed from the game data provided by the server device 5 via the input unit 13.

[0042] In step S50, the information processing device 3 obtains the coordinate data of the sound source corresponding to the frame to be processed, based on the game data input in step S40, via the input unit 13. The coordinate data of the sound source is coordinate data representing the relative position from the listener L, for example, coordinate data with the listener L as the origin. Therefore, the coordinate data of the sound source obtained in step S50 changes not only according to the movement of the sound source, but also according to the movement of the listener L.

[0043] In step S60, the information processing device 3 draws the sound source SS in a virtual three-dimensional space W based on the coordinate data acquired in step S50.

[0044] In step S70, the information processing device 3 receives audio data corresponding to multiple channels corresponding to the frame to be processed via the input unit 13, and obtains the volume of each speaker SP corresponding to the frame to be processed based on the audio data.

[0045] In step S80, the information processing device 3 uses the display unit 15 to draw each volume meter M, with a length corresponding to the volume acquired in step S70, so that it extends from the listener L towards each speaker SP.

[0046] In step S90, the information processing device 3 determines whether or not to update the frame. If the information processing device 3 updates the game data frame (step S90: YES), it returns to step S40. On the other hand, if the information processing device 3 does not update the game data frame (step S90: NO), it terminates this flowchart. Cases where the frame is not updated include, for example, when the processing of all game data frames is completed or when the volume display program application is terminated.

[0047] The processing procedure described above is merely an example, and at least some of the above procedure may be deleted or modified, or other procedures may be added. Furthermore, the order of at least some of the above procedures may be changed, or multiple procedures may be combined into a single procedure.

[0048] <4. Effects of the Embodiment> As described above, in this embodiment, multiple speakers SP representing sound-emitting locations are arranged in a predetermined array in a virtual three-dimensional space W. Furthermore, a volume meter M, representing the volume of sound emitted from each speaker SP, is plotted between the sound-receiving location (listener L's position) and each of the multiple speakers SP. Because the volume meter M is positioned between the sound-receiving location and the corresponding speaker SP, it becomes intuitively easy to understand which speaker SP is outputting how much volume. Therefore, it is possible to visualize which speaker is outputting how much volume in multi-channel audio data.

[0049] Furthermore, in this embodiment, the display unit 15 displays the volume meter M as an object that extends in the direction connecting the sound receiving position (listener L's position) and the speaker SP, and has a length corresponding to the volume of the sound emitted from the speaker SP. This allows the direction in which the volume meter M extends to indicate which speaker SP is emitting sound, and the length of the volume meter M indicates the magnitude of the volume. Therefore, it is possible to visualize, in a more intuitive and easy-to-understand manner, which speaker is emitting how much sound.

[0050] Furthermore, in this embodiment, the display unit 15 displays the volume meter M as an object extending from the sound receiving position (listener L's position) toward the speaker SP. This allows the starting point of multiple volume meters M to be common to the sound receiving position (listener L's position). As a result, each volume meter M is displayed in a manner that extends radially toward the corresponding speaker SP from the sound receiving position (listener L's position), making it easier to compare the lengths of each volume meter M and determine their relative sizes. Therefore, it is possible to visualize, in a more intuitive and easy-to-understand manner, how much volume is being output from which speaker.

[0051] Furthermore, in this embodiment, the display unit 15 displays the volume meter M as an object that gradually narrows towards the position where sound is received from the speaker SP (the position of the listener L). This prevents interference between multiple volume meters M at the central position where sound is received (listener L) when they are displayed radiating outwards from the sound receiving position (listener L) towards the corresponding speaker SP. Therefore, each volume meter M can be clearly separated (distinguished) and displayed.

[0052] Furthermore, in this embodiment, a listener L representing the position where sound is received is displayed in a virtual three-dimensional space W. This makes it possible to intuitively and easily display the positional relationship between the sound receiving position and each speaker SP.

[0053] Furthermore, in this embodiment, the display unit 15 displays the sound source SS, which represents the position of the sound source, in a movable manner relative to the listener L and the speaker SP. This makes it possible to debug the audio data by referring to the positional relationship and length of the volume meters M of each speaker SP, for example, to check whether the sound is coming from the speaker SP as expected and to correct any problems. For example, if the volume meter M of the speaker SP in the direction corresponding to the sound source SS is longer than that of the other speaker SPs, it can be determined that it is normal, and if the volume meter M of the speaker SP in the direction not corresponding to the sound source SS is longer than that of the other speaker SPs, it can be determined that it is abnormal.

[0054] Furthermore, in this embodiment, the display unit 15 changes the volume meter M based on the relative position of the sound source SS with respect to the listener L and speaker SP. This makes it possible to visualize the changes in volume of each speaker SP caused by the movement of the sound source SS.

[0055] Furthermore, in this embodiment, the display unit 15 changes the volume meter M based on the relative position of the listener L with respect to the sound source SS. This makes it possible to visualize the changes in volume of each speaker SP caused by the movement of the listener L.

[0056] Furthermore, in this embodiment, the display unit 15 displays the speaker SP in an arrangement corresponding to multiple channels of audio data. This makes it possible to display the speaker SP in an arrangement corresponding to multi-channel audio data.

[0057] <5. Variations, etc.> It should be noted that the present invention is not limited to the embodiments described above, and various modifications are possible without departing from its spirit and technical concept.

[0058] (5-1. When changing the color of an object, etc., according to the volume.) In the above embodiment, the length of the volume meter M is changed according to the volume, but other display contents besides the length of the volume meter M may also be changed. For example, the display unit 15 may change at least one of the color of the volume meter M, the size of the speaker SP, and the color of the speaker SP based on the volume of the sound emitted from the speaker SP.

[0059] Figure 8 shows an example of object display by the display unit 15 in this modified example. In the example shown in Figure 8, for example, when the volume of speaker SP1 is low, the length of the volume meter M1 is short, the size of speaker SP1 is small, and the color of the volume meter M and speaker SP1 is displayed in a color corresponding to low volume (e.g., blue). On the other hand, when the volume of speaker SP1 increases, the length of the volume meter M1 increases, the size of speaker SP1 increases, and the color of the volume meter M1 and speaker SP1 changes to a color corresponding to high volume (e.g., red).

[0060] In Figure 8, the color of the volume meter M1, the size of the speaker SP1, and the color of the speaker SP1 are all changed, but at least one of these may be changed. Also, the shape of the volume meter M1 and the speaker SP1 may be changed according to the volume.

[0061] According to this modified version, changes in volume can be represented in various display modes, making it possible to visualize changes in the volume output from the speaker SP in a more intuitive and easy-to-understand manner.

[0062] (5-2. When changing the color of an object according to the frequency) For example, the object may be changed according to the frequency of the sound output from speaker SP. In this modified example, the display unit 15 changes at least one of the color of the volume meter M and the color of speaker SP based on the frequency of the sound emitted from speaker SP. For example, the display unit 15 may be color-coded based on whether or not the sound frequency exceeds a threshold. Note that "sound frequency" refers to the frequency component with the highest output (fundamental frequency) among the multiple frequency components contained in the sound.

[0063] Figure 9 shows an example of object display by the display unit 15 in this modified example. In the example shown in Figure 9, for example, two sound sources SS1 and SS2 are displayed. Sound source SS1 is located at approximately the same position as speaker SP1, and sound source SS2 is located midway between speaker SP4 and speaker SP5. In this example, the frequency of the sound generated by sound source SS1 is higher than the threshold, and the frequency of the sound generated by sound source SS2 is lower than the threshold. For example, this corresponds to a case where sound source SS1 is the voice of a female character and sound source SS2 is the voice of a male character.

[0064] In this case, each speaker SP1 to SP16 outputs a combined sound from both sound sources SS1 and SS2. However, the closer a speaker SP is to sound source SS1, the greater the proportion of sound from sound source SS1, and the closer a speaker SP is to sound source SS2, the greater the proportion of sound from sound source SS2. Therefore, since speakers SP1, SP2, SP8, SP9, SP13, etc. are closer to sound source SS1, the volume meters M1, M2, M8, M9, M13 are displayed in a color corresponding to a high frequency (e.g., red) or a similar color. On the other hand, since speakers SP4, SP5, SP11, SP15, etc. are closer to sound source SS2, the volume meters M4, M5, M11, M15 are displayed in a color corresponding to a low frequency (e.g., blue) or a similar color. Speakers SP other than those mentioned above are located in an intermediate position between sound sources SS1 and SS2, so they are displayed in an intermediate color between the colors corresponding to high frequencies and low frequencies, depending on the proportion of sound from sound sources SS1 and SS2. Intermediate colors may be represented by gradients or other methods that correspond to the proportion of each color.

[0065] Note that in Figure 9, only the color of the volume meter M is changed according to the sound frequency, but the color of the speaker SP may also be changed in the same way, in addition to or instead of the color of the volume meter M.

[0066] This modified version allows for the intuitive and easy visualization of the high and low frequencies of sound output from the speaker SP. Furthermore, if there are multiple sound sources with different frequencies, the speaker SP and volume meter M can be colored differently for each sound source. This allows for different colors for the speaker SP and volume meter M depending on the type of sound source; for example, a male character's sound source could be colored for lower frequencies, and a female character's sound source for higher frequencies.

[0067] (5-3. Others) In the above embodiment, the listener L is displayed, but as shown in Figure 10, for example, the speaker SP and volume meter M may be displayed without displaying the listener L. In this case, in step S30 of Figure 7, the speaker SP is drawn in a virtual three-dimensional space W, and in step S80, each volume meter M is drawn so as to extend from the sound receiving position (the position of the listener L) toward each speaker SP.

[0068] Furthermore, in the above embodiment, each volume meter M was displayed as an object extending from the listener L toward each speaker SP, but it may also be displayed as an object extending from each speaker SP toward the listener L.

[0069] Furthermore, in the above embodiment, the coordinate data of the sound source is acquired for each frame and the sound source SS is displayed, but the manner in which the volume is displayed is not limited to the above. For example, the volume of each speaker SP may be acquired for each frame and the volume meter M may be displayed without displaying the sound source SS.

[0070] Furthermore, in the above embodiment, the speakers SP were arranged in an array corresponding to the number of audio data channels, but the arrangement of the speakers SP is not limited to the above. For example, the speakers may be arranged based on the location information of actual speakers installed by the player. The location information of actual speakers may be input by the player, or it may be automatically acquired by speech recognition processing or image recognition processing, etc.

[0071] Furthermore, in addition to what has already been described above, the methods of the above embodiments and their respective modifications may be used in appropriate combinations. While not explicitly exemplified here, the above embodiments and their respective modifications may be implemented with various modifications, without departing from their intended purpose.

[0072] The problems and effects that the embodiments and modifications described above aim to solve are not limited to those stated above. The embodiments and modifications may solve problems not mentioned above, or produce effects not mentioned above, and may solve only some of the problems described or produce only some of the effects described.

[0073] <6. Hardware Configuration of Information Processing Equipment> Next, an example of the hardware configuration of the information processing device 3 will be described using Figure 11.

[0074] As shown in Figure 11, the information processing device 3 includes, for example, a CPU 301, a ROM 303, a RAM 305, a dedicated integrated circuit 307 built for a specific application such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), an input device 313, an output device 315, a recording device 317, a drive 319, a connection port 321, and a communication device 323. These components are connected to each other via a bus 309, an input / output interface 311, etc., enabling them to transmit signals to one another.

[0075] The volume display program can be stored in a recording device such as a ROM 303, RAM 305, or hard disk 317.

[0076] Furthermore, the volume display program can also be temporarily or permanently (non-temporarily) recorded on a removable recording medium 325, such as a magnetic disk like a flexible disk, an optical disk like various CDs, MO disks, or DVDs, or a semiconductor memory. Such a recording medium 325 can also be provided as so-called packaged software. In this case, the volume display program recorded on these recording media 325 may be read by the drive 319 and recorded on the recording device 317 via the input / output interface 311 or bus 309, etc.

[0077] Furthermore, the volume display program can also be stored on, for example, a download site, another computer, or another recording device (not shown). In this case, the volume display program is transferred via a network such as a LAN or the Internet, and the communication device 323 receives this program. The program received by the communication device 323 may then be recorded on the recording device 317 via the input / output interface 311 or bus 309, etc.

[0078] Furthermore, the volume display program can also be stored, for example, on an appropriate external device 327. In this case, the volume display program may be transferred via an appropriate connection port 321 and recorded on the recording device 317 via an input / output interface 311 or bus 309, etc.

[0079] Then, the CPU 301 executes various processes according to the program recorded in the recording device 317, thereby realizing the processing performed by the input unit 13, display unit 15, etc. In this case, the CPU 301 may, for example, directly read and execute the program from the recording device 317, or it may load it into the RAM 305 first and then execute it. Furthermore, if the CPU 301 receives a program via the communication device 323, drive 319, or connection port 321, it may execute the received program directly without recording it in the recording device 317.

[0080] Furthermore, the CPU 301 may, if necessary, perform various processes based on signals and information input from an input device 313 such as a microphone, mouse, or keyboard (not shown).

[0081] The CPU 301 then outputs the result of the above processing from the output device 315, which includes the display 3a. Furthermore, the CPU 301 may, if necessary, transmit this processing result via the communication device 323 or connection port 321, or record it on the recording device 317 or recording medium 325. [Explanation of Symbols]

[0082] 1. Game System 3. Information Processing Device 5. Server equipment 7 Terminal devices 9 Input device 11 Display device 13 Input section 15 Display section L Listener (an example of a third object) M Volume meter (an example of a second object) M1~M16 Volume meter (Example of a second object) SP Speaker (Example of the first object) SP1~SP16 Speaker (Example of the first object) SS sound source (an example of the 4th object) SS1 Sound Source (Example of the 4th object) SS2 sound source (an example of a fourth object) W Virtual 3D space

Claims

1. Information processing equipment, A display unit that displays, based on audio data corresponding to multiple channels, a plurality of first objects representing the position from which the sound is emitted, arranged in a predetermined array based on the position from which the sound is received in a virtual three-dimensional space, and a second object representing the volume of the sound emitted from each of the plurality of first objects, between the position from which the sound is received and each of the plurality of first objects. A volume display program designed to function as such.

2. The aforementioned display unit is The second object is displayed as an object that extends in the direction connecting the sound receiving position and the first object, and has a length corresponding to the volume of the sound emitted from the first object. The volume display program according to claim 1.

3. The aforementioned display unit is The second object is displayed as an object extending toward the first object, starting from the position where the sound is received. The volume display program according to claim 2.

4. The aforementioned display unit is The second object is displayed as an object that gradually narrows in shape from the first object towards the position where the sound is received. The volume display program according to claim 3.

5. The aforementioned display unit is A third object representing the position where the aforementioned sound is received is displayed. The volume display program according to claim 4.

6. The aforementioned display unit is A fourth object representing the location of the sound source, which is the source of the sound, is displayed so as to be movable relative to the position where the sound is received and the first object. A volume display program according to any one of claims 1 to 5.

7. The aforementioned display unit is The second object is changed based on the position where the sound is received and the relative position of the fourth object to the first object. The volume display program according to claim 6.

8. The aforementioned display unit is The second object is changed based on the relative position of the sound receiving position to the fourth object. The volume display program according to claim 6.

9. The aforementioned display unit is The first object is displayed in an array corresponding to the multiple channels of the audio data. A volume display program according to any one of claims 1 to 5.

10. The aforementioned display unit is Based on the volume of the sound emitted from the first object, the color of the second object, the size of the first object, and at least one of the colors of the first object are changed. A volume display program according to any one of claims 1 to 5.

11. The aforementioned display unit is Based on the frequency of the sound emitted from the first object, the color of the second object and at least one of the colors of the first object are changed. A volume display program according to any one of claims 1 to 5.

12. A method for displaying volume performed by an information processing device, A step of displaying, based on audio data corresponding to multiple channels, a plurality of first objects representing the position from which the sound is emitted, arranged in a predetermined array based on the position from which the sound is received in a virtual three-dimensional space, and second objects representing the volume of the sound emitted from each of the plurality of first objects, between the position from which the sound is received and each of the plurality of first objects, A method for displaying volume.

13. A display unit that displays, based on audio data corresponding to multiple channels, a plurality of first objects representing the position from which the sound is emitted, arranged in a predetermined array based on the position from which the sound is received in a virtual three-dimensional space, and a second object representing the volume of the sound emitted from each of the plurality of first objects, between the position from which the sound is received and each of the plurality of first objects. An information processing device having

Images