Information processing device

Abstract

To provide an information processing device that can deliver a high level of realism with minimal computational effort. [Solution] In the signal processing device 11, the coordinate transformation processing unit 22 and the object attenuation processing unit 23 determine corrected object gain information for each object to adjust the level of the object signal according to its positional relationship with other objects. The decoding processing unit 21 receives and decodes the input bitstream and outputs the object metadata and object signal. The coordinate transformation processing unit 22 also generates object spherical coordinate position information based on object position information and user position information. The object attenuation processing unit 23 also obtains corrected object gain information based on the object spherical coordinate position information and object metadata. The rendering processing unit 24 adjusts the level of the object signal based on the corrected object gain information during rendering.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, and more particularly to an information processing apparatus capable of obtaining a high sense of presence with a small amount of computation.

Background Art

[0002] Conventionally, object audio technology has been used in movies, games, etc., and encoding methods capable of handling object audio have also been developed. Specifically, for example, the MPEG (Moving Picture Experts Group)-H Part 3: 3D audio standard, which is an international standard, is known (see, for example, Non-Patent Document 1).

[0003] In such an encoding method, together with the conventional two-channel stereo method and multi-channel stereo methods such as 5.1 channels, a moving sound source or the like is treated as an independent audio object, and the position information of the object is encoded as metadata together with the signal data of the audio object.

[0004] By doing so, reproduction can be performed in various viewing environments with different numbers and arrangements of speakers. Also, it is possible to easily process the sound of a specific sound source at the time of reproduction, such as adjusting the volume of the sound of a specific sound source or adding an effect to the sound of a specific sound source, which was difficult with conventional encoding methods.

[0005] For example, in the standard of Non-Patent Document 1, a method called 3D VBAP (Vector Based Amplitude Panning) (hereinafter simply referred to as VBAP) is used for rendering processing.

[0006] This is one of the rendering techniques generally called panning, and is a method of performing rendering by distributing gains to the three speakers closest to an audio object that also exists on the spherical surface among the speakers existing on the spherical surface with the user position as the origin.

[0007] In addition to VBAP, there are also rendering processes using panning techniques, such as the Speaker-anchored coordinates panner, which distributes the gain to the x, y, and z axes respectively (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] INTERNATIONAL STANDARD ISO / IEC 23008-3 First edition 2015-10-15 Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio [Non-Patent Document 2] ETSI TS 103 448 v1.1.1(2016-09) [Overview of the Initiative] [Problems that the invention aims to solve]

[0009] Incidentally, in the rendering method described above, the object signals of multiple audio objects are rendered for each individual audio object, and the changes in sound due to the relative positions of the audio objects are not taken into account at all. Therefore, it was not possible to achieve a high level of realism during audio playback.

[0010] For example, suppose that, from the listener's perspective, sound is emitted from a second audio object behind a first audio object. In such a case, the attenuation effects of the sound from the second audio object, caused by sound reflection, diffraction, and absorption at the first audio object, are completely ignored.

[0011] Furthermore, in the rendering method described above, since the user's position is fixed, it is possible to adjust the level of the object signals in advance, for example, based on the positional relationship between the user's position and multiple audio objects.

[0012] Such level adjustments make it possible to express changes in sound due to the relative positional relationship between audio objects. Therefore, for example, by calculating the attenuation effects caused by sound reflection, diffraction, and absorption in audio objects based on physical laws, and then adjusting the level of the object signals of the audio objects in advance based on the calculation results, a high level of realism can be achieved.

[0013] However, calculating the attenuation effects caused by sound reflection, diffraction, and absorption based on physical laws would be impractical when there are many audio objects, as it would require a large amount of computation.

[0014] Furthermore, in a fixed viewpoint where the user's position is fixed, it is possible to generate object signals that take into account sound reflection and diffraction by performing level adjustments in advance. However, in a free viewpoint where the user's position can be moved, such prior level adjustments become completely meaningless.

[0015] This technology was developed in light of these circumstances, and aims to provide a high level of realism with minimal computational effort. [Means for solving the problem]

[0016] One aspect of this technology is an information processing device which includes attenuation disable information indicating whether or not to attenuate the signal of a predetermined object; a gain determination unit which determines the amount of attenuation based on the positional relationship between the predetermined object and other objects and determines the gain of the signal of the predetermined object based on the amount of attenuation; and a rendering processing unit which performs rendering processing based on the determined gain and the object signal of the predetermined object.

[0017] The information processing method or program according to one aspect of the present technology includes steps of: determining attenuation invalid information indicating whether to attenuate the signal of a predetermined object; determining an attenuation amount based on the positional relationship between the predetermined object and other objects; determining the gain of the signal of the predetermined object based on the attenuation amount; and performing rendering processing based on the determined gain and the object signal of the predetermined object.

[0018] In one aspect of the present technology, attenuation invalid information indicating whether to attenuate the signal of a predetermined object is provided, an attenuation amount is determined based on the positional relationship between the predetermined object and other objects, the gain of the signal of the predetermined object is determined based on the attenuation amount, and rendering processing is performed based on the determined gain and the object signal of the predetermined object.

Advantages of the Invention

[0019] According to one aspect of the present technology, a high sense of presence can be obtained with a small amount of computation.

[0020] Note that the effects described here are not necessarily limited, and any effect described in the present disclosure may be applicable.

Brief Description of the Drawings

[0021] [Figure 1] A diagram for explaining VBAP. [Figure 2] A diagram showing a configuration example of a signal processing device. [Figure 3] A diagram for explaining coordinate transformation. [Figure 4] A diagram for explaining coordinate transformation. [Figure 5] A diagram for explaining a coordinate system. [Figure 6] A diagram for explaining the attenuation distance and the radius ratio. [Figure 7] A diagram for explaining metadata. [Figure 8]This is a diagram illustrating the attenuation table. [Figure 9] This is a diagram illustrating the correction table. [Figure 10] This is a flowchart explaining audio output processing. [Figure 11] This is a diagram showing an example of a computer configuration. [Modes for carrying out the invention]

[0022] The following describes embodiments to which this technology is applied, with reference to the drawings.

[0023] <First Embodiment> <About this technology> This technology enables the rendering of audio objects to achieve a sufficiently high level of realism with minimal computation by determining the gain information of audio objects based on the positional relationships of multiple audio objects in space.

[0024] Furthermore, this technology is applicable not only to the rendering of audio objects, but also to adjusting the parameters of multiple objects present in space according to their relative positions. For example, this technology can be applied to determining the amount of adjustment for parameters such as brightness (light intensity) related to the image signal of objects, depending on their relative positions.

[0025] The following explanation will continue with the rendering of audio objects as a concrete example. Furthermore, in the following, audio objects will simply be referred to as "objects."

[0026] For example, during rendering, a rendering process using a predetermined method such as VBAP is performed. In VBAP, gain is distributed to the three speakers closest to an object that also exists on the surface of a sphere with the user's position in space as the origin.

[0027] For example, as shown in Figure 1, suppose there is a user U11, who is the listener, in a three-dimensional space, and three speakers SP1 to SP3 are positioned in front of the user U11.

[0028] Furthermore, the position of the user U11's head is defined as the origin O, and speakers SP1 to SP3 are positioned on the surface of a sphere centered on the origin O.

[0029] Now, let's assume that an object exists within a region TR11 on the surface of a sphere, surrounded by speakers SP1 through SP3, and that we want to localize the sound image at the position VSP1 of that object.

[0030] In such cases, VBAP will distribute the gain of the object to speakers SP1 through SP3 located around position VSP1.

[0031] Specifically, in a three-dimensional coordinate system with the origin O as the reference point (origin), the position VSP1 will be represented by a three-dimensional vector P that starts at the origin O and ends at the position VSP1.

[0032] Furthermore, if we define vectors L1 to L3 as three-dimensional vectors starting from the origin O and ending at the positions of each speaker SP1 to SP3, then vector P can be expressed as a linear combination of vectors L1 to L3, as shown in equation (1).

[0033]

number

[0034] Here, by calculating the coefficients g1 to g3 that are multiplied by vectors L1 to L3 in equation (1), and by using these coefficients g1 to g3 as the gains of the sound output from speakers SP1 to SP3, the sound image can be localized at position VSP1.

[0035] For example, a vector whose elements are coefficients g1 to g3 123 Let =[g1,g2,g3] and let the vectors L1 through L3 be elements. 123 If we set =[L1,L2,L3], we can rearrange equation (1) above to obtain equation (2).

[0036]

number

[0037] By using the coefficients g1 to g3 obtained by calculating equation (2) as gains, the object signal, which is the sound signal of the object, is output to each speaker SP1 to speaker SP3, thereby allowing the sound image to be localized at position VSP1.

[0038] Furthermore, the placement positions of each speaker SP1 to SP3 are fixed, and the information indicating the positions of these speakers is known information, therefore the inverse matrix L 123 -1 This can be determined in advance. Therefore, VBAP makes it possible to perform rendering with relatively simple calculations, that is, with a small amount of computation.

[0039] However, as mentioned above, when rendering using VBAP or similar methods, if there are multiple objects in space, the resulting changes in sound due to the relative positions of those objects are not taken into account at all. As a result, it was not possible to achieve a high level of realism during audio playback.

[0040] It is also conceivable to pre-adjust the level of object signals, but calculating the attenuation effect for level adjustment based on physical laws would be computationally intensive and impractical. Furthermore, in a free viewpoint, the user's position changes, making pre-adjustment of levels completely meaningless.

[0041] Therefore, this technology utilizes information about object decay to adjust the level of the object signal on the sound playback side, thereby enabling a high level of realism with less computation.

[0042] In particular, this technology determines gain information for adjusting the level of object signals based on the relative positional relationship between objects, thereby enabling the creation of attenuation effects caused by sound reflection, diffraction, and absorption—that is, changes in acoustics—with minimal computation. This allows for a high level of realism to be achieved.

[0043] <Example of signal processing device configuration> Next, we will describe an example of a signal processing device configuration that applies this technology.

[0044] Figure 2 shows an example configuration of one embodiment of a signal processing device to which this technology is applied.

[0045] The signal processing device 11 shown in Figure 2 includes a decoding processing unit 21, a coordinate transformation processing unit 22, an object attenuation processing unit 23, and a rendering processing unit 24.

[0046] The decoding processing unit 21 receives the transmitted input bitstream, decodes it, and outputs the resulting object metadata and object signal.

[0047] Here, the object signal is an audio signal used to reproduce the sound of the object. The metadata also includes object position information, object outer diameter information, object attenuation information, object attenuation disable information, and object gain information for each object.

[0048] Object position information is information that indicates the absolute position of an object in the space in which the object exists (hereinafter also referred to as the listening space).

[0049] For example, object position information is considered to be coordinate information indicating the object's position, represented by the x, y, and z coordinates of a three-dimensional Cartesian coordinate system, i.e., an xyz coordinate system, with a predetermined origin.

[0050] Object outer diameter information is information that indicates the outer diameter of an object. For example, here we assume that the object is spherical, and the radius of that sphere is considered the object outer diameter information.

[0051] In the following explanation, we will assume that the object is spherical, but the object can have any shape. For example, the object may have a shape with diameters in the x, y, and z axes, and the information indicating the radius of the object in each of those axes may be used as the object's outer diameter information.

[0052] Alternatively, the outer diameter information for spreading may be used as object outer diameter information. For example, the MPEG-H Part 3:3D audio standard employs a technique called spreading to expand the size of the sound source, and the format allows for recording the outer diameter information of each object in order to expand the size of the sound source. Therefore, such outer diameter information for spreading may be used as object outer diameter information.

[0053] Object attenuation information is information about the amount of sound attenuation that occurs when sound from other objects is attenuated due to an object. By using object attenuation information, it is possible to obtain the amount of attenuation of object signals from other objects at a given object, depending on the positional relationship between objects.

[0054] Object attenuation disable information indicates whether or not attenuation processing is performed on the sound of an object, i.e., the object signal, in other words, whether or not the object signal is attenuated.

[0055] For example, if the value of the object attenuation disable information is 1, attenuation processing for object signals is disabled. In other words, if the value of the object attenuation disable information is 1, no attenuation processing is performed on object signals.

[0056] For example, if a sound source creator intends for a certain object to be important and does not want the sound of that object to be attenuated due to its position relative to other objects, the value of the object attenuation disable information will be set to 1. In the following, an object with an object attenuation disable information value of 1 will also be referred to as an attenuation-disabled object.

[0057] In contrast, if the value of the object attenuation disable information is 0, attenuation processing is performed on the object signal according to the positional relationship between the object and other objects. In the following, objects that are subject to attenuation processing and for which the object attenuation disable information value is 0 will also be referred to as attenuation processing objects.

[0058] Object gain information is information that indicates the gain used to adjust the level of an object signal, as predetermined by the sound source producer. For example, object gain information may be a decibel value indicating the gain.

[0059] Once the decoding process 21 has decoded and obtained the object signal and metadata for each object, the decoding process 21 supplies the obtained object signal to the rendering process 24.

[0060] Furthermore, the decoding processing unit 21 supplies the object position information of the metadata obtained by decoding to the coordinate transformation processing unit 22. In addition, the decoding processing unit 21 supplies the object outer diameter information, object attenuation information, object attenuation disabled information, and object gain information of the metadata obtained by decoding to the object attenuation processing unit 23.

[0061] The coordinate transformation processing unit 22 generates object spherical coordinate position information based on the object position information supplied from the decoding processing unit 21 and the user position information supplied from an external source, and supplies it to the object attenuation processing unit 23. In other words, the coordinate transformation processing unit 22 transforms the object position information into object spherical coordinate position information.

[0062] Here, user position information refers to the absolute position of the user who is the listener within the listening space where the object exists, that is, the absolute position of the listening point desired by the user, and is represented as coordinate information using the x, y, and z coordinates of the xyz coordinate system.

[0063] This user location information is not information contained in the input bitstream, but rather information supplied from, for example, an external user interface connected to the signal processing device 11.

[0064] Furthermore, object spherical coordinate position information is information that indicates the relative position of an object as seen from the user in the listening space, expressed in spherical coordinates, i.e., spherical coordinates.

[0065] The object attenuation processing unit 23 obtains corrected object gain information by appropriately correcting the object gain information based on the object spherical coordinate position information supplied from the coordinate transformation processing unit 22 and the object outer diameter information, object attenuation information, object attenuation disabled information, and object gain information supplied from the decoding processing unit 21.

[0066] In other words, the object attenuation processing unit 23 functions as a gain determination unit that determines corrected object gain information based on object spherical coordinate position information, object outer diameter information, object attenuation information, object attenuation invalidation information, and object gain information.

[0067] Here, the gain value indicated by the corrected object gain information is obtained by appropriately correcting the gain value indicated by the object gain information, taking into account the positional relationship of the objects.

[0068] This type of corrected object gain information is intended to enable level adjustment of object signals that takes into account attenuation caused by sound reflection, diffraction, and absorption occurring within objects due to their positional relationships, i.e., changes in acoustics.

[0069] In the rendering processing unit 24, a process called attenuation is performed to adjust the level of the object signal based on the corrected object gain information during rendering. This attenuation process can be described as a process that attenuates the level of the object signal in response to sound reflection, diffraction, and absorption.

[0070] The object attenuation processing unit 23 supplies object spherical coordinate position information and corrected object gain information to the rendering processing unit 24.

[0071] In the signal processing device 11, the coordinate transformation processing device 22 and the object attenuation processing device 23 function as information processing devices that determine corrected object gain information for adjusting the level of the object signal according to the positional relationship of each object with other objects.

[0072] The rendering processing unit 24 generates an output audio signal based on the object signal supplied from the decoding processing unit 21 and the object spherical coordinate position information and corrected object gain information supplied from the object attenuation processing unit 23, and supplies it to subsequent speakers, headphones, recording units, etc.

[0073] Specifically, the rendering processing unit 24 generates the output audio signal by performing a panning process such as VBAP as part of the rendering process.

[0074] For example, when VBAP is performed as a panning process, calculations similar to those in equation (2) above are performed based on the object's spherical coordinate position information and the placement information of each speaker to obtain gain information for each speaker. Then, the rendering processing unit 24 adjusts the level of the object signal for the channel corresponding to each speaker based on the obtained gain information and the corrected object gain information to generate an output audio signal consisting of signals from multiple channels. If there are multiple objects, the signals from the same channel of each of those objects are added together to form the final output audio signal.

[0075] The rendering process performed by the rendering processing unit 24 may be any method, such as VBAP used in the MPEG-H Part 3:3D audio standard, or a panning technique called Speaker-anchored coordinates panner.

[0076] Furthermore, while VBAP rendering uses spherical coordinate position information for objects, Speaker-anchored coordinates panner rendering uses position information in a Cartesian coordinate system for direct rendering. Therefore, when rendering using a Cartesian coordinate system, the coordinate transformation processing unit 22 should be configured to obtain Cartesian coordinate position information, which indicates the position of each object as seen from the user's position, through coordinate transformation.

[0077] <Regarding the determination of coordinate transformation and corrected object gain information> Next, we will explain in more detail the coordinate transformation performed in the coordinate transformation processing unit 22 and the processing performed in the object attenuation processing unit 23.

[0078] In the coordinate transformation processing unit 22, the object position information and user position information are taken as input, a coordinate transformation is performed, and the object spherical coordinate position information is output.

[0079] Here, the object position information and user position information used as input for the coordinate transformation are expressed in coordinates of a three-dimensional Cartesian coordinate system using the x, y, and z axes, i.e., an xyz coordinate system, as shown in Figure 3.

[0080] In Figure 3, the coordinates indicating the position of user LP11 as seen from the origin O of the xyz coordinate system are considered the user position information. Similarly, the coordinates indicating the position of object OBJ1 as seen from the origin O of the xyz coordinate system are considered the object position information for object OBJ1, and the coordinates indicating the position of object OBJ2 as seen from the origin O of the xyz coordinate system are considered the object position information for object OBJ2.

[0081] During coordinate transformation, the coordinate transformation processing unit 22 first performs a parallel translation of all objects within the listening space so that the position of the user LP11 becomes the position of the origin O, as shown in Figure 4, and then transforms the coordinates of all those objects in the xyz coordinate system to the coordinates of the spherical coordinate system. Note that in Figure 4, the same reference numerals are used for parts corresponding to the case in Figure 3, and their explanations are omitted as appropriate.

[0082] Specifically, the coordinate transformation processing unit 22 determines a movement vector MV11 that moves the position of user LP11 to the origin O of the xyz coordinate system based on the user position information. This movement vector MV11 is a vector that starts at the position of user LP11 indicated by the user position information and ends at the position of the origin O.

[0083] Furthermore, the coordinate transformation processing unit 22 sets the movement vector MV12 to be the same magnitude (length) and direction as the movement vector MV11, and with the position of object OBJ1 as its starting point. Then, based on the object position information of object OBJ1, the coordinate transformation processing unit 22 moves the position of object OBJ1 by the amount indicated by the movement vector MV12.

[0084] Similarly, the coordinate transformation processing unit 22 sets a vector MV13 that has the same magnitude and direction as the movement vector MV11 and starts from the position of object OBJ2, and moves the position of object OBJ2 by ​​the amount indicated by the movement vector MV13 based on the object position information of object OBJ2.

[0085] Furthermore, the coordinate transformation processing unit 22 determines the spherical coordinate system coordinates indicating the position of object OBJ1 after movement as viewed from the origin O, and uses the obtained coordinates as the object spherical coordinate position information for object OBJ1. Similarly, the coordinate transformation processing unit 22 determines the spherical coordinate system coordinates indicating the position of object OBJ2 after movement as viewed from the origin O, and uses the obtained coordinates as the object spherical coordinate position information for object OBJ2.

[0086] Here, the relationship between the spherical coordinate system and the xyz coordinate system is as shown in Figure 5. Note that in Figure 5, the parts corresponding to those in Figure 4 are denoted with the same reference numerals, and their explanations are omitted as appropriate.

[0087] In Figure 5, the x, y, and z axes, which pass through the origin O and are perpendicular to each other, form the axes of the xyz coordinate system. For example, in the xyz coordinate system, the position of object OBJ1 after movement by the movement vector MV12 is expressed as (X1, Y1, Z1), using the x coordinate X1, the y coordinate Y1, and the z coordinate Z1.

[0088] In contrast, in the spherical coordinate system, the position of object OBJ1 is represented using the azimuth angle, the elevation angle, and the radius, position_radius.

[0089] Let r be the line connecting the origin O and the position of object OBJ1, and let L be the line obtained by projecting this line r onto the xy-plane.

[0090] In this case, the angle θ between the x-axis and the line L is defined as the azimuth angle (position_azimuth) that indicates the position of object OBJ1. Also, the angle φ between the line r and the xy-plane is defined as the elevation angle (position_elevation) that indicates the position of object OBJ1, and the length of the line r is defined as the radius (position_radius) that indicates the position of object OBJ1.

[0091] Therefore, the spherical coordinate information of the object, consisting of the azimuth angle, elevation angle, and radius relative to the user's position, i.e., the origin O, becomes the object's spherical coordinate position information. More specifically, the object's spherical coordinate position information can be determined by, for example, assuming that the positive x-axis is the direction the user is facing.

[0092] Next, we will explain the processing performed in the object decay processing unit 23.

[0093] For the sake of simplicity, this explanation will assume that only two objects, object OBJ1 and object OBJ2, exist in the listening space.

[0094] Specifically, as shown in Figure 6, for example, object OBJ1 and object OBJ2 exist in the listening space, and the corrected object gain information for object OBJ1 is determined. Note that in Figure 6, the same reference numerals are used for parts corresponding to those in Figure 4, and their explanations are omitted as appropriate.

[0095] In the example in Figure 6, object OBJ1 is assumed to be an object that is not an object with attenuation disabled, i.e., an attenuation processing object whose object attenuation disabled information value is 0.

[0096] In determining the corrected object gain information for object OBJ1, first, a vector OP1 indicating the position of object OBJ1 is obtained.

[0097] This vector OP1 is a vector that starts at the origin O and ends at position O11, which is indicated by the spherical coordinate position information of object OBJ1. A user located at the origin O will hear sound radiated from object OBJ1 at position O11 toward the origin O. More specifically, position O11 indicates the center position of object OBJ1.

[0098] Next, objects that are closer to the origin O than object OBJ1, that is, objects that are closer to the user's position (origin O) than object OBJ1, are selected as attenuated objects. Attenuated objects are objects that, because they are located between the attenuation processing object and the origin O, can become a factor in attenuating sound from the attenuation processing object.

[0099] In the example in Figure 6, object OBJ2 is located at position O12, which is indicated by the object's spherical coordinate position information. This position O12 is located closer to the origin O than the position O11 of object OBJ1. That is, the magnitude of vector OP2, which starts at the origin O and ends at position O12, is smaller than the magnitude of vector OP1.

[0100] Therefore, in the example in Figure 6, object OBJ2, which is located closer to the origin O than object OBJ1, is selected as the attenuated object relative to object OBJ1. More specifically, position O12 indicates the center position of object OBJ2.

[0101] The shape of object OBJ2 is a sphere with radius OR2, indicated by the object's outer diameter information, centered at position O12. Object OBJ2 is not a point source, but an object with a predetermined size.

[0102] Next, for object OBJ2, which is the attenuated object, the normal vector N2_1 from object OBJ2, that is, from position O12 to vector OP1, is obtained.

[0103] If we let position P2_1 be the intersection point of a line passing through position O12 and perpendicular to vector OP1 with vector OP1, then the normal vector N2_1 is the vector that starts at position O12 and ends at position P2_1. In other words, position P2_1 is the intersection point of vector OP1 and the normal vector N2_1.

[0104] Furthermore, the normal vector N2_1 is compared with the radius OR2, which is determined by the object outer diameter information of object OBJ2, to determine whether the magnitude of the normal vector N2_1 is less than or equal to the radius OR2, which is half the outer diameter of object OBJ2, the object being attenuated.

[0105] This determination process determines whether or not object OBJ2, which is an attenuated object, is in the path of sound emitted from object OBJ1 and traveling towards the origin O.

[0106] In other words, this determination process can be described as determining whether the center position O12 of object OBJ2 is located within a predetermined distance from the line connecting the user's position (origin O) and the center position O11 of object OBJ1.

[0107] The range of the predetermined distance referred to here is determined by the size of the object OBJ2. Specifically, the predetermined distance is the distance from position O12 on the object OBJ2 to the end of the line connecting the origin O and position O11, i.e., the radius OR2.

[0108] For example, in the example in Figure 6, the magnitude of the normal vector N2_1 is less than or equal to the radius OR2. That is, vector OP1 intersects with object OBJ2. Therefore, sound radiated from object OBJ1 toward the origin O is attenuated by reflection, diffraction, and absorption at object OBJ2, and then travels toward the origin O.

[0109] Therefore, the object attenuation processing unit 23 determines corrected object gain information to attenuate the level of the object signal of object OBJ1, according to the relative positional relationship between object OBJ1 and object OBJ2. In other words, the object gain information is corrected and becomes corrected object gain information.

[0110] Specifically, the corrected object gain information is determined based on the attenuation distance and radius ratio, which are information indicating the relative positional relationship between object OBJ1 and object OBJ2.

[0111] Note that the attenuation distance is the distance between object OBJ1 and object OBJ2.

[0112] In this case, if we define vector OP2_1 as a vector starting at the origin O and ending at position P2_1, then the difference between the magnitudes of vector OP1 and vector OP2_1, that is, the distance from position P2_1 to position O11, is the attenuation distance of object OBJ1 with respect to object OBJ2. In other words, |OP1|-|OP2_1| is the attenuation distance.

[0113] Furthermore, in this case, the radius ratio is the ratio of the distance from position O12, which is the center position of object OBJ2, to the line connecting the origin O and position O11, to the distance from position O12 to the edge of object OBJ2 on that line.

[0114] Here, since the shape of object OBJ2 is spherical, the radius ratio of object OBJ2 is the ratio of the magnitude of the normal vector N2_1 to the radius OR2, i.e., |N2_1| / OR2.

[0115] The radius ratio is information that indicates the amount of deviation of position O12, which is the center position of object OBJ2, from vector OP1, that is, the amount of deviation of position O12 from the line connecting the origin O and position O11. Such a radius ratio can be said to be information that indicates the positional relationship between object OBJ2 and object OBJ1, which depends on the size of object OBJ2.

[0116] Here, we will explain an example where the radius ratio is used as information indicating the positional relationship that depends on the size of the object. However, other information, such as the distance from the line connecting the origin O and position O11 to the edge of object OBJ2 on that line, may also be used.

[0117] In the object attenuation processing unit 23, for example, a correction value for the object gain information of object OBJ1 is determined based on the attenuation table index and correction table index as object attenuation information of metadata, as well as the attenuation distance and radius ratio. Then, the object attenuation processing unit 23 corrects the object gain information of object OBJ1 using this correction value to obtain corrected object gain information.

[0118] Here, we will explain the attenuation table shown by the attenuation table index and the correction table shown by the correction table index.

[0119] For example, the metadata for a given time frame included in the input bitstream is as shown in Figure 7.

[0120] In the example in Figure 7, the text "Object 1 Position Information" indicates the object position information of object OBJ1, the text "Object 1 Gain Information" indicates the object gain information of object OBJ1, and the text "Object 1 Attenuation Disable Information" indicates the object attenuation disabled information of object OBJ1.

[0121] Additionally, the text "Object 2 Position Information" indicates the object position information of object OBJ2, the text "Object 2 Gain Information" indicates the object gain information of object OBJ2, and the text "Object 2 Attenuation Disabled Information" indicates the object attenuation disabled information of object OBJ2.

[0122] Furthermore, the text "Object 2 Outer Diameter Information" indicates the outer diameter information of object OBJ2, the text "Object 2 Attenuation Table Index" indicates the attenuation table index of object OBJ2, and the text "Object 2 Correction Table Index" indicates the correction table index of object OBJ2.

[0123] Here, the attenuation table index and the correction table index represent object attenuation information.

[0124] The attenuation table index is an index used to identify the attenuation table that shows the amount of attenuation of an object signal according to the attenuation distance described above.

[0125] The amount of sound attenuation by the attenuated object changes depending on the distance between the attenuation processing object and the attenuated object. To easily obtain an appropriate attenuation amount based on the attenuation distance with minimal computation, an attenuation table is used, which associates attenuation distance with attenuation amount.

[0126] For example, the absorption rate of sound, diffraction, and reflection effects differ depending on the material of the object. Therefore, multiple attenuation tables are pre-prepared according to the material and shape of the object, the frequency band of the object signal, etc. The attenuation table index is an index that indicates one of these multiple attenuation tables, and the sound source creator specifies the appropriate attenuation table index for each object according to the material of the object, etc.

[0127] Furthermore, the correction table index is an index for identifying the correction table that shows the correction rate for the attenuation of the object signal according to the radius ratio described above.

[0128] The radius ratio indicates how far the straight line representing the path of sound emitted from the attenuating object deviates from the center of the attenuating object.

[0129] Even if the attenuation distance is the same, the actual amount of attenuation changes depending on the amount of deviation of the attenuated object from the path of the sound emitted from the attenuating object, i.e., the radius ratio.

[0130] For example, generally, when the line connecting the origin O and the attenuated object passes through the outer part of the object far from its center, the amount of attenuation is less due to diffraction compared to when the line passes through the center of the object. Therefore, to correct the attenuation of the object signal according to the radius ratio, a correction table is used in which the radius ratio and the correction rate are associated.

[0131] Similar to the case of attenuation tables, the appropriate correction rate according to the radius ratio changes depending on the material of the object, etc. Therefore, multiple correction tables are prepared in advance according to the material and shape of the object, the frequency band of the object signal, etc. The correction table index is an index that indicates one of these multiple correction tables, and the sound source producer specifies the appropriate correction table index for each object according to the material of the object, etc.

[0132] In the example shown in Figure 7, object OBJ1 is processed as a point source without object outer diameter information, so only object position information, object gain information, and object attenuation disable information are provided as metadata for object OBJ1.

[0133] In contrast, object OBJ2 contains object outer diameter information and acts as an object that attenuates sound radiated from other objects. Therefore, in addition to object position information, object gain information, and object attenuation disable information, object outer diameter information and object attenuation information are also provided as metadata for object OBJ2.

[0134] In particular, object attenuation information is provided here, including an attenuation table index and a correction table index. These attenuation table indexes and correction table indexes are used to calculate the correction values ​​for object gain information.

[0135] For example, the attenuation table shown by a single attenuation table index contains information that shows the relationship between attenuation distance and attenuation amount, as shown in Figure 8.

[0136] In Figure 8, the vertical axis represents the decibel value of the attenuation, and the horizontal axis represents the distance between objects, i.e., the attenuation distance. For example, in the example shown in Figure 6, the distance from position P2_1 to position O11 is the attenuation distance.

[0137] In the example in Figure 8, the amount of attenuation increases as the attenuation distance decreases, and the change in attenuation is greater in response to the change in attenuation distance as the attenuation distance decreases. From this, it can be seen that the closer the object being attenuated is to the object being attenuated, the greater the attenuation of the sound from the object being attenuated.

[0138] Furthermore, for example, a correction table indicated by a single correction table index contains information showing the relationship between the radius ratio and the correction rate, as shown in Figure 9.

[0139] In Figure 9, the vertical axis represents the attenuation correction rate, and the horizontal axis represents the radius ratio. For example, in the example shown in Figure 6, the ratio of the magnitude of the normal vector N2_1 to the radius OR2 is the radius ratio.

[0140] For example, if the radius ratio is 0, sound traveling from the attenuation object towards the origin O, i.e., the user, will pass through the center of the attenuated object. If the radius ratio is 1, sound traveling from the attenuation object towards the origin O will pass through the boundary of the attenuated object.

[0141] In this example, the correction rate decreases as the radius ratio increases, and the change in the correction rate in response to the change in the radius ratio also increases as the radius ratio increases. For example, if the correction rate is 1.0, the attenuation amount obtained from the attenuation table is used as is, and if the correction rate is 0, the attenuation amount obtained from the attenuation table is set to 0, resulting in a attenuation effect of 0. Note that if the radius ratio is greater than 1, sound traveling from the attenuation processing object toward the origin O does not pass through a region containing the attenuated object, so no attenuation processing is performed.

[0142] Based on the attenuation distance and radius ratio, the attenuation amount and correction rate corresponding to those attenuation distances and radius ratios are obtained. Then, a correction value is determined based on those attenuation amounts and correction rates, and the object gain information is corrected.

[0143] Specifically, the correction value is obtained by multiplying the attenuation amount by the correction factor, i.e., (correction factor × attenuation amount). This correction value is the final attenuation amount obtained by correcting the attenuation amount with the correction factor. Once the correction value is obtained, the object gain information is corrected by adding this correction value to the object gain information. The corrected object gain information obtained in this way, i.e., the sum of the correction value and the object gain information, is called the corrected object gain information.

[0144] The correction value, which is the product of the correction rate and the attenuation amount, can be said to represent the amount of attenuation of the object signal that corresponds to the attenuation of the sound of one object in other objects, determined based on the positional relationship between objects.

[0145] Here, we have described an example where pre-prepared attenuation table indexes and correction table indexes are included in the metadata as object attenuation information. However, the object attenuation information can be anything as long as the attenuation amount and correction rate can be obtained, such as using the change points of the line graphs corresponding to the attenuation tables and correction tables shown in Figures 8 and 9 as object attenuation information.

[0146] Alternatively, for example, multiple damping functions, which are continuous functions that take damping distance as input and output damping amount, and correction rate functions, which are continuous functions that take radius ratio as input and output correction rate, may be prepared, and an index indicating one of these multiple damping functions and an index indicating one of the multiple correction rate functions may be used as object damping information. Furthermore, multiple continuous functions that take damping amount and radius ratio as input and output correction values ​​may be prepared in advance, and an index indicating one of these functions may be used as object damping information.

[0147] <Explanation of audio output processing> Next, the specific operation of the signal processing device 11 will be described. Specifically, the audio output processing by the signal processing device 11 will be described below with reference to the flowchart in Figure 10.

[0148] In step S11, the decoding processing unit 21 decodes the received input bitstream to obtain metadata and object signals.

[0149] The decoding processing unit 21 supplies the object position information of the obtained metadata to the coordinate transformation processing unit 22, and also supplies the object outer diameter information, object attenuation information, object attenuation disabled information, and object gain information of the obtained metadata to the object attenuation processing unit 23. The decoding processing unit 21 also supplies the obtained object signal to the rendering processing unit 24.

[0150] In step S12, the coordinate transformation processing unit 22 performs a coordinate transformation for each object based on the object position information supplied from the decoding processing unit 21 and the user position information supplied from an external source to generate object spherical coordinate position information, and supplies it to the object attenuation processing unit 23.

[0151] In step S13, the object attenuation processing unit 23 selects one attenuation object to be processed based on the object attenuation invalidation information supplied from the decoding processing unit 21 and the object spherical coordinate position information supplied from the coordinate transformation processing unit 22, and determines the position vector of that attenuation object.

[0152] For example, the object attenuation processing unit 23 selects one object whose object attenuation disable information value is 0 and designates that object as the attenuation processing object. Then, based on the object spherical coordinate position information of the attenuation processing object, the object attenuation processing unit 23 calculates a position vector that starts at the origin O, i.e., the user's position, and ends at the position of the attenuation processing object.

[0153] Therefore, for example, in the example shown in Figure 6, if object OBJ1 is selected as the attenuation processing object, vector OP1 is obtained as the position vector.

[0154] In step S14, the object attenuation processing unit 23 selects one object that is at a shorter distance from the origin O than the attenuation processing object, based on the object spherical coordinate position information of the attenuation processing object to be processed and other objects, as the object to be attenuated for that attenuation processing object.

[0155] For example, in the example in Figure 6, if object OBJ1 is selected as the object to be attenuated, then object OBJ2, which is located closer to the origin O than object OBJ1, will be selected as the object to be attenuated.

[0156] In step S15, the object attenuation processing unit 23 determines the normal vector from the center of the object to be attenuated to the position vector of the object to be attenuated, based on the position vector of the object to be attenuated obtained in step S13 and the object spherical coordinate position information of the object to be attenuated.

[0157] For example, in the example shown in Figure 6, if object OBJ1 is selected as the attenuation object and object OBJ2 is selected as the object to be attenuated, the normal vector N2_1 will be calculated.

[0158] In step S16, the object attenuation processing unit 23 determines whether the magnitude of the normal vector is less than or equal to the radius of the object to be attenuated, based on the normal vector obtained in step S15 and the object outer diameter information of the object to be attenuated.

[0159] For example, in the example shown in Figure 6, if object OBJ1 is selected as the object to be attenuated and object OBJ2 is selected as the object to be attenuated, it is determined whether the magnitude of the normal vector N2_1 is less than or equal to the radius OR2, which is half the outer diameter of object OBJ2.

[0160] If it is determined in step S16 that the magnitude of the normal vector is not less than or equal to the radius of the object being attenuated, the object being attenuated is not on the path of the sound traveling from the attenuation processing object toward the origin O (user), so steps S17 and S18 are not performed, and the process proceeds to step S19.

[0161] In contrast, if it is determined in step S16 that the magnitude of the normal vector is less than or equal to the radius of the object being attenuated, the object being attenuated is on the path of sound traveling from the attenuation processing object toward the origin O (user), and the process proceeds to step S17. In this case, the attenuation processing object and the object being attenuated are located in approximately the same direction from the user's perspective.

[0162] In step S17, the object attenuation processing unit 23 calculates the attenuation distance based on the position vector of the attenuated object obtained in step S13 and the normal vector of the attenuated object obtained in step S15. The object attenuation processing unit 23 also calculates the radius ratio based on the object outer diameter information and normal vector of the attenuated object.

[0163] For example, in the example shown in Figure 6, if object OBJ1 is selected as the object to be attenuated and object OBJ2 is selected as the object to be attenuated, the distance from position P2_1 to position O11, i.e., |OP1|-|OP2_1|, is calculated as the attenuation distance. Furthermore, in this case, the ratio of the magnitude of the normal vector N2_1 to the radius OR2, |N2_1| / OR2, is calculated as the radius ratio.

[0164] In step S18, the object attenuation processing unit 23 determines the corrected object gain information for the attenuation processing object based on the object gain information of the attenuation processing object, the object attenuation information of the attenuated object, and the attenuation distance and radius ratio obtained in step S17.

[0165] For example, if the metadata includes the aforementioned attenuation table index and correction table index as object attenuation information, the object attenuation processing unit 23 pre-stores multiple attenuation tables and correction tables.

[0166] In this case, the object attenuation processing unit 23 reads the amount of attenuation determined for the attenuation distance from the attenuation table, which is indicated by the attenuation table index as object attenuation information for the object to be attenuated.

[0167] Furthermore, the object attenuation processing unit 23 reads a correction rate determined for the radius ratio from a correction table indicated by a correction table index as object attenuation information for the object to be attenuated.

[0168] Then, the object attenuation processing unit 23 obtains a correction value by multiplying the read attenuation amount by a correction rate, and obtains corrected object gain information by adding this correction value to the object gain information of the attenuation processing object.

[0169] In this way, the process of determining corrected object gain information can be described as a process that determines a correction value indicating the amount of attenuation of the object signal based on the attenuation distance and radius ratio, that is, the positional relationship between objects, and further determines corrected object gain information, which is the gain for adjusting the level of the object signal, based on that correction value.

[0170] Once the correction object gain information is obtained, the process proceeds to step S19.

[0171] If the processing in step S18 has been performed, or if it is determined in step S16 that the magnitude of the normal vector is not less than or equal to the radius, then in step S19, the object attenuation processing unit 23 determines whether or not there are any unprocessed attenuated objects for the attenuation processing object to be processed.

[0172] If it is determined in step S19 that there are still unprocessed attenuated objects, the process returns to step S14, and the process described above is repeated.

[0173] In this case, in step S18, the correction value obtained for the new attenuated object is added to the already obtained correction object gain information, and the correction object gain information is updated. Therefore, if there are multiple attenuated objects whose normal vector magnitude is less than or equal to the radius of the attenuated object, the final correction object gain information will be obtained by adding the correction values ​​obtained for each of the multiple attenuated objects to the object gain information.

[0174] Furthermore, if it is determined in step S19 that there are no unprocessed attenuated objects, that is, that processing has been performed on all attenuated objects, the process proceeds to step S20.

[0175] In step S20, the object attenuation processing unit 23 determines whether or not all attenuation processing objects have been processed.

[0176] If it is determined in step S20 that not all attenuation objects have been processed yet, the process returns to step S13, and the process described above is repeated.

[0177] If, in step S20, it is determined that all attenuation processing objects have been processed, the process proceeds to step S21.

[0178] In this case, the object attenuation processing unit 23 uses the object gain information of objects for which the processing in steps S17 and S18 was not performed, i.e., objects for which attenuation processing was not performed, as the corrected object gain information.

[0179] Furthermore, the object attenuation processing unit 23 supplies the object sphere reference position information of all objects, supplied from the coordinate transformation processing unit 22, and the corrected object gain information to the rendering processing unit 24.

[0180] In step S21, the rendering processing unit 24 performs rendering processing based on the object signal supplied from the decoding processing unit 21 and the object spherical coordinate position information and corrected object gain information supplied from the object attenuation processing unit 23, and generates an output audio signal.

[0181] Once the output audio signal is obtained in this way, the rendering processing unit 24 outputs the obtained output audio signal to the next stage, and the audio output processing is completed.

[0182] As described above, the signal processing device 11 corrects the object gain information according to the positional relationship between objects and converts it into corrected object gain information. In this way, a high level of realism can be obtained with a small amount of computation.

[0183] In other words, when multiple objects are located in roughly the same direction from the user's perspective within the listening space, instead of calculating the attenuation effects due to sound absorption, diffraction, and reflection of the objects based on physical laws, a simple calculation using a table to determine a correction value according to the attenuation distance and radius ratio can be performed to obtain an effect roughly equivalent to that obtained when calculations based on physical laws are performed. Therefore, even when the user moves freely within the listening space, it is possible to provide the user with a highly immersive 3D sound effect with a small amount of computation.

[0184] While this explanation focused on the case of a free viewpoint where the user can move to any position within the listening space, a high level of realism can be achieved with less computation even in the case of a fixed viewpoint where the user's position within the listening space is fixed, in the same way as in the free viewpoint case.

[0185] In such cases, the user position indicated by the user position information is always the position of the origin O, so the coordinate transformation processing by the coordinate transformation processing unit 22 is unnecessary, and the object position information is expressed as position information in spherical coordinates. In particular, in this case, the object position information is information indicating the position of the object as seen from the origin O. Furthermore, the processing by the object attenuation processing unit 23 may be performed on the client side receiving the content, or on the server side distributing the content.

[0186] <Variations> Furthermore, while the above describes the case where the object attenuation disable information is 0 or 1, the object attenuation disable information may be set to one of three or more values. In such cases, for example, the value of the object attenuation disable information may indicate not only whether or not it is an object with attenuation disabled, but also the amount of attenuation correction. Therefore, for example, the correction value obtained from the correction rate and the attenuation amount may be further corrected according to the value of the object attenuation disable information to obtain the final correction value.

[0187] Furthermore, while the above describes an example where object attenuation disable information is defined to indicate whether or not attenuation processing should be disabled for each object, it is also possible to define whether or not attenuation processing should be disabled for a region within the listening space.

[0188] For example, if the sound source producer intends to prevent the attenuation effect of an object from occurring in a specific spatial region within the listening space, then instead of storing object attenuation disable information, object attenuation disable region information indicating the spatial region where the attenuation effect will not occur should be stored in the input bitstream.

[0189] In such cases, the object attenuation processing unit 23 designates objects whose position, indicated by the object position information, falls within the spatial region indicated by the object attenuation invalidation region information as objects with attenuation disabled. This makes it possible to achieve audio playback that reflects the intentions of the sound source creator.

[0190] Furthermore, the positional relationship between the user and the object may also be taken into consideration, for example, by designating objects located approximately in front of the user as objects with attenuation disabled, and objects located behind the user as objects with attenuation applied. In other words, whether or not an object is designated as an object with attenuation disabled may be determined based on the positional relationship between the user and the object.

[0191] In addition, while the above describes an example where the object signal is attenuated according to the relative positional relationship between objects, it is also possible to add reverberation effects to the object signal according to the relative positional relationship between objects.

[0192] It has long been known that reverberation effects are caused by trees in forests, and Kuttruff has modeled forest reverberation by treating trees as spheres and solving the diffusion equation.

[0193] Therefore, for example, if a certain number of objects exist within a certain space that includes the user's position and the position of the sound source object, it is conceivable to apply a specific reverberation effect to the object signal of each object within that space.

[0194] In this case, by including parametric reverb coefficients for adding reverberation effects in the input bitstream and changing the mixing ratio of direct sound and reverberation sound according to the relative relationship between the user's position and the position of the sound source object, it becomes possible to add reverberation effects.

[0195] <Example of computer configuration> Incidentally, the series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up that software are installed on a computer. Here, a computer includes computers built into dedicated hardware, as well as general-purpose personal computers that can perform various functions by installing various programs.

[0196] Figure 11 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program.

[0197] In a computer, the CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, and RAM (Random Access Memory) 503 are interconnected by a bus 504.

[0198] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0199] The input unit 506 consists of a keyboard, mouse, microphone, image sensor, etc. The output unit 507 consists of a display, speaker, etc. The recording unit 508 consists of a hard disk, non-volatile memory, etc. The communication unit 509 consists of a network interface, etc. The drive 510 drives a removable recording medium 511 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0200] In a computer configured as described above, the CPU 501 loads, for example, a program stored in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504, and executes it, thereby performing the series of processes described above.

[0201] The program executed by the computer (CPU 501) can be provided by recording it on a removable recording medium 511, such as a packaged media. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0202] In a computer, a program can be installed in the recording unit 508 via the input / output interface 505 by inserting the removable recording medium 511 into the drive 510. Alternatively, the program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Furthermore, the program can be pre-installed in the ROM 502 or the recording unit 508.

[0203] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0204] Furthermore, the embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the gist of this technology.

[0205] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0206] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0207] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0208] Furthermore, this technology can also be configured as follows:

[0209] (1) The system includes a gain determination unit that determines the amount of attenuation based on the positional relationship between a predetermined object and other objects, and determines the gain of the signal of the predetermined object based on the amount of attenuation. Information processing device. (2) The aforementioned other object is located closer to the user's position than the predetermined object. (1) The information processing device described above. (3) The aforementioned other object is located within a predetermined distance from the straight line connecting the user's position and the predetermined object. The information processing device described in (1) or (2). (4) The range is determined by the size of the other objects. (3) The information processing device described above. (5) The predetermined distance is the distance from the center of the other object to the end of the straight line on the other object. The information processing device described in (3) or (4). (6) The aforementioned positional relationship is a positional relationship that depends on the size of the other objects. An information processing device as described in any one of items (3) through (5). (7) The aforementioned positional relationship is the amount of deviation of the center of the other object from the aforementioned straight line. (6) The information processing device described above. (8) The aforementioned positional relationship is the ratio of the distance from the center of the other object to the straight line to the distance from the center of the other object to the end of the other object on the straight line side. (6) The information processing device described above. (9) The gain determination unit determines the amount of attenuation based on the positional relationship and the attenuation information of the other object. An information processing device as described in any one of items (1) through (8). (10) The aforementioned attenuation information is information for obtaining the amount of attenuation of the signal in relation to the other object, according to the positional relationship. (9) The information processing device described above. (11) The positional relationship is the distance between the other object and the predetermined object. An information processing device as described in any one of items (1) through (10). (12) The gain determination unit determines the amount of attenuation based on attenuation disable information indicating whether or not to attenuate the signal of the predetermined object, and the positional relationship. An information processing device as described in any one of items (1) through (11). (13) The signal of the predetermined object is an audio signal. An information processing device as described in any one of items (1) through (11). (14) Information processing device, The amount of attenuation is determined based on the positional relationship between a predetermined object and other objects, and the gain of the signal of the predetermined object is determined based on the amount of attenuation. Information processing methods. (15) The amount of attenuation is determined based on the positional relationship between a predetermined object and other objects, and the gain of the signal of the predetermined object is determined based on the amount of attenuation. A program that causes a computer to execute a process that includes steps. [Explanation of Symbols]

[0210] 11 Signal processing unit, 21 Decoding unit, 22 Coordinate transformation unit, 23 Object attenuation unit, 24 Rendering unit

Claims

[Claim 1] The system includes a gain determination unit that determines the amount of attenuation based on attenuation information from among multiple pieces of attenuation information corresponding to information about an object, and the positional relationship between a predetermined object and other objects, and determines the gain of the signal of the predetermined object based on the amount of attenuation. Information processing device.

Images