Rendering of audio objects with complex shape

Abstract

A method (900) for representing an audio object in an extended reality scene. The method includes obtaining (s902) first metadata that describes a first three-dimensional (3D) shape associated with the audio object. The method also includes transforming (s904) the obtained first metadata to produce transformed metadata that describes a two-dimensional (2D) plane or a one-dimensional (1D) line, wherein the 2D plane or 1D line represents at least a portion of the audio object.

Description

Technical Field

[0001] Examples relating to the rendering of audio objects with complex shapes are disclosed. Background Technology

[0002] Spatial audio rendering is used to present audio within extended reality (XR) scenes (e.g., virtual reality (VR), augmented reality (AR), or mixed reality (MR) scenes) to give listeners the impression that the sound originates from a physical source within the scene and has a certain size and shape (i.e., extent). This can be achieved through headphone speakers or other loudspeakers. If presented via headphone speakers, the processing used is called binaural rendering and utilizes spatial cues from human spatial hearing to determine the direction from which the sound is coming. Cueing involves interaural time delay (ITD), interaural level difference (ILD), and / or spectral differences.

[0003] The most common form of spatial audio rendering is based on the concept of point sources, where each sound source is defined as emitting sound from a specific point. Because each sound source is defined as emitting sound from a specific point, the sound source has no size or shape. Different methods have been developed to render sound sources with a certain range (size and shape).

[0004] One known approach is to create multiple copies of a mono audio object at locations surrounding the audio object. This arrangement creates the perception of a spatially uniform object of a certain size. This concept is used, for example, in the “object expansion” and “object divergence” features of the MPEG-H 3D audio standard (see references [1] and [2]) and the “object divergence” feature of the EBU Audio Definition Model (ADM) standard (see reference [4]). As described in reference [7], the idea of ​​using a mono audio source has been further developed, in which the area-volume geometry of the sound object is projected onto a sphere surrounding the listener, and the sound is rendered to the listener using a pair of head-related (HR) filters that are evaluated as the integral of all HR filters covering the geometric projection of the object onto the sphere. For a spherical volume source, this integral has an analytical solution. However, for arbitrary area-volume source geometry, the integral is evaluated by sampling the projected source surface on the sphere using so-called Monte Carlo ray sampling.

[0005] Another rendering method renders a spatial diffusion component in addition to the mono audio signal, which creates the perception of a slightly diffused object that does not have a distinct pin-point position compared to the original mono audio object. This concept is used, for example, in the “object diffusion” feature of the MPEG-H 3D audio standard (see reference [3]) and the “object diffusion” feature of EBU ADM (see reference [5]).

[0006] Combinations of the two methods are also known. For example, the “object range” feature of EBU ADM combines the creation of multiple copies of a mono audio object with the addition of a diffusion component (see reference [6]).

[0007] In many cases, the actual shape of an audio object can be well described using basic shapes (e.g., a sphere or a box). However, sometimes the actual shape is more complex and needs to be described in more detail (e.g., a mesh structure or a parametric description format). In such cases, the rendering of the audio object requires a real-time evaluation of how the audio object should be rendered based on the listener's current relative position to the audio object. This real-time evaluation can be processing-intensive. Summary of the Invention

[0008] Existing methods for rendering volumetric audio objects typically represent the audio object using a single point source and set the position of that point source as the center of the audio object (see example...). Figure 1 This means that the height and width of the audio object are not represented. More detailed methods exist for rendering volumetric audio objects. These methods attempt to represent the shape of the audio object, but they are limited to mono sources. This results in the audio object being rendered either spatially uniformly or spatially diffusely. In both cases, the rendering is based on the mono recording without spatial information.

[0009] Another option for rendering volumetric audio objects is to represent the audio object using multiple point sources that together constitute the sound emanating from it. However, this method is very inefficient due to the number of point sources required to accurately represent the shape of the object from different angles. The number of point sources will also need to increase proportionally with the size of the object.

[0010] Therefore, this disclosure supports the use of audio objects with defined sizes and shapes, represented by multi-channel audio signals, where different channels represent spatial information of the audio object in different dimensions. An example is stereo recording, which represents spatial information along the horizontal dimension of the audio object. More specifically, this disclosure provides a method for simplifying the three-dimensional (3D) shape of an audio object into a one-dimensional (1D) or two-dimensional (2D) representation describing the width and / or height of the audio object as seen from the listener's position. Furthermore, the multi-channel audio signal is rendered such that the perceived spatial range matches the simplified spatial range.

[0011] Therefore, in one aspect, a method for representing an audio object in an extended reality scene is provided. The method includes obtaining first metadata describing a first three-dimensional (3D) shape associated with the audio object. The method also includes transforming the obtained first metadata to produce transformed metadata describing a two-dimensional (2D) plane or a one-dimensional (1D) line. The 2D plane or 1D line represents at least a portion of the audio object.

[0012] On the other hand, a computer program including instructions is provided that, when executed by processing circuitry, causes the processing circuitry to perform the methods described above. In one embodiment, a carrier containing a computer program is provided, wherein the carrier is one of electronic signals, optical signals, radio signals, and computer-readable storage media.

[0013] On the other hand, an apparatus is provided for representing an audio object in an extended reality scene. The apparatus is adapted to obtain first metadata describing a first three-dimensional (3D) shape associated with the audio object. The apparatus is further adapted to transform the obtained first metadata to produce transformed metadata describing a two-dimensional (2D) plane or a one-dimensional (1D) line. The 2D plane or 1D line represents at least a portion of the audio object. In one embodiment, the apparatus includes processing circuitry and a storage unit for storing instructions for configuring the apparatus to perform any of the processes disclosed herein.

[0014] When a listener moves around in a six-degrees-of-freedom (6DoF) XR scene, the embodiments described herein provide a more efficient way to dynamically represent both the width and height of an audio object compared to existing methods of rendering audio objects. Attached Figure Description

[0015] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments.

[0016] Figure 1 This is an exemplary representation of an audio object.

[0017] Figure 2 An example of a simplified range is shown.

[0018] Figure 3A and Figure 3B An exemplary process for finding descriptive points is shown.

[0019] Figure 4 An exemplary process for defining a 2D plane is shown.

[0020] Figure 5 This illustrates a scenario where a very small movement by the listener triggers a jump in the anchor point's position.

[0021] Figures 6A-6C An example rendering configuration is shown.

[0022] Figure 7 A system according to some embodiments is shown.

[0023] Figure 8A and Figure 8B A system according to some embodiments is shown.

[0024] Figure 9 This is a process according to some embodiments.

[0025] Figure 10 An apparatus according to some embodiments is shown. Detailed Implementation

[0026] To render audio objects with complex ranges (i.e., complex shapes) in a 6DoF virtual environment, the shape of the object needs to be evaluated in real time as the listener's position changes. The shape of the object relative to the listener's position should affect the apparent position, width, and height of the audio object to match the behavior of the real sound source.

[0027] When rendering audio of an audio object without affecting the quality of perceptual space, the level of detail of the range that needs to be considered will vary depending on, for example, the distance from the listener's position to the audio object. As the listener moves away from the audio object, the detail of the shape becomes less important. This can be taken advantage of by efficiently simplifying the shape description of the range.

[0028] Typically, the scope of a volumetric audio object is described either by a basic shape (e.g., a sphere or a box) or by a more detailed description based on a combination of shape descriptions such as the basic shape, a mesh structure, or other forms. To efficiently render these audio objects with complex shape descriptions in real time, it is desirable to simplify the object's shape description in a way that retains important cues perceived by the auditory system but requires less complexity. In many cases, this simplification is possible because very fine shape details are often perceptually irrelevant to audio rendering, and therefore considering these fine shape details during rendering would be a waste of computational resources.

[0029] This disclosure describes a method for simplifying complex descriptions of audio objects so that a vague impression of the audio object can be rendered to the listener at low computational cost.

[0030] In one embodiment, the method is divided into three basic steps:

[0031] (1) An optional preprocessing step that simplifies a complex description of the extent (i.e., shape) of an audio object into a simpler one. For example, a simpler description could be a mesh structure with fewer vertices, a basic shape (e.g., a box, sphere, ellipse, or cylinder), or a combination of several basic shapes. Instead of the original shape description, this simplified shape description may be used either continuously or only when, for example, the distance to the listener's position is sufficiently long. This step is described in Section 1.

[0032] (2) Real-time simplification steps, where the range is represented by a 2-dimensional plane or a 1-dimensional line. The listener's instantaneous listening position is considered here. This step is described in Section 2.

[0033] (3) Render the audio object given its simplified geometric extent (as determined in step 2). During the rendering phase, processing can be added to add perceptual effects such as distance gain. This step is described in Section 3.

[0034] 1. Simplified pre-rendering

[0035] The pre-render simplification step is completed before rendering begins, or alternatively as a parallel process during rendering, but its results are not used until this step is complete. The purpose of this step is to find one or more simplified descriptions of the range that can be used, based on the amount of detail required at a given moment. Several alternative range descriptions can be stored in a list so that the real-time rendering step can select the most appropriate range description at each moment. Because the pre-render simplification step is not performed in real time, it can involve complex calculations and optimizations if necessary. This step can even be done as part of the content creation step, where several alternative representations are pre-computed. Alternatively, this step can be performed by the encoding node or the pre-rendering node to offload the listener's device.

[0036] The amount of detail required to describe the range of high-quality rendering for a listener in an extended reality scene depends on many things that can change in real time, such as: i) the distance between the audio object and the listener's listening position in the extended reality scene; ii) the size of the audio object; iii) how many other audio objects are currently active; iv) the current audio energy level of the audio object; and v) the current CPU load of the renderer that will be used to render the audio object.

[0037] If the distance is large enough, any audio object can be represented well enough with a simple point source. However, the closer the listener is to the audio object, the more detail is needed to provide a subjectively accurate audio rendering of that object.

[0038] By pre-determining one or more simplified descriptions of the range, the description that provides the most effective trade-off between detail and complexity can be chosen at any time. As an example, a complex mesh structure describing the range of an audio object can be simplified into two alternative simplified shapes: one that remains a mesh structure but with a reduced number of vertices, and another whose dimensions are simply set to match the original shape as a box. Examples of this simplification are found in... Figure 2 It is shown in the middle, Figure 2 A complex mesh structure 202, a first less complex version 204 of the complex mesh structure 202, and a second less complex version 206 of the complex mesh structure 202 are shown.

[0039] like Figure 2 As shown, mesh structure 202 has a relatively high number of vertices. In the example shown, mesh structure 202 represents a forest, as seen from above. In this example, a first, less complex version 204 of the complex mesh structure 202 is itself a mesh structure, but mesh structure 204 has fewer vertices than mesh structure 202; a second, less complex version 206 of the complex mesh structure 202 is a simple box shape, which is used to represent the shape of a forest. That is, in Figure 2In the example, two simplified shapes were created: one by reducing the number of vertices in the original mesh 202, and the other based on a basic box 206. These two alternative shape descriptions, 204 and 206, represent different levels of simplification and, depending on the desired amount of detail, either of these representations can be chosen as the basis for a live simplification step. For box 206, the box has been rotated so that a size represents the maximum size of the forest (i.e., mesh 202), and the width and height of the box have been adjusted to match mesh 202 as closely as possible. Even simple shapes such as spheres or boxes can be simplified to point sources when the distance to the listener is sufficient.

[0040] Many methods for reducing the number of vertices in a mesh are known (see, for example, reference [8]). The problem of decomposing an arbitrary shape into one or more basic shapes such as boxes is also well known, and this problem can be solved, for example, by using minimum volume bounding box decomposition (see, for example, reference [9]). As described herein, at any time during real-time rendering, multiple alternative representations are pre-computed along with the parameterization method for selecting which representation to use.

[0041] The parameterization of which representation to use as the basis for real-time simplification (step 2) can be based on the distance between the audio object and the listener's listening position. Examples of this are described in Table 1.

[0042] Table 1: Tables used when choosing between representations with complex ranges

[0043]

[0044] Here D l D1 and D2 are threshold distance parameters, which can be constants or variables and are controlled, for example, by the number of currently active audio objects or the current CPU load. Design parameters can depend on the size of the audio objects. For example, the listener's position may need to be far from large audio objects, as a single-point representation of a large audio object may suffice. In the examples shown in Table 1, there are three different representations for audio objects, and two parameters (i.e., D1, D2, D2, D3, D4, D5, D6, D7, D8, D9, D1, D9, D1, D2 ... l D2 and D2 are used to indicate when the renderer switches between these three different representations. Any additional representations (such as D2) Figure 2 The simplified mesh structure (represented in 204) requires additional parameters to specify when this representation should be used.

[0045] Here are some ways to choose D. l Example of D2: D l =2S max And D2 = 20S max , of which Smax It is the size within its maximum dimensions (e.g., length, width, height, or depth).

[0046] In some cases (e.g., when the computational resources available for rendering are below a certain lower limit), D1 can be very small or even 0 (i.e., the original mesh is always simplified to at least the first simplification level—the box shape in the examples in Table 1).

[0047] 2. Real-time simplified steps

[0048] When the listener's current listening position is known, the real-time simplification step can be run as part of the rendering process. This step can typically run at a uniform rate relative to the frame size used by the rest of the renderer (e.g., it can be performed each time a new audio frame is received or processed). Alternatively, this step can be triggered only when the position or rotation of the audio object or the listener changes.

[0049] Because this step is performed in real time, one of the different representations determined in step 1 can be selected based on the parameterized selection rules and the listener's current listening position. Other parameters that can be considered when selecting one of these representations include the number of active audio objects, the renderer's CPU load, and the current audio energy level of the rendered audio objects.

[0050] After selecting one of these representations (i.e., one of the original or simplified descriptions), the selected representation can be further simplified to a 2-dimensional (2D) plane or a 1-dimensional (1D) line. In this further simplification, the depth of the audio object is not considered because the depth of the audio object placed in front of the listener cannot be subjectively perceived in the same way as the width and height of the audio object.

[0051] When an audio object is directly in front of the listener, the perceived horizontal width is primarily determined by time and level difference cues. However, the perceived vertical height of the object is primarily determined by less accurate spectral cues.

[0052] Therefore, listeners are generally able to estimate the horizontal width of an audio object more accurately than its vertical height. The perceptual estimation of the depth of an audio object is even less accurate, and is often impossible. Therefore, the depth dimension of an audio object typically does not need to be included in its rendering.

[0053] Therefore, a 2D plane representing both the width and height of an audio object captures most of the important spatial cues. In many cases, a 1D line representation of only the position and horizontal width of the audio object will be an effective representation for capturing the most important spatial cues of the original audio object.

[0054] This simplification step simplifies the geometry of the range, but this may not directly correspond to the perceived width and height experienced by the listener. These perceptual effects should be handled by the rendering step, where the perceived width and height are calculated from the geometry and shape of the range.

[0055] 2.1 Finding the descriptive point

[0056] The process of finding the 2-dimensional plane representing an audio object begins by finding anchor points on, within, or near the range as seen from the listener's position, as well as the outer edge of that range. Therefore, the following steps can be performed:

[0057] (1) Locate anchor points on, within, or near the range. The 2D plane defined in the following steps will be fitted such that it passes through the anchor points. The line between the listener's listening position and this point will serve as a reference direction, and the angles of the following points will be calculated based on this reference direction.

[0058] (2) Locate the point representing the right edge of the range from the listener's position. This point corresponds to the point with the maximum azimuth angle relative to the reference direction.

[0059] (3) Locate the point representing the left edge of the range from the listener's position. This point corresponds to the point with the smallest azimuth angle relative to the reference direction.

[0060] (4) Locate the point representing the top edge of the range from the listener's position. This point corresponds to the point with the maximum elevation angle relative to the reference direction.

[0061] (5) Locate the point representing the bottom edge of the range from the listener's position. This point corresponds to the point with the minimum elevation angle relative to the reference direction.

[0062] The five points found during this process will be marked as descriptive points for the representation of that range. Figure 3A An example of the process of finding descriptive points is shown. That is, Figure 3A This demonstrates how to find the descriptive point for a grid-based range 302. For simplicity, Figure 3A This is a bird's-eye view, meaning the elevation axis is not shown. Line 310 between the listener's position and anchor point (A) on range 302 is used as the basis for angle measurements, where the azimuth in the direction of the anchor point is 0 and increases to the right and decreases to below zero to the left. Furthermore, the right (B) and left (C) are found as points on this range with the maximum corresponding minimum azimuth. The top and bottom edges are points with the maximum corresponding minimum elevation.

[0063] 2.1.1 Finding Anchor Points

[0064] An anchor point is a point that will serve as the base point of the resulting 2D plane or 1D line. The plane or line will be defined so that it passes through this point, thus the anchor point will largely define the distance and angle to the listener. Different methods for finding anchor points are described below.

[0065] 2.1.1.1 Use the nearest point as the anchor point

[0066] A straightforward method for finding anchor points is to use the point on the range closest to the listener's position. The correct method for finding the closest point on a range will vary depending on the type of representation used. For example, a basic cylinder has no vertices, so a parametric model based on its radius, length, position, and rotation is needed to find the closest point on its surface. A box shape has vertices, but the closest point might be somewhere on one of its surfaces rather than exactly on one of the vertices. For some shapes, the closest point is easy to calculate (such as for a sphere), while for others, approximations may be necessary because identifying the exact closest point would be overly complex.

[0067] For complex mesh shapes, a finite number of points can be evaluated, representing both vertices and points on the surface. Then, each of these points is evaluated to identify which point is the closest by calculating its Euclidean distance D to the listening position: D = |P| k -P L |, where P k P is the 3-dimensional position of point k within this range. L It is the listener's listening position.

[0068] 2.1.1.2 Alternative methods for determining anchor points

[0069] As an alternative to using the nearest point as the anchor point, the spatial average of points over a range (or, in some embodiments, within a range) can be used to select the anchor point.

[0070] In one particular embodiment, the anchor point is found by determining the geometric "centroid" of a portion of the range "visible" from the listener's listening location.

[0071] The "visible" portion of this range can be defined by the largest outline on the surface of the object that has a direct line of sight from the listener's current listening position. For example, for a spherical range, the outline would be a circle with the same radius as the sphere, and the "visible" portion of the range would be the front hemisphere of the spherical range relative to the listener's listening position.

[0072] For a cube of length L, where one of the cube's vertical sides faces the listener, the outline will be of size L. The rectangle represents the portion of the cube's surface that faces the listener. For more arbitrary shapes, the outline can also be of any shape, and the visible portion of the range's surface will typically be more than half of the total surface area of ​​the range.

[0073] Once the "visible" portion is determined, the anchor point can be found by locating the geometric centroid of that visible portion (e.g., by spatially averaging the position vectors of all points on the visible portion of the range). In the example of a spherical range, the anchor point would therefore be located at a point on the line between the listener's listening position and the center of the range, somewhere between the surface and the center of the spherical range. Thus, in this example, the anchor point is located on the same line as the listener and the nearest point used in the first embodiment, but at a greater distance from the listener. This is generally not the case for ranges of arbitrary shapes—that is, for ranges of arbitrary shapes, the anchor point and the nearest point derived as described above will generally not be located on the same line as the listener's listening position.

[0074] For the example of elliptical range 304, Figure 3B The diagram illustrates the difference between using the nearest point (A) on the range as the anchor point and using the geometric centroid (B) of the visible portion of the range as the anchor point. Note that the visible portion of the range is represented by a thick line segment in this figure.

[0075] When the range is defined in a discrete manner (e.g., by a set of grid vertices) and the distribution of points is non-uniform (as seen from the listener's position), this non-uniformity of spatial sampling of the range surface can be illustrated by appropriate weighting of the position vectors used in the spatial averaging process to determine the anchor points.

[0076] In other alternative embodiments, the anchor point can be determined as the geometric centroid of the entire spatial extent rather than just the "visible" portion of that extent. This will produce an anchor point located at the spatial centroid of the volume enclosed by the extent. In a simple example of a spherical extent, it would be located at the center of the sphere. Figure 3B An alternative anchor point (C) for an example elliptical range is also shown. This alternative embodiment can be applied to smaller volumetric sound sources, while the embodiment using the "visible" portion of the range is more suitable for larger volumetric sound sources.

[0077] In yet another alternative embodiment, the anchor point is determined as the geometric centroid of a finite region surrounding the range of the nearest point as determined by the method described in Section 2.1.1.1.

[0078] One advantage of the alternative embodiments described in this section is that the position of the anchor point changes more smoothly with variations in the listener's listening position and / or the object's position / or orientation, especially for irregularly shaped applications, compared to when the nearest point is used as the anchor point. Therefore, the position and orientation of the fitted 2D plane also change more smoothly with variations in the listener's position and / or the object's position / or orientation. In particular, these alternative embodiments do not exhibit the problem of abrupt changes in the fitted 2D plane due to discontinuous jumps in the anchor point, as will be discussed in more detail below. This advantage comes at the cost of more complex calculations of the anchor point's position, although, on the other hand, this embodiment may not require the smoothing processes described below in Section 2.3.

[0079] 2.1.2 Finding the description point given the anchor point

[0080] Given an anchor point, there are many ways to determine other descriptive points. For example, the following describes one method using a grid shape.

[0081] The azimuth angle can be determined by evaluating the vector between the listening position and the point being evaluated, relative to the vector between the listening position and the anchor point of the range. Find the left and right sides of this range: azimuth.

[0082]

[0083] Among them, v k It is the vector between the listener's location and a point k within that range, v c This is the vector between the listener's location and the anchor point within that range, with the elevation components of both vectors set to zero. In this example, such as... Figure 3A As depicted in the text, the azimuth angle is defined as zero in the direction of the anchor point, increasing to the right and decreasing to the left.

[0084] Similarly, the top and bottom edges are found by evaluating the elevation angle of the vectors from the listening position and the evaluated point relative to the horizontal plane of the scene. The point v within this range... k (with coordinates (x, y, z)) Elevation angle θ relative to the listening position k It can be calculated as: elevation angle,

[0085]

[0086] Here, a coordinate system is defined such that the z-component represents the upward direction in the scene.

[0087] 2.2 Definition of 2-dimensional plane or 1-dimensional line representation

[0088] After determining the descriptive points, the next step may include defining a 2-dimensional plane. Figure 4An example of this process is shown. For simplicity, Figure 4 This is a bird's-eye view, which means that the elevation axis is not shown. For example... Figure 4 As shown, plane 402 is defined such that it passes through the anchor point (A) of range 302 at a horizontal angle α, which matches the angle between the line from the listener's position to the anchor point and the line from the right (B) to the left (C) of the range. The outer edge of this plane should be aligned with the minimum and maximum azimuth angles of the corresponding outer edge of the range. The same method is used when calculating the height and vertical tilt of this plane, using the top and bottom descriptor points as a basis.

[0089] In another embodiment, a simplified 1-dimensional representation is used, where the height aspect of the plane is ignored and the representation is a line passing through the anchor point (A) at a horizontal angle α, which matches the angle between the line from the listener's listening position to the anchor point and the line from the right (B) to the left (C) of the range. The end of this line should be aligned with the minimum and maximum azimuth angles of the corresponding outer edges of the range. Based on this simplified representation, top and bottom edge description points are not required.

[0090] When the range representation selected from the simplified representation derived in the pre-rendering simplification step is a point source, the computation of 2-dimensional plane or 1-dimensional line representation can be bypassed.

[0091] Alternatively, the 2D plane can be fitted through the anchor point and oriented perpendicular to the line from the listener's position to the anchor point. In this case, the points of the aforementioned edges are still used to determine the "opening angle" of the range and its 2D planar representation (as seen from the listener's position), but the orientation of this 2D plane is always perpendicular to the line from the listener to the anchor point. For certain range shapes, this representation may be preferable.

[0092] 2.3 Range Description of Movement Synchronization Smoothness

[0093] A potentially important aspect of real-time simplification is that changes in position and range may need to be smoothed so that listeners do not experience unexpected, abrupt changes in the spatial representation of audio objects.

[0094] As the listener or audio object moves or rotates, the descriptive point of the range representation will sometimes jump from one point to another. This can also occur when switching between different representations, as defined in the pre-rendering simplification step. Any such abrupt jumps need to be smoothed out to avoid generating unnatural audio artifacts. However, normal temporal smoothing is not suitable here, as it may make audio objects appear to be moving even when they are not moving.

[0095] A better approach is to use smoothing that is synchronized with the relative movement of the listener and the audio object. This means that changes in the range description are only updated when the listener moves or the audio object moves or rotates. This avoids situations where the range of the audio object changes when the listener is stationary, or where very small listener movements cause large changes in the position or shape of the audio object, which could occur in conjunction with time smoothing.

[0096] Figure 5 This illustrates an example where a very small movement by the listener triggers a jump in the anchor point's position. By measuring the change in the relative distance between the listener and the center position of the audio object, the change in the descriptive points can be synchronized with this change. This is accomplished by imposing constraints on the changes in the descriptive points so that they will never move beyond the change in relative distance. Figure 5 In the example, the nearest point is used as the anchor point. When the listener starts from t... n-1 Move the position to t n At this point, the nearest point of the range suddenly jumps from one edge to another (i.e., from the "previous anchor point" to the "target anchor point"). In this situation, very small changes in the listening position will cause a large discrepancy in the location of the range, which will feel unnatural. Even using time smoothing techniques cannot solve this problem. This can be addressed by using dD based on the change in the relative distance between the listener and the audio object. R Smoothing allows for more natural behavior. In this example, only the listener is moving, so changes in relative position are the same as changes in the listener's position. Smoothing will move the anchor point in the direction toward the target location, but only the movement is affected by changes in relative distance dD. R Limited distance.

[0097] The listener's rotation will not affect the range description, but the rotation of the audio object may. Therefore, when limiting changes in the range description, the rotation of the audio object should also be taken into account.

[0098] Now, a concrete example of smoothness proportional to changes in relative position will be described. Let D... R dD indicates the relative distance between the audio object and the listener's listening position. R Indicates the incremental change in relative distance. dD R Calculated as:

[0099] dD R =|(p AEn -p Ln )|-|(p AEn-1 -p Ln-1 )|

[0100] Where, p Aen and p Lnp represents the 3-dimensional position of the audio object and the listener at time n. AEn-1 and p Ln-1 It is the corresponding position at the previous moment.

[0101] The five distinct descriptive points of the range representation should all be interpolated / smoothed so that the maximum positional variation of each point is limited to dD. R This can be accomplished by first calculating the incremental change at each point as: dD vSmooth =min(dD) v ,αdD R ), where dD v The distance between the old location of the marker and the new target point, dD vSmooth The indicator is the smoothed distance used to update the descriptor point; α controls the update rate and dD. R The design constant for the relationship between them. In most cases, a value between 1.0 and 2.0 is preferred.

[0102] Therefore, the description point is updated to:

[0103]

[0104] Where, p pn It is the target description point, p pn-1 This is the previous descriptive point. Therefore, the change in descriptive point occurs in the direction of the new target descriptive point, but the distance is limited to a maximum change as large as the change in the relative position between the listener and the audio object. See also Figure 5 For example.

[0105] To include the rotation of audio objects as part of the smoothing, additional terms can be added to dD. R In the calculation, the rotation is calculated as a circular motion around the midpoint of a range at the radius of the anchor point. Furthermore, dD R Calculated as:

[0106] dD R =|(p AE1 -p L1 )|-|(p AE1 -p L1 )|+|dθr|

[0107] Here, dθ is the rotational change since the last iteration of the method, and r is the distance from the center point of the range to the anchor point to the listener's listening position. To capture rotations in all directions, dθ can be calculated as the sum of rotations around all axes.

[0108] 3. Rendering

[0109] The two steps described earlier, simplifying the 3D geometry of the audio object into a 2D plane or a 1D line, have been completed. The rendering step uses this simplified description of the geometry and spatial rendering techniques to render the audio object to the listener. That is, this rendering step uses the simplified description and spatial rendering techniques to render the audio signal (e.g., a multi-channel audio signal) representing the audio object. Different spatial rendering techniques may be used depending on the speaker configuration used; for example, binaural rendering is typically used for headphones, while speaker translation techniques are typically used for other speaker setups. In some cases, rendering is done in intermediate formats, such as Ambisonics or certain specific multi-channel audio formats.

[0110] To convey spatial information from a multi-channel audio object, different channels should be rendered so that they represent the expected size of the object. For a stereo audio object, two channels typically represent spatial information in a horizontal plane, and therefore should be rendered as the left and right sides of the object. This can be done by using two sources, one representing the right side and the other the left side, respectively, representing a simplified range. In the simplest implementation, these sources can be point sources, but in more advanced implementations, they can use head-related transfer functions (HRTFs) that better describe the range of a 2-dimensional plane. For multi-channel sources that include channels that also describe spatial information in the elevation dimension, more sources are needed to represent the elevation dimension.

[0111] An example of the rendering configuration described above is in Figure 6A , Figure 6B and Figure 6C The figures are shown in the diagram. These figures illustrate three different methods for rendering audio objects represented by a 2-dimensional plane using multiple virtual sound sources. Figure 6A In the example, two point sources are used to render an audio object with a stereo audio signal. Figure 6B In the example, two virtual sound sources with associated HRTFs, representing sub-regions of a 2-dimensional plane, are used to render an audio object with a stereo audio signal. Specifically, Figure 6B This illustrates how a 2D plane can be divided into segments that collectively cover the entire 2D plane, and how each segment can be rendered using a virtual sound source with an associated HRTF representing the entire segment. Figure 6C In the example, four virtual point sources are used to render an audio object with a four-channel audio signal, where different channels represent spatial information in both the horizontal and vertical planes.

[0112] Additional processing of the virtual sound source signal may be required before binaural rendering to achieve a perceptually correct spatial distribution on the plane (e.g., using mid / side blending techniques). If the shape is relatively long in one dimension, it can be rendered as a line source, where the special behavior of line sources can be taken into account.

[0113] 4. Example Implementation

[0114] Figure 7 An example system 700 (also known as a renderer) for generating sound for XR scenes is shown. System 700 includes a controller 701, a signal modifier 702 for modifying an audio signal 751 (e.g., a multi-channel audio signal), a left speaker 704, and a right speaker 705. Although in Figure 7 An audio signal and two speakers are shown, but this is for illustrative purposes only and does not limit the embodiments of this disclosure in any way.

[0115] The controller 701 can be configured to receive one or more parameters and trigger the modifier 702 to modify the audio signal 751 based on the received parameters (e.g., increase or decrease the volume level). The received parameters include (1) information 753 about the listener's location (e.g., direction and distance to the audio source) and (2) metadata 754 about the audio object, such as transformed metadata describing a 2D plane or 1D line as described herein (in some embodiments, the controller 701 generates the metadata 754 itself).

[0116] In some embodiments of this disclosure, information 753 can be obtained from... Figure 8A One or more sensors are provided in the XR system 800 shown. Figure 8A As shown, the XR system 800 is configured to be worn by the listener. Figure 8BAs shown, the XR system 800 may include a direction sensing unit 801, a position sensing unit 802, and a processing unit 803 coupled to a controller 851 of the system 800. The direction sensing unit 801 is configured to detect changes in the listener's orientation and provide information about the detected changes to the processing unit 803. In some embodiments, given a detected change in orientation detected by the direction sensing unit 801, the processing unit 803 determines the absolute orientation (relative to a coordinate system). Different systems may also be used to determine orientation and position, such as systems using a lighthouse tracker (LiDAR). In one embodiment, given a detected change in orientation, the direction sensing unit 801 may determine the absolute direction (relative to a coordinate system). In this case, the processing unit 803 may multiplex only the absolute orientation data from the direction sensing unit 801 and the absolute position data from the position sensing unit 802. In some embodiments, the direction sensing unit 801 may include one or more accelerometers and / or one or more gyroscopes.

[0117] Figure 9 This is a flowchart illustrating a process 900 for representing an audio object in an extended reality scene. Process 900 may begin at step s902.

[0118] Step s902 includes obtaining first metadata that describes a first three-dimensional (3D) shape associated with the audio object.

[0119] Step s904 includes transforming the obtained first metadata to produce transformed metadata that describes a two-dimensional (2D) plane or a one-dimensional (1D) line. The 2D plane or 1D line represents at least a portion of the audio object.

[0120] In some embodiments, obtaining the first metadata includes selecting the first metadata from a set of metadata that includes the first metadata and the second metadata, wherein the second metadata includes a description of a second 3D shape associated with an audio object, and the second 3D shape described by the second metadata takes the form of a mesh structure, a box shape, a sphere, an ellipsoid, or a cylinder with fewer vertices than the first 3D shape described by the first metadata.

[0121] In some embodiments, the metadata set further includes third metadata that describes a single point in an extended reality scenario.

[0122] In some embodiments, the selection is based on one or more of the following: the distance between the audio object and the listener's listening position in the extended reality scene and at least one threshold distance parameter (e.g., D1 or D2), the size of the audio object, the number of currently active audio objects in the extended reality scene, the current load of the renderer that will be used to render the audio object, or the current audio energy level of the audio object.

[0123] In some embodiments, transforming the obtained first metadata to produce transformed metadata includes: determining a set of descriptive points, wherein the set of descriptive points includes anchor points; and using these descriptive points to determine a 2D plane or 1D line, wherein the 2D plane or 1D line passes through the anchor point. For many 3D shapes, the anchor point is inside or on the 3D shape, but for some other 3D shapes (e.g., concave shapes), the anchor point is not on or inside the 3D shape but is typically close to the 3D shape.

[0124] In some embodiments, the anchor point is a point on the surface of the 3D shape that is closest to the listener's listening position in the extended reality scene.

[0125] In some embodiments, the anchor point is a spatial average of points on or within a 3D shape.

[0126] In some embodiments, a portion of the shape is visible to a listener in an extended reality scene, and the anchor point is the centroid of the portion of the shape visible to the listener.

[0127] In some embodiments, the set of descriptive points further includes: a first point on the first 3D shape, representing a first side of the first 3D shape relative to the listener's listening position, and a second point on the first 3D shape, representing a second side of the first 3D shape relative to the listening position.

[0128] In some embodiments, the method further includes: determining the dimensions (e.g., width, height, length) of the 2D plane or 1D line based on the first point and the second point.

[0129] In some embodiments, the method further includes: determining the horizontal angle of a 2D plane or a 1D line based on a first point and a second point.

[0130] In some embodiments, the set of descriptive points further includes: a third point on the first 3D shape, representing a third side of the first 3D shape relative to the listening position, and a fourth point on the first 3D shape, representing a fourth side of the first 3D shape relative to the listening position.

[0131] In some embodiments, the method further includes: determining the dimensions (e.g., width, height, length) of the 2D plane or 1D line based on the third and fourth points.

[0132] In some embodiments, the method further includes: determining the vertical angle of a 2D plane or a 1D line based on a third point and a fourth point.

[0133] In some embodiments, the audio object is represented by a multi-channel audio signal, and the method further includes rendering the multi-channel audio signal using transformed metadata.

[0134] Figure 10 This is a block diagram of an apparatus 1000 for performing the methods disclosed herein, according to some embodiments (e.g., the renderer 700 may be implemented using the apparatus 1000). Figure 10 As shown, device 1000 may include: a processing circuit (PC) 1002, which may include one or more processors (P) 1055 (e.g., a general-purpose microprocessor and / or one or more other processors, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), which may coexist in a single enclosure or a single data center, or may be geographically distributed (i.e., device 1000 may be a distributed computing device); at least one network interface 1048, which includes a transmitter (Tx) 1045 and a receiver (Rx) 1047, for enabling device 1000 to transmit data to and receive data from other nodes of network 110 (e.g., an Internet Protocol (IP) network) to which network interface 1048 is (directly or indirectly) connected (e.g., network interface 1048 may be wirelessly connected to network 110, in which case network interface 1048 is connected to an antenna device); and a storage unit (also referred to as a "data storage system") 1008, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 1002 includes a programmable processor, a computer program product (CPP) 1041 may be provided. CPP 1041 includes a computer-readable medium (CRM) 1042 storing a computer program (CP) 1043 including computer-readable instructions (CRI) 1044. CRM 1042 may be a non-transitory computer-readable medium, such as a magnetic medium (e.g., a hard disk), an optical medium, a memory device (e.g., random access memory, flash memory), etc. In some embodiments, the CRI 1044 of the computer program 1043 is configured such that, when executed by PC 1002, these CRIs cause device 1000 to perform the steps described herein (e.g., the steps described herein with reference to the flowcharts). In other embodiments, device 1000 may be configured to perform the steps described herein without requiring code. That is, for example, PC 1002 may consist only of one or more ASICs. Therefore, the features of the embodiments described herein may be implemented in hardware and / or software.

[0135] 5. Overview of various embodiments

[0136] A1. A method (900) for representing an audio object relative to a listener's listening position in an extended reality scene, the method comprising: obtaining (s902) first metadata, the first metadata describing a first three-dimensional (3D) shape associated with the audio object; and transforming (s904) the obtained first metadata to produce transformed metadata, the transformed metadata describing a two-dimensional (2D) plane or a one-dimensional (1D) line, wherein the 2D plane or the 1D line represents at least a portion of the audio object.

[0137] A2a. The method according to embodiment A1, wherein obtaining the first metadata includes: selecting the first metadata from a set of metadata including the first metadata and the second metadata, wherein the second metadata describes a second 3D shape associated with the audio object (in one embodiment, the second 3D shape described by the second metadata has fewer vertices than the first 3D shape described by the first metadata).

[0138] A2b. The method according to embodiment A2a, wherein the second 3D shape takes the form of one of the following: a mesh structure having fewer vertices than the first 3D shape described by the first metadata, a box shape, a sphere, an ellipsoid, or a cylinder.

[0139] A3. The method according to embodiment A2a or A2b, wherein the metadata set further includes third metadata, the third metadata describing a single point in the extended reality scenario.

[0140] A4. The method according to embodiments A2a, A2b or A3, wherein the selection is based on one or more of the following: the distance between the audio object and the listener's listening position in the extended reality scene and at least one threshold distance parameter, the size of the audio object, the number of currently active audio objects in the extended reality scene, the current load of the renderer to render the audio object, or the current audio energy level of the audio object.

[0141] A5. The method according to any one of embodiments A1-A4, wherein transforming the obtained first metadata to produce transformed metadata includes: determining a set of description points, wherein the set of description points includes anchor points; and using the description points to determine the 2D plane or 1D line, wherein the 2D plane or 1D line passes through the anchor points.

[0142] A6. The method according to embodiment A5, wherein the anchor point is a point on the surface of the 3D shape that is closest to the listening position of the listener in the extended reality scene.

[0143] A7. The method according to embodiment A5, wherein the anchor point is a spatial average of points on or within the 3D shape.

[0144] A8. The method according to embodiment A5, wherein a portion of the shape is visible to the listener in the extended reality scene, and the anchor point is the centroid of the portion of the shape visible to the listener.

[0145] A9. The method according to any one of embodiments A5-A8, wherein the set of descriptive points further includes: a first point on the first 3D shape, which represents a first side of the first 3D shape relative to the listening position of the listener, and a second point on the first 3D shape, which represents a second side of the first 3D shape relative to the listening position of the listener.

[0146] A10. The method according to embodiment A9 further includes: determining the dimensions (e.g., width, height, length) of the 2D plane or 1D line based on the first point and the second point.

[0147] A11. The method according to embodiment A9 or A10 further includes: determining the horizontal angle of the 2D plane or 1D line based on the first point and the second point.

[0148] A12. The method according to any one of embodiments A5-A11, wherein the set of descriptive points further includes: a third point on the first 3D shape, which represents the third side of the first 3D shape relative to the listening position of the listener, and a fourth point on the first 3D shape, which represents the fourth side of the first 3D shape relative to the listening position.

[0149] A13. The method according to embodiment A12 further includes: determining the dimensions (e.g., width, height, length) of the 2D plane or 1D line based on the third point and the fourth point.

[0150] A14. The method according to embodiment A12 or A13 further includes: determining the vertical angle of the 2D plane or 1D line based on the third point and the fourth point.

[0151] A15. The method according to any one of embodiments A5-A14, wherein the position of at least one descriptive point is smoothed by using a technique in which the magnitude of the positional change between two time instances is limited to being at most proportional to the change in the relative distance between the audio object and the listener between the same two time instances.

[0152] A16. The method according to embodiment A16, wherein the rotational change of the range of the audio object is considered by calculating the expected positional change of the anchor point, and wherein the positional change of the descriptive point is limited to being proportional at most to the sum of the following two: the magnitude of the expected positional change of the anchor point, and the magnitude of the change in the relative position between the audio object and the listener.

[0153] A17. The method according to any one of embodiments A1-A16, wherein the audio object is represented by a multi-channel audio signal, and the method further comprises: rendering the multi-channel audio signal using the transformed metadata.

[0154] A18. The method according to embodiment A17, wherein a virtual source representing the edges of the 2D shape or 1D line is used to render the multi-channel audio signal.

[0155] A19. The method according to embodiment A17 or A18, wherein the multi-channel audio signal is rendered using a virtual sound source having an associated head-related transfer function (HRTF) representing a sub-region of the 2D shape or 1D line.

[0156] B1. An apparatus (1000) for representing an audio object in an extended reality scene, the apparatus being configured to: obtain (s902) first metadata, the first metadata describing a first three-dimensional 3D shape associated with the audio object; and transform (s904) the obtained first metadata to produce transformed metadata, the transformed metadata describing a two-dimensional 2D plane or a one-dimensional 1D line, wherein the 2D plane or the 1D line represents at least a portion of the audio object.

[0157] B2. The apparatus according to embodiment B1, wherein the apparatus is further configured to perform the method according to any one of embodiments A2-A19.

[0158] B3. An apparatus (1000) for representing an audio object in an extended reality scene, the apparatus comprising: a storage unit (1008); and processing circuitry (1002) coupled to the storage unit, wherein the apparatus is configured to: obtain (s902) first metadata, the first metadata describing a first three-dimensional 3D shape associated with the audio object; and transform (s904) the obtained first metadata to generate transformed metadata, the transformed metadata describing a two-dimensional 2D plane or a one-dimensional 1D line, wherein the 2D plane or the 1D line represents at least a portion of the audio object.

[0159] B4. The apparatus according to embodiment B3, wherein the storage unit (1008) includes a memory (1042) storing instructions for configuring the apparatus to perform the method according to any one of embodiments A1-A19.

[0160] C1. A computer program (1043) comprising instructions that, when executed by a processing circuit (1002), cause the processing circuit to perform the method according to any one of embodiments A1-A19.

[0161] C2. A carrier comprising a computer program according to embodiment C1, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium (1042).

[0162] While various embodiments have been described herein, it should be understood that they are presented by way of example only and not as limitations. Therefore, the breadth and scope of this disclosure should not be limited to any of the exemplary embodiments described above. Furthermore, unless otherwise stated herein or otherwise clearly contradicted by the context, this disclosure includes any combination of the objects described above in all possible variations thereof.

[0163] Furthermore, although the process described above and illustrated in the accompanying figures is shown as a series of steps, this is merely for illustrative purposes. Therefore, it is conceivable that some steps can be added, some steps can be omitted, the order of the steps can be rearranged, and some steps can be performed in parallel.

[0164] References

[0165] [1] MPEG-H3D Audio, Section 8.4.4.7: "Spreading"

[0166] [2] MPEG-H3D Audio, Section 18.1: "Element Metadata Preprocessing"

[0167] [3] MPEG-H3D Audio, Item 18.11: "Diffuseness Rendering"

[0168] [4] EBU ADM Renderer Technology 3388, Section 7.3.6: "Divergence"

[0169] [5] EBU ADM Renderer Technology 3388, Section 7.4: "Decorrelation Filters"

[0170] [6] EBU ADM Renderer Technology 3388, Section 7.3.7: "Extent Panner"

[0171] [7] "Efficient HRTF-based Spatial Audio for Area and Volumetric Sources", IEEE Transactions on Visualization and Computer Graphics, 22(4): 1-1, January 2016

[0172] [8] "Surface Simplification Using Quadric Error Metrics", ACM SIGGRAPH Computer Graphics, July 1997

[0173] [9] "Minimum volumebounding box decomposition for shape approximation in robot grasping", IEEE International Conference on Robotics and Automation, 2008.

Claims

1. A method (900) for representing an audio object relative to a listener's listening position in an extended reality scene, the method comprising: Obtain (s902) first metadata, the first metadata describing a first 3D shape associated with the audio object, wherein the audio object is represented by a multi-channel audio signal; and The obtained first metadata is transformed (s904) to produce transformed metadata, which describes a 2D plane, wherein, The 2D plane represents at least a portion of the audio object, and Transforming the obtained first metadata to produce the transformed metadata includes: Define a set of descriptive points, wherein the set of descriptive points includes Anchor point A first point on the first 3D shape, representing a first side of the first 3D shape relative to the listener's listening position, wherein the first point on the first 3D shape is determined such that the first point has the maximum azimuth angle relative to a reference direction. A second point on the first 3D shape, representing a second side of the first 3D shape relative to the listener's listening position, wherein the second point on the first 3D shape is determined such that the second point has a minimum azimuth angle relative to the reference direction. A third point on the first 3D shape, representing the third side of the first 3D shape relative to the listener's listening position, wherein the third point on the first 3D shape is determined such that the third point has the maximum elevation angle relative to the reference direction, and A fourth point on the first 3D shape, representing the fourth side of the first 3D shape relative to the listening position, wherein the fourth point on the first 3D shape is determined such that the fourth point has a minimum elevation angle relative to the reference direction. Wherein, the reference direction is the line between the listener's listening position and the anchor point; The 2D plane is determined using the described point, wherein the 2D plane passes through the anchor point; and The transformed metadata is used to render the multi-channel audio signal.

2. The method according to claim 1, wherein, Obtaining the first metadata includes: selecting the first metadata from a metadata set that includes the first metadata and second metadata, wherein the second metadata describes a second 3D shape associated with the audio object.

3. The method according to claim 2, wherein, The second 3D shape takes one of the following forms: A mesh structure having fewer vertices than the first 3D shape described by the first metadata. Box-shaped spherical, ellipsoidal Cylindrical.

4. The method according to claim 2 or 3, wherein, The metadata set further includes third metadata, which describes a single point in the extended reality scenario.

5. The method according to claim 2 or 3, wherein, The selection is based on one or more of the following: In the extended reality scenario, the distance between the audio object and the listener's listening location, and at least one threshold distance parameter. The size of the audio object The number of currently active audio objects in the extended reality scene. The current load of the renderer used to render the audio object, or The current audio energy level of the audio object.

6. The method according to claim 1, wherein, The anchor point is a point on the surface of the 3D shape that is closest to the listener's listening position in the extended reality scene.

7. The method according to claim 1, wherein, The anchor point is the spatial average of points on or within the 3D shape.

8. The method according to claim 1, wherein, A portion of the shape is visible to the listener in the extended reality scene, and The anchor point is the centroid of the portion of the shape that is visible to the listener.

9. The method of claim 1, further comprising: Based on the first point and the second point, the first dimension of the 2D plane is determined.

10. The method according to claim 1 or 9, further comprising: Based on the first point and the second point, the horizontal angle of the 2D plane is determined.

11. The method of claim 1, further comprising: Based on the third and fourth points, the second dimension of the 2D plane is determined.

12. The method according to claim 1 or 11, further comprising: Based on the third and fourth points, the vertical angle of the 2D plane is determined.

13. The method according to claim 1, wherein, The following technique is used to smooth the position of at least one descriptive point: the magnitude of the positional change between two time instances is limited to being at most proportional to the change in the relative distance between the audio object and the listener between the same two time instances.

14. The method according to claim 13, wherein, The rotational variation of the audio object's range is considered by calculating the expected positional change of the anchor point, wherein the positional change of the descriptive point is limited to being proportional at most to the sum of the following two: the magnitude of the expected positional change of the anchor point, and the magnitude of the change in the relative position between the audio object and the listener.

15. The method according to claim 1, wherein, The multi-channel audio signal is rendered using a virtual source representing the edges of a 2D shape.

16. The method according to claim 1 or 15, wherein, The multi-channel audio signal is rendered using a virtual sound source with an associated head-related transfer function (HRTF) representing a sub-region of a 2D shape.

17. An apparatus (1000) for representing audio objects in an extended reality scene, the apparatus being configured to: Obtain (s902) first metadata, the first metadata describing a first 3D shape associated with the audio object, wherein, The audio object is represented by multi-channel audio signals; as well as The obtained first metadata is transformed (s904) to produce transformed metadata, which describes a 2D plane, wherein, The 2D plane represents at least a portion of the audio object, and Transforming the obtained first metadata to produce the transformed metadata includes: Define a set of descriptive points, wherein the set of descriptive points includes Anchor point A first point on the first 3D shape represents a first side of the first 3D shape relative to the listener's listening position, wherein the first point on the first 3D shape is determined such that the first point has the maximum azimuth angle relative to a reference direction. A second point on the first 3D shape, representing a second side of the first 3D shape relative to the listener's listening position, wherein the second point on the first 3D shape is determined such that the second point has a minimum azimuth angle relative to the reference direction. A third point on the first 3D shape, representing the third side of the first 3D shape relative to the listener's listening position, wherein the third point on the first 3D shape is determined such that the third point has the maximum elevation angle relative to the reference direction, and A fourth point on the first 3D shape, representing the fourth side of the first 3D shape relative to the listening position, wherein the fourth point on the first 3D shape is determined such that the fourth point has a minimum elevation angle relative to the reference direction. Wherein, the reference direction is the line between the listener's listening position and the anchor point; The 2D plane is determined using the described point, wherein the 2D plane passes through the anchor point; and The transformed metadata is used to render the multi-channel audio signal.

18. The apparatus according to claim 17, wherein, The apparatus is further configured to perform the method according to any one of claims 2-16.

19. An apparatus (1000) for representing audio objects in an extended reality scene, the apparatus comprising: Storage unit (1008); as well as A processing circuit (1002) coupled to the memory cell, wherein the device is configured to: Obtain (s902) first metadata, the first metadata describing a first 3D shape associated with the audio object, wherein the audio object is represented by a multi-channel audio signal; and The obtained first metadata is transformed (s904) to produce transformed metadata, which describes a 2D plane, wherein, The 2D plane represents at least a portion of the audio object, and Transforming the obtained first metadata to produce the transformed metadata includes: Define a set of descriptive points, wherein the set of descriptive points includes Anchor point A first point on the first 3D shape represents a first side of the first 3D shape relative to the listener's listening position, wherein the first point on the first 3D shape is determined such that the first point has the maximum azimuth angle relative to a reference direction. A second point on the first 3D shape, representing a second side of the first 3D shape relative to the listener's listening position, wherein the second point on the first 3D shape is determined such that the second point has a minimum azimuth angle relative to the reference direction. A third point on the first 3D shape, representing the third side of the first 3D shape relative to the listener's listening position, wherein the third point on the first 3D shape is determined such that the third point has the maximum elevation angle relative to the reference direction, and A fourth point on the first 3D shape, representing the fourth side of the first 3D shape relative to the listening position, wherein the fourth point on the first 3D shape is determined such that the fourth point has a minimum elevation angle relative to the reference direction. Wherein, the reference direction is the line between the listener's listening position and the anchor point; The 2D plane is determined using the described point, wherein the 2D plane passes through the anchor point; and The transformed metadata is used to render the multi-channel audio signal.

20. The apparatus according to claim 19, wherein, The storage unit (1008) includes a memory (1042) that stores instructions for configuring the device to perform the method according to any one of claims 2-16.

21. A computer program product comprising a computer-readable storage medium having instructions stored thereon, the instructions causing the processing circuit (1002) to perform the method according to any one of claims 1-16 when executed by the processing circuit (1002).

22. A computer-readable storage medium (1042) having instructions stored thereon that, when executed by a processing circuit (1002), cause the processing circuit to perform the method according to any one of claims 1-16.

Images