Rendering Audio Elements

Abstract

A method for rendering an audio element (e.g., a spatially heterogeneous audio element), the audio element having a range and represented with a set of virtual speakers including a central virtual speaker, the method including selecting a position of the central virtual speaker based on a position of a listener and / or calculating an attenuation coefficient of the central virtual speaker.

Description

[Technical field]

[0001] SUMMARY OF THE DISCLOSURE Embodiments related to rendering audio elements are disclosed. [Background technology]

[0002] Spatial audio rendering is a process used to present audio in an extended reality (XR) scene (e.g., a virtual reality (VR), augmented reality (AR), mixed reality (MR) scene) that gives the listener the impression that the sound comes from a physical source in the scene at a specific location and has a specific size and shape (i.e., extent). Presentation can be through headphone speakers or other speakers. When presentation is through headphone speakers, the process used is called binaural rendering and uses spatial cues of human spatial hearing that allow identifying from which direction a sound is coming. The cues involve interaural time delay (ITD), interaural level difference (ILD), and / or spectral differences.

[0003] The most common form of spatial audio rendering is based on the concept of point sources, where each sound source is defined as emanating from one specific point. Because each sound source is defined as emanating from one specific point, the sound sources do not have any size or shape. Different methods have been developed to render sound sources with extent (size and shape).

[0004] One such known method is to create multiple copies of a mono audio element at positions around the audio element. This configuration creates the perception of a spatially homogeneous object with a certain size. This concept is used for example in the "object spread" and "object divergence" features of the MPEG-H 3D Audio standard (see references [1] and [2]) and in the "object divergence" feature of the EBU Audio Definition Model (ADM) standard (see reference [4]). This idea of ​​using mono audio sources has been further developed as described in reference [7], where the area-volume geometry of the sound object is projected onto a sphere around the listener and the sound is rendered to the listener using a pair of head-related (HR) filters that are evaluated as the integral of all HR filters that cover the geometric projection of the object on the sphere. For spherical volumetric sources, this integral has an analytical solution. However, for arbitrary area-volume source geometries, the integral is evaluated by sampling the projected source surface on the sphere using so-called Monte Carlo ray sampling.

[0005] Another rendering method is to render a spatially diffuse component on top of the mono audio signal, which creates the perception of somewhat diffuse objects that, in contrast to the original mono audio elements, do not have a distinct pinpoint location. This concept is used, for example, in the "object diffuseness" feature of the MPEG-H 3D Audio standard (see Reference [3]) and the "object diffuseness" feature of the EBU ADM (see Reference [5]).

[0006] Combinations of the two above methods are also known: for example, the "object extent" feature of EBU ADM combines the creation of multiple copies of a mono audio element with the addition of a diffuse component (see reference [6]).

[0007] In many cases, the actual shape of an audio element can be adequately described by a basic shape (e.g., a sphere or a box). However, the actual shape may be more complex and may need to be described in a more detailed form (e.g., a mesh structure or a parametric description form).

[0008] However, these methods do not allow the rendering of audio elements with distinct spatially non-uniform characteristics, i.e. audio elements with an amount of spatial source variation within their spatial extent. Often these sources are composed of a sum of multiple sources (e.g. forest sounds or cheering crowd sounds). Most of these known solutions are only able to create objects with spatially homogeneous (i.e. no spatial variation within the element) or spatially diffuse characteristics that are too limited to render some of the above examples in a convincing way.

[0009] For heterogeneous audio elements, as described in Reference [8], the audio element contains at least two audio channels (i.e., audio signals) to describe the spatial variation over its range. Techniques exist for rendering these heterogeneous audio elements, where the audio element is represented by a multi-channel audio recording and the rendering uses a number of virtual speakers to represent the audio element and the spatial variation therein. By placing the virtual speakers at positions corresponding to the range of the audio element, the illusion of audio emanating from the audio element can be conveyed.

[0010] The number of virtual speakers required to achieve a reasonable spatial rendering of a spatially heterogeneous audio element depends on the range of the audio element. If the spatially heterogeneous audio element is small or at a certain distance from the listener, a two-speaker setup may be sufficient. However, for audio elements that are large and / or close to the listener, as shown in FIG. 1, a two-speaker setup may be too sparse and cause a psychoacoustical hole between the left speaker (SP-L) and the right speaker (SP-R) as the speakers are placed far apart. Adding a third center speaker helps to improve this effect. For this reason, a center speaker is used in most standardized multi-channel speaker setups. The simplest way to render a spatially heterogeneous audio element is by representing each of its audio channels as a virtual speaker, although the number of speakers may be less or more than the number of audio channels. If the number of virtual speakers is less than the number of audio channels, a downmixing step is required to derive the signals for each virtual speaker. If the number of virtual speakers is greater than the number of audio channels, an upmixing step is required to derive a signal for each virtual speaker. One implementation is to simply use two virtual speakers at fixed positions. Summary of the Invention [Problem to be solved by the invention]

[0011] Currently, there are several challenges. For example, rendering spatially heterogeneous audio elements usually requires the use of several virtual speakers. Although using a large number of speakers can be beneficial to have an evenly distributed audio representation of the range, when the source signal has a limited number of channels (e.g., a stereo signal), upsampling to a large number of speakers can cause problems where the spatial quality does not increase with more speakers. Also, using a large number of virtual speakers leads to undesirable high complexity. On the other hand, using too few virtual speakers can seriously impair the spatial characteristics of the audio elements, and the rendering may no longer be able to represent the corresponding audio elements well. Therefore, selecting the number of virtual speakers to render spatially heterogeneous audio elements is a trade-off between complexity and quality.

[0012] The aforementioned problem of the psychoacoustic hole between two speakers is well known and is especially problematic when the listener is not exactly located in the sweet spot of the speakers. For example, typical multi-speaker setups designed for home theater use are built under the assumption that the listener will be located somewhere around the sweet spot. These systems often have a center speaker centered between the left and right front speakers, but for XR audio rendering, when the listener is free to move around with six degrees of freedom, a static speaker setup is not ideal. In particular, the psychoacoustic hole problem can be accentuated as the listener approaches the range of the sound source.

[0013] Therefore, designing a static speaker setup with a limited number of speakers that provides a good spatial representation that works for any listening position is problematic. [Means for solving the problem]

[0014] Thus, in one aspect, a method is provided for rendering an audio element (e.g., a spatially heterogeneous audio element). The audio element has a range and is represented with a set of virtual speakers including a central virtual speaker. The method includes at least one of the steps of selecting a position of the central virtual speaker based on a position of a listener and calculating an attenuation coefficient of the central virtual speaker.

[0015] In another aspect, a computer program is provided that includes instructions that, when executed by a processing circuit of an audio renderer, cause the audio renderer to perform the above-mentioned method. In one embodiment, a carrier is provided that includes the computer program. The carrier is one of an electrical signal, an optical signal, a wireless signal, and a computer-readable storage medium. In another aspect, a rendering apparatus is provided that is configured to perform the above-mentioned method. The rendering apparatus may include a memory and a processing circuit coupled to the memory.

[0016] An advantage of the embodiments disclosed herein is that they provide an adaptive method for the placement of virtual speakers for rendering spatially heterogeneous audio elements having a range. The embodiments allow adapting the position of a virtual speaker representing the center of the range to the current listener position, such that the spatial distribution over the range is preserved with a small number of virtual speakers. Compared to simpler methods that distribute speakers evenly over the range of an audio element, the embodiments provide a more efficient solution that works for all listening positions without using a large number of virtual speakers. [Brief description of the drawings]

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate various embodiments.

[0018] [Figure 1] A diagram showing a psychoacoustic hall.

[0019] [Figure 2A]FIG. 1 illustrates an exemplary virtual speaker setup.

[0020] [Figure 2B] FIG. 1 illustrates an exemplary virtual speaker setup that can create a psychoacoustic hall.

[0021] [Diagram 3] FIG. 1 illustrates an exemplary virtual speaker setup.

[0022] [Figure 4] FIG. 1 illustrates an exemplary virtual speaker setup in which the center speaker is placed near the end speakers.

[0023] [Diagram 5] FIG. 1 illustrates one embodiment of how a preferred position for a center speaker may be determined.

[0024] [Figure 6] FIG. 13 illustrates another embodiment of how the preferred position of the center speaker may be determined.

[0025] [Figure 7] FIG. 1 illustrates one embodiment of how a preferred position for a center speaker may be determined when the audio element area is rectangular in shape.

[0026] [Figure 8] 1 is a flow chart illustrating a process according to some embodiments.

[0027] [Figure 9] 1 is a flow chart illustrating a process according to some embodiments.

[0028] [Figure 10] 1 is a flow chart illustrating a process according to some embodiments.

[0029] [Figure 11A] FIG. 1 illustrates a system according to some embodiments. [Figure 11B] FIG. 1 illustrates a system according to some embodiments.

[0030] [Figure 12] FIG. 1 illustrates a system according to some embodiments.

[0031] [Figure 13] FIG. 2 illustrates a signal modifier according to one embodiment.

[0032] [Figure 14] 1 is a block diagram of an apparatus according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] The present disclosure proposes, among other things, various methods for adapting the positions of virtual loudspeakers (or "speakers" for short) used to represent audio elements with ranges (e.g., spatially heterogeneous audio elements with a certain size and shape). By using several (e.g., two or more) speakers to represent the outer edges of the audio element and at least one speaker (hereinafter "middle speaker") adaptively positioned between the ends of the audio element or the ends of a simplified shape representing the audio element, an optimized rendering can be achieved with a few speakers with audio energy perceptually evenly distributed over the range of the audio element. Furthermore, the potential problem of undesirable excess energy due to the overlap with the extra center speaker and one of the side speakers is addressed.

[0034] overview

[0035] The objective is to render audio elements with a range such that for any listening position, the sound is perceived by the listener as being evenly distributed across the range. Embodiments avoid or reduce psychoacoustic hall problems by using as few speakers as possible.

[0036] In some embodiments, a set of speakers representing the ends of an audio element and a center speaker that is adaptively positioned to represent the center of the audio element are used to render the audio element, where the placement of the center speaker and / or the attenuation factor (e.g., gain factor) of the center speaker take into account the listening position (e.g., the position of the listener in virtual space relative to the audio element).

[0037] An example of such a rendering setup is shown in Figure 2A, which shows a region 200 representing an audio element. The region 200 representing an audio element may be the region of the audio element (i.e., the region 200 has the same size and shape as the actual region of the audio element) or may be a simplified region derived from the region of the audio element (e.g., a line or a rectangle). International Patent Application WO2021180820 describes various methods for generating such simplified regions.

[0038] FIG. 2A further shows that the left speaker 202 is located at the left endpoint 212 of the range 200, the right speaker 204 is located at the right endpoint 214 of the range, and the center speaker 203 is located somewhere between the left and right endpoints.

[0039] Advantageously, in one embodiment, the positioning of the center speaker 203 is controlled such that the center speaker 203 is located at or near a midpoint (MP) 220 between the first and second endpoints of the range when the listener is at least a certain distance (D) from the audio element. The distance D typically depends on the size of the range 200.

[0040] As shown in Figure 2B, when the listener moves closer to the audio element, keeping the center speaker 203 at or near the midpoint 220 can lead to problems with the psychoacoustic hall 240. Therefore, in this situation, the center speaker 203 is moved to a new position, as shown in Figure 3, so that a uniform spatial distribution is maintained.

[0041] Adaptive placement of center speaker

[0042] Some embodiments herein adaptively position the center speaker 203 based on the position of the listener. In many situations, the goal is to place the center speaker on a selected "anchor point" for the audio element, which can move with the listener. For example, in one embodiment, the anchor point is the point of the range 200 that is closest to the listener. However, in other embodiments, the anchor point can be defined differently. International Patent Application WO2021180820 describes different methods for selecting anchor points for audio elements. However, there are situations in which it is not advantageous to place the center speaker on the anchor point.

[0043] For example, when a listener is close to one of the ends of an audio element, if the center speaker is placed on the anchor point, the center speaker and the corresponding side speaker will overlap, resulting in an undesirable increase in energy from the corresponding side, as shown in Figure 4. In this situation, it is advantageous to place the center speaker closer to the midpoint between the left and right end points (e.g., the positions of the left and right speakers) in order to have more evenly distributed audio energy.

[0044] As another example, when the distance between the listener and an audio element is greater than a threshold (which may depend on the range of the audio element), experiments show that placing the center speaker closer to the midpoint 220 provides a more perceptually significant output.

[0045] In summary, if the listener is close to the range but not close to one of the range's ends, it is usually preferable to place the center speaker at the anchor point. In other listening positions, the preferred center speaker location is at or near the midpoint 220.

[0046] Therefore, the proposed embodiment provides an adaptive placement of the centre speaker, optimizing its position depending on the current listening position.

[0047] More specifically, in one embodiment, placing the center speaker at the anchor point is avoided when the anchor position is close to one of the end speakers. Thus, in addition to considering the anchor point (A) 590 (see FIG. 5), the positioning of the center speaker 203 may also depend on the midpoint (MP) 220.

[0048] In one embodiment, the preferred position of the center speaker 203 on line 599, indicated by "M" 591 from one endpoint of the range 200 (e.g., left endpoint 212) to another endpoint of the range 200 (e.g., right endpoint 214), is calculated using equation (1). M = α*A+(1-α)*MP (Eq. 1) where A is the position of the anchor point on line 599, MP is the position of the midpoint on line 599, and α ∈ [0,1] is a coefficient that controls the weighting of the anchor point and the midpoint in the positioning of the center speaker on line 599.

[0049] The value of α is such that when the listener is close to the range and not close to the end of the range, M is near or above the anchor point (α→1), and when the listener is close to the end of the range 200 or far from the range, M is near or above the midpoint (α→0). Thus, α takes into account the movement of the listener in both the x and z directions at the same time. Once point M 591 is determined, the center speaker 203 may be "placed" at point M. That is, the position of the center speaker 203 is set to point M.

[0050] In one embodiment, α is a function of two variables, xm_w and zm_w, i.e. α=f(xm_w, zm_w) (Eq. 2)

[0051] The first variable, xm_w, is a weight that reflects the movement of the listener in the x direction, and zm_w, a weight that reflects the movement of the listener in the z direction. The parameters β and λ, shown in Figure 5, are used to set the values ​​of xm_w and zm_w.

[0052] In one embodiment, xm_w is a function of λ and β, i.e.

number

[0053] From equation 3 and Figure 5, it can be seen that as the listener moves towards either end of the range 200, either λ → 0 or β → 0, and therefore xm_w → 0. Similarly, as the listener moves towards the center, λ and β approach each other and xm_w approaches 1.

[0054] The weight in the Z direction, zm_w, is also a function of λ and β. That is, zm_w=f2(λ,β)=sin((λ+β) / 2) (Equation 4)

[0055] From equation (4) and Figure 5, we can see that as the listener gets closer to the range, λ and β become larger; at very close distances, λ+β approaches 180 degrees and therefore zm_w approaches 1; and as the listener gets further away from the range, λ and β become smaller and therefore zm_w approaches 0.

[0056] In one embodiment, α is defined as: α=f3(xm_w,zm_w)=min(1,d*xm_w*zm_w) (Equation 5)

[0057] where d∈(0,+∞) is a tuning factor that controls the factor α. Experiments have shown that d≈2.2 produces desirable results. The above derivation of α is just one way of using information that represents the relative position of the listener to the range. Other ways of using λ and β (e.g., the cosine or tangent of λ and β) or other parameters that can reflect the relative position of the listener to the range (e.g., the coordinates of the listener, the midpoint, the anchor point, the left and right ends of the range) may be used to derive α.

[0058] Another approach to deal with excess energy when the listener is at the edge of the audio element may be to attenuate the energy of the signal played through the center speaker instead of changing its position. That is, the center speaker 203 is placed at the anchor point, but the audio signal for the center speaker 203 is attenuated when the listener approaches the edge of the audio element. To do so, the angles described in FIG. 5 may be used as follows:

number

[0059] where X is the original time-domain audio signal of the center speaker 203 and X′ is the time-domain signal reproduced by the center speaker 203. Although this approach alleviates the excess energy problem, it may not improve the spatial perception of the audio elements.

[0060] Adaptive placement of center speaker based on equal angles

[0061] In another embodiment, the placement of the center speaker is controlled only by the angles relative to the left and right endpoints. In FIG. 6, these angles are φ=λ+θ and Φ=β-θ, respectively. In one embodiment, point M 591 is selected such that Φ=φ. If the distance (dA) between the listener and the anchor point is known, the distance (dM) from point M 591 to point 212 can be found by calculating dM=dA*(tan(λ)+tan(θ)), so that the x-coordinate of point M is equal to the x-coordinate of the left endpoint 212+dM.

[0062] If dA is unknown, but v (the distance between the listener and the left endpoint 212) and w (the distance between the listener and the right endpoint 214) are known, then the location of point M can be found by calculating M=(v*Re+w*Le) / (v+w), where Re is the x-coordinate of the right endpoint 214 and Le is the x-coordinate of the left endpoint 212.

[0063] Adaptive placement of center loudspeaker in two dimensions

[0064] The examples provided above have shown one-dimensional audio element ranges. The techniques described above also apply to two-dimensional audio element ranges, such as the range 700 shown in FIG. 7. In this example, the range 700 has a rectangular shape and four endpoints are defined: top endpoint 701, right endpoint 702, bottom endpoint 703, and left endpoint 704. In this example, for each endpoint, the endpoint is located at the midpoint of the side on which it is located. Thus, the left endpoint 704 is exactly halfway between the top left and bottom left corners of the range 700, the right endpoint 702 is exactly halfway between the top right and bottom right corners of the range 700, the top endpoint 701 is exactly halfway between the top left and top right corners of the range 700, and the bottom endpoint 703 is exactly halfway between the bottom left and bottom right corners of the range 700.

[0065] In one embodiment, for each defined edge point 701-704, a speaker can be placed at that point. Thus, in one embodiment, four speakers are used to represent the top, bottom, left, and right edges of this two-dimensional plane 700. In another embodiment, a speaker is placed at each corner point of the plane 700.

[0066] In addition, a center speaker can be used and placed using the same principles as already explained, i.e., the coordinates (Mx, My) of the center speaker can be found by calculating: Mx = αx * A1x + (1-αx) * MPx, My = αy*A2y+(1-αy)*MPy where αx=f3(xm_w,zm1_w), αy=f3(ym_w,zm2_w), A1x is the x-coordinate of anchor point A1, A2y is the y-coordinate of anchor point A2, MPx is the x-coordinate of the midpoint, MPy is the y-coordinate of the midpoint, xm_w=f1(λx,βx), ym_w=f1(λy,βy), zm1_w=f2(λx,βx), zm2_w=f2(λy,βy).

[0067] 8 is a flow diagram illustrating a process 800 according to one embodiment for rendering an audio element (e.g., a spatially heterogeneous audio element). The audio element has a range and is represented with a set of virtual speakers including a center virtual speaker. Process 800 begins at step s802, which includes selecting a position of the center virtual speaker based on the position of the listener and / or calculating an attenuation coefficient for the center virtual speaker.

[0068] 9 is a flow diagram illustrating a process 900 according to one embodiment for rendering an audio element (e.g., a spatially heterogeneous audio element). The audio element has a range and is represented using a set of virtual speakers including a central virtual speaker. Process 900 begins at step s902.

[0069] Step s902 includes selecting an anchor point for the audio element. Typically, the anchor point is on a line passing through a first endpoint of a range associated with the audio element (e.g., the actual range of the audio element or the simplified range of the audio element), and the second endpoint of the range and the anchor point are dependent on the position of the listener. In some embodiments where the audio element has a complex range, selecting the anchor point is part of a process for creating a simplified range for the audio element based on the listener position and the range of the audio element.

[0070] Step s904 includes positioning a first speaker (e.g., a right speaker) and positioning a second speaker (e.g., a left speaker). That is, the positions of the first and second speakers are determined. In one embodiment, the speakers are positioned at opposite ends of a range, i.e., the left speaker is located at the left endpoint of the range and the right speaker is located at the right endpoint of the range. In an embodiment where the range associated with the audio element is a rectangle, the speakers are positioned at each corner of the rectangle.

[0071] Step s906 involves determining the midpoint between the two speakers.

[0072] Step s908 includes determining a first angle (λ) between a line from the listener to the anchor point and a line from the listener to the left speaker, and a second angle (β) between a line from the listener to the anchor point and a line from the listener to the right speaker (examples of λ and β are shown in FIG. 5).

[0073] Step s910 includes determining x weight values ​​(xm_w) and z weight values ​​(zm_w) (eg, calculating xm_w=f1(λ,β) and zm_w=f2(λ,β)).

[0074] Step s912 includes determining a coefficient (α) based on xm_w and zm_w, i.e., α is a function of xm_w and zm_w (e.g., α=f3(xm_w,zm_w)).

[0075] Step s914 involves calculating M=α*A+(1−α)*MP, where M is the x-coordinate of the preferred position of the center speaker, A is the x-coordinate of the anchor point, and MP is the x-coordinate of the midpoint between the left and right speakers.

[0076] 10 is a flow diagram illustrating a process 1000 according to one embodiment for rendering an audio element (e.g., a spatially heterogeneous audio element). The audio element has a range and is represented using a set of virtual speakers including a central virtual speaker. Process 1000 begins at step s1002.

[0077] Step s1002 includes selecting an anchor point for the audio element (see step s902 above).

[0078] Step s1004 includes placing a first speaker (e.g., a right speaker) and placing a second speaker (e.g., a left speaker), i.e., the positions of the first and second speakers are determined (see step s904 above).

[0079] Step s1006 includes determining a first angle (λ) between a line from the listener to the anchor point and a line from the listener to the left speaker, and a second angle (β) between a line from the listener to the anchor point and a line from the listener to the right speaker (examples of λ and β are shown in FIG. 5).

[0080] Step s1008 includes calculating a gain factor (g) using λ and β, for example, g=f4(λ,β).

[0081] Step s1010 involves processing the center speaker signal (X) with a gain factor (g) to generate a modified signal X', eg, X'=g*X.

[0082] Usage example

[0083] 11A shows an XR system 1100 to which the embodiments disclosed herein may be applied. The XR system 1100 includes speakers 1104 and 1105 (which may be speakers of headphones worn by a listener) and an XR device 1110 that includes a display for displaying images to a user and is configured to be worn by the listener in some embodiments. In the illustrated XR system 1100, the XR device 1110 has a display and is designed to be worn on the user's head, and is generally referred to as a head-mounted display (HMD).

[0084] As shown in FIG. 11B, the XR device 1110 may include an orientation detector 1101, a position detector 1102, and a processor 1103 coupled (directly or indirectly) to an audio renderer 1151 to generate output audio signals (e.g., a left audio signal 1181 for the left speaker and a right audio signal 1182 for the right speaker as shown).

[0085] The orientation sensing unit 1101 is configured to detect changes in the orientation of the listener and provide information about the detected changes to the processing unit 1103. In some embodiments, the processing unit 1103 determines an absolute orientation (with respect to some coordinate system) taking into account the detected changes in orientation detected by the orientation sensing unit 1101. There may also be different systems for determining orientation and position, such as, for example, systems using lighthouse trackers (lidars). In one embodiment, the orientation sensing unit 1101 may determine an absolute orientation (with respect to a coordinate system) given the detected changes in orientation. In this case, the processing unit 1103 may simply multiplex the absolute orientation data from the orientation sensing unit 1101 and the position data from the position sensing unit 1102. In some embodiments, the orientation sensing unit 1101 may comprise one or more accelerometers and / or one or more gyroscopes.

[0086] The audio renderer 1151 generates an audio output signal based on an input audio signal 1161, metadata 1162 about the XR scene the listener is experiencing, and information about the listener's position and orientation 1163. The metadata 1162 for the XR scene includes metadata for each object and audio element included in the XR scene, and the metadata for the objects may include information about the object's dimensions. The metadata 1162 may also include control information such as reverberation time values, reverberation level values, absorption parameters, etc. The audio renderer 1151 may be a component of the XR device 1110 or may be remote with respect to the XR device 1110 (e.g., the audio renderer 1151 or components thereof may be implemented in the so-called "cloud").

[0087] 12 illustrates an example implementation of an audio renderer 1151 for generating sound for an XR scene. The audio renderer 1151 includes a controller 1201 and a signal modifier 1202 for modifying an audio signal 1161 (e.g., an audio signal of a multi-channel audio element) based on control information 1210 from the controller 1201. The controller 1201 may be configured to receive one or more parameters and trigger the signal modifier 1202 to perform a modification of the audio signal 1161 (e.g., increase or decrease a volume level) based on the received parameters. The received parameters include information 1163 regarding the position and / or orientation of the listener (e.g., direction and distance relative to the audio element) and metadata 1162 regarding the audio element (e.g., range 200) in the XR scene (in some embodiments, the controller 1201 itself generates the metadata 1162). Using the metadata and position / orientation information, the controller 1201 can calculate one or more gain factors (g) (also known as attenuation factors) for audio elements in the XR scene, as described herein.

[0088] 13 illustrates an exemplary implementation of the signal modifier 1202 according to one embodiment. The signal modifier 1202 includes a directional mixer 1304, a gain adjuster 1306, and a speaker signal generator 1308.

[0089] The directional mixer receives an audio input 1161, which in this example includes a pair of audio signals 1301 and 1302 associated with an audio element (e.g., an audio element associated with range 200 or 700), and generates a set of k virtual speaker signals (VS1, VS2, ..., VSk) based on the audio input and control information 1391. In one embodiment, the signal for each virtual speaker may be derived by, for example, an appropriate mixing of signals including the audio input 1161. For example, VS1 = α x L + β x R, where L is the input audio signal 1301, R is the input audio signal 1302, and α and β are coefficients that depend, for example, on the position of the listener relative to the audio element and the position of the virtual speaker to which VS1 corresponds.

[0090] The gain adjuster 1306 may adjust the gain of any one or more of the virtual speaker signals based on the control information 1392, which may include the above-mentioned gain factors calculated by the controller 1301. That is, for example, when the center speaker 203 is placed near another speaker (e.g., the left speaker 202 shown in FIG. 4), the controller 1301 may control the gain adjuster 1306 to adjust the gain of the virtual speaker signal relative to the center speaker 203 by providing the gain adjuster 1306 with the gain factor calculated as described above.

[0091] Using the virtual speaker signals VS1, VS2, ..., VSk, the speaker signal generator 1308 generates output signals (e.g., output signal 1181 and output signal 1182) for driving speakers (e.g., headphone speakers or other speakers). In an embodiment where the speakers are headphone speakers, the speaker signal generator 1308 may perform conventional binaural rendering to generate the output signals. In an embodiment where the speakers are not headphone speakers, the speaker signal generator may perform conventional speaker panning to generate the output signals.

[0092] FIG. 14 is a block diagram of an audio rendering device 1400 for performing methods disclosed herein, according to some embodiments (e.g., the audio renderer 1151 may be implemented using the audio rendering device 1400). As shown in FIG. 14, the audio rendering device 1400 includes a processing circuit (PC) 1402. The processing circuit (PC) 1402 may include one or more processors (P) 1455 (e.g., a general-purpose microprocessor and / or one or more other processors, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.). The multiple processors may be co-located in a single enclosure or in a single data center, or may be geographically distributed (i.e., the audio rendering device 1400 may be a distributed computing device). The audio rendering device 1400 includes at least one network interface 1448. At least one network interface 1448 may comprise a transmitter (Tx) 1445 and a receiver (Rx) 1447 for enabling data to be transmitted to and received from other nodes connected to the network 110 (e.g., an Internet Protocol (IP) network) to which the network interface 1448 is connected (directly or indirectly). In that case, the network interface 1448 may be wirelessly connected to the network 110. The network interface 1448 is connected to an antenna arrangement and a storage unit (also referred to as a "data storage system") 1408, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In an embodiment in which the PC 1402 includes a programmable processor, a computer readable storage medium (CRM) 1442 may be provided. The CRM 1442 stores a computer program (CP) 1443 that includes computer readable instructions (CRI) 1444. The CRM 1442 may be a non-transitory computer readable storage medium, such as a magnetic medium (e.g., a hard disk), an optical medium, a memory device (e.g., a random access memory, a flash memory), or the like.In some embodiments, the CRI 1444 of the computer program 1443, when executed by the PC 1402, is configured to cause the audio-rendering device 1400 to perform the steps described herein (e.g., steps described herein with reference to flow charts). In other embodiments, the audio-rendering device 1400 may be configured to perform the steps described herein without the need for code; that is, for example, the PC 1402 may consist solely of one or more ASICs. Thus, features of the embodiments described herein may be implemented in hardware and / or software.

[0093] Overview of Various Embodiments

[0094] A1. A method for rendering an audio element (e.g., a spatially heterogeneous audio element), the audio element having a range and represented using a set of virtual speakers including a central virtual speaker, the method including at least one of the following steps: selecting a position of the central virtual speaker based on a position of a listener; and calculating an attenuation coefficient of the central virtual speaker.

[0095] A2. The method of embodiment A1, the method comprising selecting the position of the central virtual speaker based on the position of the listener, wherein selecting the position of the central virtual speaker based on the position of the listener includes determining a first angle based on the position of the listener and a position of i) a first endpoint of the audio element or of a range determined based on the range of the audio element, or ii) a first virtual speaker; determining a second angle based on the position of the listener and i) a second endpoint of the audio element or of the range, or ii) a second virtual speaker; and calculating a first coordinate Mx of the central virtual speaker using the first angle and the second angle, wherein the selected position of the central virtual speaker is identified at least in part by the calculated first coordinate.

[0096] A3. The method of embodiment A2, wherein selecting the position of the central virtual speaker based on the position of the listener further comprises: determining a third angle based on the position of the listener and i) a position of a third endpoint of the audio element or of the range, or ii) a third virtual speaker; determining a fourth angle based on the position of the listener and i) a position of a fourth endpoint of the audio element or of the range, or ii) a fourth virtual speaker; and calculating a second coordinate My of the central virtual speaker using the third angle and the fourth angle, wherein the selected position is specified at least in part by the calculated first and second coordinates.

[0097] A4. The method of embodiment A2 or A3, wherein the step of calculating the first coordinate Mx of the central virtual speaker using the first angle and the second angle includes the steps of determining the first coefficient α1 using the first angle and the second angle, and calculating Mx=(α1*Ax)+((1-α1)*MPx), where Ax is a coordinate used to specify the position of the determined anchor point on a line extending between the first end point and the second end point or between the first virtual speaker and the second virtual speaker, and MPx is a coordinate used to specify the midpoint of the line.

[0098] A5. The method of embodiment A4, wherein the step of determining α1 using the first angle and the second angle includes the steps of calculating a first weight xm_w using the first angle and the second angle, calculating a second weight zm_w using the first angle and the second angle, and determining α1 based on xm_w and zm_w.

[0099] A6. The method described in embodiment A5, wherein the step of determining α1 based on xm_w and zm_w includes the steps of determining whether d*xm_w*zm_w is less than 1 when a predetermined coefficient is d, and setting α1 to 1 if d*xm_w*zm_w is not less than 1, and otherwise setting α1 to d*xm_w*zm_w.

[0100] A7. The method of embodiment A5 or A6, wherein when the first angle is λ and the second angle is β, the step of calculating xm_w using the first angle and the second angle includes the step of calculating xm_w=sin(λ) / sin(β) or xm_w=sin(β) / sin(λ).

[0101] A8. The method of embodiment A5, A6 or A7, wherein calculating zm_w using the first angle and the second angle includes calculating zm_w=sin((λ+β) / 2).

[0102] A9. The method described in embodiment A3, wherein the step of calculating the second coordinate My of the central virtual speaker using the third angle and the fourth angle includes the steps of: determining a second coefficient α2 using the third angle and the fourth angle; and calculating My=(α2*Ay)+((1-α2)*MPy), where Ay is the coordinate used to specify the position of the determined anchor point on a line extending between the third end point and the fourth end point (or between the third virtual speaker and the fourth virtual speaker), and MPy is the coordinate used to specify the midpoint of the line.

[0103] A10. The method includes selecting the position of the central virtual speaker based on the position of the listener, the step of selecting the position of the central virtual speaker based on the position of the listener comprising: 1) selecting a position of the central virtual speaker between a first point (e.g., a first end point) of the audio element or of a range determined based on the range of the audio element and a second point (e.g., a second end point) of the audio element or of the range; or 2) selecting a position of the central virtual speaker between a first virtual speaker and a second virtual speaker. selecting a position point on a line such that an angle between i) a second line extending from the position of the listener to the first point (or the first virtual speaker) and ii) a third line extending from the position of the listener to the selected position point on the first line is equal to an angle between i) a fourth line extending from the position of the listener to the second point (or the second virtual speaker) and ii) the third line.

[0104] A11. The method of embodiment A10, wherein the step of selecting the position point includes a step of calculating coordinate M of the position point by calculating M=(v*Re+w*Le) / (v+w), where v is the length of the second line, w is the length of the third line, Re is the coordinate of the first point or the first virtual speaker, and Le is the coordinate of the second point or the second virtual speaker.

[0105] A12. The method of embodiment A10 or A11, further comprising placing the central virtual speaker at the selected position point.

[0106] A13. The method according to any one of embodiments A1 to A12, wherein the method includes a step of calculating an attenuation coefficient of the center virtual speaker based on the position of the listener, the step of calculating an attenuation coefficient of the center virtual speaker based on the position of the listener including the steps of determining a first angle based on the position of the listener and i) a position of a first endpoint of the audio element or of a range determined based on the range of the audio element, or ii) a position of a first virtual speaker, determining a second angle based on the position of the listener and a position of a second endpoint of the audio element or of the range, or ii) a position of a second virtual speaker, and calculating ε=sin(λ) / sin(β) or ε=sin(β) / sin(λ), where λ is the first angle, β is the second angle, and ε is the attenuation coefficient.

[0107] A14. The method of embodiment A13, further comprising modifying the center virtual speaker signal X to generate a modified center virtual speaker signal X' such that X' = ε * X, and rendering the audio element using the modified center virtual speaker signal (e.g., generating an output signal using the center virtual speaker signal).

[0108] A15. A method according to any one of embodiments A2 to A14, wherein the set of virtual speakers further includes a first virtual speaker positioned at the first endpoint and a second virtual speaker positioned at the second endpoint, and the method further comprises rendering the audio element using information identifying the positions of the virtual speakers.

[0109] A16. The method of embodiment A3 or A9, wherein the set of virtual speakers further includes a first virtual speaker positioned at the first endpoint (e.g., a first corner point, or a first point that is a midpoint between a first pair of corner points), a second virtual speaker positioned at the second endpoint (e.g., a second corner point, or a second point that is a midpoint between another pair of corner points), a third virtual speaker positioned at the third endpoint (e.g., a third corner point, or a third point that is a midpoint between another pair of corner points), and a fourth virtual speaker positioned at the fourth endpoint (e.g., a fourth corner point, or a fourth point that is a midpoint between another pair of corner points), the method further comprising a step of rendering the audio element using information identifying the positions of the virtual speakers.

[0110] A17. The method of embodiment A1, wherein the method includes selecting the position of the central virtual speaker based on the position of the listener, the selecting the position of the central virtual speaker based on the position of the listener including: determining a distance from the position of the listener to a position of the audio element; determining whether the determined distance is greater than a threshold; and, as a result of determining that the determined distance is greater than the threshold, selecting a position of the midpoint of the audio element or the midpoint of a range determined based on the range of the audio element.

[0111] A18. The method of embodiment A1, wherein the method has a step of selecting the position of the central virtual speaker based on the position of the listener, the step of selecting the position of the central virtual speaker based on the position of the listener comprising: i) obtaining listener information indicating a coordinate (e.g., an x ​​coordinate) of the listener; ii) obtaining midpoint information indicating a coordinate (e.g., an x ​​coordinate) of a midpoint between a first point associated with the audio element (e.g., a first endpoint of a range associated with the audio element) and a second point associated with the audio element (e.g., a second endpoint of the range); and iii) selecting the position of the central virtual speaker based on the midpoint information and the listener information.

[0112] A19. The method of embodiment A18, wherein the step of selecting the position of the central virtual speaker based on the midpoint information and the listener information includes the steps of: i) determining coordinates of an anchor point; and ii) selecting the position of the central virtual speaker based on the midpoint information and anchor information indicating the coordinates of the determined anchor point.

[0113] A20. The method of embodiment A19, wherein i) the midpoint information includes a midpoint value MP indicating the coordinates of the midpoint, ii) the anchor information includes an anchor value A indicating the coordinates of the anchor point, and iii) the step of selecting the position of the central virtual speaker based on the midpoint information and the anchor information includes a step of calculating a coordinate value M of the central speaker using MP and A.

[0114] A21. The method of embodiment A20, wherein the anchor value A depends on the indicated coordinates of the listener.

[0115] A22. The method of embodiment A21, wherein A=L, where L is the indicated coordinate of the listener (as shown in Figures 5 and 6, a coordinate system may be defined for the range such that the range extends along the x-axis).

[0116] A23. The method of embodiment A20, A21, or A22, wherein the step of calculating M using MP and A includes the step of calculating M=α*A+(1-α)*MP, where α is a coefficient that depends on the indicated coordinate of the listener.

[0117] A24. A method according to any one of embodiments A18 to A23, wherein the listener information further indicates a second coordinate (e.g., a y coordinate) of the listener, the midpoint information further indicates a second coordinate (e.g., a y coordinate) of the midpoint, and the step of selecting the position of the central virtual speaker based on the midpoint information and the listener information further includes a step of determining a second coordinate of the central virtual speaker based on the second coordinate of the listener and the second coordinate of the midpoint.

[0118] A25. A method according to any one of embodiments A10 or A18 to A24, wherein a first virtual speaker is positioned at the first point and a second virtual speaker is positioned at the second point, the method further comprising rendering the audio element using information identifying the positions of the virtual speakers.

[0119] A26. A method according to any one of embodiments A1 to A25, further comprising: generating a center virtual speaker signal for the center virtual speaker based on the position of the center virtual speaker; and rendering the audio element using the center virtual speaker signal (e.g., generating an output signal using the center virtual speaker signal).

[0120] B1. A computer program comprising instructions that, when executed by a processing circuit of an audio renderer, cause the audio renderer to perform a method according to any one of the preceding embodiments.

[0121] B2. A carrier containing the computer program, the carrier being one of an electrical signal, an optical signal, a radio signal, and a computer readable storage medium.

[0122] C1. An audio rendering device configured to perform the method of any one of the preceding embodiments.

[0123] C2. The audio-rendering device of embodiment C1, wherein the audio-rendering device comprises a memory and a processing circuit coupled to the memory.

[0124] It should be understood that while various embodiments have been described herein, they are presented by way of example only, and not limitation. Thus, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above. Moreover, any combination of the above objects in all possible variations thereof is encompassed by the present disclosure unless otherwise indicated herein or clearly contradicted by context.

[0125] In addition, while the processes described above and illustrated in the figures are shown as a sequence of steps, this is done for illustrative purposes only and, as such, some steps may be added, some steps may be omitted, the order of steps may be rearranged, and some steps may be performed in parallel.

[0126] References

[0127] [1] MPEG-H 3D Audio, Section 8.4.4.7: "Spreading"

[0128] [2] MPEG-H 3D Audio, Section 18.1: "Element Metadata Preprocessing"

[0129] [3] MPEG-H 3D Audio, Section 18.11: "Diffuseness Rendering"

[0130] [4] EBU ADM Renderer Tech 3388, Section 7.3.6: "Divergence"

[0131] [5]EBU ADM Renderer Tech 3388, Clause 7.4: "Decorrelation Filters"

[0132] [6]EBU ADM Renderer Tech 3388, Clause 7.3.7: "Extent Panner"

[0133] [7]Efficient HRTF-based Spatial Audio for Area and Volumetric Sources, IEEE Transactions on Visualization and Computer Graphics 22(4):1 - 1 January 2016

[0134] [8]Patent Publication WO2020144062 "Efficient spatially - heterogeneous audio elements for Virtual Reality"

[0135] [9]Patent Publication WO2021180820 "Rendering of Audio Objects with a Complex Shape"

Claims

1. A method (800) for rendering an audio element, the audio element having a range (200) and represented using a set of virtual speakers (202, 203, 204) including a central virtual speaker (203), the method comprising: The method includes at least one of the steps of: selecting a position of the central virtual speaker based on a position of a listener; and calculating an attenuation factor of the central virtual speaker (s802).

2. The method may include selecting a location for the central virtual speaker based on the position of the listener, the selecting the location for the central virtual speaker based on the position of the listener comprising: determining a first angle based on the position of the listener and a position of i) a first endpoint of the audio element or of a range determined based on the range of the audio element, or ii) a first virtual speaker; determining a second angle based on the position of the listener and i) a second end point of the audio element or of the range, or ii) a position of a second virtual speaker; calculating a first coordinate Mx of the central virtual speaker using the first angle and the second angle, the selected position of the central virtual speaker being specified at least in part by the calculated first coordinate; The method of claim 1 , comprising:

3. Selecting the position of the central virtual speaker based on the position of the listener comprises: determining a third angle based on the position of the listener and i) a position of the audio element or of a third end point of the range, or ii) a third virtual speaker; determining a fourth angle based on the position of the listener and a position of i) a fourth end point of the audio element or of the range, or ii) a fourth virtual speaker; calculating a second coordinate My of the central virtual speaker using the third angle and the fourth angle, the selected position being specified at least in part by the calculated first and second coordinates; The method of claim 2 further comprising:

4. The step of calculating the first coordinate Mx of the central virtual speaker using the first angle and the second angle comprises: determining the first coefficient α1 using the first angle and the second angle; A step of calculating Mx=(α1*Ax)+((1-α1)*MPx), where Ax is a coordinate used when specifying the position of the determined anchor point on a line extending between the first end point and the second end point, or between the first virtual speaker and the second virtual speaker, and MPx is a coordinate used when specifying the midpoint of the line; The method of claim 2 or 3, comprising:

5. The step of determining α1 using the first angle and the second angle includes the steps of:

5. The method of claim 4, comprising: calculating a first weight xm_w using the first angle and the second angle; calculating a second weight zm_w using the first angle and the second angle; and determining α1 based on xm_w and zm_w.

6. The step of determining α1 based on xm_w and zm_w includes the steps of: determining whether d*xm_w*zm_w is less than 1, where d is a predetermined coefficient; setting α1 to 1 if d*xm_w*zm_w is not less than 1, otherwise setting α1 to d*xm_w*zm_w; The method of claim 5 , comprising:

7. When the first angle is λ and the second angle is β, The step of calculating xm_w using the first angle and the second angle includes the steps of: xm_w=sin(λ) / sin(β) or xm_w=sin(β) / sin(λ) The method of claim 5 or 6, comprising the step of calculating:

8. The step of calculating zm_w using the first angle and the second angle includes the steps of: zm_w=sin((λ+β) / 2) 8. The method of claim 5, 6 or 7, comprising the step of calculating:

9. The step of calculating the second coordinate My of the central virtual speaker using the third angle and the fourth angle comprises: determining a second coefficient α2 using the third angle and the fourth angle; When a coordinate used when specifying the position of the determined anchor point on a straight line extending between the third end point and the fourth end point or between the third virtual speaker and the fourth virtual speaker is Ay, and a coordinate used when specifying the midpoint of the straight line is MPy, My=(α2*Ay)+((1-α2)*MPy) and calculating The method of claim 3 , comprising:

10. The method may include selecting a location for the central virtual speaker based on the position of the listener, the selecting the location for the central virtual speaker based on the position of the listener comprising: selecting a position point on a first line between 1) a first point of the audio element or of a range determined based on the range of the audio element and a second point of the audio element or of the range, or 2) between a first virtual speaker and a second virtual speaker, selecting the location point such that an angle between i) a second line extending from the position of the listener to the first point or the first virtual speaker, and ii) a third line extending from the position of the listener to the selected location point on the first line is equal to an angle between i) a fourth line extending from the position of the listener to the second point or the second virtual speaker, and ii) the third line; The method of claim 1 , comprising:

11. The step of selecting a location point comprises: The length of the second straight line is v, The length of the third straight line is w, The coordinates of the first point or the first virtual speaker are Re, The coordinates of the second point or the second virtual speaker are Le, When M=(v*Re+w*Le) / (v+w), The method of claim 10, comprising calculating a coordinate M of the location point by calculating:

12. The method of claim 10 or 11, further comprising the step of positioning the central virtual speaker at the selected position point.

13. The method may include calculating an attenuation factor for the center virtual speaker based on the position of the listener, the step of calculating an attenuation factor for the center virtual speaker based on the position of the listener comprising: determining a first angle based on the position of the listener and i) a position of a first end point of the audio element or of a range determined based on the range of the audio element, or ii) a position of a first virtual speaker; determining a second angle based on the position of the listener and a position of the audio element or of a second end point of the range, or ii) a position of a second virtual speaker; Calculating ε=sin(λ) / sin(β) or ε=sin(β) / sin(λ), where λ is the first angle, β is the second angle, and ε is the damping coefficient; 13. The method of any one of claims 1 to 12, comprising:

14. 14. The method of claim 13, further comprising the steps of modifying the center virtual speaker signal X to generate a modified center virtual speaker signal X', such that X' = ε * X, and rendering the audio element using the modified center virtual speaker signal.

15. The set of virtual speakers includes: a first virtual speaker disposed at the first endpoint and a second virtual speaker disposed at the second endpoint, the method further comprising: rendering the audio element using information identifying the positions of the virtual speakers.

15. The method according to any one of claims 2 to 14.

16. The set of virtual speakers includes: a first virtual speaker disposed at the first endpoint, a second virtual speaker disposed at the second endpoint, a third virtual speaker disposed at the third endpoint, and a fourth virtual speaker disposed at the fourth endpoint, the method further comprising rendering the audio element using information identifying the positions of the virtual speakers. The method according to claim 3 or 9.

17. The method may include selecting a location for the central virtual speaker based on the position of the listener, the selecting the location for the central virtual speaker based on the position of the listener comprising: determining a distance from the position of the listener to a position of the audio element; determining whether the determined distance is greater than a threshold; selecting a position of the midpoint of the audio element or a midpoint of a range determined based on the range of the audio element as a result of determining that the determined distance is greater than the threshold value; The method of claim 1 , comprising:

18. The method may include selecting a location for the central virtual speaker based on the position of the listener, the selecting the location for the central virtual speaker based on the position of the listener comprising: i) obtaining listener information indicating coordinates of the listener; ii) obtaining midpoint information indicative of a coordinate of a midpoint between a first point associated with the audio element and a second point associated with the audio element; iii) selecting the location of the central virtual speaker based on the midpoint information and the listener information; The method of claim 1 , comprising:

19. Selecting the location of the central virtual speaker based on the midpoint information and the listener information comprises: i) determining the coordinates of an anchor point; ii) selecting the position of the central virtual speaker based on the midpoint information and anchor information indicating coordinates of the determined anchor point; 20. The method of claim 18, comprising:

20. 20. The method of claim 19, wherein i) the midpoint information includes a midpoint value MP indicating the coordinates of the midpoint, ii) the anchor information includes an anchor value A indicating the coordinates of the anchor point, and iii) selecting the position of the central virtual speaker based on the midpoint information and the anchor information includes calculating a coordinate value M of the central speaker using MP and A.

21. The method of claim 20 , wherein the anchor value A depends on the indicated coordinates of the listener.

22. The method of claim 21 , wherein A=L, where L is the indicated coordinate of the listener.

23. 23. The method of claim 20, 21 or 22, wherein the step of calculating M using MP and A comprises the step of calculating M=α*A+(1−α)*MP, where α is a coefficient that depends on the indicated coordinate of the listener.

24. 24. The method of claim 18, wherein the listener information further indicates a second coordinate of the listener, and the midpoint information further indicates a second coordinate of the midpoint, and wherein selecting the position of the central virtual speaker based on the midpoint information and the listener information further comprises determining a second coordinate of the central virtual speaker based on the second coordinate of the listener and the second coordinate of the midpoint.

25. a first virtual speaker is placed at the first point and a second virtual speaker is placed at the second point; 25. The method of claim 10 or any one of claims 18 to 24, further comprising the step of rendering the audio element using information identifying the positions of the virtual speakers.

26. generating a center virtual speaker signal for the center virtual speaker based on the position of the center virtual speaker; rendering the audio element using the center virtual speaker signal; 26. The method of any one of claims 1 to 25, further comprising:

27. A computer program (1443) comprising instructions (1444) which, when executed by a processing circuit (1402) of an audio renderer (1400), causes the audio renderer to perform a method according to any one of claims 1 to 26.

28. 30. A carrier containing the computer program of claim 27, the carrier being one of an electrical signal, an optical signal, a radio signal, and a computer readable storage medium (1442).

29. An audio rendering device (1400) for rendering an audio element, the audio element having a range (200) and represented using a set of virtual speakers (202, 203, 204) including a central virtual speaker (203), 8. The audio rendering apparatus of claim 7, further comprising: a first audio component providing a first virtual speaker position based on a position of a listener; and a second audio component providing a second virtual speaker position based on a position of a listener.

30. The audio rendering device comprises: determining a first angle based on the position of the listener and a position of i) a first endpoint of the audio element or of a range determined based on the range of the audio element, or ii) a first virtual speaker; determining a second angle based on the position of the listener and i) a second end point of the audio element or of the range, or ii) a position of a second virtual speaker; calculating a first coordinate Mx of the central virtual speaker using the first angle and the second angle, the selected position of the central virtual speaker being specified at least in part by the calculated first coordinate; 30. The audio rendering apparatus of claim 29, configured to select the position of the central virtual speaker based on the position of the listener by:

31. Selecting the position of the central virtual speaker based on the position of the listener comprises: determining a third angle based on the position of the listener and i) a position of the audio element or of a third end point of the range, or ii) a third virtual speaker; determining a fourth angle based on the position of the listener and a position of i) a fourth end point of the audio element or of the range, or ii) a fourth virtual speaker; calculating a second coordinate My of the central virtual speaker using the third angle and the fourth angle, the selected position being specified at least in part by the calculated first and second coordinates; 31. The audio rendering apparatus of claim 30, further comprising:

32. The step of calculating the first coordinate Mx of the central virtual speaker using the first angle and the second angle comprises: determining the first coefficient α1 using the first angle and the second angle; A step of calculating Mx=(α1*Ax)+((1-α1)*MPx), where Ax is a coordinate used when specifying the position of the determined anchor point on a line extending between the first end point and the second end point, or between the first virtual speaker and the second virtual speaker, and MPx is a coordinate used when specifying the midpoint of the line; 32. An audio rendering apparatus according to claim 30 or 31, comprising:

33. The step of determining α1 using the first angle and the second angle includes the steps of:

33. The audio rendering apparatus of claim 32, further comprising: calculating a first weight xm_w using the first angle and the second angle; calculating a second weight zm_w using the first angle and the second angle; and determining α1 based on xm_w and zm_w.

34. The step of determining α1 based on xm_w and zm_w includes the steps of: determining whether d*xm_w*zm_w is less than 1, where d is a predetermined coefficient; setting α1 to 1 if d*xm_w*zm_w is not less than 1, otherwise setting α1 to d*xm_w*zm_w; 34. An audio rendering apparatus according to claim 33, comprising:

35. When the first angle is λ and the second angle is β, The step of calculating xm_w using the first angle and the second angle includes the steps of: xm_w=sin(λ) / sin(β) or xm_w=sin(β) / sin(λ) 35. An audio rendering apparatus according to claim 33 or 34, comprising the step of calculating:

36. The step of calculating zm_w using the first angle and the second angle includes the steps of: zm_w=sin((λ+β) / 2) 36. An audio rendering apparatus according to claim 33, 34 or 35, comprising the step of calculating:

37. The step of calculating the second coordinate My of the central virtual speaker using the third angle and the fourth angle comprises: determining a second coefficient α2 using the third angle and the fourth angle; When a coordinate used when specifying the position of the determined anchor point on a straight line extending between the third end point and the fourth end point or between the third virtual speaker and the fourth virtual speaker is Ay, and a coordinate used when specifying the midpoint of the straight line is MPy, My=(α2*Ay)+((1-α2)*MPy) and calculating 32. An audio rendering apparatus according to claim 31, comprising:

38. selecting a position point on a first line between 1) a first point of the audio element or of a range determined based on the range of the audio element and a second point of the audio element or of the range, or 2) between a first virtual speaker and a second virtual speaker, selecting the location point such that an angle between i) a second line extending from the position of the listener to the first point or the first virtual speaker, and ii) a third line extending from the position of the listener to the selected location point on the first line is equal to an angle between i) a fourth line extending from the position of the listener to the second point or the second virtual speaker, and ii) the third line; 30. The audio rendering apparatus of claim 29, configured to select the position of the central virtual speaker based on the position of the listener by performing:

39. The step of selecting a location point comprises: The length of the second straight line is v, The length of the third straight line is w, The coordinates of the first point or the first virtual speaker are Re, The coordinates of the second point or the second virtual speaker are Le, When M=(v*Re+w*Le) / (v+w), 39. An audio rendering apparatus as claimed in claim 38, comprising the step of calculating a coordinate M of the position point by:

40. 40. An audio rendering apparatus according to claim 38 or 39, wherein the audio rendering apparatus is further configured to position the central virtual speaker at the selected position point.

41. determining a first angle based on the position of the listener and i) a position of a first end point of the audio element or of a range determined based on the range of the audio element, or ii) a position of a first virtual speaker; determining a second angle based on the position of the listener and a position of the audio element or of a second end point of the range, or ii) a position of a second virtual speaker; Calculating ε=sin(λ) / sin(β) or ε=sin(β) / sin(λ), where λ is the first angle, β is the second angle, and ε is the damping coefficient; 41. An audio rendering apparatus according to claim 29, configured to calculate the attenuation coefficient of the central virtual speaker based on the position of the listener by:

42. 42. An audio rendering apparatus according to claim 41, further configured to modify the center virtual speaker signal X to generate a modified center virtual speaker signal X', such that X' = ε * X, and to render the audio element using the modified center virtual speaker signal.

43. The set of virtual speakers includes:

43. An audio rendering apparatus according to any one of claims 30 to 42, further comprising a first virtual speaker located at the first endpoint and a second virtual speaker located at the second endpoint, the method further comprising the step of rendering the audio element using information identifying the positions of the virtual speakers.

44. The set of virtual speakers includes: a first virtual speaker disposed at the first endpoint, a second virtual speaker disposed at the second endpoint, a third virtual speaker disposed at the third endpoint, and a fourth virtual speaker disposed at the fourth endpoint, the method further comprising rendering the audio element using information identifying the positions of the virtual speakers.

38. Audio rendering apparatus according to claim 31 or 37.

45. determining a distance from the position of the listener to a position of the audio element; determining whether the determined distance is greater than a threshold; selecting a position of the midpoint of the audio element or a midpoint of a range determined based on the range of the audio element as a result of determining that the determined distance is greater than the threshold value; 30. The audio rendering apparatus of claim 29, configured to select the position of the central virtual speaker based on the position of the listener by performing:

46. i) obtaining listener information indicating coordinates of the listener; ii) obtaining midpoint information indicative of a coordinate of a midpoint between a first point associated with the audio element and a second point associated with the audio element; iii) selecting the location of the central virtual speaker based on the midpoint information and the listener information; 30. The audio rendering apparatus of claim 29, configured to select the position of the central virtual speaker based on the position of the listener by performing:

47. Selecting the location of the central virtual speaker based on the midpoint information and the listener information comprises: i) determining the coordinates of an anchor point; ii) selecting the position of the central virtual speaker based on the midpoint information and anchor information indicating coordinates of the determined anchor point; 47. An audio rendering apparatus according to claim 46, comprising:

48. 48. The audio rendering apparatus of claim 47, wherein i) the midpoint information includes a midpoint value MP indicating the coordinates of the midpoint, ii) the anchor information includes an anchor value A indicating the coordinates of the anchor point, and iii) selecting the position of the central virtual speaker based on the midpoint information and the anchor information includes calculating a coordinate value M of the central speaker using MP and A.

49. 49. An audio rendering apparatus according to claim 48, wherein the anchor value A depends on the indicated coordinates of the listener.

50. 50. An audio rendering apparatus as claimed in claim 49, wherein A=L, where L is the indicated coordinate of the listener.

51. 51. An audio rendering apparatus as claimed in claim 48, 49 or 50, wherein the step of calculating M using MP and A comprises the step of calculating M = α * A + (1 - α) * MP, where α is a coefficient dependent on the indicated coordinates of the listener.

52. 52. An audio rendering apparatus as claimed in any one of claims 46 to 51, wherein the listener information further indicates a second coordinate of the listener, and the midpoint information further indicates a second coordinate of the midpoint, and wherein selecting the position of the central virtual speaker based on the midpoint information and the listener information further comprises determining the second coordinate of the central virtual speaker based on the second coordinate of the listener and the second coordinate of the midpoint.

53. a first virtual speaker is placed at the first point and a second virtual speaker is placed at the second point; 52. An audio rendering apparatus according to any one of claims 38 or 46 to 51, wherein the method further comprises the step of rendering the audio element using information identifying the positions of the virtual speakers.

54. generating a center virtual speaker signal for the center virtual speaker based on the position of the center virtual speaker; Rendering the audio element using the center virtual speaker signal.

54. An audio rendering apparatus according to any one of claims 29 to 53, further configured to:

Images