Rendering of reverberation in connected space

Abstract

A method (400) performed by an audio renderer (151) for rendering reverberation in a second space (302) connected to a first space (301) via a portal (300). The method includes obtaining information indicative of a portal size. The method also includes determining a zoom factor using information indicative of the portal size. The method further includes rendering a first set of reverberation signals comprised of one or more reverberation signals in a second space (302) using the scaling factor.

Description

Technical Field

[0001] Embodiments are disclosed that relate to rendering reverberation in a second space connected to a first space via a portal. Background Art

[0002] Extended reality (XR) (e.g., virtual reality (VR), augmented reality (AR), mixed reality (MR), etc.) systems typically include an audio renderer for rendering audio to a user of the XR system. The audio renderer typically includes a reverb processor for generating late and / or diffuse reverberation that is rendered to the user of the XR system to provide an auditory sensation of being in the XR scene being rendered. The generated reverberation should provide the user with an auditory sensation of being in an acoustic environment corresponding to the XR scene (e.g., a living room, a gym, an outdoor environment, etc.).

[0003] Reverberation is one of the most important acoustic properties of a room. Sound generated in a room will bounce repeatedly off reflective surfaces such as the floor, walls, ceiling, windows, or tables while gradually losing energy. When these reflections mix with each other, the so-called "reverberation" phenomenon is produced. Thus, reverberation is a collection of many sound reflections.

[0004] The two most fundamental characteristics of reverberation in any (real or virtual) acoustic environment are: 1) reverberation time and 2) reverberation level, i.e., the intensity or loudness of the reverberation (e.g., relative to the power level of a sound source or the direct sound level in the space).

[0005] Reverberation time is a measure of the time it takes for the reflected sound to "die out" in an enclosed space after the sound source has stopped. It is important in defining how a room responds to acoustic sound. Reverberation time depends on the amount of sound absorption in the space, which is lower in a space with many sound-absorbing surfaces such as curtains, upholstered chairs, or even people, and higher in a space mainly containing hard reflective surfaces.

[0006] Traditionally, reverberation time has been defined as the time it takes for the sound pressure level to decrease by 60 dB after suddenly turning off the sound source. The shorthand for this amount of time is "RT60" (or sometimes T60).

[0007] Typically, for a reverb processor used in an audio renderer, these two (and other) characteristics of the generated reverberation can be controlled separately and independently. For example, a reverb processor can typically be configured to generate reverberation with a specific desired reverberation time and a specific desired reverberation level.

[0008] In an XR system, the characteristics of the reverberation generated are typically controlled by control information specified, for example, by a scene creator (e.g., special metadata included in the XR scene description), which describes many aspects of the XR scene, including its acoustic characteristics. The audio renderer receives this control information, for example, from a bitstream or a file, etc., and uses this control information to configure a reverb processor to generate a reverberation with desired characteristics. Depending on the type of reverb algorithm used by the reverb processor to generate the reverberation, the exact way in which the reverb processor obtains the desired reverberation time and reverberation level in the generated reverberation can vary.

[0009] Reverberation and connected spaces

[0010] As described above, one of the key aspects of immersive audio rendering is the realistic rendering of the reverberation associated with the virtual space of the XR scene. A particular challenge is to realistically render the reverberation in a space (aka "acoustic environment") (hereinafter referred to as the "second space") that is connected (or "coupled") to another space (hereinafter referred to as the "first space") via one or more openings (e.g., open doors, windows) and / or one or more other interfaces between the first space and the second space (e.g., including a partially transmissive wall), through which the reverberation can propagate from the first space to the second space (and vice versa). Hereinafter, the term "portal" will be used to refer to any connection interface (e.g., opening, transmissive object, etc.) through which sound can propagate from one space to another.

[0011] In real life, the nature of the reverberant sound field in the connected spaces (e.g., reverberation level, reverberation time, and other properties of the reverberant sound field in each space) is affected by the acoustic properties of each connected space (e.g., volume and absorption in each space).

[0012] In the acoustic literature, advanced models can be used to model the reverberant sound field in each of the two connected spaces, including the mutual influence of the acoustic properties of the two spaces on the reverberant sound field obtained in each space. In principle, such models can be used to derive a complete model of the energy exchange and the resulting reverberant sound field in all the connected spaces in the XR scene, but when more than two spaces are connected directly or in a "cascaded" connection manner, such models quickly become rather complex.

[0013] Thus, in a practical audio rendering system, a slightly simpler method for simulating the effect of reverberation in a connected space can typically be used, where a portion of the reverberation generated in one space (e.g., due to active sound sources present in that space) can be rendered into a second connected space as a "portal" sound source in an opening (or generally, an interface) between these spaces, which essentially simulates the propagation of the reverberation through the portal from the first space to the second space. Subsequently, the sound from the portal sound source can be further "reverberated" according to the acoustic properties of the second space in order to simulate the overall reverberation that a listener would perceive in that second space.

[0014] Although this simplified method is an approximation of what physically occurs and may not result in an exact match with the predictions of a more complex physical model, its perceived effect is generally very reasonable and its implementation is more easily fitted into existing rendering pipelines. SUMMARY OF THE INVENTION

[0015] There are certain challenges currently. For example, one problem with the above portal source method is that it can be challenging to determine the appropriate acoustic intensity of the portal source (i.e., the acoustic intensity that results in a realistic effect, i.e., an acoustic intensity that causes the reverberation to propagate into the second space as close as possible to physical reality). In other words, some acoustic coupling factor that controls the acoustic intensity of the reverberation portal source needs to be determined with a certain degree of accuracy.

[0016] Furthermore, in terms of practical audio rendering, it is necessary to determine how the determined acoustic intensity of the reverberation portal source relates to the level of the reverberant audio signal in the first space and the audio signal rendered from the reverberation portal source in the second space.

[0017] One complication is that the signal representation of the reverberation in the first space may be different from the representation of the signal rendered by the reverberation portal source into the second space. For example, the reverberation in the first space can be represented as a first set (uncorrelated) of signals that are intended to be rendered from different directions around a virtual user located in the first space (i.e., an immersive rendering representation). On the other hand, the reverberation that propagates through the portal into the second space and is rendered by the reverberation portal source can be represented as a second set of signals that can be derived from the first set of signals. For example, it can be represented by a single signal or by a set of uncorrelated signals, depending on the specific rendering algorithm used to render the reverberation portal source.

[0018] In other words, a scaling factor needs to be derived from the acoustic coupling factor that scales the reverberation portal source audio signal derived from the reverberation signal in the first space such that when these scaled reverberation portal source audio signals are rendered by the reverberation portal source, the correct audio level is produced in the second space.

[0019] Thus, in one aspect, there is provided a method for rendering reverberation in a second space (Space 2) connected to a first space (Space 1) via a portal, which is performed by an audio renderer. The method includes obtaining (e.g., exporting) information indicating the dimensions of the portal. The method further includes using the information indicating the dimensions of the portal to determine a scaling factor. The method also includes using the scaling factor to render a set of reverberation signals composed of one or more Space 2 reverberation signals in Space 2. The method may further include: determining a reverberation intensity value associated with the reverberation associated with Space 1, and using the reverberation intensity value and the scaling factor to render a set of reverberation signals composed of one or more Space 2 reverberation signals in Space 2.

[0020] In another aspect, there is provided a computer program including instructions that, when executed by a processing circuit of an audio renderer, cause the audio renderer to perform the methods disclosed herein. In an embodiment, there is provided a carrier including the computer program, wherein the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium. In another aspect, there is provided a rendering device configured to perform any of the above methods. The rendering device may include a memory and a processing circuit coupled to the memory.

[0021] The advantages of the embodiments disclosed herein are that they enable an audio renderer to produce reasonable reverberation rendering in a complex XR scenario with connected acoustic spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings incorporated herein and forming a part of the specification illustrate various embodiments.

[0023] Figure 1A A system according to some embodiments is shown.

[0024] Figure 1B A system according to some embodiments is shown.

[0025] Figure 2 A system according to some embodiments is shown.

[0026] Figure 3A A portal from a first space to a second space is shown.

[0027] Figure 3B A portal from a first space to a second space is shown.

[0028] Figure 4 A flowchart showing a process according to an embodiment is shown.

[0029] Figure 5 A block diagram of a device according to some embodiments is shown. DETAILED DESCRIPTION

[0030] Figure 1A FIG. 1 shows an XR system 100 to which embodiments disclosed herein can be applied. The XR system 100 includes speakers 104 and 105 (which can be speakers of headphones worn by a user) and an XR device 110 which can include a display for displaying images to the user and which in some embodiments is configured to be worn by a listener. In the illustrated XR system 100, the XR device 110 has a display and is designed to be worn on the user's head and is commonly referred to as a head-mounted display (HMD).

[0031] As Figure 1B shown, the XR device 110 can include an orientation sensing unit 101, a position sensing unit 102, and a processing unit 103 which is (directly or indirectly) coupled to an audio renderer 151 for generating output audio signals (e.g., a left audio signal 181 for the left speaker and a right audio signal 182 for the right speaker as shown).

[0032] The orientation sensing unit 101 is configured to detect changes in the orientation of the listener and provide information about the detected changes to the processing unit 103. In some embodiments, given a detected change in orientation detected by the orientation sensing unit 101, the processing unit 103 determines an absolute orientation (relative to some coordinate system). There may also be different systems for determining orientation and position, e.g., systems using a beacon tracker (LIDAR). In one embodiment, given a detected change in orientation, the orientation sensing unit 101 can determine an absolute orientation (relative to some coordinate system). In this case, the processing unit 103 can simply multiplex the absolute orientation data from the orientation sensing unit 101 and the position data from the position sensing unit 102. In some embodiments, the orientation sensing unit 101 can include one or more accelerometers and / or one or more gyroscopes.

[0033] The audio renderer 151 generates an audio output signal based on an input audio signal 161, metadata 162 about the XR scene the listener is experiencing, and information 163 about the position and orientation of the listener. The metadata 162 of the XR scene can include metadata for each object and audio element included in the XR scene and metadata for the XR space (the “acoustic environment”) in which the listener is virtually located. The metadata of an object can include information about the size of the object and the occlusion factor of the object (e.g., the metadata can specify a set of occlusion factors, where each occlusion factor applies to a different frequency or frequency range). The metadata 162 can also include control parameters such as a reverberation time value, a reverberation level value, and / or an absorption parameter.

[0034] The audio renderer 151 can be a component of the XR device 110, or it can be remote from the XR device 110 (e.g., the audio renderer 151 or its components can be implemented in the cloud).

[0035] Figure 2 An example implementation of the audio renderer 151 for generating the sound of an XR scene is shown. The audio renderer 151 includes a controller 201 and an audio signal generator 202 for generating an output audio signal (e.g., an audio signal of a multi-channel audio component) based on control information 210 from the controller 201 and input audio 161. In this embodiment, the controller 201 includes a reverb processor 204 for determining a scaling factor as described below.

[0036] In some embodiments, the controller 201 can be configured to receive one or more parameters and trigger the audio signal generator 202 to perform modifications (e.g., increase or decrease the volume level) on the audio signal 161 based on the received parameters. The received parameters include information 163 about the position and / or orientation of the listener (e.g., the direction and distance to the audio component) and metadata 162 about the XR scene. As described above, the metadata 162 can include metadata about the XR space in which the user is virtually located (e.g., the dimensions of the space, information about the objects in the space, and information about the acoustic properties of the space), as well as metadata about the audio components and metadata about the objects occluding the audio components. In some embodiments, the controller 201 itself generates at least a portion of the metadata 162. For example, the controller 201 can receive metadata about the XR scene and derive additional metadata (e.g., control parameters) based on the received metadata. For example, using the metadata 162 and the position / orientation information 163, the controller 201 can calculate one or more gain factors (g) for the audio components in the XR scene (e.g., the above-mentioned scaling factor or a gain derived using the scaling factor).

[0037] For the generation of the reverb signal used by the signal generator 202 to generate the final output signal, the controller 201 provides the signal generator 202 with reverb parameters (e.g., the reverb time and reverb level of the acoustic environment of the XR scene) and the above-mentioned scaling factor, such that the signal generator 202 is operable to generate a reverb signal. The reverb time of the generated reverb is most commonly provided to the reverb processor 204 as an RT60 value (generally for each frequency band), although there are other reverb time metrics and other reverb time metrics can also be used. In a typical embodiment, the metadata 162 includes all necessary reverb parameters (e.g., RT60 value and reverb level value). However, in embodiments where the metadata does not include all necessary reverb parameters, the controller 201 can be configured to generate the missing parameters.

[0038] The reverberation level can be expressed in a variety of formats. Typically, it will be expressed as a relative level. For example, it can be expressed as the energy ratio (DRR) between the direct sound component and the reverberant sound component at a certain distance from the sound source rendered in the XR environment or its reciprocal (i.e., the RDR energy ratio). Alternatively, the reverberation level can be expressed as the energy ratio between the reverberant sound and the total emitted energy or power of the source. In the following text, the term "reverberation energy ratio" is used to refer to any of these or other measures of the relative reverberation level. In still other cases, the reverberation level can be directly expressed as the level / gain of the reverberation processor.

[0039] In this context, the term "reverberation" can generally refer only to those sound field components corresponding to the diffuse part of the acoustic room impulse response of the acoustic environment, but in some embodiments, it can also include sound field components corresponding to the early part of the room impulse response, e.g., including some late non-diffuse reflections, or even all of the reflected sound.

[0040] Other metadata that can be included in the metadata 162 to describe the reverberation-related characteristics of the acoustic environment include: parameters describing the acoustic properties of the materials of the surfaces of the environment (describing, for example, the absorption, reflection, transmission, and / or diffusion properties of the materials), or specific time points of the room impulse response associated with the acoustic environment, e.g., the time after the source emission after which the room impulse response becomes diffuse (sometimes referred to as "pre-delay").

[0041] All of the above reverberation-related properties are generally frequency-dependent, and thus their associated metadata parameters are typically also provided and processed separately for multiple frequency bands.

[0042] Embodiments

[0043] As described above, the present disclosure provides embodiments for producing a reasonable rendering of reverberation in a complex XR scene with connected acoustic spaces. For example, the present disclosure provides means for determining an acoustic coupling factor that indicates the amount of reverberation propagating from a first space ("acoustic environment") through a portal (e.g., an opening or a partially transmitting surface) into a second space, the portal connecting the two spaces. In one embodiment, the determined acoustic coupling factor is determined using information indicating the size of the portal.

[0044] In some embodiments, based on the acoustic coupling factor, an appropriate signal level is set to render one or more audio signals into a second space, wherein the one or more audio signals are derived from one or more reverberant signals corresponding to a first space. For example, in some embodiments, the determined acoustic coupling factor is used to derive a scaling factor, wherein the scaling factor is used to scale one or more audio signals derived from a set of audio signals (constituted by one or more audio signals) representing reverberation in the first space. In some embodiments, the signal level for rendering one or more audio signals into the second space is also determined based on the amplitude, power, or energy of the total reverberant signal received at a certain position in the first space.

[0045] In some embodiments, the signal level of the signal rendered into the second space is determined based on one or more acoustic parameters of the first space, and more specifically, is determined based on a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0046] Theoretical framework

[0047] Figure 3A A scenario (real life or VR) consisting of the following two spaces is shown: Space 1 and Space 2. These spaces are connected to each other via a portal 300 (which may alternatively be referred to as an "opening", "aperture", "interface", etc.). A sound source S1 is located somewhere in Space 1. The sound source S1 generates a reverberant sound field in Space 1.

[0048] The portal represents the interface between Space 1 and Space 2, through which some portions of the reverberant sound energy can be exchanged between these two spaces. The portal has m 2 "acoustic" dimensions (also known as "associated" dimensions) (area). In the case where the portal is an acoustically fully transparent opening (e.g., an open door or window), the acoustic dimension of the portal is equal to the geometric dimension of the portal (e.g., the geometric area of the portal). More generally, if the portal is not fully open and is only partially acoustically transparent (e.g., a thin wall or thick curtain separating two spaces), then the acoustic dimension of the portal is the equivalent dimension of a fully transparent opening representing the same amount of energy "leakage". In other words, if the portal is not fully acoustically transparent, then the acoustic dimension of the portal will be smaller than its geometric dimension. Hereinafter, whenever the "dimension" or "area" of the portal is mentioned, unless otherwise clearly stated, it refers to the "acoustic" dimension.

[0049] A portion of the reverberant sound energy generated by the sound source in Space 1 propagates through the portal into Space 2, where a listener L located in Space 2 hears the reverberation coming from Space 1 through the portal.

[0050] To determine the level of spatial 1 reverberation perceived by listener L in space 2, it is necessary to determine the reverberant sound energy transmitted from space 1 to space 2 through the portal.

[0051] For simplicity, space 2 is first considered a "free field", which means it is either a very large open (e.g., outdoor) space or a space with very high sound absorption such that no reverberant energy propagates back from space 2 into space 1, so this is a one-way problem.

[0052] Assuming a steady-state diffuse sound field in space 1, which means that the sound energy leaving space 1 per unit time through absorption and the portal to the connected space is equal to the power of the sound source S1 in space 1, it can be shown that due to the sound source S1 in space 1, the so-called average reverberant energy density in space 1 is equal to:

[0053] (1)

[0054] where, is the sound power of the sound source S1 in space 1 (in watts), is the total absorption in space 1, including the absorption represented by the portal, expressed as an equivalent absorption area (m 2 ) and c is the speed of sound in air (in m / s).

[0055] can be further specified in terms of the absorption in space 1 (excluding the portal) and the dimensions of the portal 2 in m as follows:

[0056] (2)

[0057] It can be further shown that under diffuse steady-state conditions, the power (expressed in watts) transmitted from space 1 to space 2 through the portal is typically equal to:

[0058] (3)

[0059] Combining equations 1, 2, and 3, the power transmitted from space 1 to space 2 through the portal is obtained: (4)

[0060] The factor in equation 4 is called the "acoustic coupling factor" from space 1 to space 2, which indicates the fraction of the source power in space 1 transmitted to space 2 under steady-state conditions.

[0061] It can be seen that the acoustic coupling factor is equal to the fraction of the total absorption in space 1 due to the portal. In other words, the power transmitted from space 1 to space 2 is determined by the fraction of the total absorption in space 1 which is represented by the size of the portal .

[0062] If the absorption in space 1 (excluding the portal) is very small compared to (e.g., if the walls of space 1 are highly reflective and / or the portal is a very large opening), then the acoustic coupling factor is substantially equal to 1, and the amount of power transmitted through the portal is substantially equal to the power radiated by the source.

[0063] On the other hand, if the absorption in space 1 (excluding the portal) is very large compared to (e.g., if the walls of space 1 are highly absorptive and / or the portal is very small), then the acoustic coupling factor is equal to , i.e., the ratio of the size of the portal to the absorption in space 1 (excluding the portal), which in this case will be a very small number, i.e., only a very small fraction of the source power is transmitted through the portal.

[0064] The average steady-state reverberant energy density E in a space is directly related to the root-mean-square steady-state reverberant sound pressure p in the space as follows:

[0065] (5)

[0066] where is the mass density of air.

[0067] Therefore, by using Equation 1, the steady-state reverberant sound pressure p in space 1 due to source S1 can be written as 1 :

[0068] (6)

[0069] Combining Equation 4 and Equation 6, we find the relationship between the steady-state reverberant pressure p in space 1 1 and the power transmitted to space 2:

[0070] (7)

[0071] Therefore, Equation 7 provides an expression for the amount of power transmitted from space 1 to space 2, which involves the diffuse sound pressure p in space 1 1 and the size of the portal . Importantly, Equation 7 shows that if the diffuse sound pressure in Space 1 and the dimensions of the portal are known, this directly gives the amount of power transmitted through the portal.

[0072] Now, in order to derive the relationship expression between the diffuse sound pressure p 1 in Space 1 and the sound pressure p 2 associated with the radiation through the portal in Space 2, the power transmitted through the portal is required and the relationship between the pressure p 2 generated in Space 2.

[0073] If the portal is relatively small, it can be assumed that the reverberant energy transmitted through the portal radiates uniformly in all directions from the portal to Space 2 (i.e., radiates spherically). By this assumption, this results in the pressure p 2,1m at a distance of 1 m from the portal being equal to:

[0074] (8)

[0075] This is obtained from the relationship between the sound source power P and the pressure p at a distance of 1 m from the source radiating a spherical wave:

[0076] (9)

[0077] where, in Equation 8, a factor of 2 has been added to the power because the power radiates only into the hemisphere on the Space 2 side of the portal (therefore, the resulting pressure should be the pressure corresponding to a global radiation source with twice this power).

[0078] Combining Equations 7 and 8, the following relationship can be found between the diffuse pressure p 1 in Space 1 and the root mean square pressure p 2,1m at a distance of 1 m from the portal in Space 2:

[0079] (10)

[0080] In terms of the audio signal level in the audio rendering system, the root mean square sound pressure p is directly proportional to the root mean square signal level of the corresponding audio signal, such that Equation 10 provides a direct way to correlate the desired root mean square audio signal level in Space 2 with the root mean square reverberant audio signal level in Space 1.

[0081] Therefore, in terms of the linear sound pressure or the linear root mean square audio signal level, Equation 10 means that the reverb in Space 1 should be rendered into Space 2, where it is scaled such that at a distance of 1 m from the portal, the resulting sound pressure or root mean square audio signal level is scaled relative to the diffuse sound pressure or root mean square audio signal level of the reverb in Space 1 by a factor .

[0082] For logarithmic sound pressure levels or logarithmic root mean square audio signal levels (in dB), this means that the reverberation in space 1 is rendered in space 2 such that the level obtained at 1 m from the portal is 10*log 10 ( ) = 10*log 10 ( ) - 14 (dB) (relative to the diffuse sound pressure level or root mean square audio signal level in space 1).

[0083] As mentioned, this result is valid for portals that are small enough, so the assumption of spherical radiation of the portal is reasonable. Clearly, there are limitations to the applicability of Equation 10 because if the size of the portal exceeds 8π m 2 , then the root mean square pressure p 2,1m in space 2 will be greater than the root mean square pressure p 1 in space 1, which physically never occurs. In fact, since the acoustic power associated with pressure p 2,1m only corresponds to the part of the diffuse sound field in space 1 that is incident on the opening, i.e., its maximum value corresponds to the diffuse acoustic power of the hemisphere on the space 1 side of the opening, so it will never exceed .

[0084] The reason why Equation 10 may give physically unreasonable results for larger values of is that using Equation 9 implies that the total power transmitted through the portal is actually radiated from a single point. If this were the case, then the pressure near that single point would be much higher compared to the situation where the power would be evenly distributed and radiated from the entire portal (which is actually the case in reality).

[0085] One way to solve this problem is to modify Equation 10 to:

[0086] (11)

[0087] This modification prevents the level obtained at 1 m from the portal in space 2 from exceeding the diffuse reverberation level in space 1 minus 3 dB (= 10log 10(0.5), which is required by physics (as explained above). Note that depending on the actual rendering method used to render the reverberation from the portal, additional measures may be required at distances within 1 m of the portal to ensure that the level there does not exceed the diffuse reverberation level in Space 1 minus 3 dB. In other words, it should be ensured that the resulting level of the signal rendered from the portal does not exceed the diffuse reverberation level in Space 1 minus 3 dB anywhere in Space 2. This will also ensure a smooth transition of the reverberation level when the listener moves between spaces through the portal.

[0088] Although a rather simple measure for solving the problem regarding if exceeds 4πm 2 (= 12.6 m 2 ) then p 2,1m might become too large, the solution of Equation 11 may actually provide very reasonable results in many use cases.

[0089] If is significantly less than 4πm 2 , the assumption of spherical radiation from the opening is reasonable, and reasonable results can be obtained using / 8π as a scaling factor.

[0090] As the size of the portal increases and approaches 4πm 2 , the sound pressure level at 1 m from the portal approaches the sound pressure level of the reverberation in Space 1 minus 3 dB, so that for portals larger than 4πm 2 , this level is eventually reached and maintained. The latter seems reasonable because the reverberation experienced when standing 1 m from the center of an opening with dimensions such as 4 x 3 m (= 12 m 2 ) is very similar to the reverberation experienced when standing at the opening itself.

[0091] Based on the root mean square pressure p 2,1m derived at 1 m from the portal, the root mean square sound pressure p 2 at any position at any distance d from the portal in Space 2 is directly derived, and the equation is as follows: p 2 (d) = p 2,1m / d. This assumes that the portal source radiates as a point source.

[0092] Alternatively, instead of the spherical radiation model of Equations 8 and 9, other models can be used to model the radiation from the portal, resulting in alternative equations to Equation 8 for representing the transmitted power and the resulting pressure p 2 in Space 2 and the pressure p 1 and p 2The resulting relationship (and thus the scaling that is ultimately applied to the audio signal rendered from the portal in order to obtain the correct rendered audio signal level in space 2).

[0093] For example, a reverberant portal source can be modeled as a spatially diffuse extended sound source, e.g., a spatially diffuse line source or plane source with dimensions equal to the geometric dimensions of the portal. Acoustic radiation models for such spatially diffuse extended sources are provided in the literature, which correlate the source power with the resulting sound pressure at a given distance from the source.

[0094] Thus, in some embodiments, alternative equations of Equation 10 or 11 in the following form can be used:

[0095] (12)

[0096] where C1 is a constant, or more generally:

[0097] (13)

[0098] where is a function of the portal size .

[0099] Alternative theoretical framework

[0100] An alternative but largely equivalent theoretical view of the transmission of reverberation from space 1 to space 2 will be presented.

[0101] Since the diffuse reverberant sound field at each point in space is theoretically composed of uncorrelated plane waves of equal intensity arriving from all directions, the reverberant energy propagating from space 1 through the portal to a specific point in space 2 can be determined by geometric considerations.

[0102] If the average (root mean square) pressure of the individual reverberant plane waves arriving at an arbitrary point in space 1 from a single solid angle is denoted as p d , then the root mean square diffuse reverberant pressure p 1 at any point in space 1 is obtained by integrating p over all solid angles d :

[0103] (14)

[0104] Figure 3B Point L in space 2 is shown, and the opening angle is indicated when looking through the portal from point L into space 1. From the perspective of point L, the portal represents the solid angle (where ).

[0105] Since, by definition, the diffuse field in space 1 consists of uncorrelated plane waves of equal intensity from all directions, and since the pressure of a plane wave is constant along its path (i.e., it is independent of the distance traveled), each individual plane wave arriving at point L in space 2 through the aperture contributes the same uncorrelated pressure component p d , and thus, the resulting pressure p 2 at point L in space 2 (where ) can be determined by integrating p d over the solid angle

[0106] (15)

[0107] Combining equations 14 and 15, it can be obtained that: the pressure p 2 at point L in space 2 can be directly determined from the solid angle 1 and the diffuse pressure p

[0108] (16)

[0109] In the case where the aperture is not completely acoustically transparent but transmits a fraction T of the power incident on the aperture, p 2 is scaled accordingly.

[0110] This result can be compared with equations 10 to 12, and like these equations, it represents that the root mean square pressure in space 2 is proportional to the root mean square pressure in space 1, where the proportionality factor depends linearly on the size of the aperture, where the size of the aperture is expressed as (equivalent) area (m 2 ) in equations 10 to 12 and as a solid angle in equation 16.

[0111] The diffuse reverberant pressure p 1 in space 1 can be determined from equation 6, based on the power of the sound source and the sound absorption in space 1.

[0112] One point to note when comparing the two proposed theoretical frameworks is that while the first framework models the aperture as a "secondary" sound source radiating into space 2, the second framework directly considers the reverberant energy received from space 1 at a specific point in space 2. Since the solid angle represented by the aperture depends on the position relative to the aperture, the pressure p 2 obtained from equation 16 also depends on the relative position of that specific location.

[0113] Specifically, the pressure obtained from Equation 16 will be very different for positions directly in front of the portal and positions on the side (or above / below) of the portal.

[0114] If the portal is small enough, the solid angle represented by a flat surface portal with a geometric area of m 2 at a distance r from the observation point can be approximated as:

[0115] (17)

[0116] where and represent the position vector from the observation point to the portal and the normal vector of the portal, respectively.

[0117] At a distance r = 1 m, this becomes:

[0118] (18)

[0119] where is the observation angle relative to the normal vector of the portal.

[0120] Combining Equation 18 with Equation 16, it can be seen that for the position directly in front of the portal, and . Comparing this equation with Equation 10, it can be seen that the root mean square pressure at 1 m obtained from Equation 16 is sqrt(2) times the root mean square pressure at 1 m obtained from Equation 10. On the other hand, at a position completely on the side of the portal, , so also holds. This can be interpreted as follows: While Equation 16 represents the pressure at a specific point in Space 2, Equation 10 derived from the assumption of spherical radiation from the portal represents the average value of Equation 16 at all angles at a distance of 1 m (i.e., the average value of the solid angle at all angles at a distance of 1 m from the portal is equal to / 2).

[0121] Also note that while in the first theoretical framework, the level of reverberation in Space 2 decreases as the distance from the portal increases in a manner determined by the radiation model for the portal source (e.g., point source radiation model or diffuse plane source radiation model as discussed above), in the second theoretical framework, the decrease in this level as the distance from the portal increases is an inherent result of the solid angle represented by the portal decreasing with increasing distance.

[0122] Above, it has been assumed that Space 2 is a free field, i.e., a space that does not generate any diffuse reverberation by itself (e.g., a large outdoor space).

[0123] Rendering

[0124] The XR audio renderer can be configured to reasonably render reverberation in the connected space of the XR environment using the above model.

[0125] In one embodiment, in the second connected space 2, the rendering of the reverberation associated with the reverberation in the first space 1 is split into two stages: (1) the rendering of the reverberation from space 1 that directly reaches the listener in space 2 via the portal between the two spaces, and (2) the generation and rendering (also known as "second-order reverberation") of the reverberation generated in space 2 in response to the reverberation from space 1 entering space 2 through the portal.

[0126] In some embodiments, only the first rendering stage can be performed. In other embodiments, only the second rendering stage can be performed. In other embodiments, both rendering stages can be performed.

[0127] The first rendering stage

[0128] The first rendering stage is substantially independent of the acoustic effects of space 2. It is the rendering of the reverberant sound (e.g., a listener standing in a large open space will hear the reverberant sound coming out of the open door (i.e., the portal) of a cathedral where music is being played).

[0129] In one embodiment, a method for rendering sound in space 2 due to the reverberant sound in connected space 1 may include the following steps:

[0130] (1: Optional step) Determine a space 1 reverberation intensity value that represents the intensity of the reverberation in space 1;

[0131] (2) Derive one or more space 2 reverberation signals (e.g., downmixed signals) for rendering in space 2 from one or more reverberation signals representing the reverberation in space 1 (also known as "space 1 reverberation signals");

[0132] (3) Obtain (e.g., determine, derive, receive) the dimensions of the portal through which sound is transmitted between space 1 and space 2;

[0133] (4) Use the portal dimensions to determine a scaling factor that models the transmission of the reverberant sound from space 1 to space 2;

[0134] (5) Use the space 2 reverberation signals and the scaling factor (and optionally the space 1 reverberation intensity value) to render one or more reverberation signals in space 2.

[0135] In one embodiment, the method may include the following additional steps: before deriving the reverberation signal of Space 2 from the reverberation signal of Space 1, using the reverberation control information of Space 1 to generate the reverberation signal of Space 1. In one embodiment, the reverberation control information of Space 1 includes the reverberation level parameter or reverberation energy ratio parameter of Space 1.

[0136] For the first (optional) step, the reverberation intensity value of Space 1 can be determined in various ways.

[0137] In a simple scenario where the reverberation in Space 1 is rendered from a single (i.e., non-directional, mono) audio signal, the reverberation intensity value of Space 1 can simply be determined as the root mean square amplitude or root mean square power of the signal, or, in the case where the reverberation is rendered based on the room impulse response, as the total amount of reverberation energy contained in the room impulse response.

[0138] In the case of using multiple audio signals (e.g., multiple uncorrelated signals rendered from multiple directions around the user) to render the reverberation in Space 1, the reverberation intensity value of Space 1 can be determined as the root mean square amplitude or power of the resulting combined signal. As an example, assume that the reverberation in Space 1 is rendered to a listener in Space 1 as N uncorrelated reverberation signals from N corresponding directions, each uncorrelated reverberation signal having a root mean square amplitude of 1 / N (or 1 / N 2 of the root mean square power), then the resulting combined reverberation signal has a root mean square power of 1 / N and a root mean square amplitude of 1 / sqrt(N).

[0139] In some cases, the reverberation intensity value of Space 1 does not have to be determined based on the actual reverberation audio signal of Space 1, but can be more efficiently derived from the reverberation intensity metadata of Space 1. For example, the scene description metadata of an XR scene may contain the reverberation level parameter or reverberation energy ratio parameter of Space 1, which describes the desired reverberation level in Space 1, and this reverberation level is absolute or relative to the direct sound level or the transmitted source energy / power of the source that generates reverberation in Space 1. In this case, the (relative) reverberation level in Space 1 is known a priori (and it is the job of the renderer to generate the reverberation audio signals of Space 1 such that they produce the specified reverberation level in Space 1). For example, assume that Space 1 has associated metadata that includes the value of the reverberation-to-direct energy ratio (RDR) in Space 1, which specifies the desired ratio of the energy of the reverberation to the direct sound energy at a distance of 1 m from an omnidirectional audio source located somewhere in Space 1. Now, if the omnidirectional audio source in Space 1 has an associated audio signal with a linear root mean square signal amplitude of s and an associated linear source gain (“volume control”) of g, then the rendered linear root mean square signal amplitude of the direct sound at a distance of 1 m from the audio source is given by g*s, such that the root mean square power / energy of the direct sound signal is (g*s) 2 (versus (g*s)2 (proportional). It can be seen from this that the root mean square energy / power of the reverberation associated with the audio source should be equal to RDR*(g*s) 2 , such that the linear root mean square signal amplitude of the reverberation is sqrt(RDR)*g*s. Thus, the spatial 1 reverberation intensity value can be directly derived from the provided reverberation energy ratio (RDR) parameter for spatial 1 and the source gain and audio signal level of the audio source.

[0140] In some embodiments, the spatial 1 reverberation intensity value simply equals the value of the spatial 1 reverberation level parameter or reverberation energy ratio parameter obtained from the spatial 1 metadata.

[0141] If the source is not omnidirectional, but has an arbitrary directivity pattern associated with it (which results in the source radiating an X fraction of the power of an omnidirectional source (for the same source signal)), then this results in the power of the resulting reverberation also being an X fraction of the power of the omnidirectional source. Thus, the derived spatial 1 reverberation intensity value can be scaled accordingly, i.e., if expressed in linear root mean square signal amplitude, multiplied by the factor sqrt(X), or if expressed in root mean square signal energy / power, multiplied by the factor X.

[0142] In addition to the source gain g, signal level s, and directivity pattern discussed above, other source rendering aspects also affect the gain of the rendered direct sound level or the rendered reverberation level and can be taken into account in a similar manner when calculating the spatial 1 reverberation intensity value.

[0143] However, in many embodiments, the spatial 1 reverberation intensity value is not explicitly required and does not have to be explicitly applied. This is the case, for example, when the spatial 2 reverberation signal is directly derived from the spatial 1 reverberation signal with the correct level for rendering in spatial 1. In this case, the spatial 2 reverberation signal derived from the spatial 1 reverberation signal has already been implicitly scaled using the spatial 1 reverberation intensity. Specifically, if the provided spatial 1 reverberation level parameter or reverberation energy ratio parameter is used to generate the spatial 1 reverberation signal, then this reverberation intensity information inherently exists in the generated spatial 1 reverberation signal and thus automatically carries over to the spatial 2 reverberation signal derived from it. Thus, in this case, the optional step (1) of the above method can be omitted, and the rendering step (5) using the reverberation intensity value can be omitted.

[0144] For step 2, the step of deriving one or more spatial 2 reverberation signals for rendering in spatial 2 can be accomplished in various ways. In one embodiment, the spatial 2 reverberation signal can be a mono or stereo downmix of the spatial 1 reverberation audio signal.

[0145] In another embodiment, one or more spatial 2 reverberation signals can be directly derived from the source signal and the spatial 1 reverberation metadata parameters (e.g., reverberation time RT60 and reverberation energy ratio parameter), i.e., without the intermediate step of first generating the actual spatial 1 reverberation signal. This may be more efficient because the spatial 1 reverberation signal is not actually rendered to the listener (located in spatial 2), but is only generated as an intermediate step for generating one or more spatial 2 reverberation signals.

[0146] Regarding step 3, the size of the portal can be obtained in various ways. In some embodiments, the size of the portal can be obtained directly in the scene description data, which can explicitly specify the location and / or size of the portal in the space and the other spaces it is connected to. In other embodiments, the size can be derived from such scene description data (e.g., from geometric information). In still other embodiments, for example, a form of ray tracing algorithm can be used to heuristically detect the size of the portal.

[0147] In some embodiments, the size of the portal represents the area of the portal (in m 2 units). In some embodiments, this area is the equivalent area of an acoustically perfectly transparent opening that has the same "acoustic power leakage" as the portal.

[0148] In other embodiments, the size of the portal represents the solid angle relative to a specific location in spatial 2 corresponding to the portal. Methods for deriving the solid angle can be easily found in the literature.

[0149] The scaling factor derived in step 4 represents the desired relationship between the intensity of reverberation in spatial 1 (e.g., root mean square diffuse pressure, root mean square signal amplitude, or root mean square signal power) and the intensity of the rendered spatial 2 reverberation in spatial 2 (e.g., root mean square diffuse pressure, root mean square signal amplitude, or root mean square signal power).

[0150] In many embodiments, the basis for deriving the scaling factor can be given by any one of equations 10 to 13 or 16, according to which the scaling factor can be derived as a factor that correlates p 1 with p 2 (alternatively, correlates p 1 2 with p 2 2 ).

[0151] Thus, for example, the scaling factor can be derived according to equation 10 to be equal to (or its square root), while according to equation 16, it can be derived as (or its square root).

[0152] Finally, the one or more derived spatial 2 reverberation signals are rendered to a listener in spatial 2 using a scaling factor and an optional spatial 1 reverberation intensity value.

[0153] The scaling factor and the optional spatial 1 reverberation intensity value can be combined to determine the desired intensity of the rendered spatial 2 reverberation, for example, determined by a relationship such as the following: Desired intensity of rendered spatial 2 reverberation = Scaling factor x Spatial 1 reverberation intensity value.

[0154] After the desired intensity of the rendered spatial 2 reverberation has been determined, an appropriate scaling gain to achieve the desired intensity of the rendered spatial 2 reverberation can be determined for the spatial 2 reverberation signal.

[0155] In some embodiments, the scaling gain of the spatial 2 reverberation signal includes the scaling factor.

[0156] In some embodiments, the scaling gain of the spatial 2 reverberation signal simply equals the scaling factor.

[0157] In some embodiments, in addition to the scaling factor, the scaling gain of the spatial 2 reverberation signal can also take into account the gain effect due to the specific way the spatial 2 reverberation signal is derived from the spatial 1 reverberation signal and the gain effect due to the different signal representations and rendering methods used for the spatial 1 and spatial 2 reverberation signals respectively.

[0158] As already explained, the spatial 1 reverberation can be represented by a combination of multiple signals (and rendered to a listener virtually located in spatial 1), and the spatial 2 reverberation signal is derived from the multiple signals using a certain signal transformation (e.g., downmixing) process, which may introduce a certain transformation gain effect, i.e., the difference in the total signal intensity before and after the transformation. The scaling gain of the spatial 2 reverberation signal can compensate for this transformation gain effect.

[0159] The scaling gain of the spatial 2 reverberation signal can also compensate for the gain effect due to the specific way of combining the reverberation signals in the specific spatial 1 and spatial 2 rendering methods used.

[0160] As a simple example, referring to the earlier example below: The spatial 1 reverberation is represented by N uncorrelated signals rendered from different directions around a listener in spatial 1, where each signal has a root mean square amplitude of 1 / N. In this case, the spatial 1 reverberation intensity value is the root mean square amplitude of the sum of the N uncorrelated signals, which is equal to 1 / sqrt(N). Now assume that the spatial 2 reverberation is derived from the spatial 1 reverberation signal by simply selecting one of the N signals with a root mean square amplitude of 1 / N. If this spatial 2 reverberation signal is now rendered as a point source at a certain position within a portal and a scaling factor according to Equation 10 Then, an additional gain of sqrt(N) must be applied to the spatial 2 reverberation signal in order to obtain the correct balance between the reverberation intensities in spaces 1 and 2.

[0161] Thus, the basic idea is to scale the spatial 2 reverberation signal such that the resulting intensity of the rendered spatial 2 reverberation has the desired relationship with the intensity of the spatial 1 reverberation as represented by the scaling factor.

[0162] As previously discussed, different rendering methods can be used to render the derived spatial 2 reverberation signal.

[0163] In one embodiment, the sound transmitted through the portal is rendered to the listener as a sound source located within the portal (i.e., the portal sound source). In one embodiment, the portal sound source is an extended sound source whose size substantially corresponds to the geometric size of the portal. The extended sound source can be a uniformly extended sound source (radiating the same signal from each point within the range), a diffuse extended sound source (radiating a spatially diffuse signal from many points within the range), or a heterogeneous extended sound source (radiating a diffuse or (partially) correlated signal from different points within the range).

[0164] In another embodiment, the portal sound source is a point source. In one embodiment, the point source is located at a fixed position (e.g., the central position within the portal). In another embodiment, the point source can be dynamically located within the portal depending on the listener's position. For example, the point source can be located at the point within the portal closest to the listener's position.

[0165] Second rendering stage

[0166] In the second rendering stage, a so-called "second-order" reverberation is generated in space 2 in response to the spatial 1 reverberation entering space 2 through the portal, according to the acoustic properties of space 2 (e.g., the spatial 2 reverberation time, absorption, and / or reverberation level or reverberation energy ratio). Here, for example according to Equation 7, this rendering can be based on the amount of diffuse power transmitted from space 1 to space 2. Then, the second-order reverberation can be generated as the reverberation of a (nominal) point source located in space 2 with a source power equal to the transmitted power.

[0167] More specifically, the method for implementing the second rendering stage can include the following steps:

[0168] (1: Optional step) Determine a spatial 1 reverberation intensity value that represents the intensity of the reverberation in space 1;

[0169] (2) Derive one or more reverberation input signals for generating reverberation in space 2 from one or more spatial 1 reverberation signals representing the reverberation in space 1;

[0170] (3) Obtain (e.g., determine, derive, receive) the size of the portal through which sound is transmitted between spaces 1 and 2;

[0171] (4) Use the portal size to determine a scaling factor that models the transmission of the reverberant sound from space 1 to space 2;

[0172] (5) Render one or more space 2 reverberant signals using one or more reverberant input signals, the scaling factor, and optionally a space 1 reverberation intensity value (e.g., generate one or more space 2 reverberant signals using one or more reverberant input signals, the scaling factor, and optionally a space 1 reverberation intensity value, and use the space 2 reverberant signals to produce an output audio signal).

[0173] Thus, the steps of the second rendering stage are generally similar to those of the first rendering stage, but with some differences in details, as will be described below.

[0174] Steps 1 and 3 are the same as in the first rendering stage. Thus, if both the first and second rendering stages are performed, steps 1 and 3 need only be performed once. Additionally, as in the case of the first rendering stage, in many embodiments, it may not be necessary to explicitly perform step 1, and in such cases step 1 can be omitted. Similarly, for example, this may be the case when deriving a reverberant input signal (which is derived in step 2) from a space 1 reverberant signal that has the correct level for rendering in space 1.

[0175] In step 2, a reverberant input signal is derived that is used as an input signal to a reverberator (e.g., a reverberation processor, engine, or processing block) to produce reverberation in space 2. Typically, only a single reverberant input signal may be needed to produce reverberation. Thus, if the first rendering stage has also been performed and the step 2 in the first rendering stage produces a single (e.g., mono downmix) signal, that signal can also be used as the reverberant input signal for the second rendering stage. In principle, any signal having the general characteristics of reverberation in space 1 can be used as the reverberant input signal in the second rendering stage, e.g., a single reverberant signal among multiple space 1 reverberant signals, or a single reverberant signal from which multiple space 1 reverberant signals are generated.

[0176] In one embodiment, the method may include the following additional step: generating a space 1 reverberant signal using space 1 reverberation control information before deriving one or more reverberant input signals for generating reverberation in space 2. In one embodiment, the space 1 reverberation control information includes a space 1 reverberation level parameter or a reverberation energy ratio parameter.

[0177] In step 4, the scaling factor can be equal to This is obtained by combining Equation (7) for the diffusion power P transmitted to Space 2 with Equation (9) for the pressure at 1 m from an omnidirectional source with source power P. It can be seen that this scaling factor is 1 / 2 of the scaling factor in the first rendering stage when using the model of Equation 10. The reason is that: in the second rendering stage, the reason for adding the factor 2 in Equation 8 does not apply here, and the "normal" relationship between the source power and the pressure of the omnidirectional point source of Equation 9 should be used.

[0178] Finally, in Step 5, by using the scaled version of the derived reverberant input signal as the source signal, the reverberation of Space 2 is generated and rendered according to the reverberation characteristics corresponding to Space 2 (e.g., reverberation time, reverberation energy ratio). The scaling factor and the optional reverberation intensity value of Space 1 are used to scale the gain of the reverberant input signal used to generate the reverberation signal of Space 2. This scaling can ensure that: when the scaled reverberant input signal is to be rendered as a point source, it will have the desired level at 1 m from the point source, i.e.,

[0179] p 2,1m 2 = Scaling factor x p 1 2 。Now, a reverberator configured according to the reverberation control information of Space 2 (e.g., RT60 and reverberation energy ratio parameters) is used to generate reverberation from the scaled reverberant input signal, thereby generating reverberation with the desired intensity.

[0180] Additional rendering aspects

[0181] If, as in a typical embodiment, the reverberation from Space 1 is rendered from the portal into Space 2 as an extended sound source (also referred to as a "volume" or "dimension" sound source) located at the portal and having the same geometric dimensions as the portal, then using the result of Equation 11 will be more realistic than when the sound from the portal is rendered as a point source located at a fixed point within the portal. In such an embodiment using an extended portal sound source (e.g., as used in the MPEG-I immersive audio standard), the distance from the extended sound source (i.e., the portal) is typically not measured relative to a reference point (e.g., the center point) within the portal, but rather relative to its closest point. This means that if the user were to (virtually) walk along a path parallel to the large portal, the distance from the portal (i.e., the distance used when rendering the extended sound source to the user) would remain constant, which means that the sound level experienced by the user along that path would also remain constant, as would be expected. (Conversely, if the sound from the portal were to be rendered as a point source at a fixed location within the portal, then that distance and thus the rendered sound level would change as the user moves along the portal).

[0182] A similar effect can be achieved if the sound from the portal is rendered to the user in Space 2 as a point source at a dynamic position that moves with the user within the portal rather than at a fixed position within the portal. In this case, the portal point source is dynamically located at the position within the portal that is closest to the user.

[0183] Furthermore, in the implementation of using an extended sound source to render the sound from the portal as described above, a distance attenuation function can generally be applied to the sound rendered from the extended portal sound source, and this distance attenuation function takes into account the geometric dimensions of the extended sound source as observed from the listening position, which can make the perceived effect more realistic. For example, if the listening position is initially in front of the portal and relatively close, the extended portal source can behave as a diffused planar sound source, and if the distance from the portal increases along a trajectory perpendicular to the portal, the rendered sound level may only decrease relatively slowly. As this distance further increases, the rate of decrease of the sound level increases more and more rapidly with the increase in distance and eventually approaches the rate of decrease of a point source.

[0184] On the other hand, if the listening position is initially on one side of the portal, the "perceived" geometric dimensions of the volume source range (i.e., the geometric dimensions "observed" from the listener's position) are much smaller than when standing directly in front of it. If the distance is now increased (while keeping the angle with the portal the same), the rendered sound level decreases more rapidly with the increase in distance than in the case of the listening trajectory in front of the portal.

[0185] Cascaded connected spaces

[0186] In the case where more than two spaces are connected to each other, the propagation of reverberation from one space to all other spaces through the corresponding portals can be modeled by repeatedly applying Equation 7 and / or Equation 4, which model the amount of reverberation power transmitted from one space to the next through the portal. For example, if three spaces 1, 2, and 3 are connected via a first portal between Space 1 and Space 2 and a second portal between Space 2 and Space 3, the amount of reverberation power transmitted to Space 3 due to a sound source in Space 1 can be determined as follows: First, apply Equation 7 to determine the power transmitted from the diffused reverberation pressure in Space 1 to Space 2 through the first portal. By using this determined transmission power level, reverberation can be generated in Space 2 based on the acoustic parameters of Space 2 (e.g., RT60 and reverberation energy ratio), thus providing diffused reverberation pressure in Space 2. Then, by applying Equation 7 to the Space 2 diffused reverberation pressure, the amount of power transmitted to Space 1 through the second portal can be calculated.

[0187] As an alternative to the step of rendering reverberation in space 2 based on the determined power amount from space 1 to space 2 and thereby determining the diffuse reverberation pressure in space 2, the power amount transmitted to space 3 can also be directly determined by applying equation 4 to the result of the first step (i.e., the power amount obtained in the first step from space 1 transmitted to space 2 as P1 in equation 4). Here, the only issue is that applying equation 4 requires the absorption amount A 1,tot (or A 1,0 ), which may not be directly available as metadata. In such cases, the absorption amount can be estimated based on available parameters, in particular a combination of the reverberation energy ratio, or the reverberation time RT60 and the volume of space 2. Patent application publication No. WO / 2023 / 031182 describes methods for deriving the absorption amount from these other parameters.

[0188] Figure 4 FIG. is a flowchart showing a process 400 for rendering reverberation in a second space (space 2) connected to a first space (space 1) via a portal according to some embodiments. Process 400 may be performed by an audio renderer 151. Process 400 may begin with an optional step s402.

[0189] Optional step s402 includes determining a reverberation intensity value associated with the reverberation associated with space 1.

[0190] Step s404 includes obtaining (e.g., deriving) information indicating the size of the portal.

[0191] Step s406 includes determining a scaling factor using the information indicating the size of the portal.

[0192] Step s408 includes rendering a set of reverberation signals composed of one or more space 2 reverberation signals in space 2 using the scaling factor and optionally the reverberation intensity value.

[0193] In some embodiments, the method further includes obtaining a reverberation intensity value associated with the reverberation associated with the first space (step s402), and the step of rendering a first set of reverberation signals composed of one or more reverberation signals in the second space using the scaling factor includes: rendering a first set of reverberation signals composed of one or more reverberation signals in the second space using the scaling factor and the reverberation intensity value.

[0194] In some embodiments, the reverberation intensity value is a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0195] In some embodiments, obtaining the reverberation intensity value includes receiving metadata of the first space, where the metadata includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0196] In some embodiments, a second set of reverberation signals composed of one or more reverberation signals represents reverberation in a first space, and the method further includes: deriving a first set of reverberation signals composed of one or more reverberation signals from the second set of reverberation signals before rendering the first set of reverberation signals in a second space.

[0197] In some embodiments, obtaining a reverberation intensity value includes determining the reverberation intensity value based on a second set of reverberation signals composed of one or more reverberation signals.

[0198] In some embodiments, deriving a first set of reverberation signals composed of one or more reverberation signals for rendering in a second space includes: downmixing a second set of reverberation signals composed of one or more reverberation signals.

[0199] In some embodiments, the information indicating the portal size is a size value S portal , and determining the scaling factor includes: calculating C1 * S portal , where C1 is a predetermined value. In some embodiments, C1 is approximately 1 / (8π). In some embodiments, determining the scaling factor further includes calculating the square root of C1 * S portal . In some embodiments, determining the scaling factor further includes determining whether C1 * S portal is less than C2, where C2 is a predetermined number.

[0200] In some embodiments, the size value represents the geometric size of the portal, or the size value represents the acoustic size of the portal.

[0201] In some embodiments, the information indicating the portal size is a solid angle value Ω relative to a position in the second space portal .

[0202] In some embodiments, determining the scaling factor includes calculating C1 * Ω portal , where C1 is a predetermined value.

[0203] In some embodiments, determining the scaling factor further includes calculating the square root of C1 * Ω portal .

[0204] In some embodiments, C1 is 1 / (4π).

[0205] In some embodiments, rendering a first set of reverberation signals composed of one or more reverberation signals in a second space includes: rendering the first set of reverberation signals composed of one or more reverberation signals as an extended sound source.

[0206] In some embodiments, a second set of reverberation signals composed of one or more reverberation signals represents reverberation in a first space, and rendering the first set of reverberation signals in a second space includes: deriving one or more reverberation input signals from the second set of reverberation signals; and using the one or more reverberation input signals to generate the first set of reverberation signals composed of one or more reverberation signals.

[0207] In some embodiments, using one or more reverberation input signals to generate the first set of reverberation signals composed of one or more reverberation signals includes: using the one or more reverberation input signals and a scaling factor to generate the first set of reverberation signals composed of one or more reverberation signals.

[0208] In some embodiments, the method further includes rendering one or more reverberation input signals in the second space using a second scaling factor, the second scaling factor being determined using information indicating a portal size.

[0209] In some embodiments, deriving one or more reverberation input signals includes: downmixing the second set of reverberation signals composed of one or more reverberation signals.

[0210] In some embodiments, the method further includes: generating the second set of reverberation signals composed of one or more reverberation signals using reverberation control information associated with the first space before deriving the one or more reverberation input signals.

[0211] In some embodiments, the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0212] In some embodiments, the information indicating the portal size is a size value S portal , and the scaling factor is equal to S portal / (16π).

[0213] In some embodiments, generating the first set of reverberation signals composed of one or more reverberation signals includes: generating the first set of reverberation signals composed of one or more reverberation signals according to reverberation characteristics corresponding to the second space.

[0214] In some embodiments, generating the first set of reverberation signals composed of one or more reverberation signals according to reverberation characteristics corresponding to the second space includes: using reverberation control information associated with the second space to generate the first set of reverberation signals composed of one or more reverberation signals.

[0215] In some embodiments, the reverberation control information associated with the second space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the second space.

[0216] In some embodiments, rendering a first set of reverberation signals composed of one or more reverberation signals in a second space includes: rendering the first set of reverberation signals composed of one or more reverberation signals as an immersive sound field.

[0217] In some embodiments, the method further includes: generating a second set of reverberation signals composed of one or more reverberation signals using reverberation control information associated with a first space before exporting the first set of reverberation signals composed of one or more reverberation signals.

[0218] In some embodiments, the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0219] Figure 5 is a block diagram of an audio rendering apparatus 500 for performing the methods disclosed herein according to some embodiments (e.g., the audio renderer 151 may be implemented using the audio rendering apparatus 500). As Figure 5As shown, the audio rendering device 500 may include: a processing circuit (PC) 502, which may include one or more processors (P) 555 (e.g., a general - purpose microprocessor and / or one or more other processors, such as an application - specific integrated circuit (ASIC), a field - programmable gate array (FPGA), etc.), and these processors may be co - located in a single housing or a single data center, or may be geographically distributed (i.e., the device 500 may be a distributed computing device); at least one network interface 548, which includes a transmitter (Tx) 545 and a receiver (Rx) 547, for enabling the device 500 to send data to and receive data from other nodes connected to a network 110 (e.g., an Internet Protocol (IP) network), and in this case, the network interface 548 is (directly or indirectly) connected to the network 110 (e.g., the network interface 548 may be wirelessly connected to the network 110, and in this case, the network interface 548 is connected to an antenna arrangement); and a storage unit (also known as a "data storage system") 508, which may include one or more non - volatile storage devices and / or one or more volatile storage devices. In an embodiment where the PC 502 includes a programmable processor, a computer program product (CPP) 541 may be provided. The CPP 541 includes a computer - readable medium (CRM) 542 storing a computer program (CP) 543, and the computer program (CP) includes computer - readable instructions (CRI) 544. The CRM 542 may be a non - transitory computer - readable medium, such as a magnetic medium (e.g., a hard disk), an optical medium, a storage device (e.g., a random - access memory, a flash memory), etc. In some embodiments, the CRI 544 of the computer program 543 is configured such that when executed by the PC 502, the CRI causes the audio rendering device 500 to perform the steps described herein (e.g., the steps described with reference to the flowcharts herein). In other embodiments, the audio rendering device 500 may be configured to perform the steps described herein without code. That is, for example, the PC 502 may consist only of one or more ASICs. Thus, the features of the embodiments described herein may be implemented in hardware and / or software ways.

[0220] Overview of additional various embodiments

[0221] A1. A method for rendering reverberation in a second space (space 2) connected to a first space (space 1) via a portal by an audio renderer, the method comprising: determining a reverberation intensity value associated with the reverberation associated with space 1; obtaining (e.g., deriving) information indicating the portal size; using the information indicating the portal size to determine a scaling factor; and using the scaling factor and the reverberation intensity value to render a set of reverberation signals composed of one or more space 2 reverberation signals in space 2.

[0222] A2. The method according to embodiment A1, wherein a set of reverberation signals composed of one or more reverberation signals represents a reverberant sound field in space 1 (the set of reverberation signals composed of the one or more signals is referred to as "space 1 reverberation signals"), and the method further includes: before rendering the space 2 reverberation signals, deriving a set of reverberation signals composed of one or more space 2 reverberation signals from the space 1 reverberation signals.

[0223] A3. The method according to embodiment A2, wherein determining the reverberation intensity value includes: determining the reverberation intensity value based on the set of reverberation signals composed of one or more space 1 reverberation signals.

[0224] A4. The method according to embodiment A2 or A3, wherein deriving a set of reverberation signals composed of one or more space 2 reverberation signals for rendering in space 2 includes: downmixing the set of reverberation signals composed of one or more space 1 reverberation signals.

[0225] A5. The method according to any one of embodiments A1 to A4, wherein the information indicating the portal size is a size value S portal , and determining the scaling factor includes calculating C1 * S portal , where C1 is a predetermined value.

[0226] A6. The method according to embodiment A5, wherein C1 is approximately 1 / (8π).

[0227] A7. The method according to embodiment A5 or A6, wherein determining the scaling factor further includes calculating the square root of C1 * S portal .

[0228] A8. The method according to embodiment A5 or A6, wherein determining the scaling factor further includes determining whether C1 * S portal is less than C2, where C2 is a predetermined number (for example, 0.5).

[0229] A9. The method according to any one of embodiments A1 to A4, wherein the information indicating the portal size is a solid angle value Ω portal .

[0230] A10. The method according to embodiment A9, wherein determining the scaling factor includes calculating C1 * Ω portal , where C1 is a predetermined value (for example, C1 = 1 / (4π)).

[0231] A11. The method according to embodiment A10, wherein determining the scaling factor further includes calculating the square root of C1 * Ω portal .

[0232] B1. A computer program comprising instructions which, when executed by a processing circuit of an audio renderer, cause the audio renderer to perform the method according to any one of the above embodiments.

[0233] B2. A carrier comprising the computer program according to embodiment B1, wherein the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium.

[0234] C1. An audio rendering apparatus configured to perform the method according to any one of the above embodiments.

[0235] C2. The audio rendering apparatus according to embodiment C1, wherein the audio rendering apparatus includes a memory and a processing circuit coupled to the memory.

[0236] Although various embodiments are described herein, it should be understood that they are presented by way of example and not limitation. Accordingly, the breadth and scope of the present disclosure should not be limited by any of the above exemplary embodiments. In addition, any combination of the above objects in all possible variations is covered by the present disclosure, unless otherwise indicated or clearly conflicting with the context in some other way.

[0237] In addition, although the processes described above and illustrated in the drawings are shown as a series of steps, they are for illustrative purposes only. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be rearranged, and some steps may be performed in parallel.

Claims

1. A method (400) performed by an audio renderer (151) for rendering reverberation in a second space (302) connected to a first space (301) via a portal (300), the method comprises: obtaining (s404) information indicating the size of the portal; using (s406) the information indicating the size of the portal to determine a scaling factor; and using the scaling factor to render (s408) a first set of reverberation signals composed of one or more reverberation signals in the second space (302).

2. The method according to claim 1, wherein, the method further comprises obtaining a reverberation intensity value associated with the reverberation associated with the first space, and the step of using the scaling factor to render a first set of reverberation signals composed of one or more reverberation signals in the second space comprises: using the scaling factor and the reverberation intensity value to render the first set of reverberation signals composed of one or more reverberation signals in the second space.

3. The method according to claim 2, wherein, the reverberation intensity value is a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

4. The method according to claim 3, wherein, obtaining the reverberation intensity value comprises: receiving metadata of the first space, wherein the metadata includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

5. The method according to any one of claims 1 to 4, wherein, a second set of reverberation signals composed of one or more reverberation signals represents the reverberation in the first space (301), and the method further comprises: deriving the first set of reverberation signals composed of one or more reverberation signals from the second set of reverberation signals before rendering the first set of reverberation signals in the second space (302).

6. The method according to claim 5 when dependent on claim 2, 3 or 4, wherein, obtaining the reverberation intensity value comprises: determining the reverberation intensity value based on the second set of reverberation signals composed of one or more reverberation signals.

7. The method according to claim 5 or 6, wherein, deriving the first set of reverberation signals composed of one or more reverberation signals for rendering in the second space comprises: downmixing the second set of reverberation signals composed of one or more reverberation signals.

8. The method according to any one of claims 1 to 7, wherein, The information indicating the size of the portal is the size value S portal , and Determining the scaling factor includes: calculating C1*S portal , where C1 is a predetermined value.

9. The method according to claim 8, wherein, C1 is about 1 / (8π).

10. The method according to claim 8 or 9, wherein, Determining the scaling factor further includes: calculating the square root of C1*S portal .

11. The method according to claim 8 or 9, wherein, Determining the scaling factor further includes: determining whether C1*S portal is less than C2, where C2 is a predetermined number.

12. The method according to any one of claims 8 to 11, wherein, the size value represents the geometric size of the portal, or the size value represents the acoustic size of the portal.

13. The method according to any one of claims 1 to 7, wherein, The information indicating the size of the portal is a solid angle value Ω relative to a position in the second space portal .

14. The method according to claim 13, wherein, Determining the scaling factor includes: calculating C1*Ω portal , where C1 is a predetermined value.

15. The method according to claim 14, wherein, Determining the scaling factor further includes: calculating the square root of C1*Ω portal .

16. The method according to claim 14 or 15, wherein, C1 is 1 / (4π).

17. The method according to any one of claims 1 to 16, wherein, rendering the first set of reverberation signals composed of one or more reverberation signals in the second space includes: rendering the first set of reverberation signals composed of one or more reverberation signals as an extended sound source.

18. The method according to claim 1, wherein, a second set of reverberation signals composed of one or more reverberation signals represents reverberation in the first space (301), and rendering the first set of reverberation signals in the second space (302) includes: deriving one or more reverberation input signals from the second set of reverberation signals; and using the one or more reverberation input signals to generate the first set of reverberation signals composed of one or more reverberation signals.

19. The method according to claim 18, wherein, using the one or more reverberation input signals to generate the first set of reverberation signals composed of one or more reverberation signals includes: using the one or more reverberation input signals and the scaling factor to generate the first set of reverberation signals composed of one or more reverberation signals.

20. The method according to claim 19, wherein, the method further includes: rendering the one or more reverberation input signals in the second space using a second scaling factor, the second scaling factor being determined using information indicating the size of the portal.

21. The method according to any one of claims 18 to 20, wherein, deriving the one or more reverberation input signals includes: downmixing the second set of reverberation signals composed of one or more reverberation signals.

22. The method according to any one of claims 18 to 21, wherein, the method further includes: generating the second set of reverberation signals composed of one or more reverberation signals using reverberation control information associated with the first space before deriving the one or more reverberation input signals.

23. The method according to claim 22, wherein, the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

24. The method according to any one of claims 18 to 23, wherein, The information indicating the size of the portal is the size value S portal , and The scaling factor is equal to S portal / (16π).

25. The method according to any one of claims 18 to 24, wherein, generating the first set of reverberation signals composed of one or more reverberation signals includes: generating the first set of reverberation signals composed of one or more reverberation signals according to the reverberation characteristics corresponding to the second space.

26. The method according to claim 25, wherein, generating the first set of reverberation signals composed of one or more reverberation signals according to the reverberation characteristics corresponding to the second space includes: generating the first set of reverberation signals composed of one or more reverberation signals using reverberation control information associated with the second space.

27. The method according to claim 26, wherein, the reverberation control information associated with the second space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the second space.

28. The method according to any one of claims 18 to 27, wherein, rendering the first set of reverberation signals composed of one or more reverberation signals in the second space includes: rendering the first set of reverberation signals composed of one or more reverberation signals as an immersive sound field.

29. The method according to any one of claims 5 to 7, wherein, the method further includes: generating a second set of reverberation signals composed of one or more reverberation signals using reverberation control information associated with the first space before exporting the first set of reverberation signals composed of one or more reverberation signals.

30. The method according to claim 29, wherein, the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

31. An audio rendering device configured to execute a method for rendering reverberation in a second space (302) connected to a first space (301) via a portal (300), the method comprising: obtaining (s404) information indicating the size of the portal; using (s406) the information indicating the size of the portal to determine a scaling factor; and using the scaling factor to render (s408) a first set of reverberation signals composed of one or more reverberation signals in the second space (302).

32. The audio rendering device (500) according to claim 31, wherein, the audio rendering device is further configured to execute the method according to any one of claims 2 to 30.

33. A computer program comprising instructions which, when executed by a processing circuit of an audio renderer, cause the audio renderer to execute the method according to any one of claims 1 to 30.

34. A carrier containing the computer program according to claim 33, wherein, the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium.

Images