Rendering of reverberation in connected space

Abstract

A method (400) performed by an audio renderer (151) for rendering reverberation in a second space (302) connected to a first space (301) via a portal (300). The method includes obtaining information indicative of a portal size. The method also includes determining a zoom factor using information indicative of the portal size. The method further includes rendering a first set of reverberation signals comprised of one or more reverberation signals in a second space (302) using the scaling factor.

Description

[0001] This application is a divisional application of PCT international application PCT / EP2023 / 083744 filed on November 30, 2023, entitled “Rendering of reverberation in connected spaces”, which has entered the Chinese national phase as patent application 202380079847.0. Technical Field

[0002] An embodiment involving rendering reverb in a second space connected to a first space via a portal is disclosed. Background Technology

[0003] Extended reality (XR) systems (e.g., virtual reality (VR), augmented reality (AR), mixed reality (MR), etc.) typically include an audio renderer for rendering audio to users of the XR system. The audio renderer typically contains a reverb processor for generating post-reverbearing and / or diffused reverb, which is rendered to the user of the XR system to provide the auditory experience of being in the XR scene being rendered. The resulting reverb provides the user with the auditory experience of being in an acoustic environment corresponding to the XR scene (e.g., a living room, gym, outdoor environment, etc.).

[0004] Reverberation is one of the most important acoustic properties of a room. Sound produced in a room will repeatedly bounce off reflective surfaces such as floors, walls, ceilings, windows, or tables, gradually losing energy as it does so. When these reflections mix with each other, the phenomenon known as "reverberation" occurs. Therefore, reverberation is a collection of many sound reflections.

[0005] The two most fundamental characteristics of reverberation in any (real or virtual) acoustic environment are: 1) reverberation time and 2) reverberation level, which is the intensity or loudness of the reverberation (e.g., relative to the power level or direct sound level of the sound source in that space).

[0006] Reverberation time is a measure of the time it takes for reflected sound to "disappear" in an enclosed space after the sound source has stopped. It is important in defining how a room responds to acoustic sounds. Reverberation time depends on the amount of sound absorption in the space, which is lower in spaces with many sound-absorbing surfaces such as curtains, upholstered chairs, or even people, and higher in spaces that mainly contain hard, reflective surfaces.

[0007] Traditionally, reverberation time is defined as the time required for the sound pressure level to decrease by 60 dB after the sound source is suddenly turned off. This time is abbreviated as "RT60" (or sometimes T60).

[0008] Typically, for reverb processors used in audio renderers, these two (and other) characteristics of the generated reverb can be controlled separately and independently. For example, a reverb processor can often be configured to produce reverb with a specific desired reverb time and a specific desired reverb level.

[0009] In XR systems, the characteristics of the generated reverberation are typically controlled by control information specified by the scene creator (e.g., specific metadata contained in the XR scene description), which describes many aspects of the XR scene, including its acoustic properties. The audio renderer receives this control information, for example, from a bitstream or file, and uses it to configure the reverberation processor to produce reverberation with the desired characteristics. Depending on the type of reverberation algorithm used by the reverberation processor to generate the reverberation, the exact way the reverberation processor obtains the desired reverberation time and level in the generated reverberation can vary.

[0010] Reverberation and Connectivity

[0011] As mentioned above, one of the key aspects of immersive audio rendering is the realistic rendering of reverberation associated with the virtual space of an XR scene. A particular challenge is realistically rendering reverberation in a space (aka the “acoustic environment”) (hereinafter referred to as the “second space”) that is connected (or “coupled”) to another space (hereinafter referred to as the “first space”) via one or more openings (e.g., open doors, windows) and / or one or more other interfaces between the first and second spaces (e.g., including partially transmissive walls), through which reverberation can propagate from the first space to the second space (and vice versa). In the following text, the term “portal” will be used to refer to any connecting interface (e.g., opening, transmissive object, etc.) through which sound can propagate from one space to another.

[0012] In real life, the properties of the reverberant sound field in a connected space (e.g., reverberation level, reverberation time, and other properties of the reverberant sound field in each space) are affected by the acoustic properties of each connected space (e.g., volume and absorption in each space).

[0013] In acoustic literature, advanced models can be used to model the reverberant sound field in each of two connected spaces, including the mutual influence of the acoustic properties of the two spaces on the resulting reverberant sound field in each space. In principle, such models can be used to derive a complete model of energy exchange and the resulting reverberant sound field in all connected spaces in an XR scene, but when more than two spaces are connected directly or through a "cascaded" connection, such models quickly become quite complex.

[0014] Therefore, in practical audio rendering systems, a slightly simpler method can often be used to simulate the effect of reverberation in connected spaces. A portion of the reverberation generated in one space (e.g., due to active sound sources present in that space) can be rendered into a second connected space as a "portal" sound source in an opening (or generally, an interface) between these spaces. This essentially simulates the propagation of reverberation from the first space to the second space through this portal. Subsequently, the sound from this portal source can be further "reverberated" according to the acoustic properties of the second space to simulate the overall reverberation that a listener would perceive in that second space.

[0015] Although this simplification is an approximation of what happens physically and may not lead to an exact match with predictions from more complex physics models, its perceptual effect is generally quite reasonable and its implementation is easier to fit into existing rendering pipelines. Summary of the Invention

[0016] Several challenges exist. For example, one problem with the portal source method described above is that determining the appropriate acoustic intensity of the portal source (i.e., the acoustic intensity that yields a realistic effect, i.e., the acoustic intensity that causes reverberation to propagate into the second space to be as close as possible to physical reality) can be challenging. In other words, a certain acoustic coupling factor that controls the acoustic intensity of the reverberant portal source needs to be determined with a certain degree of accuracy.

[0017] Furthermore, in practical audio rendering, it is necessary to determine how the acoustic intensity of the determined reverberation portal source relates to the level of the reverberated audio signal in the first space and the audio signal rendered from the reverberation portal source in the second space.

[0018] One source of complexity is that the signal representation of reverberation in the first space may differ from the representation of the signal rendered into the second space by the reverberation portal source. For example, the reverberation in the first space could be represented as a first set of (uncorrelated) signals designed to be rendered from different directions around a user virtually located in the first space (i.e., an immersive rendering representation). On the other hand, the reverberation propagated through the portal to the second space and rendered by the reverberation portal source could be represented as a second set of signals that can be derived from the first set. For example, it could be represented by a single signal or by a set of uncorrelated signals, depending on the specific rendering algorithm used to render the reverberation portal source.

[0019] In other words, a scaling factor needs to be derived from the acoustic coupling factor, which scales the reverberation portal source audio signal derived from the reverberation signal in the first space, so that when these scaled reverberation portal source audio signals are rendered by the reverberation portal source, the correct audio level is produced in the second space.

[0020] Therefore, in one aspect, a method is provided, performed by an audio renderer, for rendering reverberation in a second space (space 2) connected to a first space (space 1) via a portal. The method includes obtaining (e.g., deriving) information indicating the size of the portal. The method also includes using the information indicating the portal size to determine a scaling factor. The method further includes using the scaling factor to render a set of reverberation signals in space 2, consisting of one or more space 2 reverberation signals. The method may further include: determining a reverberation intensity value associated with the reverberation associated with space 1, and using the reverberation intensity value and the scaling factor to render a set of reverberation signals in space 2, consisting of one or more space 2 reverberation signals.

[0021] In another aspect, a computer program including instructions is provided that, when executed by the processing circuitry of an audio renderer, causes the audio renderer to perform the methods disclosed herein. In an embodiment, a carrier including a computer program is provided, wherein the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium. In another aspect, a rendering apparatus configured to perform any of the above methods is provided. The rendering apparatus may include a memory and processing circuitry coupled to the memory.

[0022] The advantage of the embodiments disclosed herein is that they enable audio renderers to produce reasonable reverberation rendering in complex XR scenes with connected acoustic spaces. Attached Figure Description

[0023] The accompanying drawings, which are included in and form part of this specification, illustrate various embodiments.

[0024] Figure 1A A system according to some embodiments is shown.

[0025] Figure 1B A system according to some embodiments is shown.

[0026] Figure 2 A system according to some embodiments is shown.

[0027] Figure 3A A portal from the first space to the second space is shown.

[0028] Figure 3B A portal from the first space to the second space is shown.

[0029] Figure 4 This is a flowchart illustrating a process according to an embodiment.

[0030] Figure 5 This is a block diagram of an apparatus according to some embodiments. Detailed Implementation

[0031] Figure 1A An XR system 100 capable of applying embodiments disclosed herein is illustrated. The XR system 100 includes speakers 104 and 105 (which may be speakers of headphones worn by a user) and an XR device 110, which may include a display for displaying images to a user and is configured in some embodiments to be worn by a listener. In the illustrated XR system 100, the XR device 110 has a display and is designed to be worn on the user's head and is generally referred to as a head-mounted display (HMD).

[0032] like Figure 1B As shown, the XR device 110 may include an orientation sensing unit 101, a position sensing unit 102, and a processing unit 103, which is (directly or indirectly) coupled to an audio renderer 151 for generating output audio signals (e.g., left audio signal 181 for the left speaker and right audio signal 182 for the right speaker, as shown).

[0033] Orientation sensing unit 101 is configured to detect changes in the listener's orientation and provide information about the detected changes to processing unit 103. In some embodiments, given a detected change in orientation detected by orientation sensing unit 101, processing unit 103 determines the absolute orientation (relative to a coordinate system). Different systems may also exist for determining orientation and position, such as systems using lighthouse trackers (LIDAR). In one embodiment, given a detected change in orientation, orientation sensing unit 101 may determine the absolute orientation (relative to a coordinate system). In this case, processing unit 103 may simply reuse the absolute orientation data from orientation sensing unit 101 and the position data from position sensing unit 102. In some embodiments, orientation sensing unit 101 may include one or more accelerometers and / or one or more gyroscopes.

[0034] The audio renderer 151 generates an audio output signal based on the input audio signal 161, metadata 162 about the XR scene the listener is experiencing, and information 163 about the listener's position and orientation. The metadata 162 of the XR scene may include metadata for each object and audio element included in the XR scene, as well as metadata about the XR space (“acoustic environment”) in which the listener virtually resides. The object metadata may include information about the object's size and its occlusion factor (e.g., the metadata may specify a set of occlusion factors, where each occlusion factor applies to a different frequency or frequency range). The metadata 162 may also include control parameters such as reverberation time values, reverberation level values, and / or absorption parameters.

[0035] The audio renderer 151 may be a component of the XR device 110, or it may be located away from the XR device 110 (e.g., the audio renderer 151 or its components may be implemented in the cloud).

[0036] Figure 2 An example implementation of an audio renderer 151 for generating sound for an XR scene is shown. The audio renderer 151 includes a controller 201 and an audio signal generator 202 for generating an output audio signal (e.g., an audio signal from a multi-channel audio element) based on control information 210 from the controller 201 and input audio 161. In this embodiment, the controller 201 includes a reverb processor 204 for determining a scaling factor as described below.

[0037] In some embodiments, controller 201 may be configured to receive one or more parameters and trigger audio signal generator 202 to modify audio signal 161 based on the received parameters (e.g., increase or decrease volume level). The received parameters include information 163 about the listener's location and / or orientation (e.g., direction and distance to audio elements) and metadata 162 about the XR scene. As described above, metadata 162 may include metadata about the XR space in which the user virtually resides (e.g., the size of the space, information about objects in the space, and information about the acoustic properties of the space), metadata about audio elements, and metadata about objects obstructing audio elements. In some embodiments, controller 201 itself generates at least a portion of metadata 162. For example, controller 201 may receive metadata about the XR scene and derive additional metadata (e.g., control parameters) based on the received metadata. For example, using metadata 162 and location / orientation information 163, controller 201 may calculate one or more gain factors (g) for audio elements in the XR scene (e.g., the scaling factor described above or the gain derived using the scaling factor).

[0038] For the generation of the reverberation signal used by signal generator 202 to produce the final output signal, controller 201 provides signal generator 202 with reverberation parameters (e.g., reverberation time and reverberation level of the acoustic environment of the XR scene) and the aforementioned scaling factor, enabling signal generator 202 to operate to generate the reverberation signal. The reverberation time of the generated reverberation is most commonly provided to reverberation processor 204 as an RT60 value (generally for each frequency band), although other reverberation time metrics exist and can be used. In a typical embodiment, metadata 162 includes all necessary reverberation parameters (e.g., RT60 values ​​and reverberation level values). However, in embodiments where the metadata does not include all necessary reverberation parameters, controller 201 can be configured to generate missing parameters.

[0039] Reverberation level can be expressed in several formats. Typically, it will be expressed as a relative level. For example, it can be expressed as the energy ratio (DRR) between the direct sound component and the reverberant sound component at a distance from the sound source rendered in an XR environment, or its reciprocal (i.e., RDR energy ratio). Alternatively, the reverberation level can be expressed as the energy ratio between the reverberant sound and the total emitted energy or power of the source. In the following text, the term "reverberation energy ratio" is used to refer to any of these or other measures of relative reverberation level. In other cases, the reverberation level can be directly expressed as the level / gain of the reverberation processor.

[0040] In this context, the term "reverberation" can generally refer only to those sound field components that correspond to the diffuse portion of the acoustic room impulse response of an acoustic environment, but in some embodiments, it can also include sound field components that correspond to the earlier portion of the room impulse response, for example, including some later non-diffuse reflections, or even all reflected sound.

[0041] Other metadata that can be included in metadata 162 to describe the reverberation-related characteristics of the acoustic environment includes: parameters describing the acoustic properties of the materials on the surface of the environment (such as the absorption, reflection, transmission and / or diffusion properties of the materials), or a specific point in time of the room impulse response associated with the acoustic environment, such as the time after the source is emitted, after which the room impulse response becomes diffuse (sometimes referred to as “pre-delay”).

[0042] All of the above reverberation-related properties are usually frequency-dependent, so their relevant metadata parameters are usually provided and processed separately for multiple frequency bands.

[0043] Example

[0044] As described above, this disclosure provides embodiments for generating reasonable reverberation in complex XR scenes with connected acoustic spaces. For example, this disclosure provides means for determining an acoustic coupling factor that indicates the amount of reverberation propagating from a first space (“acoustic environment”) through a portal (e.g., an opening or a partial emitting surface) into a second space connecting the two spaces. In one embodiment, the determined acoustic coupling factor is determined using information indicating the dimensions of the portal.

[0045] In some embodiments, an appropriate signal level is set based on an acoustic coupling factor to render one or more audio signals into a second space, wherein the one or more audio signals originate from one or more reverberation signals corresponding to a first space. For example, in some embodiments, the determined acoustic coupling factor is used to derive a scaling factor, wherein the scaling factor is used to scale one or more audio signals derived from a set of audio signals (consisting of one or more audio signals) representing reverberation in the first space. In some embodiments, the signal level used to render one or more audio signals into the second space is also determined based on the amplitude, power, or energy of the total reverberation signal received at a location in the first space.

[0046] In some embodiments, the signal level of the signal rendered into the second space is determined based on one or more acoustic parameters for the first space, more specifically, based on a reverberation level parameter or reverberation energy ratio parameter associated with the first space.

[0047] Theoretical Framework

[0048] Figure 3A A scene (real-life or VR) is shown consisting of two spaces: Space 1 and Space 2. These spaces are connected to each other via a portal 300 (which may alternatively be referred to as an "opening," "aperture," "interface," etc.). Sound source S1 is located somewhere in Space 1. Sound source S1 generates a reverberant sound field in Space 1.

[0049] The portal represents the interface between space 1 and space 2, through which some portions of the reverberant acoustic energy can be exchanged between the two spaces. The portal has... m 2 The "acoustic" dimension (also known as the "associated" dimension) (area). In the case where the portal is an acoustically completely transparent opening (e.g., an open door or window), the acoustic dimension of the portal... This is equivalent to the geometric dimensions of the door (e.g., the geometric area of ​​the door). More generally, if the door is not fully open but only partially acoustically transparent (e.g., a thin wall or thick curtain separating two spaces), then the acoustic dimensions of the door are... This refers to the equivalent size of a completely transparent opening that allows for the same amount of energy "leakage." In other words, if the portal is not completely acoustically transparent, then the acoustic size of the portal... It will be smaller than its geometric dimensions. In the following text, whenever the “size” or “area” of a portal is mentioned, it refers to the “acoustic” dimensions unless otherwise explicitly stated.

[0050] A portion of the reverberant sound energy generated by the sound source in space 1 propagates into space 2 through the portal, where a listener L located in space 2 hears the reverberation transmitted from space 1 through the portal.

[0051] To determine the level of reverberation in space 1 as perceived by listener L in space 2, it is necessary to determine the reverberant sound energy transmitted from space 1 to space 2 through the portal.

[0052] For simplicity, space 2 will first be considered as a “free field”, which means it is either a very large open space (e.g., outdoors) or a space with very high sound absorption, such that no reverberant energy propagates from space 2 back to space 1, thus making it a one-way problem.

[0053] Assuming a steady-state diffused sound field in space 1, which means that the sound energy leaving space 1 through absorption and portals to the connecting space per unit time is equal to the power of sound source S1 in space 1, it can be shown that, due to sound source S1 in space 1, the so-called average reverberation energy density in space 1 is... equal:

[0054] (1)

[0055] in, It is the acoustic power (in watts) of sound source S1 in space 1. It is the total absorption in space 1, including the absorption represented by the portal, expressed in terms of equivalent absorption area (m²). 2 ) represents, and c is the speed of sound in air (in m / s).

[0056] Further analysis can be conducted based on the absorption volume in Space 1 (excluding the portal). and portal size (with m) 2 Specify by unit, as shown below:

[0057] (2)

[0058] It can be further shown that, under diffusion steady-state conditions, the power transferred from space 1 to space 2 through the portal... (In watts) is usually equal to:

[0059] (3)

[0060] Combining equations 1, 2, and 3, the power transmitted from space 1 to space 2 through the portal is obtained:

[0061] (4)

[0062] Factors in Equation 4 Known as the "acoustic coupling factor" from space 1 to space 2, it indicates the fraction of source power transmitted from space 1 to space 2 under steady-state conditions.

[0063] It can be seen that the acoustic coupling factor is equal to the fraction of the total absorption in space 1 due to the portal. In other words, the power transmitted from space 1 to space 2 is determined by the total absorption in space 1. The score is determined by the size of the portal. To express.

[0064] If the absorption in space 1 (excluding the portal) and Compared to very small (e.g., if the wall height of space 1 is reflective and / or the portal is a very large opening), the acoustic coupling factor... It is essentially equal to 1, and the amount of power transmitted through the portal is essentially equal to the power radiated by the source.

[0065] On the other hand, if the absorption in space 1 (excluding the portal) and Compared to very large factors (e.g., if the wall height of space 1 is very high and / or the portal is very small), the acoustic coupling factor is... equal This refers to the ratio of the portal's size to the amount of absorption in space 1 (excluding the portal). In this case, the ratio will be a very small number, meaning that only a small fraction of the source power is transmitted through the portal.

[0066] The average steady-state reverberation energy density E in space is directly related to the root-mean-square steady-state reverberation sound pressure p in space, as follows:

[0067] (5)

[0068] in, It is the mass density of air.

[0069] Therefore, by using Equation 1, the steady-state reverberant sound pressure p1 in space 1 caused by source S1 can be written as follows:

[0070] (6)

[0071] Combining equations 4 and 6, we find the relationship between the steady-state reverberation pressure p1 in space 1 and the power transmitted to space 2:

[0072] (7)

[0073] Therefore, Equation 7 provides an expression for the amount of power transferred from space 1 to space 2, which involves the diffuse sound pressure p1 in space 1 and the size of the portal. Importantly, Equation 7 shows that if the diffuse sound pressure in space 1 and the dimensions of the portal are known, this directly gives the amount of power transmitted through the portal.

[0074] Now, in order to derive the relationship between the diffuse sound pressure p1 in space 1 and the sound pressure p2 in space 2 associated with radiation through the portal, we need to determine the power transmitted through the portal. The relationship between the pressure p2 generated in space 2 and the pressure p2 generated in space 2.

[0075] If the portal is relatively small, it can be assumed that the reverberant energy transmitted through the portal radiates uniformly in all directions from the portal to space 2 (i.e., spherically). This assumption leads to a pressure p at a distance p of 1 m from the portal. 2,1m equal:

[0076] (8)

[0077] This is based on the relationship between the sound source power P and the pressure p at a distance of 1m from the source radiating spherical waves:

[0078] (9)

[0079] In Equation 8, a factor of 2 has already been added to the power, because the power It only radiates into the two hemispheres on either side of the portal (therefore, the resulting pressure should be the pressure corresponding to a global radiation source with twice the power).

[0080] Combining equations 7 and 8, we can find the diffusion pressure p1 in space 1 and the root mean square pressure p at a distance of 1m from the portal in space 2. 2,1m The following relationships exist between them:

[0081] (10)

[0082] Regarding the audio signal level in the audio rendering system, the root mean square sound pressure p is directly proportional to the root mean square signal level of the corresponding audio signal, making Equation 10 provide a direct way to correlate the desired root mean square audio signal level in space 2 with the root mean square reverberant audio signal level in space 1.

[0083] Therefore, in terms of linear sound pressure or linear root-mean-square audio signal level, Equation 10 means that the reverberation response of space 1 is rendered from the portal into space 2, where scaling is performed such that at a distance of 1m from the portal, the resulting sound pressure or root-mean-square audio signal level is factored relative to the diffused sound pressure or root-mean-square audio signal level of the reverberation in space 1. Scaling is performed.

[0084] In terms of logarithmic sound pressure level or logarithmic root-mean-square audio signal level (in dB), this means that the reverberation of space 1 is rendered in space 2, resulting in a level of 10*log at a distance of 1m from the portal.10 ( )=10*log 10 ( -14 (dB) (relative to the diffuse sound pressure level or root mean square audio signal level in space 1).

[0085] As mentioned, this result is valid for sufficiently small portals, therefore the assumption of spherical radiation from the portal is reasonable. Clearly, the applicability of Equation 10 is limited because if the size of the portal… More than 8πm 2 Then the root mean square pressure p in space 2 2,1m The pressure will be greater than the root mean square pressure p1 in space 1, a situation that is physically impossible. In fact, due to the pressure p... 2,1m The associated acoustic power corresponds only to the portion of the diffuse sound field in space 1 that is incident on the opening; that is, its maximum value corresponds to the diffuse acoustic power from the side hemisphere of space 1 with respect to the opening. Therefore Never more than .

[0086] Equation 10 may be correct for The reason why the larger value of gives a physically unreasonable result is that using Equation 9 implies that the total power transmitted through the portal is actually radiated from a single point. If this is indeed the case, then the pressure near that single point would indeed be much higher than if the power were evenly distributed and radiated from the entire portal (which is actually the case in reality).

[0087] One way to solve this problem is to modify equation 10 as follows:

[0088] (11)

[0089] This modification prevents the level obtained in space 2 at a distance of 1m from the portal from exceeding the diffuse reverberation level in space 1 by 3dB (=10log). 10 (0.5)), which is required by physics (as explained above). Note that, depending on the actual rendering method used to render the reverberation from the portal, additional measures may be needed at a distance of 1m from the portal to ensure that the level there does not exceed the diffuse reverberation level in space 1 minus 3dB. In other words, it should be ensured that the resulting level of the signal rendered from the portal does not exceed the diffuse reverberation level in space 1 minus 3dB anywhere in space 2. This will also ensure a smooth transition of reverberation levels as the listener moves between spaces through the portal.

[0090] Although as a solution to the question of if More than 4πm 2 (=12.6 m) 2Then p 2,1m A fairly simple measure for a problem that could become too large, but the solution to Equation 11 can actually provide very reasonable results in many use cases.

[0091] if Significantly less than 4πm 2 The assumption of spherical radiation from the opening is reasonable, and using... Using / 8π as a scaling factor yields reasonable results.

[0092] As the size of the portal increases and approaches 4πm 2 The sound pressure level at a distance of 1m from the entrance is approximately equal to the reverberant sound pressure level in space 1 minus 3dB. Therefore, for distances greater than 4πm... 2 The portal eventually reaches and maintains that level. The latter seems reasonable because when standing at a distance of, for example, 4x3m (=12m)... 2 The reverberation experienced when standing 1m from the center of the opening is very similar to the reverberation experienced when standing at the opening itself.

[0093] The root mean square pressure p derived from a distance of 1m from the portal 2,1m The root mean square sound pressure p2 at any location at any distance d from the portal in space 2 can be directly derived, with the following equation: p2(d) = p 2,1m / d. This assumes the portal source acts as a point source radiation.

[0094] Alternatively, other models for modeling radiation from the portal can be used instead of the spherical radiation model of Equations 8 and 9, resulting in an alternative equation to Equation 8 for representing the transmitted power. The relationship between the pressure p2 obtained in space 2 and the relationship between the pressure p1 and p2 obtained in equations 10 and / or 11 (therefore, the scaling finally applied to the audio signal rendered from the portal is obtained in order to obtain the correct rendered audio signal level in space 2).

[0095] For example, a reverberant portal source can be modeled as a spatially diffused sound source, such as a spatially diffused line source or a planar source with dimensions equal to the geometry of the portal. Sound radiation models for such spatially diffused sources are provided in the literature, which correlate the source power with the resulting sound pressure at a given distance from the source.

[0096] Therefore, in some embodiments, alternative equations of the form 10 or 11 can be used:

[0097] (12)

[0098] Where C1 is a constant, or more generally:

[0099] (13)

[0100] in, Portal size The function.

[0101] Alternative theoretical frameworks

[0102] An alternative, but largely equivalent, theoretical perspective on the transmission of reverberation from space 1 to space 2 will be proposed.

[0103] Since, theoretically, the diffuse reverberant sound field at each point in space consists of uncorrelated plane waves of equal intensity arriving from all directions, the reverberant energy propagating from space 1 through the portal to a specific point in space 2 can be determined by geometric considerations.

[0104] If from a single solid angle The mean (root mean square) pressure of a single reverberant plane wave reaching any point in space 1 is expressed as p. d The root mean square diffusion reverberation pressure p1 at any point in space 1 is obtained by applying all solid angles. p on d The result obtained by integration:

[0105] (14)

[0106] Figure 3B Point L in space 2 is shown, and the opening angle is indicated when "looking" from point L through the portal into space 1. From the perspective of point L, the portal represents a solid angle. (in, ).

[0107] Since, by definition, the diffusion field in space 1 consists of uncorrelated plane waves of equal intensity from all directions, and since the pressure of a plane wave is constant along its path (i.e., it is independent of the distance traveled), each individual plane wave arriving at point L in space 2 through the portal contributes the same uncorrelated pressure component p. d Therefore, the pressure p2 obtained at point L in space 2 can be obtained by considering the solid angle of the portal at point L. (in, p on ) d Determine by integration:

[0108] (15)

[0109] Combining equations 14 and 15, we can obtain that the pressure p2 at point L in space 2 can be directly calculated from the solid angle. Determined by the diffusion pressure p1 in space 1:

[0110] (16)

[0111] In cases where the portal is not completely acoustically transparent but a fraction T of the power incident on the portal is transmitted, p2 is scaled accordingly.

[0112] This result can be compared with equations 10 to 12, and like these equations, it indicates that the root-mean-square pressure in space 2 is proportional to the root-mean-square pressure in space 1, where the scaling factor depends linearly on the size of the portal, where the size of the portal in equations 10 to 12 is expressed as (equivalent) area (m²). 2 ) represents, and is expressed as a solid angle in Equation 16.

[0113] The diffusion reverberation pressure p1 in space 1 can be determined according to Equation 6, based on the power of the sound source and the amount of sound absorption in space 1.

[0114] One point to note when comparing the two proposed theoretical frameworks is that while the first framework models the portal as a “secondary” sound source radiating into space 2, the second framework directly considers the reverberant energy received from space 1 at a specific point in space 2. Since the solid angle represented by the portal depends on its position relative to the portal, the pressure p2 obtained from Equation 16 also depends on the relative position at that specific location.

[0115] Specifically, the pressure obtained from Equation 16 will be very different for positions directly in front of the portal and positions on the side (or above / below) of the portal.

[0116] If the portal is small enough, then the geometric area at a distance r from the observation point is: The solid angle represented by a portal on a flat surface of m2 can be approximated as:

[0117] (17)

[0118] in, and These represent the position vector from the observation point to the portal and the normal vector of the portal, respectively.

[0119] At a distance of r = 1m, this becomes:

[0120] (18)

[0121] in, It is the viewing angle relative to the portal normal vector.

[0122] Combining Equations 18 and 16, we can see that for the position directly in front of the portal, and Comparing this equation with Equation 10, it can be seen that the root mean square pressure at 1m obtained according to Equation 16 is sqrt(2) times the root mean square pressure at 1m obtained according to Equation 10. On the other hand, at a position completely located on the side of the portal, ,therefore, This also holds true. This can be interpreted as follows: while Equation 16 represents the pressure at a specific point in space 2, Equation 10, derived from the assumption of spherical radiation from the portal, represents the average value of Equation 16 at all angles at a distance of 1m (i.e., the solid angle at all angles at a distance of 1m from the portal). The average value is equal to / 2).

[0123] It should also be noted that, although in the first theoretical framework, the level of reverberation in space 2 decreases with increasing distance from the portal in such a manner as determined by the radiation model used for the portal source (e.g., a point source radiation model or a diffuse plane source radiation model, as discussed above), in the second theoretical framework, this decrease in level with increasing distance from the portal is an inherent consequence of the solid angle represented by the portal decreasing with increasing distance.

[0124] In the above text, it has been assumed that space 2 is a free field, that is, a space that does not produce any diffuse reverberation (e.g., a large outdoor space).

[0125] Rendering

[0126] The XR audio renderer can be configured to render reverb appropriately in the connection space of an XR environment using the model described above.

[0127] In one embodiment, in the second connecting space 2, the rendering of the reverberation associated with the reverberation in the first space 1 is split into two stages: (1) the rendering of the reverberation from space 1, which reaches the listener in space 2 directly via the portal between the two spaces, and (2) the generation and rendering of the reverberation generated in space 2 in response to the reverberation from space 1 entering space 2 through the portal (also known as "second-order reverberation").

[0128] In some embodiments, only the first rendering stage may be performed. In other embodiments, only the second rendering stage may be performed. In still other embodiments, both rendering stages may be performed.

[0129] First rendering stage

[0130] The first rendering stage is essentially unrelated to the acoustics of Space 2. It is the rendering of reverberant sounds (for example, the reverberant sounds a listener standing in a large open space would hear coming from the open doors (i.e., portals) of a cathedral where music is being played).

[0131] In one embodiment, a method for rendering sound in space 2 due to reverberation in connected space 1 may include the following steps:

[0132] (1: Optional step) Determine the reverberation intensity value of space 1, which represents the intensity of reverberation in space 1;

[0133] (2) Derive one or more space 2 reverberation signals (e.g., downmixing signals) for rendering in space 2 from one or more reverberation signals (also known as “space 1 reverberation signals”) representing reverberation in space 1.

[0134] (3) Obtain (e.g., determine, export, receive) the dimensions of the portal through which sound is transmitted between space 1 and space 2;

[0135] (4) Use the portal size to determine the scaling factor, which models the transmission of reverberant sound from space 1 to space 2;

[0136] (5) Use the space 2 reverb signal and scaling factor (and optional space 1 reverb intensity value) to render one or more reverb signals in space 2.

[0137] In one embodiment, the method may include the additional step of generating a reverberation signal for space 1 using reverberation control information for space 1 before deriving a reverberation signal for space 2 from the reverberation signal for space 1. In one embodiment, the reverberation control information for space 1 includes a reverberation level parameter or a reverberation energy ratio parameter for space 1.

[0138] For the first (optional) step, the reverberation intensity value of space 1 can be determined in various ways.

[0139] In a simple scenario where reverberation in space 1 is rendered from a single (i.e., non-directional, mono) audio signal, the reverberation intensity value of space 1 can be simply determined as the root mean square amplitude or root mean square power of that signal, or, in the case where the reverberation is rendered based on the room impulse response, as the total amount of reverberation energy contained in the room impulse response.

[0140] When reverberation in space 1 is rendered using multiple audio signals (e.g., multiple uncorrelated signals rendered from multiple directions around the user), the reverberation intensity value of space 1 can be determined as the root mean square amplitude or power of the resulting combined signal. As an example, suppose the reverberation in space 1 is rendered to a listener in space 1 as N uncorrelated reverberation signals from N corresponding directions, each uncorrelated reverberation signal having a root mean square amplitude of 1 / N (or 1 / N...). 2 If the root mean square power is 1 / N, then the resulting combined reverberation signal has a root mean square power of 1 / N and a root mean square amplitude of 1 / sqrt(N).

[0141] In some cases, the reverberation intensity value of Space 1 does not need to be determined based on the actual reverberated audio signal of Space 1, but can be more effectively derived from the reverberation intensity metadata of Space 1. For example, the scene description metadata of an XR scene may contain a reverberation level parameter or a reverberation energy ratio parameter for Space 1, which describes the desired reverberation level in Space 1, either absolute or relative to the direct sound level or emission source energy / power of the source producing the reverberation in Space 1. In this case, the (relative) reverberation level in Space 1 is known a priori (and it is the renderer's job to generate reverberated audio signals for Space 1 such that they produce the specified reverberation level in Space 1). For example, suppose Space 1 has associated metadata that includes the reverberation-to-direct energy ratio (RDR) value in Space 1, which specifies the desired ratio of the reverberation energy to the direct sound energy at a distance of 1m from an omnidirectional audio source located somewhere in Space 1. Now, if an omnidirectional audio source in space 1 has an associated audio signal with a linear root-mean-square (RMS) amplitude of s and an associated linear source gain (“volume control”) of g, then the RMS amplitude of the direct sound rendering at a distance of 1 m from the audio source is given by g*s, such that the RMS power / energy of the direct sound signal is (g*s). 2 (and (g*s)) 2 (Proportional). Therefore, the root mean square energy / power of the reverberation associated with the audio source should be equal to RDR*(g*s). 2 This makes the linear root-mean-square signal amplitude of the reverberation sqrt(RDR)*g*s. Therefore, the reverberation intensity value of space 1 can be directly derived from the provided reverberation energy ratio (RDR) parameter of space 1, as well as the source gain and audio signal level of the audio source.

[0142] In some embodiments, the reverberation intensity value of space 1 is simply equal to the value of the space 1 reverberation level parameter or reverberation energy ratio parameter obtained from space 1 metadata.

[0143] If the source is not omnidirectional, but has an arbitrary directional pattern associated with it (which results in the source radiating an X fraction of the power of an omnidirectional source (for the same source signal)), then this results in the power of the resulting reverberation also being an X fraction of the power of an omnidirectional source. Therefore, the derived space-1 reverberation intensity value can be scaled accordingly, i.e., multiplied by a factor sqrt(X) if expressed in terms of linear RMS signal amplitude, or multiplied by a factor X if expressed in terms of RMS signal energy / power.

[0144] In addition to the source gain g, signal level s, and directional pattern discussed above, other aspects of source rendering also affect the gain of the rendered direct sound level or the rendered reverberation level, and can be taken into account in a similar way when calculating the reverberation intensity value for space 1.

[0145] However, in many embodiments, the reverberation intensity value of space 1 is not explicitly required and need not be explicitly applied. This is, for example, when a space 2 reverberation signal is derived directly from a space 1 reverberation signal having the correct level for rendering in space 1. In this case, the space 2 reverberation signal derived from the space 1 reverberation signal has implicitly been scaled using the space 1 reverberation intensity. Specifically, if the space 1 reverberation signal is generated using the provided space 1 reverberation level parameter or reverberation energy ratio parameter, this reverberation intensity information is inherently present in the generated space 1 reverberation signal and therefore automatically carries over to the space 2 reverberation signal derived from it. Therefore, in this case, the optional step (1) of the above method can be omitted, and the rendering step (5) using the reverberation intensity value can be omitted.

[0146] For step 2, the step of deriving one or more space 2 reverberation signals for rendering in space 2 can be done in various ways. In one embodiment, the space 2 reverberation signal can be a mono or stereo downmix from the space 1 reverberation audio signal.

[0147] In another embodiment, one or more space 2 reverberation signals can be derived directly from the source signal and space 1 reverberation metadata parameters (e.g., reverberation time RT60 and reverberation energy ratio parameters), i.e., without the intermediate step of first generating the actual space 1 reverberation signal. This may be more efficient because the space 1 reverberation signal is not actually rendered to the listener (located in space 2), but is only generated as an intermediate step in generating one or more space 2 reverberation signals.

[0148] Regarding step 3, the portal's dimensions can be obtained in various ways. In some embodiments, the portal's dimensions can be obtained directly from scene description data, which explicitly specifies the portal's location and / or size in space, as well as other spaces it connects to. In other embodiments, the dimensions can be derived from such scene description data (e.g., from geometric information). In yet another embodiment, the portal's dimensions can be heuristically detected, for example, using some form of ray tracing algorithm.

[0149] In some embodiments, the size of the portal represents the area of ​​the portal (in meters). 2 (in units). In some embodiments, the area is the equivalent area of ​​an acoustically completely transparent opening having the same amount of "acoustic power leakage" as a portal.

[0150] In other embodiments, the size of the portal represents a solid angle relative to a specific location in space 2, corresponding to the portal. Methods for deriving solid angles are readily available in the literature.

[0151] The scaling factor derived in step 4 represents the desired relationship between the intensity of the reverberation in space 1 (e.g., root mean square diffusion pressure, root mean square signal amplitude, or root mean square signal power) and the intensity of the rendered reverberation in space 2 (e.g., root mean square diffusion pressure, root mean square signal amplitude, or root mean square signal power).

[0152] In many embodiments, the basis for deriving the scaling factor can be given by any of equations 10 to 13 or 16, according to which the scaling factor can be derived as a correlation between p1 and p2 (alternatively, p1...). 2 With p2 2 (Related factors).

[0153] Therefore, for example, the scaling factor can be derived from Equation 10 as equal to (or its square root), and according to Equation 16, it can be derived as (or its square root).

[0154] Finally, the derived one or more space 2 reverberation signals are rendered to the listener in space 2 using a scaling factor and an optional space 1 reverberation intensity value.

[0155] The scaling factor and optional space 1 reverb intensity value can be combined to determine the desired intensity of the rendered space 2 reverb, for example, by a relationship such as: desired intensity of rendered space 2 reverb = scaling factor x space 1 reverb intensity value.

[0156] Once the desired intensity of the rendered space 2 reverberation has been determined, an appropriate scaling gain can be determined for the space 2 reverberation signal to achieve that desired intensity of the rendered space 2 reverberation.

[0157] In some embodiments, the scaling gain of the space 2 reverberation signal includes a scaling factor.

[0158] In some embodiments, the scaling gain of the space 2 reverberation signal is simply equal to the scaling factor.

[0159] In some embodiments, in addition to the scaling factor, the scaling gain of the reverberation signal in space 2 may also take into account the gain effect resulting from the specific way in which the reverberation signal in space 2 is derived from the reverberation signal in space 1, as well as the gain effect resulting from the different signal representation and rendering methods used for the reverberation signals in space 1 and space 2, respectively.

[0160] As already explained, the reverberation in space 1 can be represented by a combination of multiple signals (and rendered to a virtual listener in space 1). A reverberation signal for space 2 is derived from these multiple signals using some signal transformation (e.g., downmixing) process. This may introduce a transformation gain effect, i.e., the difference in total signal strength before and after the transformation. The scaling gain of the reverberation signal for space 2 can compensate for this transformation gain effect.

[0161] The scaling gain of the space 2 reverb signal can also compensate for the gain effect caused by the specific way the reverb signals are combined in the specific space 1 and space 2 rendering methods used.

[0162] As a simple example, consider the following earlier example: Space 1 reverberation is represented by N uncorrelated signals rendered from different directions around a listener in Space 1, where each signal has a root-mean-square amplitude of 1 / N. In this case, the reverberation intensity value of Space 1 is the root-mean-square amplitude of the sum of the N uncorrelated signals, which is equal to 1 / sqrt(N). Now suppose that Space 2 reverberation is derived from the Space 1 reverberation signal by simply selecting one of the N signals with a root-mean-square amplitude of 1 / N. If this Space 2 reverberation signal is now rendered as a point source located at some position within the portal and a scaling factor according to Equation 10... Therefore, the additional gain of sqrt(N) must be applied to the reverberation signal in space 2 in order to obtain the correct balance between the reverberation intensities in space 1 and space 2.

[0163] Therefore, the basic idea is to scale the reverberation signal of space 2 so that the intensity of the rendered reverberation of space 2 has the desired relationship with the intensity of the reverberation of space 1, as expressed by the scaling factor.

[0164] As previously discussed, different rendering methods can be used to render the exported spatial 2 reverberation signal.

[0165] In one embodiment, sound transmitted through the portal is rendered to the listener as a sound source located within the portal (i.e., the portal sound source). In one embodiment, the portal sound source is an extended sound source whose size substantially corresponds to the geometry of the portal. The extended sound source can be a uniform extended sound source (radiating the same signal from every point within the range), a diffuse extended sound source (radiating a spatially diffused signal from many points within the range), or a heterogeneous extended sound source (radiating diffused or (partially) correlated signals from different points within the range).

[0166] In another embodiment, the portal sound source is a point source. In one embodiment, the point source is located at a fixed location (e.g., the center of the portal). In another embodiment, the point source may be dynamically located within the portal depending on the listener's location. For example, the point source may be located at the point within the portal closest to the listener's location.

[0167] Second rendering stage

[0168] In the second rendering stage, a so-called "second-order" reverberation is generated in space 2 in response to the reverberation of space 1 entering space 2 through the portal, based on the acoustic properties of space 2 (e.g., reverberation time, absorption, and / or reverberation level or reverberation energy ratio of space 2). Here, for example, according to Equation 7, this rendering can be based on the amount of diffusion power transmitted from space 1 to space 2. The second-order reverberation can then be generated as the reverberation of a (nominal) point source located in space 2 with a source power equal to the transmitted power.

[0169] More specifically, the method for implementing the second rendering stage may include the following steps:

[0170] (1: Optional step) Determine the reverberation intensity value of space 1, which represents the intensity of reverberation in space 1;

[0171] (2) Derive one or more reverberation input signals for generating reverberation in space 2 from one or more reverberation signals representing reverberation in space 1;

[0172] (3) Obtain (e.g., determine, export, receive) the dimensions of the portal through which sound is transmitted between space 1 and space 2;

[0173] (4) Use the portal size to determine the scaling factor, which models the transmission of reverberant sound from space 1 to space 2;

[0174] (5) Use one or more reverb input signals, scaling factors, and optional space 1 reverb intensity values ​​to render one or more space 2 reverb signals (e.g., use one or more reverb input signals, scaling factors, and optional space 1 reverb intensity values ​​to generate one or more space 2 reverb signals, and use the space 2 reverb signals to generate the output audio signal).

[0175] Therefore, the steps in the second rendering stage are largely similar to those in the first rendering stage, but some details differ, as will be explained below.

[0176] Steps 1 and 3 are the same as in the first rendering stage. Therefore, if both the first and second rendering stages are performed, steps 1 and 3 only need to be performed once. Furthermore, as with the first rendering stage, in many embodiments, it may not be necessary to explicitly perform step 1, and in such cases, step 1 can be omitted. Similarly, this may be the case, for example, when deriving the reverberation input signal (which is derived in step 2) from the reverberation signal of space 1 (which has the correct level for rendering in space 1).

[0177] In step 2, a reverb input signal is derived, which is used as the input signal for a reverb unit (e.g., a reverb processor, engine, or processing block) to generate reverb in space 2. Typically, only a single reverb input signal may be needed to generate reverb. Therefore, if a first rendering stage is also performed, and step 2 in the first rendering stage generates a single (e.g., mono downmix) signal, that signal can also be used as the reverb input signal for the second rendering stage. In principle, any signal having the general characteristics of reverb in space 1 can be used as the reverb input signal for the second rendering stage; for example, a single reverb signal among multiple space 1 reverb signals, or a single reverb signal from which multiple space 1 reverb signals are generated.

[0178] In one embodiment, the method may include the additional step of generating a reverberation signal for space 2 using reverberation control information for space 1 before deriving one or more reverberation input signals for generating reverberation in space 2 from the reverberation signal for space 1. In one embodiment, the reverberation control information for space 1 includes a reverberation level parameter or a reverberation energy ratio parameter for space 1.

[0179] In step 4, the scaling factor can be equal to This is obtained by combining equation (7) for the diffusion power P transmitted to space 2 with equation (9) for the pressure at a distance of 1m from the omnidirectional source with source power P. It can be seen that this scaling factor is 1 / 2 of the scaling factor in the first rendering stage when using the model of equation 10. The reason for this is that in the second rendering stage, the reason for adding factor 2 in equation 8 does not apply here, and the "normal" relationship between the source power and pressure of the omnidirectional point source in equation 9 should be used instead.

[0180] Finally, in step 5, the reverberation of space 2 is generated and rendered by using a scaled version of the derived reverberant input signal as the source signal, based on the reverberation characteristics (e.g., reverberation time, reverberation energy ratio) corresponding to space 2. A scaling factor and an optional reverberation intensity value for space 1 are used to scale the gain of the reverberant input signal used to generate the reverberant signal for space 2. This scaling ensures that when the scaled reverberant input signal is rendered as a point source, it will have the desired level at a distance of 1m from the point source.

[0181] p 2,1m 2 =Scaling factor xp1 2 Now, a reverberant configured according to the reverberation control information of Space 2 (e.g., RT60 and reverberation energy ratio parameters) is used to generate reverberation from the scaled reverberation input signal, thereby producing reverberation with the desired intensity.

[0182] In addition to rendering

[0183] If, as in a typical implementation, the reverberation from space 1 is rendered from the portal into space 2 as an extended sound source (also referred to as a "volume" or "size" sound source) located at the portal and having the same geometry as the portal, the result using Equation 11 will be more realistic than if the sound from the portal were rendered as a point source located at a fixed point within the portal. In this implementation using an extended portal sound source (e.g., as used in the MPEG-I immersive audio standard), the distance to the extended sound source (i.e., the portal) is typically not measured relative to some reference point (e.g., the center point) within the portal, but rather relative to its nearest point. This means that if a user walks (virtually) along a path parallel to the large portal, the distance to the portal (i.e., the distance used when rendering the extended sound source to the user) remains constant, meaning the sound level experienced by the user along that path also remains constant, as would be expected. (Conversely, if the sound from the portal were rendered as a point source at a fixed location within the portal, that distance, and therefore the rendered sound level, would change as the user moves along the portal).

[0184] A similar effect could be achieved if the sound from the portal were rendered to the user in space 2 as a point source located at a dynamic position within the portal that moves with the user, rather than a point source located at a fixed position within the portal. In this case, the portal point source is dynamically located at the position closest to the user within the portal.

[0185] Furthermore, in the implementation described above that uses an extended sound source to render sound from a portal, a distance attenuation function can typically be applied to the sound rendered from the extended portal sound source. This distance attenuation function takes into account the geometry of the extended sound source as observed from the listening position, which can make the perceived effect more realistic. For example, if the listening position is initially located in front of and relatively close to the portal, the extended portal source can behave as a diffuse planar sound source, and its rendered sound level may only decrease relatively slowly as the distance from the portal increases along a trajectory perpendicular to the portal. As the distance increases further, the rate of decrease in sound level accelerates with increasing distance, eventually approaching the rate of decrease of a point source.

[0186] On the other hand, if the listening position is initially located to one side of the portal, the "perceived" geometry of the volume source range (i.e., the geometry "observed" from the listener's position) is much smaller than the geometry when standing directly in front of it. If this distance is now increased (while maintaining the same angle with the portal), the rendered sound level decreases much faster with increasing distance than in the case of a listening trajectory directly in front of the portal.

[0187] Cascaded connection space

[0188] In cases where more than two spaces are interconnected, the propagation of reverberation through corresponding portals from one space to all other spaces can be modeled by repeatedly applying Equation 7 and / or Equation 4, which models the amount of reverberant power transmitted from one space to the next through the portals. For example, if three spaces 1, 2, and 3 are connected via a first portal between spaces 1 and 2 and a second portal between spaces 2 and 3, the amount of reverberant power transmitted to space 3 due to a sound source in space 1 can be determined as follows: First, Equation 7 is applied to determine the power transmitted from the diffuse reverberation pressure in space 1 to space 2 via the first portal. Using this determined transmission power level, reverberation can be generated in space 2 based on the acoustic parameters of space 2 (e.g., RT60 and reverberation energy ratio), thereby providing a diffuse reverberation pressure in space 2. Then, by applying Equation 7 to the diffuse reverberation pressure in space 2, the amount of power transmitted to space 1 via the second portal can be calculated.

[0189] As an alternative to the step of rendering the reverberation in space 2 based on the determined power amount from space 1 to space 2 and thereby determining the scattering reverberation pressure in space 2, the power amount transmitted to space 3 can also be directly determined by applying Equation 4 to the result of the first step (i.e., the power amount transmitted from space 1 to space 2 obtained in the first step is P1 in Equation 4). The only problem here is that applying Equation 4 requires the absorption amount A in space 2. 1,tot (or A) 1,0 This data may not be directly usable as metadata. In such cases, the absorption can be estimated based on available parameters, particularly the reverberation energy ratio, or the combination of reverberation time RT60 and the volume of space 2. Patent application publication No. WO / 2023 / 031182 describes a method for deriving the absorption from these additional parameters.

[0190] Figure 4 This is a flowchart illustrating a process 400 for rendering reverb in a second space (space 2) connected to a first space (space 1) via a portal, according to some embodiments. Process 400 can be performed by an audio renderer 151. Process 400 can begin at optional step s402.

[0191] Optional step s402 includes determining the reverberation intensity value associated with the reverberation associated with space 1.

[0192] Step s404 includes obtaining (e.g., exporting) information indicating the dimensions of the portal.

[0193] Step s406 includes using information indicating the size of the portal to determine a scaling factor.

[0194] Step s408 includes rendering a set of reverberation signals consisting of one or more reverberation signals in space 2 using a scaling factor and an optional reverberation intensity value.

[0195] In some embodiments, the method further includes obtaining a reverberation intensity value associated with the reverberation associated with the first space (step s402), and rendering a first set of reverberation signals consisting of one or more reverberation signals in a second space using a scaling factor, comprising: rendering a first set of reverberation signals consisting of one or more reverberation signals in a second space using a scaling factor and a reverberation intensity value.

[0196] In some embodiments, the reverberation intensity value is a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0197] In some embodiments, obtaining a reverberation intensity value includes receiving metadata of a first space, wherein the metadata includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0198] In some embodiments, a second set of reverberation signals, consisting of one or more reverberation signals, represents reverberation in a first space, and the method further includes: deriving a first set of reverberation signals, consisting of one or more reverberation signals, from the second set of reverberation signals before rendering the first set of reverberation signals in a second space.

[0199] In some embodiments, obtaining a reverberation intensity value includes determining the reverberation intensity value based on a second set of reverberation signals consisting of one or more reverberation signals.

[0200] In some embodiments, deriving a first set of reverberation signals consisting of one or more reverberation signals for rendering in a second space includes: downmixing a second set of reverberation signals consisting of one or more reverberation signals.

[0201] In some embodiments, the information indicating the portal size is the size value S. portal And determining the scaling factor includes: calculating C1*S portal Where C1 is a predetermined value. In some embodiments, C1 is approximately 1 / (8π). In some embodiments, determining the scaling factor further includes calculating C1*S. portal The square root of C1*S. In some embodiments, determining the scaling factor further includes determining C1*S. portal Is it less than C2, where C2 is a predetermined number?

[0202] In some embodiments, the size value represents the geometric dimensions of the portal, or the size value represents the acoustic dimensions of the portal.

[0203] In some embodiments, the information indicating the portal size is a solid angle value Ω relative to the position in the second space.portal .

[0204] In some embodiments, determining the scaling factor includes calculating C1*Ω portal Where C1 is a predetermined value.

[0205] In some embodiments, determining the scaling factor further includes calculating C1*Ω portal The square root of.

[0206] In some embodiments, C1 is 1 / (4π).

[0207] In some embodiments, rendering a first set of reverberation signals consisting of one or more reverberation signals in a second space includes: rendering the first set of reverberation signals consisting of one or more reverberation signals as an extended sound source.

[0208] In some embodiments, a second set of reverberation signals, consisting of one or more reverberation signals, represents reverberation in a first space, and rendering the first set of reverberation signals in the second space includes: deriving one or more reverberation input signals from the second set of reverberation signals; and using the one or more reverberation input signals to generate the first set of reverberation signals consisting of one or more reverberation signals.

[0209] In some embodiments, using one or more reverberation input signals to generate a first set of reverberation signals consisting of one or more reverberation signals includes: using one or more reverberation input signals and a scaling factor to generate a first set of reverberation signals consisting of one or more reverberation signals.

[0210] In some embodiments, the method further includes rendering one or more reverberant input signals in a second space using a second scaling factor determined using information indicating the portal size.

[0211] In some embodiments, deriving one or more reverberation input signals includes: downmixing a second set of reverberation signals consisting of one or more reverberation signals.

[0212] In some embodiments, the method further includes: generating a second set of reverberation signals consisting of one or more reverberation signals using reverberation control information associated with a first space before deriving the one or more reverberation input signals.

[0213] In some embodiments, the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0214] In some embodiments, the information indicating the portal size is the size value S. portal And the scaling factor is equal to S portal / (16π).

[0215] In some embodiments, generating a first set of reverberation signals consisting of one or more reverberation signals includes: generating a first set of reverberation signals consisting of one or more reverberation signals based on reverberation characteristics corresponding to a second space.

[0216] In some embodiments, generating a first set of reverberation signals consisting of one or more reverberation signals based on reverberation characteristics corresponding to a second space includes: generating a first set of reverberation signals consisting of one or more reverberation signals using reverberation control information associated with the second space.

[0217] In some embodiments, the reverberation control information associated with the second space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the second space.

[0218] In some embodiments, rendering a first set of reverberation signals consisting of one or more reverberation signals in a second space includes: rendering the first set of reverberation signals consisting of one or more reverberation signals as an immersive sound field.

[0219] In some embodiments, the method further includes: generating a second set of reverberation signals consisting of one or more reverberation signals using reverberation control information associated with a first space before deriving a first set of reverberation signals consisting of one or more reverberation signals.

[0220] In some embodiments, the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

[0221] Figure 5 This is a block diagram of an audio rendering apparatus 500 for performing the methods disclosed herein, according to some embodiments (e.g., an audio renderer 151 may be implemented using the audio rendering apparatus 500). Figure 5As shown, the audio rendering device 500 may include: a processing circuit (PC) 502, which may include one or more processors (P) 555 (e.g., a general-purpose microprocessor and / or one or more other processors, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), which may be housed in a single enclosure or a single data center, or may be geographically distributed (i.e., the device 500 may be a distributed computing device); at least one network interface 548, which includes a transmitter (Tx) 545 and a receiver (Rx) 547, for enabling the device 500 to send data to and receive data from other nodes connected to the network 110 (e.g., an Internet Protocol (IP) network), in which case the network interface 548 is (directly or indirectly) connected to the network 110 (e.g., the network interface 548 may be wirelessly connected to the network 110, in which case the network interface 548 is connected to an antenna arrangement); and a storage unit (also known as a "data storage system") 508, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 502 includes a programmable processor, a computer program product (CPP) 541 may be provided. CPP 541 includes a computer-readable medium (CRM) 542 storing a computer program (CP) 543, which includes computer-readable instructions (CRI) 544. CRM 542 may be a non-transitory computer-readable medium, such as a magnetic medium (e.g., a hard disk), an optical medium, a storage device (e.g., random access memory, flash memory), etc. In some embodiments, the CRI 544 of the computer program 543 is configured such that, when executed by PC 502, the CRI causes the audio rendering apparatus 500 to perform the steps described herein (e.g., the steps described herein with reference to the flowcharts). In other embodiments, the audio rendering apparatus 500 may be configured to perform the steps described herein without requiring code. That is, for example, PC 502 may consist only of one or more ASICs. Therefore, the features of the embodiments described herein may be implemented in hardware and / or software.

[0222] An overview of various embodiments is attached.

[0223] A1. A method performed by an audio renderer for rendering reverberation in a second space (space 2) connected to a first space (space 1) via a portal, the method comprising: determining a reverberation intensity value associated with reverberation associated with space 1; obtaining (e.g., deriving) information indicating the portal size; using the information indicating the portal size to determine a scaling factor; and rendering a set of reverberation signals consisting of one or more reverberation signals of space 2 in space 2 using the scaling factor and the reverberation intensity value.

[0224] A2. The method according to embodiment A1, wherein a set of reverberation signals consisting of one or more reverberation signals represents a reverberant sound field in space 1 (the set of reverberation signals consisting of the one or more signals is referred to as "space 1 reverberation signal"), and the method further includes: deriving a set of reverberation signals consisting of one or more space 2 reverberation signals from the space 1 reverberation signal before rendering the space 2 reverberation signal.

[0225] A3. The method according to embodiment A2, wherein determining the reverberation intensity value includes: determining the reverberation intensity value based on the set of reverberation signals consisting of one or more reverberation signals of space 1.

[0226] A4. The method according to embodiment A2 or A3, wherein deriving a set of reverberation signals consisting of one or more reverberation signals of space 2 for rendering in space 2 includes: downmixing the set of reverberation signals consisting of one or more reverberation signals of space 1.

[0227] A5. The method according to any one of embodiments A1 to A4, wherein the information indicating the portal size is the size value S. portal And determining the scaling factor includes calculating C1*S portal Where C1 is a predetermined value.

[0228] A6. The method according to embodiment A5, wherein C1 is approximately 1 / (8π).

[0229] A7. The method according to embodiment A5 or A6, wherein determining the scaling factor further includes calculating C1*S portal The square root of.

[0230] A8. The method according to embodiment A5 or A6, wherein determining the scaling factor further includes determining C1*S portal Is it less than C2, where C2 is a predetermined number (e.g., 0.5)?

[0231] A9. The method according to any one of embodiments A1 to A4, wherein the information indicating the portal size is the solid angle value Ω. portal .

[0232] A10. The method according to embodiment A9, wherein determining the scaling factor includes calculating C1*Ω portal Where C1 is a predetermined value (e.g., C1 = 1 / (4π)).

[0233] A11. The method according to embodiment A10, wherein determining the scaling factor further includes calculating C1*Ω portal The square root of.

[0234] B1. A computer program including instructions that, when executed by the processing circuitry of an audio renderer, cause the audio renderer to perform a method according to any of the embodiments described above.

[0235] B2. A carrier comprising a computer program according to embodiment B1, wherein the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium.

[0236] C1. An audio rendering apparatus configured to perform a method according to any of the above embodiments.

[0237] C2. An audio rendering apparatus according to embodiment C1, wherein the audio rendering apparatus includes a memory and processing circuitry coupled to the memory.

[0238] Although various embodiments have been described herein, it should be understood that they are presented by way of example only and not by way of limitation. Therefore, the breadth and scope of this disclosure should not be limited to any of the exemplary embodiments described above. Furthermore, any combination of the foregoing objects with all possible variations is covered in this disclosure unless otherwise indicated or otherwise explicitly conflicted by the context.

[0239] Furthermore, although the process described above and shown in the accompanying drawings is presented as a series of steps, it is for illustrative purposes only. Therefore, it is conceivable that some steps may be added, some steps may be omitted, the order of steps may be rearranged, and some steps may be performed in parallel.

Claims

1. A method (400) performed by an audio renderer (151) for rendering reverb in a second space (302) connected to a first space (301) via a portal (300), the method comprising: Obtain (s404) information indicating the size of the portal; The scaling factor is determined using the information (s406) indicating the size of the portal; A first set of reverberation signals, consisting of one or more reverberation signals, is derived from the second set of reverberation signals, wherein the second set of reverberation signals, consisting of one or more reverberation signals, represents the reverberation in the first space (301); as well as The first set of reverberation signals, consisting of one or more reverberation signals, is rendered (s408) in the second space (302) using the scaling factor.

2. The method according to claim 1, wherein, The method further includes obtaining a reverberation intensity value associated with the reverberation associated with the first space, wherein the reverberation intensity value is a reverberation level parameter or a reverberation energy ratio parameter associated with the first space, and The step of rendering a first set of reverberation signals consisting of one or more reverberation signals in the second space using the scaling factor includes: rendering the first set of reverberation signals consisting of one or more reverberation signals in the second space using the scaling factor and the reverberation intensity value.

3. The method according to claim 2, wherein, Obtaining the reverberation intensity value includes: receiving metadata of the first space, wherein the metadata includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

4. The method according to claim 1, wherein, Exporting the first set of reverberation signals, consisting of one or more reverberation signals, for rendering in the second space includes: downmixing the second set of reverberation signals, consisting of one or more reverberation signals.

5. The method according to any one of claims 1 to 4, wherein, The information indicating the size of the portal is the size value S. portal ,and Determining the scaling factor includes: calculating C1*S portal Where C1 is a predetermined value.

6. The method according to claim 5, wherein, C1 is approximately 1 / (8π).

7. The method according to claim 5 or 6, wherein, Determining the scaling factor further includes: calculating C1*S portal The square root of.

8. The method according to any one of claims 5 to 7, wherein, The size value represents the geometric dimensions of the portal, or The size value represents the acoustic size of the portal.

9. The method according to any one of claims 1 to 4, wherein, Information indicating the size of the portal is the solid angle value Ω relative to its position in the second space. portal Determining the scaling factor includes: calculating C1*Ω portal Where C1 is a predetermined value.

10. The method of claim 9, wherein, Determining the scaling factor further includes: calculating C1*Ω portal The square root of.

11. The method according to any one of claims 1 to 10, wherein, Rendering the first set of reverberation signals, which consists of one or more reverberation signals, in the second space includes: rendering the first set of reverberation signals, which consists of one or more reverberation signals, as an extended sound source.

12. The method according to any one of claims 1 to 4, wherein, The method further includes: before deriving the first set of reverberation signals consisting of one or more reverberation signals, using reverberation control information associated with the first space to generate a second set of reverberation signals consisting of one or more reverberation signals, wherein the reverberation control information associated with the first space includes a reverberation level parameter or a reverberation energy ratio parameter associated with the first space.

13. An audio rendering apparatus configured to perform a method for rendering reverberation in a second space (302) connected to a first space (301) via a portal (300), the method comprising: Obtain (s404) information indicating the size of the portal; The scaling factor is determined using the information (s406) indicating the size of the portal; A first set of reverberation signals, consisting of one or more reverberation signals, is derived from the second set of reverberation signals, wherein the second set of reverberation signals, consisting of one or more reverberation signals, represents the reverberation in the first space (301); as well as The first set of reverberation signals, consisting of one or more reverberation signals, is rendered (s408) in the second space (302) using the scaling factor.

14. The audio rendering apparatus (500) according to claim 13, wherein, The audio rendering apparatus is also configured to perform the method according to any one of claims 2 to 12.

15. A computer program comprising instructions that, when executed by processing circuitry of an audio renderer, cause the audio renderer to perform the method according to any one of claims 1 to 12.

Images