RENDERER, DECODER, ENCODER, METHODS AND BITSTREAMS USING SPATIAL AUGMENTATION SOURCES - Patent application

Spatially augmented sound sources efficiently render diffuse sound propagation in virtual reality environments by simulating sound through portals, addressing the challenge of balancing audio perception and computational efficiency in dynamic acoustic spaces.

JP2024543840AInactive Publication Date: 2024-11-26FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
JP2024527408
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-09
Filing Date
2022-11-09
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing technologies face challenges in representing sound propagation between acoustically different spaces, especially in virtual or augmented reality environments where users can move freely, leading to difficulties in achieving a balanced perception of audio scenes while efficiently transmitting and decoding data.

Method used

The use of spatially augmented sound sources, such as spatially extended sound sources (SESS), which simulate diffuse sound propagation through portals, allowing for efficient rendering of acoustic effects in multiple spatial regions by combining direct and diffuse sound rendering techniques, and incorporating occlusion effects to reduce computational complexity.

Benefits of technology

This approach provides a faithful listening experience with reduced computational burden, enabling efficient transmission and processing of audio scenes in dynamic virtual environments, supporting real-time adaptations and immersive audio experiences.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543840000001_ABST
    Figure 2024543840000001_ABST
Patent Text Reader

Abstract

An embodiment according to the invention comprises a renderer for rendering, e.g., spatially rendering, an acoustic scene, which is configured to render, e.g., reproduce, diffuse sound effects (e.g., reverberant, e.g., late reverberant) occurring in a first spatial domain (e.g., in a first acoustic homogeneous space, i.e., AHS, e.g., in a first room) in a second spatial domain (e.g., in a second acoustic homogeneous space, e.g., in a second room, e.g., in a spatial domain outside the first spatial domain) using, e.g., a homogeneous extension source algorithm, e.g., a spatial extension source, e.g., SESS, as a spatial extension source reproducing diffuse sound. Further, encoders, methods, and bitstreams are disclosed.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The embodiments relate to renderers, decoders, encoders, methods and bitstreams that use spatial enhancement sources.

[0002] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in accordance with the present invention comprises an apparatus and method for simulating diffuse sound propagation through a portal using spatially expanding sound sources. [Background technology]

[0003] For example, for virtual and augmented reality applications, a challenging task may be the representation of sound propagation between different acoustic spaces, e.g., acoustic spaces with different acoustic properties.

[0004] Such a task can be particularly challenging for virtual or augmented reality environments that have many acoustically coupled spaces. In addition, further difficulties may arise from the unstable nature of the audio scene, where a user may not have a pre-determined position but may be able to move freely in real time within the sound scene and may act as a sound source.

[0005] It is therefore desirable to provide a concept that offers a better compromise between the achievable perceived impression of a rendered audio scene, the efficiency of transmission of the data used for rendering the audio scene, and the efficiency of decoding and / or rendering of the data.

[0006] This is achieved by the subject matter of the independent claims of the present application.

[0007] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] PCT / EP2021 / 050588 [Patent Document 2] EP3879856 [Patent Document 3] "AUDIO DECORRELATOR, PROCESSING SYSTEM AND METHOD FOR DECORRELATING AN AUDIO SIGNAL", European Patent Application No. EP21162142.0, DISCH Sascha, ANEMULLER Carlotta, HERRE Jurgen Summary of the Invention [Means for solving the problem]

[0009] An embodiment according to the invention comprises a renderer for rendering, e.g. for spatially rendering, an acoustic scene, which is configured to render, e.g. reproduce, diffuse sound effects (e.g. reverberant, e.g. late reverberant) occurring in a first spatial domain (e.g. in a first acoustically homogeneous space, i.e. AHS, e.g. in a first room) in a second spatial domain (e.g. in a second acoustically homogeneous space, e.g. in a second room, e.g. in a spatial domain outside the first spatial domain) using a spatially-extending sound source, e.g. SESS, e.g. a spatially-extending sound source reproducing diffuse sound, e.g. using a homogeneous-extending sound source algorithm.

[0010] As an example, the inventors have recognised that the acoustic impact of a diffuse sound field from a first spatial region that is acoustically coupled to a second spatial region may be efficiently rendered (or represented or modelled) using spatially extending sound sources.

[0011] In other words, based on the incorporation of spatially extending sound sources into the rendering procedure, for example by calculating the sound impression caused by the spatially extending sound sources, a listening impression may be obtained in which the diffuse sound field occurring in a first spatial domain, e.g. a first room, is faithfully represented for a listener in a second spatial domain, e.g. a second room.

[0012] The inventors have recognised that the use of such spatially extending sound sources for rendering may make it possible to provide a faithful listening impression of the rendered audio scene, while limiting the adverse effects (e.g. in terms of increased data required or computational costs) on, for example, the transmission and processing (e.g. decoding and / or rendering) of data required for the provision of the audio scene.

[0013] According to a further embodiment of the invention, the renderer is configured to render direct sound sound effects of a given sound source located in the first spatial region in the second spatial region using direct sound rendering.

[0014] Further, the renderer is configured to render diffuse sound effects, e.g., diffuse sound effects, of a given sound source occurring in the first spatial domain in the second spatial domain using the spatially expanding sound source.

[0015] It should be noted that embodiments are not limited to rendering or representing diffuse and direct sound effects of the same sound source: a renderer according to embodiments may be configured to render an audio scene with multiple sound sources, some of which may provide diffuse sound and some of which may provide direct sound (or both, respectively) for each listener for whom the scene is rendered.

[0016] However, such multiple sound sources may also be modeled as a single source, with direct and diffuse sound acoustics, each of which may be an aggregated version of the acoustics of the multiple sound sources.

[0017] As an example, a sound source, such as a person speaking in a first room, may be heard by a listener in a second room, who may hear the speaker's voice as a direct sound acoustic effect and a second sound caused by late reverberation of the voice in the first room as a diffuse sound acoustic effect.

[0018] The inventors have realised that the use of distinct rendering techniques in the form of the use of direct sound rendering and the use of spatially extending sound sources makes it possible to produce a faithful listening impression.

[0019] According to a further embodiment of the present invention, the renderer is configured to apply direct source rendering, e.g. binaural rendering, which may take into account, e.g. direct propagation, occlusion, diffraction, etc., to the source signal of a given sound source located in the first spatial domain to obtain a rendered direct source response at a listener position located in the second spatial domain.

[0020] In addition, the renderer is configured to apply reverberation processing (e.g., late reverberation (effect), e.g., reverberation based on a combination of reflected signals that undergo multiple reflections, e.g., reverberation processing that generates reverberation after early reflections have disappeared among dense, statistically distributed reflections) to the source signal of the given sound source in order to obtain one or more reverberated versions of the source signal of the given sound source.

[0021] Further, the renderer is configured to apply the spatially extended source rendering to one or more reverberant versions of the source signal of a given source to obtain a rendered diffuse sound response at a listener position located within the second spatial region.

[0022] This may allow for off-loading or freeing up the bitstream since the renderer may be configured to simulate or model or represent a diffuse sound field and / or respectively a diffuse sound effect based on reverberation processing of the source signal of the sound source.

[0023] Thus, for a given sound source, only one sound source signal may need to be transmitted instead of two signals, of which, for example, a first signal would represent the direct sound signal of the source and a second signal would represent the diffuse sound signal of the source.

[0024] According to a further embodiment of the present invention, the renderer is configured to render sound effects, e.g. of reverberation, e.g. of late reverberation, excited by a sound source located in the first spatial domain (e.g. in the first acoustically homogeneous space, i.e. AHS, e.g. in the first room), in a second spatial domain (e.g. in the second acoustically homogeneous space, e.g. in the second room, e.g. in a spatial domain outside the first spatial domain) using a spatial extension source, e.g. SESS, as a spatial extension source reproducing late reverberation.

[0025] The inventors have recognized that the acoustic effects of late reverberation in acoustically coupled but separate locations can be faithfully and / or efficiently represented using space-extending sound sources.

[0026] According to a further embodiment of the present invention, the renderer is configured to render reverberant, e.g. late reverberant, diffuse sound effects using spatially extended sound sources (e.g. as spatially extended sound sources) with similar spectral content in each spatial region, e.g. SESS. Such spatially extended sound sources may thus be provided with low complexity and may, for example, better represent AHSs and / or portals between AHSs.

[0027] According to some embodiments of the present invention implementing one of the concepts described herein (sometimes also designated as "Method 2"), the renderer is configured to render diffuse sound effects using a spatially expanding sound source positioned at a portal between a first spatial region and a second spatial region and reproducing diffuse sound (or, for example, diffuse sound sound effects) originating from the first spatial region.

[0028] The acoustic coupling of rooms may be represented using portals. Such portals are geometric objects with spatial extent. In order to faithfully provide the effect of diffuse sound emanating from acoustically coupled rooms, the inventors have realized that it may be advantageous for the listener to have the impression of spatial sound sources at the interfaces of the coupled rooms.

[0029] Thus, the inventors have recognized that in some cases, locating a spatially expanding sound source at a portal between a first spatial region and a second spatial region can be used to provide such a faithful listening impression. In other words, a spatially expanding sound effect (e.g., as a representation of a diffuse sound effect) from, for example, originating in, an acoustically coupled first room to a listener in the second room may be provided.

[0030] It should further be noted that according to such inventive concept, the position of the portal within the scene may enable information about the intermediate, acoustically effective or acoustically influential, and therefore "unoccluded" interface areas of the spatial regions to be directly incorporated, and even may be such information alone, such that additional consideration of occlusion effects, e.g., of geometric boundaries such as walls of the respective spatial regions, may be excluded by the renderer.

[0031] However, the renderer may alternatively, or additionally, take into account occlusion effects, for example based on objects in the listener's room.

[0032] According to a further embodiment of the present invention implementing one of the concepts described herein (sometimes also designated as "Method 1"), the renderer is configured to render diffuse sound sound effects using a spatially expanding sound source that takes on the geometric extent, e.g., size and / or shape, of the first spatial region (e.g., taking the same shape but with the same spatial extension as the first spatial region, e.g., a contracted or downscaled version of the first spatial region, e.g., to avoid overlapping boundaries) and reproduces diffuse sound originating from the first spatial region, taking into account occlusion of the spatially expanding sound source at a listener position located in the second spatial region (e.g., by a wall between the first and second spatial regions, or by any other material that is acoustically damping or acoustically impermeable).

[0033] The inventors have realised that by setting the geometric extent of the spatially expanding sound source to the geometric extent of the first spatial region, a good trade-off between complexity and quality of the acoustic representation of the diffuse sound effect can be achieved.

[0034] As indicated above, an advantage of this approach may be that, for example, the geometric extent of a spatially expanding sound source reproducing diffuse sound originating from a first spatial region may be set simply to the geometric extent of the first spatial region, regardless of whether the listener is, for example, in a second, third or fourth spatial region.

[0035] Therefore, it may not be necessary to identify the location of the portal and therefore to place the spatially expanding sound source at the portal based on the listener position and the corresponding interface area between the listener's spatial region and the first spatial region from which the diffuse sound originates.

[0036] However, to incorporate occlusion effects, the renderer is configured to take into account the occlusion of spatially extending sound sources at a listener position that is located within the second spatial region.

[0037] As an example, this may allow for lightening the burden on the bitstream, since portal placement information may not need to be provided to the renderer, and the renderer may, for example, take into account occlusions between the listener's position and the spatially expanding sound source at its end. Furthermore, the corresponding encoding procedure may be simplified.

[0038] For example, using this technique, the space (or room) itself is the portal, and its overall radiated volume is "clipped" by occlusion / shadowing calculations in the virtual reality system (or in the renderer).

[0039] According to a further embodiment of the invention, the first spatial region is a first acoustically homogeneous space, e.g. a space or region having the same late reverberation characteristics, e.g. a late reverberation characteristic. Alternatively or additionally, the second spatial region is a second acoustically homogeneous space, e.g. a space or region having the same late reverberation characteristics.

[0040] For example, the inventors have recognised that the concepts of the present invention may be applied to particular advantage to acoustically homogeneous spaces, with respect to the ability of the embodiments to provide a listening impression that is faithful to a diffuse sound field emanating from and / or provided in an acoustically homogeneous space.

[0041] According to a further embodiment of the invention, the first and second spatial regions are rooms, e.g. physically adjacent or physically separate rooms, comprising a telepresence structure as a portal, and such rooms are acoustically coupled via the portal, e.g. via a door and / or via one or more walls that are at least partially permeable to sound, or via the telepresence structure.

[0042] This may make it possible to provide an immersive listening experience.

[0043] According to a further embodiment of the present invention, the renderer is configured to render multiple spatially expanding sound sources using the same rendering algorithm, taking into account occlusions between the listener position and the one or more spatially expanding sources that are far from the listener position and may for example take up the complete space (or a constricted part) of a respective acoustically homogeneous space or room, and one or more spatially expanding sources inside which the listener position is located and may for example take up the complete space (or a constricted part) of a respective homogeneous space or room.

[0044] Generally, it should be noted that according to an embodiment, a spatially expanding sound source or portal (e.g., a spatially expanding sound source representing a portal) may be obtained, for example, by slightly shrinking the geometry of a corresponding spatial space, for example, to avoid overlap between the geometry of the spatially expanding sound source or portal and, for example, potentially occluding boundaries of a spatial region.

[0045] According to a further embodiment of the invention, the renderer is configured to perform binaural rendering, which may enable faithful delivery of the listening experience to a headphone user.

[0046] According to a further embodiment of the present invention, the renderer is configured to determine (e.g. taking into account occlusion and / or attenuation, e.g. using a ray tracing based technique) in which spatial region, e.g. in which horizontal / vertical or azimuth / altitude region as seen from the listener's point of view, the spatially augmented sound source for diffuse sound reproduction is located relative to the listener's position and / or listener's orientation, and render the spatially augmented sound source accordingly.

[0047] This may allow a listener to be provided with a precise spatial listening experience. Furthermore, the influence of additional acoustically relevant scene objects and / or properties between the listener and the spatially extending sound sources may be taken into account.

[0048] According to a further embodiment of the present invention, the renderer is configured to determine, e.g. using a ray tracing based approach, taking into account e.g. occlusion and / or attenuation, in which spatial regions, e.g. horizontal / vertical or azimuth / altitude regions, as seen from the listener's point of view, relative to the listener's position and / or listener's orientation, and to render the spatially expanding sound sources accordingly.

[0049] Thus, for the rendering of audio scenes, occlusion effects may be accurately incorporated.

[0050] According to a further embodiment of the invention, the renderer is configured to determine using a ray tracing based technique in which spatial region, e.g. in which horizontal / vertical or azimuth / altitude region as seen from the listener's point of view, the spatially augmented sound source for the reproduction of diffuse sound is located relative to the listener's position and / or listener's orientation.

[0051] According to a further embodiment of the invention, the renderer is configured to determine using a ray tracing based approach in which spatial regions, e.g. in which horizontal / vertical or azimuth / altitude regions as seen from the listener's point of view, a spatially augmented sound source for the reproduction of diffuse sound is occluded relative to the listener's position and / or listener's orientation.

[0052] The inventors have recognised that a ray tracing based approach may enable efficient determination of the location of spatially extending sound sources relative to a listener, as well as intermediate acoustically related objects (e.g., for further occlusion effects), and thus enable accurate rendering of the audio scene for the listener.

[0053] According to a further embodiment of the present invention, the renderer is configured to determine, for a plurality of areas (e.g. areas on a surface having a predefined relationship with the listener's position or areas on a hull surrounding the listener's position), for example taking occlusion into account, whether a light ray associated with the respective area and extending, for example through the respective area, away from the listener's position, for example outwards, or starting on the respective area, hits a spatially extending sound source (whose geometry may for example be determined by mapping a geometrical definition in coordinates relative to the auditory scene (or relative to the coordinate system origin of the auditory scene) to coordinates relative to the listener), thereby determining in which spatial region, for example in which horizontal / vertical region or in which azimuth / altitude region, as seen, for example from the listener's point of view, the spatially extending sound source for the reproduction of diffuse sound is located relative to the listener's position and / or orientation.

[0054] As an example, rays may be used to help render Spatially Extended Sound Sources (SESSs). For example, in a virtual 3D scene, which may optionally comprise only the meshes of all SESSs (e.g., including portals), a predefined number of rays may be shot in all directions. This may be done in each update cycle, assuming that any relevant scene objects or listener positions have changed. For each extended source / portal, ray hits may be stored. This information is then used in later stages dealing with occlusions and / or homogeneous ranges.

[0055] Within an update cycle, several primary rays may be shot in all directions and measured relative to the listener's orientation. A list of ray directions may be stored in a list in the source code. All ray hits caused by ray intersections with source range geometries (including portal geometries or spatially extended source geometries) may be stored. However, there may be differences between rays that hit, for example, outside or inside a range geometry. If one ray hits the same range geometry multiple times, optionally only the nearest hit may be taken into account, for example.

[0056] For each primary ray, several additional rays may be emitted, for example in a pattern, for example in a circular pattern. These secondary rays may start at the same point as the primary ray and may pass through several points, for example equally distributed on a circle of a given radius on a plane, perpendicular to the direction of the primary ray, for example at a given distance from the listener.

[0057] The primary ray and all additional rays may be given equal weighting. For each ray that hits the source range geometry, its weight may be added to the total weight associated with that primary ray's ID.

[0058] All rays with non-zero weights may be stored in an item, such as a rendering item, RI, or encoder item, for consumption by a later stage.

[0059] In a second loop over all RIs, additional refined rays may be cast, for example, for range geometries that are hit by fewer rays than defined by the threshold, and for each primary ray that hits that geometry, several secondary rays may be cast in a pattern, for example, a circular pattern.

[0060] All primary and secondary rays may, for example, be given equal weight. For each ray that hits the source range geometry, its weight may be added to the total weight associated with that primary ray's ID. In the record associated with the primary ray's ID, for each ray, a bit may be set to 1 if the corresponding ray hits that geometry, and 0 otherwise.

[0061] According to a further embodiment of the present invention, the renderer is configured to determine one or more auditory cue information items (e.g. inter-channel correlation values, and / or inter-channel phase difference values, and / or inter-channel time difference values, and / or inter-channel level difference values, and / or one or more gain values) depending on the spatial region in which the spatially extended sound source for the reproduction of diffuse sound is located, for example using a look-up table that maps different spatial regions (e.g. spatial regions at different positions relative to the user and / or spatial regions of different extension) to values ​​of one or more cue information items.

[0062] Further, the renderer is configured to process the one or more audio signals representing the diffuse sound using one or more auditory cue information items to obtain a rendered version of the diffuse sound, e.g., rendered for a listener at a listening position.

[0063] The inventors have realised that based on the determination and processing of auditory cue information items the listening impression of a rendered version of diffuse sound can be improved.

[0064] According to a further embodiment of the invention, the renderer is configured to update in response to a movement of the listener, e.g. in response to a change in the listener's position and / or in response to a change in the listener's looking direction, the determination of in which spatial region, e.g. in which horizontal / vertical or azimuth / altitude region, the spatially expanding sound source for the reproduction of diffuse sound is located relative to the listener's position and / or listener's orientation, e.g. as seen from the listener's point of view.

[0065] Alternatively or additionally, the renderer is configured to update its determination of one or more auditory cue information items in response to movement of the listener, e.g., in response to a change in the listener's position and / or in response to a change in the direction the listener is looking.

[0066] Further, alternatively or additionally, the renderer is configured to update the determination of one or more cue information items in response to changes in the spatial region within which the spatially enhancing sound source for the reproduction of diffuse sound is located.

[0067] In general, a renderer according to an embodiment may be configured to take into account changes in the relative positions of, for example, listeners, spatial regions, portals, and / or spatially extending sources for rendering of the respective audio scene.

[0068] For example, the inventors have recognized that the inventive concept of using portals and spatially expanding sound sources, with the portal and / or sound source locations having spatial extents (or contracted versions) of corresponding spatial regions, may enable efficient incorporation of dynamic changes in a scene, for example based on movement of a listener and / or changes in the spatial region within which the spatially expanding sound sources are located.

[0069] Thus, embodiments may enable real-time adaptation of dynamic audio scenes.

[0070] Furthermore, the inventors have recognised that for such adaptation, in order to efficiently represent the respective changes in the audio scene, not only can direct, latent updates, e.g. a determination of in which spatial region a spatially extending sound source is located, be performed, but alternatively or additionally, the determination of the auditory cue information items can also be updated.

[0071] A further embodiment according to the present invention comprises an audio decoder comprising a renderer according to any of the embodiments as disclosed herein, the audio decoder being configured to obtain a geometric description of a portal, e.g. of one or more spatially extending sound sources for diffuse sound reproduction, from the bitstream and to map the geometry of the portal onto a listener-centric coordinate system in order to obtain a geometric description of a spatially extending sound source for diffuse sound reproduction.

[0072] Therefore, in general, it should be noted that according to an embodiment, a portal may be one or more spatially extended sound sources or may have the functionality thereof. Thus, the geometric description of a portal may be used as or for the geometric description of a spatially extended sound source. According to some embodiments of the present invention, portal and SESS may be used interchangeably.

[0073] Furthermore, the inventors have recognized that if such geometric descriptions are provided in the bitstream, so that the corresponding renderer does not need to determine the geometric descriptions of each such portal, computational power at the renderer or decoder side can be saved.

[0074] Thus, the inventors have recognised that for efficient cooperation, for example, between an encoder providing the bitstream and a renderer, the mapping functionality described above may advantageously be present within a decoder.

[0075] Thus, and by way of example, the inventors have recognised that in order to efficiently render an audio scene for each listener, a renderer may represent the audio scene in a listener-centric coordinate system.

[0076] According to a further embodiment of the invention, the audio decoder is configured to obtain two or more signals which are at least partially decorrelated for rendering of a spatially extending sound source derived from an output of the late reverb generator.

[0077] The inventors have recognized that a spatially extending sound source may be efficiently rendered using or based on two or more signals that are at least partially decorrelated. Optionally, both signals may have the same power spectral density.

[0078] According to a further embodiment of the present invention, the audio decoder is configured to obtain two or more signals for rendering a spatially extended sound source using a feedback delay network reverberator, the two or more signals may, for example, serve as signals representing diffuse sound.

[0079] The inventors have recognised that a feedback delay network reverberator may provide an efficient means for providing at least partially decorrelated signals. Optionally, both signals may have the same power spectral density.

[0080] According to a further embodiment of the invention, the decoder is configured to use the source signal and a de-correlated version of the source signal which may be derived from the source signal using a de-correlator which may be part of the audio decoder, e.g. for rendering spatially extended sound sources, and the source signal and the de-correlated source signal may serve as signals representing, e.g., diffuse sound.

[0081] The inventors have realized that a single signal can be processed to provide two signals that are at least partially and / or approximately decorrelated for rendering of spatially expanding sound sources. Therefore, less input signals may be required. Optionally, both signals may have the same power spectral density.

[0082] According to a further embodiment of the invention, the decoder is configured to exclude or attenuate occluded spatial regions when rendering spatially extending sound sources, e.g. using equalization or attenuation depending on the absorption properties of the occluder.

[0083] In general, and by way of example, a decoder according to an embodiment may comprise a pre-processing unit for the renderer, which may be configured to provide a decorrelated signal for rendering spatially augmented sound sources in order to equalize or attenuate their acoustic impact, and / or may be configured to perform spatial pre-processing, e.g. comprising determining the relative locations of acoustically related objects.

[0084] According to a further embodiment of the present invention, the decoder is configured to enable smooth transitions in, out and between multiple spatial regions, e.g. between multiple acoustically homogeneous spaces, e.g. by fading out spatially augmented sound sources representing diffuse sound and fading in non-localized renderings of diffuse sound when the listener is approaching a transition between a first and a second spatial region, e.g. a portal.

[0085] This may make it possible to provide a faithful listening impression to the listener.

[0086] In the following, embodiments related to an encoder are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments related to a decoder. Thus, the following embodiments may have the same, similar or corresponding features, functionality and details as the above-disclosed embodiments, both individually and in combination.

[0087] A further embodiment according to the invention comprises an audio encoder for encoding an audio scene, the audio encoder being configured to provide an encoded representation of one or more audio signals, e.g. for encoding the one or more audio signals as part of the encoded representation of the audio scene.

[0088] Further, the audio encoder may be configured to identify a plurality of acoustically homogeneous spaces and, based thereon, provide a definition, e.g., a geometric description of a spatially augmented sound source, the geometric properties, e.g., position and / or dimensions, of the spatially augmented sound source being identical to the geometric properties, e.g., position and / or dimensions, of the identified acoustically homogeneous spaces, and the audio encoder may be configured, for example, to include a definition of the spatially augmented sound source in an encoded representation of the audio scene, e.g., in a bitstream.

[0089] It should be noted that only some geometrical characteristics may be identical, such as the position (e.g. as the center of the area) and / or the shape, while other characteristics may differ, e.g. the outer dimensions of the spatially extending sound source, which may be a scaled version of the identified acoustically homogeneous space.

[0090] According to a further embodiment of the present invention, the audio encoder is configured to provide a definition, e.g. a geometric description, of an acoustic obstacle, e.g. a wall or other occlusion, between an acoustically homogeneous space, and the audio encoder may be configured to include the definition of the acoustic obstacle in the encoded representation of the audio scene, e.g. in the bitstream.

[0091] Optionally, the audio encoder may be configured to selectively provide a definition of acoustic obstructions between acoustically homogeneous spaces.

[0092] For example, based on ray tracing, the renderer may efficiently select the acoustically relevant obstacles placed in order to provide a faithful listening impression to the listener.

[0093] According to a further embodiment of the present invention, the audio encoder is configured to provide an encoded representation of one or more audio signals, e.g. for encoding the one or more audio signals as part of an encoded representation of an audio scene.

[0094] Further, the audio encoder is configured to provide a definition, e.g., a geometric description, of one or more spatially extending sound sources, the geometric characteristics, e.g., location and / or orientation and / or dimensions of the spatially extending sound sources being based on, e.g., equal to, the geometric characteristics of, e.g., a portal (e.g., an opening, or a door, or an acoustically permeable material, or any medium allowing sound propagation between two spatial regions or between two acoustically homogeneous spaces), e.g., physically and / or logically, e.g., between adjacent, acoustically homogeneous spaces.

[0095] Optionally, the audio encoder may be configured to, for example, include a definition of the spatial extension sound sources in the encoded representation of the audio scene, for example in the bitstream.

[0096] According to a further embodiment of the present invention, the audio encoder is configured to identify a plurality of acoustically homogeneous spaces and one or more portals between the acoustically homogeneous spaces, e.g. by analysing geometric relations between the acoustically homogeneous spaces, and based thereon provide a definition, e.g. a geometric description of one or more spatially extending sound sources, wherein geometric properties, e.g. position and / or orientation and / or dimensions, of the one or more spatially extending sound sources are based on the dimensions of the identified portals.

[0097] Optionally, the audio encoder may be configured to, for example, include a definition of the spatial extension sound sources in the encoded representation of the audio scene, for example in the bitstream.

[0098] Optionally, the audio encoder may be configured, for example, to provide a definition, for example a geometric description of an acoustic obstacle, for example a wall or other occlusion, between an acoustically homogeneous space, and the audio encoder may be configured, for example, to include the definition of the acoustic obstacle in the encoded representation of the audio scene, for example in the bitstream.

[0099] In the following, embodiments relating to a method are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above embodiments relating to a decoder and / or an encoder. Thus, the following embodiments may have the same, similar or corresponding features, functionality and details as the above disclosed embodiments, both individually and in combination.

[0100] A further embodiment according to the invention comprises a method for rendering, e.g. spatially rendering, an acoustic scene, comprising rendering, e.g. reproducing, e.g. reverberant, e.g. late reverberant, diffuse sound sound effects occurring in a first spatial domain (e.g. in a first acoustic homogeneous space, i.e. AHS, e.g. in a first room) using a spatial extension source, e.g. SESS, as a spatial extension source reproducing diffuse sound, e.g. using a homogeneous extension source algorithm, in a second spatial domain (e.g. in a second acoustic homogeneous space, e.g. in a second room, e.g. in a spatial domain outside the first spatial domain).

[0101] A further embodiment according to the invention comprises a method for encoding an audio scene, the method comprising providing an encoded representation of one or more audio signals, e.g. for encoding the one or more audio signals as part of the encoded representation of the audio scene.

[0102] The method comprises identifying a plurality of acoustically homogeneous spaces and, based thereon, providing a definition, e.g., a geometric description of a spatially extending sound source, the geometric characteristics, e.g., position and / or dimensions, of the spatially extending sound source being identical to the geometric characteristics, e.g., position and / or dimensions, of the identified acoustically homogeneous spaces.

[0103] Optionally, the audio encoder may be configured to, for example, include a definition of the spatial extension sound sources in the encoded representation of the audio scene, for example in the bitstream.

[0104] A further embodiment according to the invention comprises a method for encoding an audio scene, the method comprising providing an encoded representation of one or more audio signals, e.g. for encoding the one or more audio signals as part of the encoded representation of the audio scene.

[0105] The method further comprises providing a definition, e.g., a geometric description, of one or more spatially-extending sound sources, the geometric characteristics, e.g., location and / or orientation and / or dimensions, of the spatially-extending sound sources being based on, e.g., equal to, the geometric characteristics of, e.g., a portal (e.g., an opening, or a door, or an acoustically permeable material, or any medium allowing sound propagation between two spatial regions or between two acoustically homogeneous spaces), e.g., physically and / or logically, e.g., between adjacent, acoustically homogeneous spaces.

[0106] Optionally, the audio encoder may be configured to, for example, include a definition of the spatial extension sound sources in the encoded representation of the audio scene, for example in the bitstream.

[0107] A further embodiment according to the invention comprises a computer program for performing the method according to any of the embodiments as disclosed herein, when the computer program runs on a computer.

[0108] In the following, embodiments related to bitstreams are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments related to decoders, encoders, and / or methods. Thus, the following embodiments may have the same, similar, or corresponding features, functionality, and details as the above-disclosed embodiments, both individually and in combination.

[0109] Further embodiments according to the invention comprise an audio bitstream comprising an encoded representation of one or more audio signals and an encoded representation of one or more spatially extending sound sources for rendering, e.g. for reproducing, e.g. reverberant, e.g. late reverberant, diffuse sound sound effects originating in a first spatial domain (e.g. in a first acoustically homogeneous space, i.e. AHS, e.g. in a first room) and rendered in a second spatial domain (e.g. in a second acoustically homogeneous space, e.g. in a second room, e.g. in a spatial domain outside the first spatial domain).

[0110] A further embodiment according to the invention comprises an audio bitstream comprising an encoded description of one or more spatial regions, e.g. a plurality of spatial regions, e.g. an acoustic description of one or more spatial regions and / or a geometric description of one or more spatial regions, and an encoded representation of information describing an acoustic relationship between at least two spatial regions, e.g. between at least two spatial regions described by the encoded descriptions.

[0111] Optionally, the bitstream may also comprise an encoded representation of, e.g., one or more audio signals or audio channels, e.g., representing audio sources located in one or more of the spatial domains.

[0112] The inventors have recognised that providing information describing the acoustic relationship between at least two spatial regions can improve the quality of a rendered acoustic scene comprising at least two spatial regions, since the incorporation of acoustic coupling effects between the spatial regions can be simplified for each renderer.

[0113] According to a further embodiment of the invention, the coded representation of the spatial domain comprises a description of a portal between the two spatial domains, e.g. a description of the size of the opening between the two spatial domains and / or a description of the attenuation rate of the opening or acoustic boundary between the two spatial domains.

[0114] Such portals for coupling of spatial regions may therefore be provided to the renderer via the bitstream, in this way computational power for the determination of such portals, for example to incorporate acoustic coupling effects between spatial regions, may be saved in the renderer.

[0115] According to a further embodiment of the present invention, the audio bitstream comprises a coded representation of propagation coefficients describing an acoustic propagation from a first spatial domain to a second acoustic domain.

[0116] The inventors have recognized that incorporating propagation coefficients into the bitstream may make it possible, for example, to provide information about acoustic coupling in the spatial domain with low transmission costs and evaluation effort while enabling faithful rendering of each acoustic scene.

[0117] According to a further embodiment of the present invention, the audio bitstream comprises propagation coefficients describing the amount / proportion of acoustic energy of a first spatial domain, e.g., Space #1, that is radiated into a second spatial domain, e.g., Space #2, and optionally vice versa.

[0118] According to a further embodiment of the present invention, the audio bitstream comprises a propagation coefficient describing the ratio between the connected surface area between the first space and the second space and the overall absorbing surface area of ​​the first space.

[0119] The inventors have realised that the definition of a transmission coefficient for acoustic energy and / or a ratio between connected surface areas may allow for an efficient expression of the acoustic coupling effect.

[0120] According to a further embodiment of the invention, the audio bitstream comprises parameters describing the range, eg extent, of a transition area between two spatial domains, eg between two acoustically homogeneous spaces.

[0121] This may provide information for the geometric extent of the portal or, respectively, the SESS, so that the rendering procedure may be simplified by providing such information already in the bitstream.

[0122] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings: [Brief description of the drawings]

[0123] [Figure 1] FIG. 2 is a schematic diagram of a renderer according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram of a renderer having additional optional features according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram of a decoder according to an embodiment of the present invention; [Figure 4] 2 is a schematic diagram of an encoder according to an embodiment of the present invention; [Diagram 5] FIG. 4 is a schematic diagram of an encoder according to a further embodiment of the present invention; [Figure 6] 1 is a schematic block diagram of a method for rendering an audio scene according to an embodiment of the present invention; [Figure 7] 1 is a schematic block diagram of a method for encoding an audio scene according to an embodiment of the present invention; [Figure 8] 1 is a schematic block diagram of a method for encoding an audio scene according to an embodiment of the present invention; [Figure 9] FIG. 2 is a schematic block diagram of a bitstream according to an embodiment of the present invention; [Figure 10] FIG. 2 is a schematic block diagram of a pipeline of the inventive method according to an embodiment of the present invention; [Figure 11] FIG. 1 is a schematic diagram of an example of a portal detection method 1 according to an embodiment of the present invention. [Figure 12] FIG. 2 is a schematic diagram of an example of a portal detection method 2 according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0124] One or more elements having equal or equivalent functionality are indicated in the following description by equal or equivalent reference numbers even if they appear in different figures.

[0125] In the following description, numerous details are set forth to provide a more complete description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention. In addition, features of different embodiments described later in this specification may be combined with each other unless otherwise indicated.

[0126] Fig. 1 shows a schematic diagram of a renderer according to an embodiment of the invention. Fig. 1 shows a renderer 100 for rendering, e.g. spatially rendering, an audio scene, comprising a rendering unit 110. The renderer 100 may thus provide a rendered, e.g. spatially rendered, audio scene 101.

[0127] The renderer 100 is configured to render, for example, a diffuse sound effect occurring in a first spatial domain in a second spatial domain using a spatial extension sound source using the rendering unit 110. Thus, the renderer 100 is provided with spatial extension sound source information 102.

[0128] Optionally, the spatially extended sound source information 102 may comprise, for example, a complete set of parameters defining the SESS, or, for example, only some parameters, e.g., geometric information that can be complemented or extended using processing results of a renderer or a corresponding decoder comprising a renderer (e.g., geometric portal information corresponding to the geometric SESS information, e.g., location information, e.g., sound level information).

[0129] As an optional feature, additional scene information 103 is shown, which may be information based on which rendered sound scene 101 is to be provided (taking into account diffuse sound effects) and thus comprises, for example, information about spectral values, or time domain audio information and / or metadata information about the sound scene to be rendered.

[0130] FIG. 2 shows a schematic diagram of a renderer with additional optional features according to an embodiment of the invention.

[0131] FIG. 2 shows a renderer 200 comprising a rendering unit 210 which, as optional features, comprises a direct sound rendering unit 212, a SESS rendering unit 214, and a rendering fusion unit 216.

[0132] As explained in the context of Fig. 1, the renderer 200 is configured to render diffuse sound effects occurring in a first spatial domain into a second spatial domain using spatially augmented sound sources using a rendering unit 210. The rendering unit 210 is thus configured to provide a rendered sound scene 201. As an optional feature, an optional rendering fusion unit is configured to provide the rendered sound scene 201.

[0133] Thus, as an optional feature, in order to render a diffuse sound effect, the SESS rendering unit 214 is provided with spatially extended sound source information 202 (e.g. according to its counterpart 102 in FIG. 1), which may comprise, for example, information about portals, e.g. according to method 1 or method 2 as described with respect to FIGs. 11 and 12, and / or absolute and / or relative position information regarding the listener. Optionally, the spatially extended sound source information 202 may comprise any information suitable for defining a spatially extended sound source to provide a rendered diffuse sound response.

[0134] As an optional feature, the direct sound rendering unit 212 is configured to render direct sound acoustics of a given sound source located in the first spatial domain in the second spatial domain using direct sound rendering. Furthermore, as another optional feature, the SESS rendering unit 214 is configured to render diffuse sound acoustics of the given sound source in the second spatial domain using spatially extended sound sources.

[0135] Thus, as an optional feature, a direct sound rendering unit 212 is provided with a source signal 203 of a given source to which direct source rendering is applied in order to obtain a rendered direct source response 213 at a listener position located in the second spatial region. As another optional feature, a SESS rendering unit 214 may also be provided with the signal 203.

[0136] As another optional feature and as illustrated in Fig. 2, the SESS rendering unit 214 is provided with one or more reverberant versions 221 of a source signal of a given sound source. Furthermore, the SESS rendering unit 214 is configured to apply spatially extended source rendering to the one or more reverberant versions 221 of the source signal of the given sound source to obtain a rendered diffuse sound response 215 at a listener position located in the second spatial region.

[0137] For providing one or more reverberated versions 221 of the source signal, the renderer comprises, as an optional feature, a reverberation processing unit 220 configured to provide one or more reverberated versions 221 of the source signal 221 based on the source signal 203.

[0138] In other words, the reverberation processing unit 220 is configured to apply reverberation processing to the source signal 203 of a given source to obtain one or more reverberated versions 221 of the source signal of the given source.

[0139] As an optional feature, the rendering fusion unit is configured to fuse the rendered direct sound response 213 and the rendered diffuse sound response 215 to obtain the rendered acoustic scene 201 .

[0140] Thus, as an example, based on the source signal 203, the renderer may be configured to determine a diffuse version, in the form of a reverberant version of the source signal, based on which diffuse sound response can be efficiently and faithfully provided to a listener.

[0141] As another optional feature, the SESS rendering unit 214 is configured to render in the second spatial domain a sound effect of late reverberation excited by a sound source located in the first spatial domain using a space-expanding sound source that reproduces the late reverberation.

[0142] In other words, and as an example, based on the spatially extended sound source information 202, the SESS rendering unit 214 may render the spatially extended sound source to represent the effect of late reverberation of the sound source.

[0143] As another optional feature, the spatially extended sound sources, for example as defined by the spatially extended sound source information 202, may have similar spectral content in each spatial region. As an example, the inventors have recognized that a SESS having a uniformly distributed spatial frequency distribution may be used to efficiently represent the diffuse sound field effect.

[0144] As another optional feature, based on information about the portal, for example included in the spatial extension sound source information 202, the SESS rendering unit 214 is configured to render a diffuse sound sound effect using a spatial extension sound source positioned at the portal between the first spatial region and the second spatial region and reproducing diffuse sound originating from the first spatial region.

[0145] As another optional feature, the renderer 200 is configured to render diffuse sound effects using a spatially extending sound source, e.g., using the SESS rendering unit 214, that takes into account the geometric extent of the first spatial region and reproduces diffuse sound emanating from the first spatial region, taking into account occlusion of the spatially extending sound source at a listener position located in the second spatial region.

[0146] Thus, as an optional example, additional scene information 204 comprising, for example, spatial acoustic information, for example information about walls, openings, doors, materials, may be provided to the SESS rendering unit 214 and optionally to the direct sound rendering unit 212.

[0147] Based on such information, the SESS rendering unit 214 may be configured to determine occlusion effects in order to faithfully render the audio scene.

[0148] As another optional feature, the renderer 200 is configured to determine in which spatial regions relative to the listener's position and / or listener's orientation the spatial extension sources for diffuse sound reproduction are located and / or occluded, and render the spatial extension sources accordingly.

[0149] Thus, the renderer 200 comprises a spatial domain determination unit 230 which is provided with the spatially extended sound source information 202 and optionally additional scene information 204 and is configured to provide spatial domain information 231 identifying the relative location of the listener and the spatially extended sound source, e.g., orientation and altitude, e.g., φ, θ, with respect to the listener and / or a listener-centered coordinate system.

[0150] Accordingly, the information 231 is provided as an optional feature to the SESS rendering unit 214 for their evaluation, as well as for incorporating information about relative position and / or occlusion into the rendering procedure.

[0151] As another optional feature, the renderer 200 is configured to determine the spatial domain information 231 using a ray tracing-based approach. Thus, as an optional feature, the renderer 200 comprises a ray tracing unit 240. As optionally shown, the ray tracing unit 240 may be provided with the spatially extended sound source information 202 and optional additional scene information 204. Based thereon, the ray hit information 241 may be determined and provided to the spatial domain determination unit 230. The ray tracing unit may be configured to determine a two-dimensional approximation of acoustically significant objects and / or features from the listener's viewpoint based on a simulation of a number of light rays in a three-dimensional sound scene (e.g., the scene to be rendered). Thus, based on information about light rays hitting modeled entities such as spatially extended sound sources and / or objects, information about the relative position between the listener and the spatially extended sound field and / or about occlusion effects to be taken into account (e.g., based on occluding objects hit by light rays) may be obtained.

[0152] As another optional feature, the renderer is configured to determine, for example using the ray tracing unit 240, for a number of areas whether a ray associated with each area and extending away from the listener's position hits a spatially extending sound source, thereby determining in which spatial region relative to the listener's position and / or listener's orientation the spatially extending sound source for the reproduction of diffuse sound is located.

[0153] As another optional feature, the SESS rendering unit 214 comprises an auditory cue information unit 216 .

[0154] Thus, optionally, the renderer is configured to determine, e.g., using the auditory cue information unit 216, one or more auditory cue information items depending on a spatial region within which a spatial extension sound source for the reproduction of the diffuse sound is located, e.g., using the SESS rendering unit 214, to process the one or more audio signals representing the diffuse sound, e.g., using the one or more auditory cue information items, to obtain a rendered version of the diffuse sound, e.g. in the form of a rendered diffuse sound response.

[0155] The auditory cue information item may comprise, for example, information on at least one of Inter-Channel Coherence (ICC), Inter-Channel Phase Difference (ICPD), and / or Inter-Channel Level Difference (ICLD). Such information entities may, for example, enable the binaural rendering to be adapted to provide a faithful listening experience to the listener.

[0156] As another optional feature, the renderer 200 is configured to update the determination of which spatial region relative to the listener's position and / or listener's orientation the spatially augmented sound source for the reproduction of diffuse sound is located in in response to listener movement.

[0157] Alternatively or additionally, the renderer 200 is configured to update the determination of one or more auditory cue information items in response to a movement of the listener.

[0158] Alternatively or additionally, the renderer is configured to update the determination of one or more items of cue information in response to changes in the spatial region within which the spatially enhancing sound source for the reproduction of diffuse sound is located.

[0159] Therefore, as an optional feature, the spatial region determination unit 230, the ray tracing unit 240 and the auditory cue information unit 216 are provided with optional listener movement information 205 (e.g. comprising listener position information) that may trigger such updates.

[0160] In the following, further examples for the embodiment according to Fig. 2 are described in simple terms. As an example, a source signal 203 comprising spectral values ​​and / or time domain samples of the audio signal of a source to be rendered may be provided to the renderer 200. A listener for whom the source is to be represented may be located in a different spatial region than the source, e.g. in a room. For a faithful representation of the listening impression, the renderer 200 therefore comprises a direct sound rendering unit 212 and a SESS rendering unit 214, the first taking into account the direct sound response and the latter taking into account the diffuse sound effects from the source to the listener. The inventors have realised that diffuse sound effects, such as those caused by vibrating side walls of the listener's room, for example, halfway between the listener's room and the source's room, may be efficiently represented using SESS. Optionally, the diffuse sound impression of the sound signal may be approximated based on reverberation processing. Furthermore, such a SESS may be advantageously placed, for example, at the position of the vibrating side walls between the rooms, relative to the listener's position. Thus, information about spatial characteristics of the audio scene to be rendered may be provided to the renderer, for example as additional scene information 204. Based thereon, and based on, for example, geometric and / or positional information of the SESS contained in the SESS information 202 and / or listener information 205 (e.g. comprising the listener's position), spatial domain information may be determined, for example using ray tracing techniques. Based on such information, the renderer may accurately "place" the listener, the SESS (e.g. representing a vibrating sidewall), and / or further obscuring or attenuating objects in a correct constellation and based thereon may realistically render the scene for the listener.

[0161] Fig. 3 shows a schematic diagram of a decoder according to an embodiment of the invention. Fig. 3 shows a decoder 300 comprising a renderer 310, for example according to the renderer 200 from Fig. 2 or the renderer 100 from Fig. 1, or according to any renderer configuration as disclosed herein. The renderer 310 is thus arranged to provide a rendered audio scene 301.

[0162] The decoder 300 is configured to obtain the portal geometry description 321 from the bitstream 302 and map the portal geometry onto a listener-centric coordinate system to obtain a spatially augmented sound source geometry description 331 for diffuse sound reproduction.

[0163] Thus, as an optional feature, the decoder 300 comprises an information extraction unit 320 configured to extract a geometric description of the portal from the bitstream 302. As a further optional feature, listener movement information 322, additional scene information 323 and / or sound source signals 324 may additionally be extracted from the bitstream 302. As optionally indicated, these information entities may be provided to the renderer 310 and may be processed, for example as described in the context of FIG.

[0164] For mapping into a listener-centric coordinate system, the decoder 300 comprises, as an optional feature, a mapping unit 330 arranged to provide a geometric description 331 of the spatially extended sound sources to a SESS information providing unit 340 .

[0165] The SESS information providing unit 340 is configured to provide spatially extended sound source information 341 to the renderer 310. The spatially extended sound source information 341 may, for example, comprise geometry information (e.g., for SESS and / or audio signal information, e.g., for a representation of an audio signal).

[0166] As another optional feature, the audio decoder is configured to obtain two or more signals 351, which are at least partially decorrelated for rendering of the spatial extension sound source derived from the output of the late reverberation generator. Thus, as an optional feature, the audio decoder 300 comprises a late reverberation generator 350. As shown, the two or more signals may be provided from the late reverberation generator 350 to the SESS information providing unit 340 and included in the spatial extension sound source information 341.

[0167] As another optional feature, the audio decoder 300 is configured to obtain two or more signals 361 for rendering the spatially extended sound source using a feedback delay network reverberator, i.e., FDNR. Thus, as an optional feature, the decoder 300 comprises an FDNR 360. As shown, the two or more signals may be provided from the FDNR 360 to the SESS information providing unit 340 and included in the spatially extended sound source information 341.

[0168] As another optional feature, the decoder 300 is configured to use the source signal and a decorrelated version of the source signal for rendering the spatially extended sound source. The decoder 300 therefore comprises, as an optional feature, a decorrelator to which the source signal 324 is provided. As shown, two signals 371 may be provided from the decorrelator 370 to the SESS information providing unit 340 and may be included in the spatially extended sound source information 341.

[0169] It should be noted that the three approaches, eg, using a late reverberator 350, an FDNR 360, and / or a decorrelator 370, may be used, for example, as alternatives.

[0170] Based on these signals and, for example, optionally based on auditory cue information items, SESS information may be obtained, for example in a SESS information supply unit 340. Such auditory cue information items may, for example, be included in additional scene information 323, which may be provided to the unit 340.

[0171] As another optional feature, the decoder 300 is configured to exclude or attenuate occluded spatial regions when rendering spatially extended sound sources. As an optional feature, the SESS information providing unit 340 is thus provided with additional scene information 323, which may comprise spatial acoustic scene information, such that the SESS information providing unit may be configured to provide information in the spatially extended sound source information 341 for excluding or attenuating occluded spatial regions.

[0172] Thus, the decoder 300 may be configured to allow smooth transitions in, out, and between multiple spatial regions.

[0173] Fig. 4 shows a schematic diagram of an encoder according to an embodiment of the invention. Fig. 4 shows an encoder 400 for encoding an audio scene, the audio encoder being arranged to provide a coded representation of one or more audio signals.

[0174] Thus, as an optional feature, the encoder 400 comprises a bitstream providing unit 410 arranged to provide a bitstream 401 comprising an encoded representation of one or more audio signals 403 .

[0175] Furthermore, the audio encoder 400 is configured to identify a number of acoustically homogeneous spaces, i.e., AHSs, and based thereon provide a definition of a spatially extending sound source, the geometrical characteristics of which are identical to the geometrical characteristics of the identified acoustically homogeneous spaces.

[0176] Thus, as an optional feature, the encoder 400 comprises an AHS identification unit 420 to which the (eg additional) acoustic scene information 402 is provided, and an optional SESS definition providing unit 430 to which the AHS information is provided from unit 420 .

[0177] Based thereon, as an optional feature, a SESS definition providing unit 430 is configured to provide a SESS definition 431 to a bitstream providing unit in order to provide said definition in the bitstream.

[0178] The SESS definition 431 may comprise geometric information about the SESS to be used for rendering.

[0179] As another optional feature, the audio encoder 400 is configured to provide definitions 442 of acoustic obstacles between acoustic homogeneous spaces. Thus, as an optional feature, the encoder 400 comprises an acoustic obstacle definitions providing unit 440, which is optionally provided with the acoustic scene information 402 and provides acoustic obstacle definitions 442 to a bitstream providing unit 410, which may optionally incorporate said information in the bitstream 401.

[0180] Fig. 5 shows a schematic diagram of an encoder according to a further embodiment of the invention. Fig. 5 shows an encoder 500 for encoding an audio scene, the audio encoder being arranged to provide a coded representation of one or more audio signals.

[0181] Thus, as an optional feature, the encoder 500 comprises a bitstream providing unit 510 arranged to provide a bitstream 501 comprising an encoded representation of one or more audio signals 503 .

[0182] Furthermore, the encoder 500 is configured to provide one or more spatially extending sound source definitions 531, the geometrical properties of which are based on the geometrical properties of portals between the acoustically homogeneous spaces.

[0183] Therefore, as an optional feature, the encoder 500 comprises an AHS and portal identification unit 520 to which the optional additional acoustic scene information 502 is optionally provided. The AHS and portal identification unit 520 is optionally configured to identify the AHS and to provide portal information 521 for identifying portals between the AHSs. The portal information 521 comprises information about geometrical properties of portals between acoustic homogeneous spaces.

[0184] Furthermore, as an optional feature, and as previously explained, the encoder 500 comprises a SESS definitions providing unit 530 to which portal information is provided in order to provide definitions 531. As optionally shown, these definitions 531 may be provided to a bitstream providing unit 510 in order to be incorporated into the bitstream 501.

[0185] Thus, in other words, optionally, the audio encoder 500 is configured to identify a plurality of acoustically homogeneous spaces and one or more portals between the acoustically homogeneous spaces, and based thereon, provide a definition of one or more spatially extending sound sources, the geometric characteristics of which are based on the dimensions of the identified portals.

[0186] 6 shows a schematic block diagram of a method for rendering an audio scene according to an embodiment of the present invention. The method 600 comprises rendering (610) a diffuse sound effect occurring in a first spatial domain in a second spatial domain using a spatially expanding sound source.

[0187] 7 shows a schematic block diagram of a method for encoding an audio scene according to an embodiment of the present invention. The method 700 comprises providing an encoded representation of one or more audio signals (710), identifying a plurality of acoustically homogeneous spaces (720), and providing a definition of a spatially extending sound source based thereon (730), the geometrical characteristics of the spatially extending sound source being identical to the geometrical characteristics of the identified acoustically homogeneous spaces.

[0188] 8 shows a schematic block diagram of a method for encoding an audio scene according to an embodiment of the present invention. The method 800 comprises providing an encoded representation of one or more audio signals (810) and providing a definition of one or more spatially extending sound sources (820), the geometrical properties of which are based on the geometrical properties of portals between acoustically homogeneous spaces.

[0189] Fig. 9 shows a schematic block diagram of a bitstream according to an embodiment of the present invention. The bitstream 900 comprises an encoded representation 910 of one or more audio signals and an encoded representation 920 of one or more spatially enhancing sound sources for rendering a diffuse sound effect originating in a first spatial domain and rendered in a second spatial domain.

[0190] As an optional feature, the bitstream 900 comprises a coded description 930 of one or more spatial regions and a coded representation 940 of information describing an acoustic relationship between at least two spatial regions.

[0191] Optionally, the encoded representation may additionally comprise an encoded representation of one or more audio signals or audio channels representing audio sources located in one or more of the spatial regions.

[0192] Optionally, the encoded representation of the spatial regions comprises a description of a portal between the two spatial regions.

[0193] As another optional feature, the audio bitstream 900 comprises a coded representation 950 of propagation coefficients describing acoustic propagation from a first spatial domain to a second acoustic domain.

[0194] Optionally, the propagation coefficient may describe the amount / proportion of acoustic energy of a first spatial region that is radiated into a second spatial region and / or the ratio between the connected surface area between the first and second spaces and the overall absorbing surface area of ​​the first space.

[0195] As another optional feature, the audio bitstream 900 comprises a parameter 960 that describes the range of the transition area between the two spatial domains.

[0196] Notes: Various embodiments and aspects of the present invention are described below or further described, for example, in the "Overview" section, in the "Objectives of the Invention" section, in the "Description of the Invention" section, and in the "Aspects of the Invention" section.

[0197] Further embodiments are also defined by the enclosed claims.

[0198] It should be noted that any embodiment as defined by the claims or the above description may be optionally supplemented by any of the details (features and functionality) described in the preceding sections.

[0199] Also, the embodiments described in the above sections can be used individually and can be augmented by any of the features in another section, by any of the features in another section of the above description, and / or by any feature included in the claims.

[0200] It should also be noted that each aspect described herein may be used individually or in combination, and thus details may be added to each of the individual aspects without adding details to another of the aspects.

[0201] Moreover, features and functionality disclosed herein relating to a method may also be used in an apparatus (configured to perform such functionality). Moreover, any features and functionality disclosed herein with respect to an apparatus may also be used in the corresponding method. In other words, the methods disclosed herein may optionally be augmented by any of the features and functionality and details described with respect to the apparatus.

[0202] Also, as described in the "Implementation Alternatives" section, any of the features and functionality described herein may be implemented in hardware or software, or using a combination of hardware and software.

[0203] Moreover, it should be noted that the audio bitstream (or, in a similar sense, the encoded audio representation) may be optionally augmented with any of the features, functionality, and details disclosed in this specification, both individually and in combination.

[0204] Implementation Alternatives: Generally, some aspects are described in the context of an apparatus, but it will be apparent that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0205] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. Implementations may be performed using a digital storage medium having electronically readable control signals stored thereon, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, which cooperates (or can cooperate) with a programmable computer system such that the respective methods are executed. Thus, the digital storage medium may be computer readable.

[0206] Some embodiments according to the invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system such that one of the methods described herein is performed.

[0207] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0208] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0209] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0210] A further embodiment of the inventive method is therefore a data carrier (or a digital storage medium or a computer readable medium) comprising a computer program recorded thereon for performing one of the methods described herein. The data carrier, digital storage medium or recorded medium is typically tangible and / or non-transitory.

[0211] A further embodiment of the inventive method is therefore a data stream, or a sequence of signals, representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be arranged to be transferred via a data communication connection, for example via the Internet.

[0212] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0213] A further embodiment comprises a computer having the computer program installed thereon for performing one of the methods described herein.

[0214] Further embodiments according to the invention comprise an apparatus or a system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may comprise, for example, a file server for transferring the computer program to the receiver.

[0215] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0216] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0217] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least in part in hardware and / or software.

[0218] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0219] Any of the methods described herein, or components of the apparatus described herein, may be at least partially implemented by hardware and / or software.

[0220] The above-described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the configurations and details described herein will be apparent to others skilled in the art. It is therefore intended that the same be limited only by the scope of the forthcoming patent claims, and not by the specific details presented throughout the description and explanation of the embodiments herein.

[0221] Below, a section is provided that provides an "Overview" of embodiments.

[0222] According to one aspect, a computationally efficient approach is described for segmenting huge and complex sound scenes and rendering several realistic diffuse sound fields based on their topological relationships, for example, by modeling acoustic spaces with similar diffuse sound characteristics as homogeneous extended sound sources and then simply simulating their sound propagation in real time, for example, depending on the range of the homogeneous extended sound sources and their distance from the listener.

[0223] Previous work has proposed ideas like pre-rendered geometry-based methods (which cannot handle source movement in real time and are computationally heavy) or reverberation graph approaches (accumulated impulse responses just as point sources).

[0224] According to one aspect, the proposal (eg, the present proposal) leverages existing homogeneous extended source algorithms to achieve both efficiency and quality.

[0225] The following sections may provide context for a better understanding of embodiments in accordance with the present invention.

[0226] In the following, some background information is provided, but it should be noted that any of the features, functionality and details disclosed herein may be optionally used both individually and in combination in embodiments according to the present invention. Moreover, reference is also made to PCT / EP2021 / 050588, which describes an apparatus and method for reproducing a spatially extended sound source or an apparatus and method for generating a description for a spatially extended sound source using anchoring information.

[0227] According to one aspect, the invention relates to audio signal processing, in particular to the encoding or decoding, i.e. reproduction, of diffuse sounds in an audio scene as spatially extended sound sources (SESS).

[0228] The reproduction of sound sources over several loudspeakers or headphones has long been studied. The simplest way to reproduce sound sources over such a setup is to render them as point sources, i.e. extremely (ideally infinitely) small. However, it has been found that this theoretical concept is hardly capable of modeling existing physical sound sources in a realistic way. For example, a grand piano has a large vibrating wooden lid with many spatially distributed strings inside, and therefore appears much larger to the ear's perception than a point source (especially when the listener (and microphone) is close to the grand piano). It is recognized that many sound sources in the real world have a considerable size ("spatial extent"), such as musical instruments, machines, orchestras or choirs, or ambient sounds (such as the sound of a waterfall).

[0229] Accurate / realistic reproduction of such sound sources, even binaurally (i.e. using the so-called Head-Related Transfer Function HRTF or Binaural Room Impulse Response BRIR), using headphones or traditionally using loudspeaker setups ranging from, for example, two speakers arranged in a horizontal plane ("stereo") to many speakers ("surround sound") and many speakers surrounding the listener in all three dimensions ("3D audio"), is the target of many sound reproduction methods.

[0230] According to one aspect, it is an object of the present invention to provide a concept for encoding or reproducing spatially extended sound sources, possibly having complex geometric shapes.

[0231] The following section may be titled "2D Source Width."

[0232] This section describes, by way of example, methods involving rendering augmented sound sources on a 2D surface facing the listener's perspective, for example, within several ranges of azimuth at 0 degrees elevation (as is the case in traditional stereo / surround sound), or several ranges of azimuth and elevation (as is the case in 3D audio or virtual reality with three degrees of freedom of user movement ["3DoF"], i.e., head rotation in pitch / yaw / roll axes).

[0233] Increasing the apparent width of an audio object panned between two or more loudspeakers (producing a so-called phantom image or phantom source) can be achieved by decreasing the correlation of the channel signals involved (Blauert, 2001, s.241-257). With decreasing correlation, the spread of the phantom source increases until it covers the entire range between the loudspeakers for correlation values ​​close to 0 (and not too wide an aperture angle).

[0234] The decorrelated version of the source signal is obtained, for example, by deriving and applying a suitable decorrelation filter. For example, Lauridsen (Lauridsen, 1954) proposed adding / subtracting a time-delayed and scaled version of the source signal to itself to obtain two decorrelated versions of the signal. More complex approaches were proposed, for example, by Kendall (Kendall, 1995), who iteratively derived paired decorrelation all-pass filters based on combinations of random number sequences. Faller et al. proposed a suitable decorrelation filter ("diffuser"), for example, in (Baumgarte and Faller, 2003). Also, Zotter et al. derived a filter pair in which a frequency-dependent phase or amplitude difference was used to achieve broadening of the phantom source (Zotter and Frank, 2013). Furthermore, for example, (Alary, Politis, & Valimaki, 2017) proposed a decorrelation filter based on velvet noise, which was further optimized by, for example, (Schlecht, Alary, Valimaki, & Habets, 2018).

[0235] Besides reducing the correlation of the corresponding channel signals of the phantom sources, the source width can also be increased, for example, by increasing the number of phantom sources resulting from audio objects. In (Pulkki, 1999), for example, the source width is controlled by panning the same source signals in (slightly) different directions. This method was originally proposed to stabilize the perceived phantom source spread of VBAP panned (Pulkki, 1997) source signals as they are moved in the sound scene. This is advantageous, for example, because the rendered sources are played by two or more speakers, which may result in undesirable modifications of the perceived source width, depending on the source direction.

[0236] For example, Virtual World DirAC (Pulkki, Laitinen, & Erkut, 2009) is an extension of traditional Directional Audio Coding (DirAC) (Pulkki, 2007) techniques for sound synthesis within virtual worlds. To render spatial coverage, directional sound components of a source are, for example, panned randomly within some range around the original direction of the source, where the panning direction varies, for example, with time and frequency.

[0237] A similar approach has been pursued, for example, in (Pihlajamaki, Santala, & Pulkki, 2014), where spatial coverage is achieved by randomly distributing frequency bands of the source signal into different spatial directions, a method that aims to produce spatially distributed veiling sounds that come equally from all directions, rather than controlling the exact degree of coverage.

[0238] For example, Verron et al. captured the spatial extent of the sources by not using panned correlated signals, but by synthesizing multiple incoherent versions of the source signal, distributing them uniformly on a circle around the listener, and mixing between them (Verron, Aramaki, Kronland-Martinet, & Pallone, 2010). The number and gain of simultaneously active sources determines, for example, the strength of the broadening effect. This method has been implemented, for example, as a spatial extension to an environmental sound synthesizer.

[0239] The following may be entitled "3D Source Width."

[0240] This section describes methods related to rendering, for example, augmented sound sources in 3D space, i.e., volumetrically, as required (or at least advantageous) for virtual reality with, for example, six degrees of freedom ("6DoF"), which means, for example, six degrees of freedom of user movement, i.e., head rotation in pitch / yaw / roll axes, plus three translational movement directions x / y / z.

[0241] For example, Potard et al. extended the concept of source extent (i.e. its width between two loudspeakers) as a one-dimensional parameter of the source by considering the perception of the source shape (Potard, 2003). They applied (time-varying) decorrelation techniques to the original source signal and then generated, for example, multiple incoherent point sources by placing the incoherent sources in different spatial locations, which gave them a three-dimensional extent (Potard and Burnett, 2004).

[0242] For example, in MPEG-4 Advanced AudioBIFS (Schmidt and Schroder, 2004), volumetric objects / shapes (shells, boxes, ellipsoids, and cylinders) can be filled with several equally distributed and decorrelated sound sources to induce a three-dimensional source range.

[0243] To increase and control source range using Ambisonics, Schmele et al. (Schmele and Sayin, 2018) proposed, for example, a mixture of reducing the Ambisonics order of the input signal, which essentially increases the apparent source width, and dispersing decorrelated copies of the source signal around the listening space.

[0244] Another approach has been introduced, for example, by Zotter et al., where they adopted the principles proposed in (Zotter & Frank, 2013) for Ambisonics (i.e., deriving filter pairs that introduce, for example, frequency-dependent phase and amplitude differences to capture source range in a stereo playback setup) (Zotter F., Frank, Kronlachner, & Choi, 2014).

[0245] For example, a common drawback of panning-based methods (e.g., (Pulkki, 1997) (Pulkki, 1999) (Pulkki, 2007) (Pulkki, Laitinen, and Erkut, 2009)) is their dependence on the listener's position. Even a small deviation from the sweet spot collapses the spatial image into the loudspeaker closest to the listener. This significantly limits their application in the context of virtual and augmented reality with six degrees of freedom (6DoF), where the listener is assumed to move freely in the surroundings. Additionally, distributing the time-frequency bins in DirAC-based methods (e.g., (Pulkki, 2007) (Pulkki, Laitinen, and Erkut, 2009)) does not always guarantee a correct rendering of the spatial extent of the phantom source. Moreover, it usually significantly degrades the timbre of the source signal.

[0246] Decorrelation of source signals is, for example, typically achieved by one of the following methods: i) deriving filter pairs with complementary amplitudes (e.g., (Lauridsen, 1954)); ii) using all-pass filters with constant amplitude but (randomly) scrambled phase (e.g., (Kendall, 1995) (Potard and Burnett, 2004)); or iii) randomly distributing the time-frequency bins of the source signals in space (e.g., (Pihlajamaki, Santala, and Pulkki, 2014)).

[0247] All techniques come with their own implications: complementary filtering of the source signal according to i) usually leads to a modification of the perceived timbre of the decorrelated signal, whereas all-pass filtering as in ii) preserves the timbre of the source signal, while phase scrambling disrupts the original phase relationships, causing severe time dispersion and smearing artifacts, especially for transient signals. Spatially dispersing the time-frequency bins has proven effective for some signals, but also modifies the perceived timbre of the signal. Moreover, it has been shown to be highly signal-dependent and introduces severe artifacts for impulsive signals.

[0248] Embedding multiple decorrelated versions of source signals into volumetric shapes, as proposed for example in Advanced AudioBIFS ((Schmidt and Schroder, 2004) (Potard, 2003) (Potard and Burnett, 2004)), assumes the availability of a large number of filters (typically more than 10 point sources per volumetric shape are used) that generate mutually decorrelated output signals. However, finding such filters is not a trivial task, and the more such filters are needed, the more difficult it becomes. Furthermore, if the source signals are not completely decorrelated and, for example, in a (virtual reality) scenario, a listener moves around such a shape, the individual source distances to the listener correspond to different delays of the source signals, and their superposition at the listener's ears results in, for example, position-dependent comb-filtering potentially introducing troublesome and unstable coloration of the source signals.

[0249] Controlling source width using Ambisonics-based techniques in (Schmele and Sayin, 2018) has been shown to have an audible effect only for transitions from 2nd to 1st or 0th order, for example by reducing the Ambisonics order. Moreover, these transitions are often perceived not only as source widening, but also as the movement of phantom sources. Adding decorrelated versions of the source signals can help stabilize the perception of apparent source width, but it also introduces comb-filter effects that alter the timbre of the phantom sources.

[0250] An efficient method for binaural rendering of spatially augmented sound sources, which may optionally be used in embodiments according to the invention, was disclosed in EP3879856, for example using the following: One (mono) input waveform signal. A decorrelator (optional) to generate a decorrelated version of this signal. A cue calculation stage that calculates target binaural (and tonal) cues for e.g. spatially extended sound sources depending on the size of the source (e.g. given as an azimuth-elevation range depending on the position and orientation of the spatially extended sound sources and the listener). A binaural cue conditioning stage that uses the target cues from the cue computation stage to generate, for example, a binaurally rendered output signal, for example from the input signal and a decorrelated version of it.

[0251] The following section may be entitled "Topological Sound Propagation."

[0252] Modeling sound propagation is important (and in some cases even crucial) for virtual acoustic and virtual reality applications. In particular, the concept of topological sound propagation has been found to be important for modeling, for example, the propagation of sound between different acoustic rooms, possibly having different acoustic characteristics. Aspects of the present invention are particularly focused on room reverberation effects, resulting, for example, from sound scattering off walls, and how to accurately and efficiently model these effects for virtual environments.

[0253] Despite a considerable research history in acoustic simulation, most acoustic modeling approaches have mostly focused on single acoustic spaces such as concert halls or auditoriums. For complex scenes with numerous rooms and corridors, accurate simulations often require heavy computations that are impossible to achieve in real time. As a result, precomputed simulations are frequently used. Also, for such environments, it is advantageous to split the geometric model into separate rooms that are connected to each other by portals (Vorlonder and Schroder, 2007).

[0254] For example, Efstathios et al. proposed a reverberation graph method in which a complex geometry is first subdivided into a series of coupled spaces connected by portals, and then represents them as point sources while precomputing "transport operators" offline using geometric acoustic techniques. In other words, the method traces paths, e.g., from sources to portals, between portals, and from portals to the listener, to simulate the overall propagation route (Stavrakis, Tsingos, & Calamia, 2009).

[0255] For example, another different approach by Tsingos utilizes precomputed image source gradients to generate location-dependent reverberation in real time without access to complex 3D geometry data (Tsingos, 2009).

[0256] All these proposals can be implemented to realize topological sound propagation for diffuse sound in real time.

[0257] According to one aspect, the method (or device or concept) of the present invention proposes a new technique that, for example, improves upon two shortcomings found in previous solutions. 1. Precomputed simulations are only valid for previously known source and listener locations (source / listener location combinations), and therefore limit the movement of either or both of the source and listener. 2. The portal is represented as a point source, which is not true in real-world scenarios. In other words, the sound perceived in one room, as propagating from an adjacent room, is not coming from a general opening between the two rooms, but is located in one specific location (i.e., the location of the portal's point source) (e.g., the latter may be the case according to embodiments of the present invention). This makes the obtained acoustic impression unrealistic, especially when the listener is close to the portal.

[0258] In the following paragraphs, the objectives of embodiments according to the invention are explained.

[0259] According to one aspect, it is an object of the present invention to provide an efficient and realistic rendering of diffuse sound and its topology propagation as portals, e.g. using spatially augmented sound sources, e.g. as described in detail in EP3879856. The proposed algorithm provides an integrated solution for smooth rendering of e.g. multiple acoustically homogeneous spaces (AHS), e.g. regardless of the position and movement of the sound sources and listeners. In particular, according to one aspect, the present invention addresses the need for realistic and efficient rendering of virtual sounds, as well as for a bitrate efficient representation of these sound aspects, which can be transmitted, e.g. from an encoder to a (possibly remote) VR renderer.

[0260] In the following section entitled "Description of an embodiment", an embodiment of the present invention is described.

[0261] A summary of one embodiment of the method of the present invention is provided below.

[0262] FIG. 10 shows a schematic block diagram of the pipeline of the method of the present invention. As an example, the block diagram of FIG. 10 may demonstrate one example of the pipeline of the method of the present invention, where an encoder, a bitstream, and a decoder may be optionally used as separate embodiments. FIG. 10 shows, as an example, the metadata and signal flow of the method (or concept) of the present invention in three main components: an encoder (e.g., 1010), a bitstream (e.g., 1020), and a decoder (e.g., 1030). For example, at the very beginning of the pipeline, a scene with 3D geometry is provided as input (e.g., 1002), and the final output (e.g., output audio 1004), generated by the decoder, may be, for example, left and right binaural signals L bin and R bin(1004a and 1004b). It should be noted that, therefore, a renderer according to an embodiment, for example included in the decoder 1030, as shown in FIG. 10, may be configured to perform binaural rendering.

[0263] The method is described in three successive sections corresponding to the three components mentioned above.

[0264] 1. Encoder (e.g., 1010): (All aspects, examples, and details of the present invention are optional.) For example, each AHS in the input scene (e.g., 1002) is given a geometry outlining its extent, possibly (optionally) with additional geometry such as walls and ceilings. Using this information, for example, two different types of methods may be used to detect or create portal geometries. Details of both methods, and what a "portal" represents for each of them, are described below (e.g., with reference to Figures 11 and 12). o For example, the first method takes the overall geometry of each AHS as the geometric description of its corresponding portal. Figure 11 shows an overview of an audio scene with three acoustically coupled spatial regions in the form of spaces A, B, and C. In other words, Figure 11 shows an example where there are three such spaces A, B, and C (e.g., 1110, 1120, 1130). Figure 11 may show an example of portal detection method 1, for example, according to an embodiment where a spatially expanding sound source may take the geometric extent of the first spatial region. As can be seen in Figure 11, the portals (e.g., 1112, 1122, 1132) have, for example, the same geometry as their AHSs (e.g., the same shape, but, for example, a contracted area). Moreover, as shown in this example, the first and / or second spatial regions may be acoustically homogeneous spaces, as previously explained. A great advantage of this method is that, for example, the AHSs in which the listener is located may simply be identified as portals. This means, for example, that only one algorithm is needed to render all AHSs throughout the entire scene, regardless of where the listener (e.g., 1140) is located (compared to, for example, the second method). If the listener moves, for example, to space C, the three same portals still represent their respective AHSs. Occlusion of these radiating portals may (or may have to) be addressed, for example, in a separate occlusion stage that is usually part (for example) of a virtual 6DoF hearing environment, and beyond the scope of this description, for example, the description of this paragraph. As previously described, by way of example, ray tracing may be performed according to an embodiment to take into account occlusion effects (e.g., of walls 1150).Further, as shown in FIG. 11 and in accordance with the above description, it should be noted that a renderer, for example, such as that included in the decoder 1030, may be configured to render multiple spatial augmentation sources, including one or more spatial augmentation sources that are far from the listener position (e.g., spatial augmentation sources such as or representing the portals 1122 and 1132) and one or more spatial augmentation sources inside which the listener position is located (e.g., spatial augmentation sources such as or representing the portal 1112), using the same rendering algorithm, taking into account occlusions between the listener position and one or more spatial augmentation sources that are far from the listener position. o For example, the second method identifies and utilizes the connected parts between two AHSs to generate a geometric description of the portal. The portal serves, for example, as a representation of the neighboring AHSs and radiates, for example, its sound with a precise spatial extent into the listener space. For example, an algorithm can be used to analyze the geometric relationship between all AHSs in the scene and detect possible portals. An example is given in FIG. 12. FIG. 12 shows an overview of an audio scene with three acoustically coupled spatial regions in the form of spaces A, B, and C, as described in FIG. 11. However, in contrast, FIG. 12 may show an example of portal detection method 2, for example, according to an embodiment in which a spatially expanding sound source is placed in a portal between the first and second spatial regions. Thus, as an optional feature for an embodiment of the present invention, the first and second spatial regions may be rooms that are acoustically coupled through a portal, as shown in FIG. 12. For example, when a listener (e.g., 1140) is in space A (e.g., 1110), a wall shared by space A and space B is identified as a portal for representing AHS B. [This is shown, for example, in FIG. 12 by the orange portal_wall (e.g., 1160) drawn between A (e.g., 1110) and B (e.g., 1120).] For example, in the case of space C (e.g., 1130), the connected portion of space C and space A includes a section of the wall, and also a doorway (e.g., no geometry, only an area of ​​empty space). This results, for example, in two portals with different emission characteristics for representing AHS C [for example, the orange portal_wall (e.g., 1170) and the red portal_door (e.g., 1180) drawn between spaces A and C in FIG. 12].This method requires, for example, more geometric processing (or even being written by the user directly by hand), but also provides more flexibility in creating complex sound scenes, where type 2 portals can be interpreted as a medium that allows sound propagation between any pair of, for example, AHSs, that may or may not have a close relationship in physical space. That is, this type of portal allows, for example, to write them based on real geometric relationships but also artistic intent. Thus, they provide, for example, more flexible rendering options.

[0265] 10 may be configured to detect portals corresponding to AHSs, e.g., as described with respect to method 1, or may be configured to detect portals corresponding to interfaces between AHSs, e.g., as described with respect to method 2. The portal geometry description unit 1014 may thus be configured to determine a respective geometric description of each portal, e.g., according to an identical shape as the corresponding AHSs, e.g., with a contracted outer boundary (e.g., in the case of method 1), or, e.g., according to an intersection between the AHSs (e.g., in the case of method 2).

[0266] Furthermore, according to some embodiments of the present invention, the SESS and the portal may be used interchangeably. Thus, the SESS may be located at the location of the portal, or the portal may be described or represented or rendered using or by the SESS. Furthermore, according to some embodiments, the AHS and the portal may be used interchangeably with respect to at least some properties. The portal may, for example, share the same shape as the corresponding AHS, but, for example, with a contracted boundary.

[0267] Optionally, the portal may be rendered as or using SESS. Thus, a portal representing an AHS may be rendered as or using SESS.

[0268] 2. Bitstream (e.g., 1020): (All aspects, examples, and details of the present invention are optional.) The generated portal geometry (e.g., with associated metadata, if desired) is (optionally) quantized and (optionally) serialized into a bitstream and signaled as Portal Information (e.g., 1022). This allows, for example, data to be efficiently transmitted from an encoder (e.g., 1010) to a remote decoder (e.g., 1030).

[0269] 3. Decoder (e.g., 1030): (All aspects, examples, and details of the present invention are optional.) At the decoder, the geometrical description of the Portals from the bitstream is e.g. unpacked and reconstructed in the scene. To convert these 3D geometries into usable metadata, e.g. for the Hom.SESS synthesis algorithm, e.g. in real time, a process is performed, e.g. mapping the geometry into a listener-centric coordinate system and finding what spatial region this geometry occupies (e.g. from the listener's point of view, e.g. using the mapping unit 1032). For example, a preferred implementation of the method of the present invention uses a ray tracing based approach to perform the mapping. For example, first the listener coordinate system is segmented into multiple areas (or grids), for example based on perceptual relevance, and then rays are cast out from each grid, for example. A ray hit on a 3D geometry indicates that the corresponding grid is within the boundaries of its 2D projection from the listener's viewpoint. In other words, these grids are spatial regions that should be included in, for example, the SESS processing. For example, apart from the metadata, the Hom.SESS synthesis algorithm (for example executed in a Hom.SESS synthesis unit 1034, which corresponds to or is a SESS rendering unit) also requires, for example, one or two audio signals to audibly render the portal, for example as a spatial enhancement source. For example, to fulfill the prerequisites of the SESS algorithm, the two input signals should (ideally) be completely decorrelated (for example as shown with the decorrelated input signal 1036). An example of this type of signal is two downmixed signals from the output of a feedback delay network reverberator, which is a natural choice for the generation of late reverberation, for example, taking into account that the method of the invention is designed to simulate, for example, an acoustically homogeneous space and the propagation between them. For example, if the input signal to be prepared is not fully decorrelated (or there will only be a mono signal available, as shown for example with signal 1038), a fully decorrelated second signal can be derived from one existing input signal using for example a decorrelator (for example decorrelator 1040) such as the one described in European Patent Application EP21162142.0 entitled "AUDIO DECORRELATOR, PROCESSING SYSTEM AND METHOD FOR DECORRELATING AN AUDIO SIGNAL" (inventors: DISCH Sascha, ANEMULLER Carlotta, HERRE Jurgen). This allows the user to have two valid signals to input to the Hom.SESS synthesis algorithm. For example, as a final step, both the metadata and the audio signal are provided as input to a Hom.SESS synthesis (or Homogeneous Spatial Augmented Source Rendering, or Spatial Augmented Source Rendering), which renders the binaural output of the Portal, for example as described in EP3879856.

[0270] In the section entitled "Aspects of the Invention" below, embodiments are described or, respectively, further described as follows.

[0271] First, aspects of a renderer according to an embodiment that may (optionally) be controlled or are controlled, for example, possibly by bitstream elements, e.g., bitstream elements according to an embodiment of the present invention: A renderer, Equipped to render virtual acoustic effects, for example the propagation of reverb in one room as perceived from outside this room (e.g. from another adjacent room) in two or more acoustically homogeneous environments, as a sound source with spatial extent / size (rather than a point source). o In a preferred implementation, a sized source is (optionally) rendered as described in EP3879856, for example to render a reverb portal as a spatially enhancing source. For example, either 1. a two (or more) decorrelated downmix of the outputs, for example from a reverberator (e.g. a feedback delay network) or 2. using a single channel signal together with its decorrelated version as input to a spatially augmented source algorithm. · Optionally mapping the geometry of the portal (eg a representation of an acoustically homogeneous space) onto a listener-centric coordinate system, for example to identify the spatial sector covered by it for the listener. o In a preferred implementation, the mapping method is (optionally) a ray tracing based algorithm. Optionally simulating portals (e.g. of the two types below) as spatially expanding sound sources, e.g. according to the listener's position and orientation. A Type 1 Portal represents, for example, an AHS with its entire geometric shape. It is characterized by, for example, a seamless rendering of all AHS in a scene regardless of the listener's position. When, for example, the listener is outside the Portal, its exact perceived size can be calculated based on, for example, its projection on the listener's coordinate system. On the other hand, when, for example, the listener is inside the Portal, it covers, for example, the entire listener's head sphere. As a result, a Type 1 Portal can fully represent, for example, all AHS in a scene. o Type 2 Portals represent, for example, an AHS with its parts connected to the AHS in which the listener is located. For example, this type of Portal outlines only the actual geometrical extent radiating sound from the represented AHS into the listener AHS (rather than the full volume AHS as with Type 1). As a result, for example, the list of Portals may need to be updated each time the listener enters a different AHS to ensure that all AHSs are stably and correctly represented relative to the listener's position. In addition, radiation characteristics may also be optionally assigned on each corresponding Portal to ensure, for example, that the sound propagating therefrom is attenuated and properly colored. In other words, no further occlusion processing is required on Type 2 Portals. Optionally modeling occlusion of Type 1 portals, for example, by excluding or attenuating occluded spatial regions of the portal, for example through equalization depending on the absorption properties of the occluder. o In a preferred implementation, the occlusion processing optionally reuses ray tracing information, e.g. obtained during a previous geometry mapping step, to save computations. Optionally, allowing smooth transitions in, out of and between multiple acoustically homogeneous spaces. o In a preferred embodiment, the range of the transition area is optionally controlled by a parameter and can be optionally transmitted in the bitstream.

[0272] In the following, aspects of the bitstream according to embodiments of the present invention are described.

[0273] For example, a bitstream containing the following information (or at least some of them): An acoustic description of the acoustically homogeneous space (e.g. control parameters of the reverberators) (optional). A geometric description of the acoustically homogeneous space (e.g. a mesh of a box or the intersections and faces of an area) (optional). Detailed information about the acoustic relationships between spaces. As an example, for any pair of such spaces having a connecting portal, a propagation coefficient is transmitted from space #1 to space #2 (and, for example, vice versa) as a measure of how much of the acoustic energy of space #1 is radiated into space #2. In a preferred embodiment, this may optionally be calculated based on the ratio of the connected surface area between the two spaces to the overall absorbing surface area of ​​space #1. As a second example, the range of the transition area between AHSs is optionally controlled by a parameter that may optionally be transmitted in the bitstream.

[0274] In the following, embodiments according to the present invention are further described.

[0275] It should be noted that any of the features, functionality, and details as disclosed below may be incorporated or used both individually and in combination with any of the embodiments as disclosed herein. Thus, any of the features, functionality, and details as disclosed in the above embodiments may be incorporated or used both individually and in combination with any of the following embodiments.

[0276] An embodiment according to the invention, for example a renderer, may be configured to manage status updates and signal mixing of portals. A portal may be, for example, a representation of an Acoustic Environment (AE) or AHS as seen from the perspective of a listener outside the AE or AHS. A portal may be rendered as a homogeneous extended sound source or SESS.

[0277] Thus, embodiments in accordance with the invention may use one or more of the following data elements and variables: ReverbId A unique ID for each AE or AHS in a Scene. PortalItems A map that stores key-value pairs, where the key is the ID of an RI, eg a rendering item, and the value is the RI. PortalMap A map that stores key-value pairs, where the key is the ReverbId of an AE or AHS, and the value is a vector of PortalItems that should be active when the listener is inside the AE or AHS. PortalBySource A map storing key-value pairs, where the key is the ReverbId of an AE or AHS, and the value is a vector of PortalItems whose audio signal should be downmixed from the reverb output of the respective AE. PortalRI An entry in PortalItems that is a key-value pair, where the key is the ID of the RI and the value is the RI. listenerReverbId The unique ID of the AE or AHS the listener is in. allReverbIdsInScene A vector with the unique IDs of all AEs or AHSs in the scene. currentSignal The output signal frame from the current reverb instance (e.g. 15 channels). reverbSignalOutput A vector of output signal frames from all reverb instances in the scene. portalSignalBuffer RI signal buffer.

[0278] In the following, a stage of a renderer or a decoder comprising a renderer according to an embodiment is described.

[0279] During initialization, all Portals and their associated AE or AHS data may be read, for example, from the bitstream. Each Portal structure from the Encoder may be reconstructed into a Renderer representation of PortalItems. The following description is split into two sections, describing metadata processing in the update thread and signal processing in the audio thread, respectively.

[0280] Below, optional examples are provided for update thread processing according to an embodiment.

[0281] For each update, the stage may, for example, activate and deactivate PortalItems based on the AE or AHS the listener is in. This may be done by looking up the PortalMap with a key that is the ReverbId of the AE or AHS the listener is in. If the ID of the RI in the PortalItems is included in the value, the RI is relevant for this AE or AHS and therefore may be, for example, activated. Otherwise, the RI may be, for example, deactivated.

[0282] Working Example: for portalRI in PortalItems { if portalRI.Id is in PortalMap[listenerReverbId]{ portalRI.activate(); } else { portalRI.deactivate(); } }

[0283] Portals may be representations of, for example, an AE or AHS, so that the audio signals of the PortalItems are copied from the reverb output of the corresponding AE or AHS.

[0284] In the following, optional audio thread processing according to an embodiment is described.

[0285] For example, initially there may be a predetermined number, say 15, output channels from each reflector instance and only two signals may be required to render the portal as a homogenous area.

[0286] As mentioned in the update thread processing description above, the signal outputs of the reverb instances, and furthermore, each reverb instance, may be mapped to a corresponding RI, for example, in PortalItems.

[0287] Working Example: for ReverbId in allReverbIdsInScene{ currentSignal = reverbSignalOutput[ReverbId]; for portalRI in PortalBySource[ReverbId]{ portalSignalBuffer[portalRI.id].copyFrom(currentSignal); } }

[0288] Further, an encoder according to an embodiment is further described.

[0289] Optional Portal Creation by Embodiment: This section describes how an encoder according to an embodiment may generate portals, for example, based on the acoustic environment (AE or AHS) in a scene. An important concept to keep in mind here is that a portal may be a representation of an AE or AHS. When a listener is not in a particular AE or AHS, but the AE or AHS is still acoustically relevant, the AE or AHS may be represented as a portal.

[0290] For example, there could be three steps covering the main process of generating a portal.

[0291] Creating the portal geometry (optional): One portal geometry with a unique portalExtentId may be generated, for example, from each AE or AHS in the scene. The geometry may be obtained, for example, by slightly shrinking the geometry of the corresponding portalParentEnvironment, which may be done to avoid overlaps between the portal geometry and potential occluding boundaries (e.g., walls).

[0292] Identifying the connection state between two Portals (optional): For example, there may be three possible states of a connection between two AEs or AHSs: unconnected, connected with an opening, connected with an occlusion (or, for example, in other words, closed). This step may utilize ray tracing and / or voxelization techniques, for example, to identify potential empty spaces or geometry between each pair of AEs or AHSs, or between one AE or AHS and the "outside" environment. Additionally, it may provide, for example, information of isConnectedWithOpening, and also the location of the opening if this variable is true, i.e., openingPosX, openingPosY, and openingPosZ.

[0293] Create a portal struct with all its metadata to be encoded (optional): The metadata obtained through the above two steps, or even, for example, all metadata, may be organized, for example, into a structure for bitstream serialization. This step may deal, for example, with a) creating one portal structure with a unique portalId for each portal geometry, b) assigning them under an associated acousticEnvironmentId (if a portal is not created from a given AE or AHS, it may be associated, for example, with respect to a particular acoustic environment), and c) calculating a portalFactor for each opened connection based on the area of ​​the opening, the volume of the source AE or AHS, and the absorption coefficient of the source AE or AHS estimated from the RT60. (References) TIFF2024543840000002.tif226157TIFF2024543840000003.tif87158 [Explanation of symbols]

[0294] 100 Renderer 101 Sound Scene 102 Spatial expansion sound source information 103 Additional Scene Information 110 rendering units 200 Renderer 201 Rendered Sound Scenes 202 Spatial expansion sound source information 203 Sound Source Signal 204 Additional Scene Information 205 Listener Movement Information 210 rendering units 212 Direct Sound Rendering Unit 213 Rendered direct source response 214 SESS rendering unit 215 Rendered diffuse sound response 216 Rendering fusion unit, Auditory cue information unit 220 Reverberation Processing Unit 221 Reverberant version of the source signal 230 Spatial Region Determination Unit 231 Spatial region information 240 Ray Tracing Units 241 Rayhit Information 300 Decoder, Audio Decoder 301 Rendered Sound Scenes 302 Bitstream 310 Renderer 320 Information Extraction Unit 321 Geometric description of portals 322 Listener Movement Information 323 Additional Scene Information 324 Sound Source Signal 330 Mapping Units 331 Geometric description of spatially extended sound sources 340 SESS Information Supply Unit 341 Spatial expansion sound source information 350 Late Reverberation Generator 351 Signal 360 Feedback Delay Network Reverberator (FDNR) 361 signal 370 Decorrelator 371 Signal 400 Encoder, Audio Encoder 401 Bitstream, Audio Bitstream 402 Sound Scene Information 403 Audio Signal 410 Bitstream Supply Unit 420 AHS Identification Unit 430 SESS defined supply unit 431 SESS definition 440 Acoustic Obstacle Definition Supply Unit 442 Acoustic Obstacle Definition 500 Encoder, Audio Encoder 501 bitstream, audio bitstream 502 Sound Scene Information 503 Audio Signal 510 Bitstream Supply Unit 520 AHS and Portal Identification Unit 521 Portal Information 530 SESS defined supply unit 531 Definition of spatially augmented sound source 900 bitstream, audio bitstream 910 Coding Representation of Audio Signals 920 Coding Representation of Spatial Expansion Sound Sources 930 Coding Description of Spatial Domain 940 Coding of information describing acoustic relations 950 Coded Representation of Propagation Coefficients 960 Parameters 1002 Input Scene 1004 Output Audio 1004a Binaural Signal 1004b Binaural signal 1010 Encoder, Audio Encoder 1012 Portal Detection Unit 1014 Portal Geometry Description Unit 1020 bitstream, audio bitstream 1022 Portal Information 1030 Decoder, Audio Decoder 1032 mapping units 1034 Hom.SESS synthesis unit 1036 Decorrelated Input Signal 1038 signal 1040 Decorrelator 1110 Space A, second spatial region 1112 Portal, space expansion sound source, space expansion source 1120 Space B, first spatial region 1122 Portal, spatial expansion sound source, spatial expansion source 1130 Space C, the first spatial region 1132 Portal, spatial expansion sound source, spatial expansion source 1140 Listener, Listener Position 1150 Wall 1160 portal_wall, spatial expansion sound source, portal 1170 portal_wall, spatial expansion sound source, portal 1180 portal_door, spatial expansion sound source, portal

Claims

1. A renderer (100, 200) for rendering an audio scene, comprising: configured to render in a second spatial domain (1110) diffuse sound effects generated in a first spatial domain (1120, 1130) using spatially extending sound sources (1112, 1160, 1170, 1180); renderer(100, 200).

2. configured to render in the second spatial domain (1110) direct sound sound effects of a given sound source located in the first spatial domain (1120, 1130) using direct sound rendering; configured to render a diffuse sound effect of the given sound source in the second spatial region using the spatially expanding sound source. The renderer (100, 200) of claim 1.

3. configured to apply direct source rendering to a source signal (203, 324) of a given sound source located in the first spatial domain (1120, 1130) at a listener position (1140) located in the second spatial domain (1110) to obtain a rendered direct source response (213); configured to apply a reverberation process to the source signal of the given sound source to obtain one or more reverberated versions (221) of the source signal of the given sound source; configured to apply spatially extended source rendering to the one or more reverberant versions of the source signal of the given sound source to obtain a rendered diffuse sound response (215) at the listener position (1140) located within the second spatial region. A renderer (100, 200) according to claim 1 or 2.

4. configured to render in the second spatial domain (1110) a sound effect of late reverberation excited by a sound source located in the first spatial domain (1120, 1130) using the space-expanding sound source reproducing the late reverberation, A renderer (100, 200) according to any one of claims 1 to 3.

5. configured to render the diffuse sound effect using spatially extending sound sources having similar spectral content in each spatial region; A renderer (100, 200) according to any one of claims 1 to 4.

6. configured to render the sound effect of the diffuse sound using a spatially expanding sound source arranged at a portal (1160, 1170, 1180) between the first spatial region (1120, 1130) and the second spatial region (1110), the spatially expanding sound source reproducing the diffuse sound originating from the first spatial region; A renderer (100, 200) according to any one of claims 1 to 5.

7. configured to render the sound effect of the diffuse sound using spatially extending sound sources (1122, 1132) taking into account the geometric extent of the first spatial domain (1120, 1130) and reproducing the diffuse sound originating from the first spatial domain, taking into account the occlusion of the spatially extending sound sources at a listener position (1140) located within the second spatial domain (1110); A renderer (100, 200) according to any one of claims 1 to 6.

8. said first spatial region (1120, 1130) being a first acoustically homogeneous space; and / or the second spatial region (1110) is a second acoustically homogeneous space; A renderer (100, 200) according to any one of claims 1 to 7.

9. The first spatial region (1120, 1130) and the second spatial region (1110) are rooms that are acoustically coupled via portals (1160, 1170, 1180). A renderer (100, 200) according to any one of claims 1 to 8.

10. configured to render a plurality of spatially extending sound sources, comprising one or more spatially extending sources (1122, 1132) located far from a listener position (1140), using the same rendering algorithm, taking into account occlusions between the listener position (1140) and the one or more spatially extending sources (1122, 1132) located far from the listener position, and one or more spatially extending sources (1112) located inside the listener position; A renderer (100, 200) according to any one of claims 1 to 9.

11. configured to perform binaural rendering; A renderer (100, 200) according to any one of claims 1 to 10.

12. configured to determine in which spatial region relative to a listener's position (1140) and / or listener's orientation the spatial extension sound source for the reproduction of the diffuse sound is located and to render the spatial extension sound source accordingly; A renderer (100, 200) according to any one of claims 1 to 11.

13. configured to determine in which spatial region the spatial extension sound source for the reproduction of the diffuse sound is occluded relative to a listener's position (1140) and / or listener's orientation, and to render the spatial extension sound source accordingly; A renderer (100, 200) according to any one of claims 1 to 12.

14. configured to determine in which spatial region relative to a listener's position (1140) and / or listener's orientation the spatially expanding sound source for the reproduction of the diffuse sound is located using a ray tracing based technique; A renderer (100, 200) according to any one of claims 1 to 13.

15. configured to determine, using a ray tracing based technique, in which spatial region relative to a listener's position and / or listener's orientation the spatially expanding sound source for the reproduction of the diffuse sound is occluded; A renderer (100, 200) according to any one of claims 1 to 14.

16. configured to determine, for a plurality of areas, whether a light ray associated with each area and extending away from a listener position (1140) hits the spatially expanding sound source, thereby determining in which spatial region relative to the listener position and / or listener orientation the spatially expanding sound source for the reproduction of the diffuse sound is located. A renderer (100, 200) according to any one of claims 1 to 15.

17. configured to determine one or more auditory cue information items depending on the spatial region within which the spatially expanding sound source for the reproduction of the diffuse sound is located; - configured to process one or more sound source signals (203) representing the diffuse sound using the one or more auditory cue information items to obtain a rendered version (215) of the diffuse sound. A renderer (100, 200) according to any one of claims 1 to 16.

18. configured to update the determination in which spatial region relative to a listener's position (1140) and / or listener's orientation the spatially expanding sound source for the reproduction of the diffuse sound is located in response to the listener's movement; and / or configured to update the determination of the one or more auditory cue information items in response to movement of the listener; and / or configured to update the determination of the one or more cue information items in response to changes in the spatial region within which the spatially expanding sound source for the reproduction of the diffuse sound is located. A renderer (100, 200) according to any one of claims 1 to 17.

19. An audio decoder (300, 1030), A renderer (100, 200) according to any one of claims 1 to 18, configured to obtain a geometric description (321) of a portal (1160, 1170, 1180) from a bitstream (302, 401, 501, 900, 1020) to obtain a geometric description (331) of the spatially extended sound source for the reproduction of the diffuse sound, and to map the geometry of the portal onto a listener-centered coordinate system, Audio Decoder(300, 1030).

20. configured to obtain two or more signals (351) derived from an output of a late reverb generator (350) to be at least partially de-correlated for said rendering of said spatially extending sound sources; An audio decoder (300, 1030) according to claim 19.

21. configured to obtain two or more signals (360) for said rendering of said spatially expanding sound source using a feedback delay network reverberator (360); An audio decoder (300, 1030) according to claim 19 or 20.

22. configured to use a source signal (203, 324) and a decorrelated version of the source signal for the rendering of the spatially extended sound source, An audio decoder (300, 1030) according to any one of claims 19 to 21.

23. configured to exclude or attenuate occluded spatial regions when rendering the spatially expanding sound source; An audio decoder (300, 1030) according to any one of claims 19 to 22.

24. configured to allow smooth transitions in, out, and between multiple spatial regions; An audio decoder (300, 1030) according to any one of claims 19 to 23.

25. An audio encoder (400, 1010) for encoding an audio scene, comprising: configured to provide a coded representation of one or more audio signals; configured to identify a plurality of acoustically homogeneous spaces and, based thereon, provide a definition (431) of a spatially extended sound source, the geometrical characteristics of which are identical to the geometrical characteristics of the identified acoustically homogeneous spaces; Audio Encoder(400, 1010).

26. configured to provide a definition of an acoustic obstacle (442) between the acoustically homogeneous spaces; An audio encoder (400, 1010) according to claim 25.

27. An audio encoder (500, 1010) for encoding an audio scene, comprising: configured to provide a coded representation of one or more audio signals; configured to provide one or more spatially expanding sound source definitions (531), the geometrical characteristics of which are based on the geometrical characteristics of portals (1160, 1170, 1180) between the acoustically homogeneous spaces; Audio Encoder(500, 1010).

28. configured to identify a plurality of acoustically homogeneous spaces and one or more portals (1160, 1170, 1180) between said acoustically homogeneous spaces and, based thereon, provide a definition (531) of one or more spatially extending sound sources, the geometric characteristics of which are based on dimensions of said identified portals; An audio encoder (500, 1010) according to claim 27.

29. A method (600) for rendering an audio scene, comprising: Rendering (610) a diffuse sound effect generated in a first spatial domain (1120, 1130) in a second spatial domain (1110) using a spatially augmented sound source. A method (600).

30. A method (700) for encoding an audio scene, comprising: providing (710) an encoded representation of one or more audio signals, a step of identifying (720) a plurality of acoustically homogeneous spaces and, based thereon, providing (730) a definition of a spatially extended sound source, the geometrical characteristics of the spatially extended sound source being identical to the geometrical characteristics of the identified acoustically homogeneous spaces, Method (700).

31. A method (800) for encoding an audio scene, comprising: providing (810) an encoded representation of one or more audio signals, providing (820) definitions of one or more spatially extending sound sources, the geometrical characteristics of which are based on the geometrical characteristics of portals (1160, 1170, 1180) between the acoustically homogeneous spaces; Method (800).

32. 32. A computer program for carrying out the method according to any one of claims 29 to 31, when the computer program runs on a computer.

33. a coded representation (910) of one or more audio signals; a coded representation (920) of one or more spatially augmented sound sources for rendering a diffuse sound effect generated in a first spatial domain (1120, 1130) and rendered in a second spatial domain (1110); An audio bitstream (302, 401, 501, 900, 1020).

34. a coded description (930) of one or more spatial regions; a coded representation (940) of information describing the acoustic relationship between at least two spatial regions; An audio bitstream (302, 401, 501, 900, 1020).

35. the encoded representation of the spatial regions comprises a description of portals (1160, 1170, 1180) between two spatial regions, An audio bitstream (302, 401, 501, 900, 1020) as claimed in claim 34.

36. an encoded representation (950) of propagation coefficients describing acoustic propagation from a first spatial domain (1120, 1130) to a second acoustic domain; An audio bitstream (302, 401, 501, 900, 1020) according to claim 34 or 35, comprising:

37. A propagation coefficient that describes the amount / fraction of acoustic energy in the first spatial region (1120, 1130) that is radiated into the second spatial region (1110) An audio bitstream (302, 401, 501, 900, 1020) according to any one of claims 34 to 36, comprising:

38. A propagation coefficient that describes the ratio between the connected surface area between a first space and a second space and the overall absorbing surface area of ​​said first space. An audio bitstream (302, 401, 501, 900, 1020) according to any one of claims 34 to 37, comprising:

39. Parameter describing the range of the transition area between two spatial domains (960) An audio bitstream (302, 401, 501, 900, 1020) according to any one of claims 34 to 38, comprising:

Citation Information

Patent Citations

  • Audio apparatus and method therefor

    EP3712788A1

  • Acoustic control device

    JP1986257099A

  • Reflected sound extraction device

    JP1993046193A

  • Sound field space information transmission / reception method, sound field space information transmission device and sound field reproduction device

    JP2000322069A

  • Method for generating and consuming 3D sound scenes with spatially extended sound sources

    JP2006503491A