Audio device and method of operation thereof

Abstract

The audio device includes a first receiver 501 that receives audio data of an audio source of a scene including many rooms. A determiner 509 determines the room including the listening position and the neighboring rooms. A second receiver 503 receives spatial audio transmission data describing several transmission boundary regions of the listening room having an acoustic transmission level of sound from the neighboring rooms to the listening room exceeding a threshold. A first reverberator 511 determines the reverberant audio signals of the neighboring rooms. A sound source circuit 513 determines the sound source positions in the neighboring rooms relative to the transmission boundary regions. A renderer 507 renders an audio signal for the listening position including an audio component generated by rendering the reverberant audio signal of the second room from the sound source positions. An improved rendering of the multi-room scene is achieved.

Description

[Technical field]

[0001] The present invention relates to an apparatus and method for generating audio signals, and in particular, but not exclusively, to an apparatus and method for rendering audio of a multi-room scene, for example as part of an augmented reality experience. [Background technology]

[0002] The variety and range of audiovisual content-based experiences has increased significantly in recent years, and new services and ways of using and consuming such content are continually being developed and introduced. In particular, many spatial and interactive services, applications, and experiences are being developed to provide users with more complex and immersive experiences.

[0003] An example of such an application is Extended Reality (XR), a common term for Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) applications, which is rapidly becoming mainstream, with several solutions aimed at the consumer market. Several standards are also under development by several standards bodies. Such standardization activities are actively developing standards for various aspects of VR / AR / MR systems, including, for example, streaming, broadcasting, rendering, etc.

[0004] VR applications tend to provide a user experience that corresponds to the user being in a different world / environment / scene, while AR (including mixed reality MR) applications tend to provide a user experience that corresponds to the user being in a current environment, but with the addition of additional information or virtual objects or information. Thus, VR applications tend to provide fully immersive synthetically generated worlds / scenes, while AR applications tend to provide partially synthetic worlds / scenes that are superimposed on the real scene in which the user is physically present. However, the terms are often used interchangeably and have a high degree of overlap. In the following, the term extended reality / XR is used to refer to both virtual reality and augmented / mixed reality.

[0005] As an example, an increasingly popular service is one in which a user can actively and dynamically interact with the system to change the parameters of the rendering, thereby providing images and audio in a manner that adapts to movements and changes in the user's position and orientation. A very attractive feature in many applications is the ability to change the viewer's effective viewing position and direction, for example, allowing the viewer to move and "look around" within the scene being presented.

[0006] Such functionality allows, in particular, providing a virtual reality experience to the user, whereby the user can move (relatively) freely around the virtual scene, dynamically changing the user's position and where the user is looking. Typically, such virtual reality applications are based on a three-dimensional model of the scene, dynamically evaluating the model to provide a specific requested view. This approach is well known from gaming applications for computers and consoles, for example in the category of computer and console first-person shooter games.

[0007] Furthermore, particularly in virtual reality applications, it is generally desirable for the presented images to be three-dimensional images presented using a stereoscopic display. Indeed, to optimize the viewer's immersion, it is generally preferred for the user to experience the presented scene as a three-dimensional scene. Indeed, the virtual reality experience preferably allows the user to select his or her own position, viewpoint, and point in time relative to the virtual world.

[0008] In addition to the visual rendering, most XR applications also provide a corresponding audio experience. In many applications, the audio preferably provides a spatial audio experience in which the audio source is perceived to arrive from an object location corresponding to the location of the corresponding object in the visual scene. Thus, the audio and video scene are preferably perceived as congruent, and both provide a complete spatial experience.

[0009] For example, many immersive experiences are provided by virtual audio scenes generated by headphone playback using binaural audio rendering techniques. In many scenarios, such headphone playback is based on head tracking, where the rendering is done in response to the user's head movements. This greatly enhances the sense of immersion.

[0010] An important feature of many applications is the ability to generate and / or distribute audio that provides a natural and realistic perception of the audio scene. For example, when generating audio for virtual reality applications, it is important that not only are the desired audio sources generated, but that they are also generated in a way that provides a realistic perception of the audio environment, including attenuation, reflections, coloration, etc.

[0011] In room / ambient acoustics, reflections of sound waves from walls, floors, ceilings, objects, etc. result in delayed and attenuated (generally frequency-dependent) versions of the source signal that reach the listener (i.e., the user of an XR system) via different paths. The combined effect is modeled by an impulse response called the Room Impulse Response (RIR).

[0012] As shown in Figure 1, RIR generally consists of a direct sound, which depends on the distance from the sound source to the listener, followed by a reflected portion that characterizes the acoustic properties of the room. The size and shape of the room, the position of the sound source and the listener within the room, and the reflective properties of the room's surfaces all contribute to the characteristics of this reverberant portion.

[0013] The reflection part is divided into two time domains, which usually overlap. The first domain contains the so-called early reflections, which represent isolated reflections of the sound source off walls or obstacles in the room before reaching the listener. As the time delay / (propagation) delay increases, the number of reflections present within a certain time interval increases, and the path includes secondary or higher order reflections (e.g. reflections can be from several walls, or both walls and ceiling, etc.).

[0014] The second region, called the reverberant part, is where the density of these reflections increases to the point where they can no longer be separated by the human brain. This region is commonly called the diffuse reverberation, late reverberation, or reverberant tail, or simply reverberation.

[0015] The RIR contains cues that give the hearing system information about the distance of a sound source as well as the size and acoustic properties of the room. The energy in the reverberant part relative to the energy in the anechoic part largely determines the perceived distance of a sound source. The level and delay of the earliest reflections provide cues as to how close a sound source is to a wall, and anthropometric filtering enhances the assessment of a particular wall, floor, or ceiling.

[0016] The density of (early) reflections contributes to the perceived size of a room. The time it takes for the energy level of the reflections to drop by 60 dB is called the reverberation time T60 Reverberation time is denoted by and is a frequently used measure of how quickly reflections dissipate in a room. Reverberation time provides information about the acoustic properties of a room, such as whether the walls are highly reflective (e.g., a bathroom) or highly sound absorbing (e.g., a bedroom with furniture, carpets, and curtains).

[0017] Furthermore, the RIR, if it is part of the Binaural Room Impulse Response (BRIR), depends on the anthropometric characteristics of the user since the RIR is filtered by the head, ears and shoulders, i.e. the Head Related Impulse Response (HRIR).

[0018] Since reflections in the late reverberation cannot be separated distinctly by a listener, they are often simulated and represented parametrically, for example using parametric reverberators that use feedback delay networks, as in the well-known Jot reverberator.

[0019] For early reflections, the incidence direction and distance-dependent delays are important cues for humans to extract information about the room and about the relative position of the sound source. Therefore, the simulation of early reflections must be clearer than late reverberation. Therefore, in efficient acoustic rendering algorithms, early reflections are simulated separately from late reverberation. A well-known method for early reflections is to mirror the sound source at each of the room boundaries to generate virtual sound sources that represent the reflections.

[0020] Early reflections involve the position of the user and / or sound source relative to the room boundaries (walls, ceiling, floor), while in late reverberation the acoustic response of the room tends to be diffuse and therefore homogenous throughout the room, which allows the simulation of late reverberation to be often more computationally efficient than early reflections.

[0021] Two main properties of late reverberation are the slope and amplitude of the impulse response at the time above a given threshold. These properties tend to be strongly frequency dependent in natural rooms. Reverberation is often described using parameters that characterize these properties.

[0022] An example of parameters characterizing reverberation is shown in Figure 2. Examples of parameters traditionally used to indicate the slope and amplitude of the impulse response corresponding to diffuse reverberation include the well-known T 60 These include the amplitude level / energy and the reverberation level / energy. More recently, other measures of amplitude level have been proposed, in particular parameters indicating the ratio between the diffuse reverberation energy and the total emitted source energy.

[0023] In particular, the diffuse-to-source ratio DSR is used to represent the amount of diffuse reverberant energy or level of a sound source received by a user as a ratio of the total emitted energy of that sound source. DSR represents the ratio between the emitted sound source energy and the diffuse reverberant characteristics, such as in particular the energy or (initial) level of the diffuse reverberant signal.

number

[0024] Henceforth, this will be referred to as DSR (Diffusion-to-Source Ratio).

[0025] Such known techniques tend to provide an efficient description of audio propagation in a room and tend to result in a rendering of audio that is perceived as natural for the room in which the listener is (virtually) located. Summary of the Invention [Problem to be solved by the invention]

[0026] However, conventional approaches for representing and rendering sounds in a room, while providing an adequate perception in many embodiments, tend not to be suitable for all possible scenarios. In particular, in audio scenes that include different acoustic regions / rooms, audio signals generated using the described reverberation approaches may not result in an optimal experience or perception. This may generally result in situations where audio from other rooms is not fully or accurately represented by the rendered audio, resulting in a perception that does not fully reflect the acoustic scenario and scene.

[0027] In fact, reverberation is typically modeled for a listener in a room, taking into account the characteristics of the room. If the listener is outside the room or in a different room, the reverberators are turned off or reconfigured for other room characteristics. Even when multiple reverberators are operated in parallel, the output of the reverberators is typically a diffuse binaural (or multi-loudspeaker) signal intended to be presented to a listener located inside the room. However, such an approach tends to result in audio being generated that is often not perceived as an accurate representation of the real environment. This can result, for example, in a perceived disconnect or even a clash between the visual perception of the scene and the associated audio being rendered.

[0028] Thus, while common approaches to rendering audio are suitable for rendering environmental audio in many embodiments, they tend to be suboptimal in some scenarios, including particularly when rendering audio for scenes that include rooms with different acoustics.

[0029] Accordingly, improved techniques for rendering the audio of a scene would be advantageous, particularly techniques that allow for improved operation, increased flexibility, reduced complexity, easier implementation, improved audio experience, improved audio quality, reduced computational burden, improved suitability for various locations, improved performance for virtual / mixed / augmented reality applications, increased processing flexibility, improved representation and rendering of audio and audio characteristics for multiple rooms, improved audio rendering of multi-room scenes, and / or improved performance and / or operation.

[0030] Accordingly, the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination. [Means for solving the problem]

[0031] According to one aspect of the invention, a first receiver configured to receive audio data of an audio source of a scene including many rooms, a location circuit configured to determine a listening position within the scene, a determiner configured to determine a first room including the listening position and a second room that is a neighboring room of the first room, a second receiver configured to receive spatial acoustic transfer data between the first room and the second room, the spatial acoustic transfer data describing several transfer boundary regions of the first room, each transfer boundary region having an acoustic transfer level of sound exceeding a threshold from the second room to the first room, and a determiner configured to determine a few rooms in the second room. An audio device is provided that includes a first reverberator configured to determine a second room reverberant audio signal for a second room from at least one audio source and at least one of a geometric characteristic of the second room and an acoustic characteristic of the second room, a source circuit configured to determine a source position of an audio source in the second room with respect to at least a first transmission boundary region of a number of transmission boundary regions, and a renderer configured to render an audio signal for a listening position to include a first audio component generated by rendering the second room reverberant audio signal from the source position.

[0032] This approach allows for an improved user experience and provides improved rendering of the audio of a scene in many scenarios. This approach allows for improved audio rendering of multi-room scenes. A more natural and / or accurate audio perception of the scene is achieved in many scenarios.

[0033] The present invention provides improved and / or facilitated rendering of audio including reverberant components, often achieved with reduced complexity and reduced computational resource requirements.

[0034] This approach provides improved, enhanced, and / or facilitated flexibility and / or adaptability of processed and / or rendered audio.

[0035] In many embodiments, the renderer is configured to render the first audio component as a localized audio source, which may be a (spatially) extended localized audio source or may be, for example, a point source.

[0036] The first reverberator is a diffusion reverberator. The first reverberator includes (or is) a parametric reverberator such as a feedback delay network (FDN) reverberator, in particular a Jot reverberator.

[0037] The audio source may be an audio source of a second room reverberation signal for a listening position in the first room.

[0038] The acoustic transmission level is the acoustic gain and / or transparency.

[0039] In accordance with an optional feature of the invention, the rendering depends on at least one of a geometric characteristic of the first transmission bounding region and an acoustic characteristic of the first transmission bounding region.

[0040] This provides improved performance and / or facilitated implementation in many scenarios, which helps provide an improved user experience when perceiving audio in a multi-room scene. The geometric properties are also referred to as spatial properties.

[0041] According to an optional feature of the invention, the sound source circuitry is configured to determine the sound source location subject to a requirement that the distance from the sound source location to the first transmission boundary region is greater than or equal to one tenth of the maximum distance within the first transmission boundary region.

[0042] This provides improved performance and / or facilitated implementation in many scenarios, which helps provide an improved user experience when perceiving audio in a multi-room scene.

[0043] According to an optional feature of the invention, the sound source circuitry is configured to determine the sound source location subject to a requirement that the distance from the sound source location to the first transmission boundary region is greater than or equal to one tenth of the maximum distance in the second room.

[0044] This provides improved performance and / or facilitated implementation in many scenarios, which helps provide an improved user experience when perceiving audio in a multi-room scene.

[0045] According to an optional feature of the invention, the sound source circuitry is configured to determine the sound source location subject to a requirement that the distance from the sound source location to the first transmitting boundary region is greater than or equal to 20 cm.

[0046] This provides improved performance and / or facilitated implementation in many scenarios, which helps provide an improved user experience when perceiving audio in a multi-room scene.

[0047] According to an optional feature of the invention, the renderer is configured to generate the first audio component by rendering a reverberant audio signal of the second room along an acoustic path from the sound source position to the listening position that passes through the first transmission boundary region.

[0048] This allows for improved performance in many embodiments and allows for improved audio rendering and / or user experience. It generally allows for rendering that provides the perception of the second room reverberation signal as coming from a localized sound source. The acoustic path is a direct acoustic path.

[0049] According to an optional feature of the invention, the rendering includes generating the second audio component by rendering the second room reverberation signal as a reverberant audio component.

[0050] This allows for improved performance in many embodiments, allowing for improved audio rendering and / or user experience. It allows for more flexible adaptation providing a more natural sounding audio scene. A reverberant audio component is a diffuse audio component. A reverberant audio component is a component that does not have spatial cues. A reverberant audio component is one that does not have spatial cues that indicate the spatial source location of the reverberant audio component.

[0051] According to an optional feature of the invention, the rendering includes adapting a level of the first audio component relative to a level of the second audio component depending on a listening position relative to a position of the first transmission boundary region.

[0052] This allows for improved performance in many embodiments, allowing for an improved audio rendering and / or user experience, for example allowing for a more natural sounding, flexible transition between a first room audio experience and a second room audio experience, for example providing a gradual transition when the listening position changes from within the first room to within the second room.

[0053] According to an optional feature of the invention, the rendering includes increasing a level of the first audio component relative to a level of the second audio component for an increasing distance of the second room from the listening position.

[0054] This allows for improved performance in many embodiments, allowing for improved audio rendering and / or user experience.

[0055] According to an optional feature of the invention, the rendering includes level adapting the first audio component relative to a level of the second audio component in response to a size of the first transmission bounding region.

[0056] This allows for improved performance in many embodiments, allowing for an improved audio rendering and / or user experience, for example allowing for a more natural sounding, flexible transition between a first room audio experience and a second room audio experience, for example providing a gradual transition when the listening position changes from within the first room to within the second room.

[0057] In some embodiments, the rendering includes level adapting the first audio component relative to a level of the second audio component depending on geometric / spatial characteristics of the first transmission bounding region.

[0058] According to an optional feature of the invention, the renderer is configured to render second room reverberant audio signals from the source positions as spatially extended sources.

[0059] This allows in many scenarios an improved audio scene to be rendered.

[0060] According to an optional feature of the invention, the renderer includes a path renderer for rendering audio of an acoustic path; a plurality of reverberators configured to generate a reverberation signal of a room, the plurality of reverberators including a first reverberator configured to generate a reverberated audio signal of a second room; a combining circuit for combining the reverberation signals from the plurality of reverberators to the path renderer; a combining circuit for combining the reverberation signals from the plurality of reverberators including the reverberated audio signal of the second room with an output signal from the path renderer to generate a combined audio signal; and an adapter for adapting levels of the reverberation signals for combining by the combining circuit and for combination by the combining circuit.

[0061] This provides improved performance and / or facilitated implementation in many scenarios, which helps provide an improved user experience when perceiving audio in a multi-room scene. This approach also enables highly efficient and low complexity implementation in many embodiments.

[0062] According to an optional feature of the invention, the adapter is configured to adapt a level of the reverberation signal in response to at least one of metadata received together with the audio data of the audio source, acoustic characteristics of the first transmission boundary region, geometric characteristics of the first transmission boundary region, the listening position, the acoustic distance from the listening position to the sound source position, and a size of the first transmission boundary region.

[0063] This provides improved performance and in particular in many embodiments improved and / or more flexible and / or adaptable rendering of multi-room audio scenes.

[0064] According to an optional feature of the invention, the first reverberator is further configured to generate a second room reverberation signal in response to the first room reverberation signal.

[0065] This allows in many scenarios an improved audio scene to be rendered.

[0066] According to an optional feature of the invention, in the audio device described above, the sound source circuitry is configured to generate the sound source position based on a position of the first transmission boundary region.

[0067] This allows an improved audio scene to be rendered in many scenarios. The location of the first transmission boundary region can be the closest location, the center location, the edge location, a predetermined location, etc.

[0068] In some embodiments, the rendering includes adapting a level of the first audio component relative to a level of the second audio component depending on the distance between the listening position and a position of the first transmission boundary region.

[0069] In some embodiments, the sound source circuitry is configured to generate a sound source position as a function of the position of the first transmission boundary region. This allows an improved audio scene to be rendered in many scenarios. The position of the first transmission boundary region can be a nearest position, a center position, an edge position, a predetermined position, etc.

[0070] According to one aspect of the present invention, there is provided a method of operating an audio device, the method comprising the steps of receiving audio data of an audio source of a scene including a number of rooms; determining a first room including a listening position and a second room being a neighboring room of the first room; receiving spatial acoustic transmission data between the first room and the second room, the spatial acoustic transmission data describing a number of transmission boundary regions of the first room, each transmission boundary region having an acoustic transmission level of sound from the second room to the first room exceeding a threshold; determining a second room reverberation audio signal for the second room from at least one audio source in the second room and at least one of a geometric characteristic of the second room and an acoustic characteristic of the second room; determining a source position of the audio source in the second room for at least a first transmission boundary region of the number of transmission boundary regions; and rendering the audio signal for the listening position to include a first audio component generated by rendering the second room reverberation audio signal from the source position.

[0071] These and other aspects, features and advantages of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

[0072] Embodiments of the present invention will now be described, by way of example only, with reference to the drawings, in which: [Brief description of the drawings]

[0073] [Figure 1] FIG. 2 is a diagram illustrating an example of a room impulse response. [Diagram 2] FIG. 2 is a diagram illustrating an example of a room impulse response. [Diagram 3] FIG. 1 illustrates an example of elements of a virtual reality system. [Figure 4] FIG. 1 shows an example of a scene with three rooms. [Diagram 5] FIG. 1 illustrates an example of an audio device for generating audio output according to some embodiments of the present invention. [Figure 6] FIG. 2 illustrates an example of a renderer. [Figure 7] FIG. 2 illustrates an example of a transition area between two rooms according to one embodiment of the present invention. [Figure 8] FIG. 2 illustrates an example of a transition area between two rooms according to one embodiment of the present invention. [Figure 9] FIG. 2 illustrates an example of a transition area between two rooms according to one embodiment of the present invention. [Figure 10] FIG. 1 illustrates an example of an audio device for generating audio output according to some embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0074] The following description focuses on audio processing and rendering for augmented reality applications, but it will be understood that the principles and concepts described may be used in many other applications and embodiments.

[0075] Virtual experiences that allow users to move around in virtual worlds are becoming increasingly popular, and services are being developed to meet such demand.

[0076] In some systems, the VR application is provided locally to the viewer, e.g., by a standalone device that does not use or even have access to remote VR data or processing. For example, the device, such as a game console, includes a store for storing scene data, an input for receiving / generating viewer poses, and a processor for generating corresponding images from the scene data.

[0077] In other systems, the VR application may be implemented and executed remotely from the viewer. For example, a device local to the user detects / receives motion / pose data, which is sent to a remote device that processes the data to generate the viewer's pose. The remote device then generates view images and corresponding audio signals appropriate for the user pose based on scene data describing the scene. The view images and corresponding audio signals are then sent to a device local to the viewer where they are presented. For example, the remote device directly generates a video stream (typically a stereoscopic / 3D video stream) and a corresponding audio stream that is directly presented by the local device. Thus, in such an example, the local device does not perform any VR processing except for sending motion data and presenting the received video data.

[0078] In many systems, functionality is distributed across local and remote devices. For example, the local device processes the received input and sensor data to generate a user pose that is continuously transmitted to the remote VR device. The remote VR device then generates a corresponding view image and a corresponding audio signal and transmits these to the local device for presentation. In other systems, the remote VR device does not directly generate a view image and a corresponding audio signal, but instead selects relevant scene data and transmits it to the local device, which then generates the presented view image and corresponding audio signal. For example, the remote VR device identifies the nearest capture point, extracts the corresponding scene data (e.g., a set of object sources and their position metadata) and transmits it to the local device. The local device then processes the received scene data to generate an image and audio signal for a particular current user pose. A user pose generally corresponds to a head pose, and a reference to a user pose generally corresponds to a reference to a head pose, which can be considered equivalently.

[0079] In many applications, particularly broadcast services, a source transmits or streams scene data in the form of images (including video) and audio representations of a scene that are independent of the user pose. For example, signals and metadata corresponding to audio sources within the boundaries of a particular virtual room are transmitted or streamed to multiple clients. Each client then locally synthesizes an audio signal that corresponds to the current user pose. Similarly, the source transmits an overview of the audio environment, including a description of the audio sources in the environment and the acoustic properties of the environment. An audio representation is then generated locally and presented to the user, for example using binaural rendering and processing.

[0080] Figure 3 shows one such example of a VR system, where a remote VR client device 301 contacts a VR server 303 over a network 305, such as the Internet. The server 303 is configured to support a potentially large number of client devices 301 simultaneously.

[0081] The VR Server 303 supports the broadcast experience by, for example, transmitting an image signal including an image representation in the form of image data that is used by the client device to locally synthesize a view image corresponding to the appropriate user pose (pose refers to position and / or orientation). Similarly, the VR Server 303 transmits an audio representation of the scene, allowing the audio to be synthesized locally for the user pose. Specifically, as the user moves around in the virtual environment, the images and audio that are synthesized and presented to the user are updated to reflect the user's current (virtual) position and orientation within the (virtual) environment.

[0082] Therefore, in many applications, such as that of FIG. 3, it is desirable to model the scene and generate efficient image and audio representations that are efficiently included in a data signal, which can then be transmitted or streamed to various devices that locally synthesize views and audio for different poses other than the capture pose.

[0083] In some embodiments, a model representing the scene is, for example, stored locally and used locally to synthesize appropriate images and audio. For example, an audio model of a room includes characteristics of the audio sources heard in the room as well as indications of the acoustic properties of the room. The model data is then used to synthesize appropriate audio for a particular position.

[0084] In many scenarios, a scene includes several different acoustic environments or regions having different acoustic characteristics, in particular different reverberation characteristics. In particular, a scene includes or is divided into different acoustic environments / regions, each having homogenous reverberation, but with different reverberation between them. For all positions within an acoustic environment / region, the reverberation components of the audio received at that position are homogenous, in particular substantially the same (potentially except for gain differences). An acoustic environment / region is a set of positions where the reverberation components of the audio are homogenous. An acoustic environment / region is a set of positions where the reverberation components of the audio propagation impulse response of an audio source in the acoustic environment are homogenous. In particular, an acoustic environment / region is a set of positions where the reverberation components of the audio propagation impulse response of an audio source in the acoustic environment have the same frequency-dependent slope and / or amplitude characteristics, except for possibly gain differences. In particular, an acoustic environment / region is a set of positions where the reverberation components of the audio propagation impulse response of an audio source in the acoustic environment are the same, except for possibly gain differences.

[0085] An acoustic environment / space is generally a set of locations (generally a 2D or 3D space) that have the same rendering reverberation parameters. The reverberation parameters used to render the reverberant components are the same for all locations of the acoustic environment / space. In particular, all locations that have the same reverberation decay parameters (e.g. T 60 ) or the diffuse-to-source ratio DSR is applied to all positions within the acoustic environment / area.

[0086] The impulse response differs between different locations in a room / acoustic environment / area due to its "noisy" character resulting from the many different reflections of different orders that cause reverberation, but even in such cases the frequency-dependent slope and / or amplitude characteristics are the same (except perhaps for gain differences), especially when represented for example by the reverberation time (T60) or reverberation coloration.

[0087] The acoustic environment / area is also called an acoustic room or simply a room. A room is considered an environment / area as described above.

[0088] In many embodiments, scenes are provided in which the acoustic rooms correspond to different virtual or real rooms through which the user moves (e.g., virtually). An example of a scene with three rooms, A, B, and C, is shown in Figure 4. In this example, the user moves between and outside the three rooms through doorways and openings.

[0089] Rooms with substantial reverberant properties tend to represent spatial regions that are well bounded by geometric surfaces that have wholly or partially reflective properties, such that a substantial portion of the reflections in the room continue to reflect back into the region, creating a diffuse reflection field within the region that has no significant directionality. The geometric surfaces do not need to be aligned with any visual elements.

[0090] Audio rendering aimed at providing a natural and realistic effect to the listener generally involves a rendering of the acoustic scene. In many environments, this includes a representation and rendering of the diffuse reverberation present in the environment, such as the room the listener is in. Such a rendering and representation of the diffuse reverberation has been found to have a significant effect on the perception of the environment, such as whether the audio is perceived as representing a natural and realistic environment.

[0091] In situations where the scene contains many rooms, this approach typically renders audio and reverberation only for the room in which the listener is present, and ignores any audio from other rooms. However, this tends to result in an audio experience that is not perceived as optimal, and tends not to provide an optimal natural experience, especially when the user transitions between rooms. Some applications have been implemented to include rendering of audio from adjacent rooms, but this has been found to be suboptimal.

[0092] In the following, an advantageous technique for rendering an audio scene containing many rooms is described. For clarity and brevity, the technique is mainly described with reference to the example scenario / scene of Figure 4, where three adjacent rooms A, B, and C are included in the scene.

[0093] Fig. 5 shows an example of an audio device configured to render an audio scene. The audio device receives audio data describing the audio and audio sources in the scene. In a particular example, the audio device receives the audio data of the scene of Fig. 4. Based on the received audio data, the audio device renders an audio signal representative of the scene at a given listening position. The rendered audio includes both contributions from audio generated in the room in which the listener is located as well as contributions from other nearby, typically adjacent, rooms.

[0094] The audio device is configured to generate an audio output signal representative of audio in a scene. In particular, the audio device generates audio representative of the audio perceived by a user moving around in the scene with a number of audio sources and given acoustic characteristics. Each audio source is represented by an audio signal representative of the sound from the audio source, as well as metadata describing the characteristics of the audio source (such as providing a level indication of the audio signal). Additionally, metadata is provided to characterize the scene.

[0095] The renderer, in this example, is a part of an audio device that is configured to receive audio data and metadata for a scene and, based on the received data, render audio that represents at least a portion of the environment.

[0096] The audio device of Figure 5 includes a first receiver 501 configured to receive audio data of audio sources in a scene. Typically, for example, some point sources are provided with audio data reflecting the sound rendered from those audio point sources. In some embodiments, audio data is also provided for more diffuse audio sources, for example background or ambient sound sources, or spatially widespread sound sources.

[0097] The audio device includes a second receiver 503 configured to receive metadata characterizing the scene. The metadata describes, for example, room dimensions, acoustic properties of the rooms (e.g., T60, DSR, material properties), relationships between rooms, etc. The metadata further describes the positions and orientations of some or all of the audio sources.

[0098] The metadata includes spatial acoustic transmission data of the different rooms. In particular, it includes data describing one or more transmission boundary regions for at least one room, generally all rooms, of the scene. A transmission boundary region is specifically a region where the acoustic transmission level of a sound from another room to the room in which it is located exceeds a threshold. In particular, a transmission boundary region defines a region (generally a zone) of a boundary between two rooms, where the attenuation by / across the boundary is less than a given threshold, while outside the region it is greater.

[0099] Thus, a transmission boundary region defines an area of ​​a boundary between two rooms where acoustic propagation / transmission / transmission / coupling exceeds a threshold. Parts of the boundary that are not included in the transmission boundary region may have acoustic propagation / transmission / transmission / coupling above the threshold. Correspondingly, a transmission boundary region defines an area of ​​a boundary between two rooms where acoustic attenuation is below a threshold. Parts of the boundary that are not included in the transmission boundary region may have acoustic attenuation above the threshold.

[0100] Thus, a transmission boundary region refers to an area of ​​a boundary where the acoustic transparency is relatively high, while the acoustic transparency is low outside the area. A transmission boundary region corresponds, for example, to an opening in the boundary. For example, in a conventional room, a transmission boundary region corresponds, for example, to a doorway, an open window, or a hole in a wall separating two rooms.

[0101] The transmitting boundary region may be a three-dimensional region or a two-dimensional region. In many embodiments, the boundaries between rooms are represented as two-dimensional objects (e.g., walls that are considered to have no thickness), and the transmitting boundary region is in such a case the two-dimensional shape or area of ​​the boundary that has low acoustic attenuation.

[0102] Acoustic transparency is expressed on a scale: complete transparency means there is no acoustic suppression (e.g. an open doorway). partial transparency introduces attenuation to the energy transferring from one room to another (e.g. thick curtains in a doorway, or single-pane windows). At the other end of the scale are room-separating materials (e.g. thick concrete walls) that do not allow any (significant) acoustic leakage between rooms.

[0103] Thus, this approach provides acoustic link metadata (in the form of a transmission boundary region) that describes how two rooms are acoustically linked. This data can be locally derived or obtained, for example, from the received bitstream. The data can be manually provided by the content creator or indirectly derived from a geometric description of the rooms (e.g., a box, mesh, voxel representation, etc.) that includes acoustic properties such as material properties that indicate how much audio energy is transmitted through materials or coupled into material vibrations that cause an acoustic link from one room to another.

[0104] The transmission boundary region is often considered to represent room leakage, where acoustic energy is exchanged between two rooms. It can be a binary measure (opening of the boundary between the rooms) or a scalar measure (reflecting that some of the energy is transmitted).

[0105] It will be appreciated that in many cases the audio data and metadata are received as part of the same bitstream and the first receiver 501 and the second receiver 503 are implemented with the same functionality, and in effect the same receiver functionality implements both the first and second receivers. The audio apparatus of Figure 5 specifically corresponds to or is part of the client device 301 of Figure 3, which receives the audio data and metadata in a single bitstream transmitted from the server 303.

[0106] The device further includes a position circuit 505 configured to determine a listening position within the scene. The listening position generally reflects the (virtual) position of the user within the scene. For example, the third receiver 506 is coupled to a user tracking device such as a VR headset, an eye tracking device, a motion capture camera, etc., thereby receiving data of the user's movements (including, or perhaps limited to, head movements and / or eye movements). The position circuit 505 continuously determines the current listening position from this data.

[0107] This listening position is alternatively represented or augmented by controller input that causes the user to move or teleport the listening position within the scene.

[0108] Many approaches and techniques are known and used for determining listening positions in a scene for various applications, and it will be appreciated that any suitable approach may be used without detracting from the invention.

[0109] The audio device includes a renderer 507 configured to generate an audio output signal that represents the audio of the scene at the listening position. Typically, the audio signal is generated to include audio components of various different audio sources in the scene. For example, point audio sources in the same room are rendered as point audio sources with a direct acoustic path, reverberant components are rendered or generated, etc.

[0110] In the following, we describe a technique in which the rendered audio signal includes an audio signal / component representing audio from a room other than the room containing the listening position. The description focuses on the generation of this audio component, but it will be appreciated that the rendered audio signal presented to the user may include many other components and audio sources. These are generated and processed according to any suitable algorithm or technique, and it will be appreciated that those skilled in the art will be aware of many such techniques.

[0111] The following description focuses on generating / rendering audio signals (components) that reflect the audio in one or more other rooms other than the room containing the current listening position.

[0112] The audio device specifically includes a room determiner 509 configured to determine a first room that includes the listening position, a second room that is a nearby room, and generally an adjacent room of the first room. The room determiner 509 receives the listening position data from the location circuitry 505 and determines a current room for the listening position. The audio device then proceeds to select an adjacent room for the current room, and the audio device proceeds to generate an audio signal component for the listening position related to this adjacent room. As a specific example, consider a scenario where the listening position is currently in room B of FIG. 4, and the location circuitry 505 identifies room A as an adjacent room. The audio device then proceeds to render the audio / sound of the audio signal component from room A as heard from the listening position in room B.

[0113] It will be appreciated that the same process is used for room C, which is also identified as being an adjacent room to room B.

[0114] The audio device includes a reverberator 511 configured to generate a reverberant audio signal of the determined neighboring room, ie room A in the specific example.

[0115] The room determiner 509 provides information of the reverberation characteristics of the determined room, i.e. room A, to the reverberator 511. It does so directly or indirectly. For example, the room determiner 509 indicates a selected neighboring room to the reverberator 511, which then extracts the reverberation parameters of the selected room (i.e. room A in this example) from the received metadata. The reverberator 511 then proceeds to generate a reverberation signal corresponding to the reverberation present in the neighboring room.

[0116] Thus, the reverberator 511 proceeds to generate a reverberant audio signal of a neighboring room based on at least one audio source of the neighboring room and at least one characteristic of the second room, such as geometric characteristics (size, distance between boundary / reflective walls, etc.) or acoustic characteristics (attenuation, frequency response, etc.).

[0117] For example, the reverberator 511 may use the T 60 The reverberator 511 extracts parameters of the DSR or the DSR. It then proceeds to select all sound sources in the nearby rooms and provide audio data relating to them as input to the reverberation process. The reverberation signal is then generated according to a suitable reverberation algorithm. Many algorithms and techniques are known and it will be appreciated that any suitable technique may be used. For example, the reverberator 511 implements a parametric reverberator such as the Jot reverberator.

[0118] The nearby room reverberation signal is provided together with audio source data for audio sources in the listening room (i.e., the room containing the listening position, i.e., room B in the specific example) to a renderer 507. The renderer 507 then proceeds to render an audio signal for the listening position that further includes components corresponding to nearby room reverberation in addition to components of the audio sources in the listening room.

[0119] The rendering of the near-room reverberant signal specifically involves rendering the near-room reverberant signal as a localized audio source rather than as a diffuse non-spatial background source. Thus, the near-room reverberator 511 is not simply rendered as (always) diffuse reverberant audio, but is rendered as a localized source or even a point source. A localized source may be a point source or may be widespread but spatially limited.

[0120] The audio device includes a source circuit 513 configured to determine a source location of the near-room reverberation signal. The renderer 507 is then configured to render the near-room reverberation signal from this source location as a localized source. The rendering of the near-room reverberation signal may be rendered as a point source from the source location, or as a spatially extended audio source (i.e., a spread audio source) positioned according to and / or including the source location.

[0121] The sound source positions are specifically arranged to determine the sound source positions based on the transmission boundary regions, in particular by generating one sound source position per transmission boundary region, and rendering of the nearby room reverberation signal is performed for each sound source position / transmission boundary region.

[0122] The sound source location is determined based on the transmission boundary region, but is located within the neighboring room. Thus, reverberant audio from the neighboring room is also rendered to the listening position within the listening room, but is rendered as being perceived to originate from a location within the neighboring room. It is therefore not merely perceived as a diffuse non-spatial sound, but rather provides a spatial component. This provides a more realistic perception of the scene in many scenarios.

[0123] In many embodiments, the sound source circuitry 513 is configured to generate a sound source position relative to the transmission boundary region based on the position of the transmission boundary region. In many embodiments, the sound source circuitry 513 is configured to generate a sound source position relative to the transmission boundary region as a function of the position of the transmission boundary region. The position of the transmission boundary region can be a closest position, a center position, an edge position, a predetermined position, etc.

[0124] In a multi-room scenario, each room generally has its own reverberation characteristics: a sound source inside each room contributes very strongly to the room in which it resides, but very weakly to the reverberation of other rooms. Hence, the balance between all the sound sources in all the rooms also differs between the rooms.

[0125] Additionally, when simulating a room, the configuration is generally given by T60 and DSR, but the dimensions of the room also influence how fast and what pattern reflections occur.

[0126] When a listener is moving from one room to another, the reverberation of the room in which the listener is located is rendered as a diffuse reverberation as known in the art. However, the reverberation of other rooms is rendered according to the described approach as a localizable source close to the boundaries between the rooms where there is significant acoustic transparency between the rooms. As a result, the reverberation of these rooms is still perceived, but instead of being perceived as diffuse and non-spatial, it is perceived as localizable in the direction of the transparent part of the boundary between the rooms. The reverberation of the neighboring rooms is perceived as coming from and through an opening in the wall between the rooms. Furthermore, the level of reverberation from other rooms decreases as the distance from the listener to the reverberation source increases, similar to the experience in a physical situation.

[0127] The sound source positions are determined to be within the neighboring room, i.e. the neighboring room reverberation signal is not rendered from within the listening room or even at the boundary between the two rooms, but rather from a position within the neighboring room.

[0128] Such an approach is counter-intuitive since one would expect the rendering of audio in a room to be performed to reflect the geometric and acoustic properties of the room. However, the inventors have realized that by determining that the sound source locations are in neighboring rooms, an improved spatial perception is perceived. In particular, this has been found to result in a more natural sound perception in multi-room scenes, especially in transitions between rooms where the listening position is on or close to a transmission boundary region.

[0129] The sound source position is in many embodiments determined to be within the room by a given minimum distance. The minimum distance is the distance to the nearest transmission boundary region and / or the nearest boundary point (i.e. a point on the boundary). The minimum distance is at least 20 cm, or possibly 30 cm, 50 cm or 1 meter. The minimum distance is the scene distance. The scene generally corresponds to a real scene in the sense that the distances measured correspond to real distances. The minimum distance is determined with reference to these.

[0130] In many embodiments, the minimum distance is a relative distance, and in particular, the minimum distance depends on the size of the transmission boundary region. In many embodiments, the minimum distance from the sound source position to the transmission boundary region is equal to or greater than one tenth of the maximum distance of the transmission boundary region. In some embodiments, it is equal to or greater than one fifth, half, or the maximum distance of the transmission boundary region.

[0131] Such an approach offers particularly advantageous operation in many scenarios, and generally results in a rendering that is perceived as providing a natural impression of the scene.

[0132] In some embodiments, the minimum distance is a relative distance to the listening room and / or neighboring rooms.

[0133] In many embodiments, the minimum distance from the sound source position to the transmission boundary region is at least one tenth of the maximum distance of the listening room and / or neighboring rooms, in some embodiments it is at least one fifth, half, or the maximum distance of the listening room and / or neighboring rooms.

[0134] Such an approach offers particularly advantageous operation in many scenarios, and generally results in a rendering that is perceived as providing a natural impression of the scene.

[0135] The sound source position is in many embodiments determined to be within the room by a given maximum distance. The maximum distance is the distance to the nearest transmission boundary region and / or the nearest boundary point. The maximum distance is 1 m, or possibly 3 m, 5 m or 10 meters or less. The maximum distance is the scene distance. The scene generally corresponds to a real scene in the sense that the distances measured correspond to real distances. The maximum distance is determined with reference to these.

[0136] In many embodiments, the maximum distance is a relative distance, and in particular, the maximum distance depends on the size of the transmission boundary region. In many embodiments, the maximum distance from the sound source position to the transmission boundary region is less than or equal to half the maximum distance of the transmission boundary region. In some embodiments, it is more than or equal to 1, 2, or 5 times the maximum distance of the transmission boundary region.

[0137] Such an approach offers particularly advantageous operation in many scenarios, and generally results in a rendering that is perceived as providing a natural impression of the scene.

[0138] In some embodiments, the maximum distance is relative to the listening room and / or neighboring rooms.

[0139] In many embodiments, the maximum distance from the sound source position to the transmission boundary region is more than half the maximum distance of the listening room and / or neighboring rooms, in some embodiments it is more than one-fifth or one-third of the maximum distance of the listening room and / or neighboring rooms.

[0140] Such an approach offers particularly advantageous operation in many scenarios, and generally results in a rendering that is perceived as providing a natural impression of the scene.

[0141] In many embodiments, the distance is selected based on a combination of scalability considerations, for example the sound source is positioned at a distance of one-fifth of the maximum transmission boundary area, but at least 20 cm, and no greater than one-third of the smallest room dimension of the neighboring rooms.

[0142] Locating a sound source representing a nearby room reverberation signal within the nearby room, specifically close to the boundary but not too close, provides a very advantageous experience.

[0143] Positioning the sound source in the neighboring room slightly away from the boundary results in a more realistic transition where the directionally received reverberation originates from the room and not from the boundary, especially at the boundary where it is more realistic not to overlap with the directional source, thereby achieving an improved user experience, for example, when the user moves from one room to a neighboring room.

[0144] In particular, the described approach often enables a user position-based transition from omnidirectional diffuse reverberation to directional reverberation before reaching a transmission boundary between two rooms, where the reverberation is substantially directional and originates from the room, which is consistent with physical rooms, where there is no reverberation contribution from the direction of the transmission region, and where the reverberation is very little diffuse.

[0145] If a sound source location is exactly on the boundary and contributes to the audio signal at the listening position, then as it overlaps with the listening position, the localization will be perceived as unrealistic and even coming from the wrong room. It should be treated with extreme caution that a listener may move across the boundary and reverse the localization from one side of the listener to the other.

[0146] Moreover, localizable sound sources are often rendered using a non-zero reference distance where there is no distance attenuation required for the signal, which, when the sound source is some distance from the boundary, acts realistically for listening positions around the boundary and within the listening room.

[0147] In addition, when a sound source is located inside a neighboring room, rendering with a direct path renderer (as described below) conveniently models occlusion and / or diffraction when, for example, a door is fully or partially closed in the transmission boundary region: if the sound source is on the boundary, there is a risk that the sound source will not be occluded by the door shape.

[0148] When rendering the reverberant signal from a position in a neighboring room, the renderer 507 is configured to render the reverberant signal to include several spatial cues related to the sound source position. Specifically, the rendering includes rendering a sound path from the sound source position to the listening position, the sound path passing through a first transmission boundary region. The sound path may be a direct sound path from the sound source position to the listening position or a reflected sound path. Such a reflected sound path typically includes one, two, three, or five or fewer reflections. The reflections may be, for example, from walls or boundaries of the listening room.

[0149] 6 shows an example of elements of the renderer 507. In this example, the renderer 600 includes a path renderer 601 for each audio source. Each path renderer 601 is configured to generate a direct path signal component that represents a direct path from the audio source to the listener. The direct path signal component is generated based on the position of the listener and the audio source, in particular by potentially frequency-dependent scaling of the audio signal of the audio source depending on the distance of the audio source in a particular direction to the user (e.g., for non-omnidirectional sources) and, for example, the relative gain.

[0150] In many embodiments, the path renderer 601 also generates a direct path signal based on occlusion or diffraction (virtual) elements intermediate the sound source position and the user position.

[0151] In many embodiments, the path renderer 601 also generates further signal components for each path that include one or more reflections, for example by evaluating reflections off walls, ceilings, etc., as known to those skilled in the art. The direct and reflected path components are combined into a single output signal for each path renderer, such that a single signal representing the direct path and early / discrete reflections is generated for each audio source.

[0152] In some embodiments, the output audio signals for each audio source are binaural signals, so that each output signal includes both a left-ear (sub-) signal and a right-ear (sub-) signal, and includes a directional rendering of that direction relative to the user's orientation (e.g., by applying a Head Related Transfer Function (HRTF), a Binaural Room Impulse Response (BRIR), or a loudspeaker panning algorithm).

[0153] The output signals from the path renderers 601 are fed to a combiner 603, which combines the signals from the different path renderers 601 to generate a single combined signal. In many embodiments, a binaural output signal is generated and the combiner performs a combination, such as a weighted combination, of the individual signals from the path renderers 601, i.e. adding all right ear signals from the path renderers 601 together to generate a combined right ear signal and adding all left ear signals from the path renderers 601 together to generate a combined left ear signal.

[0154] The path renderer 601 and combiner 603 are typically implemented in any suitable manner, including as executable code for processing on a suitable computational resource, such as a microcontroller, microprocessor, digital signal processor, or central processing unit including supporting circuitry, such as memory, etc. It will be appreciated that multiple path renderers may be implemented as parallel functional units, for example a bank of dedicated processing units, or as repeated operations for each audio source. Typically the same algorithm / code is executed for each audio source / signal.

[0155] In addition to the individual path audio components, the renderer 507 is further configured to generate a signal component representing the diffuse reverberation in the environment. The diffuse reverberation signal is generated, in a particular example, by combining the source signal with a downmix signal and then applying a reverberation algorithm to the downmix signal to generate the diffuse reverberation signal.

[0156] The audio device of Fig. 6 comprises a downmixer 605, which receives audio signals of multiple sound sources (typically all sound sources inside the acoustic environment in which the reverberator is simulating diffuse reverberation) and metadata for combining the audio signals into a downmix (metadata is provided, for example, by a content creator as part of the audiovisual data stream). The downmixer combines the audio signals into a downmix, which accordingly reflects all sounds generated in the environment. The coefficients / weights of the individual audio signals are, for example, set to reflect the (relative) levels of the corresponding sound sources and are optionally combined with the DSR to control the level of reverberation.

[0157] In many embodiments, sound sources located outside the room modeled by the reverberator also contribute to the reverberation, however, they typically contribute much less than sources inside the room, since only a portion of these outside sound sources reaches the room through the transmission boundary region.

[0158] The downmix is ​​fed to a reverberation renderer / reverberator 607 configured to generate a diffuse reverberation signal based on the downmix. The reverberator 607 is in particular a parametric reverberator such as a Jot reverberator. The reverberator 607 is coupled to a combiner 603 to which the diffuse reverberation signal is fed. The combiner 603 then proceeds to combine the diffuse reverberation signal with the path signals representing the individual paths to generate a combined audio signal representative of the combined sound in the environment as perceived by a listener.

[0159] In this example, all audio signals from the audio sources in the listening room are fed into a path renderer 507 which proceeds to generate an output signal that includes contributions from all of these, including contributions corresponding to the direct path, the reflected paths and the diffuse reverberation.

[0160] In addition, however, the output of the reverberator 511, i.e. the nearby room reverberation signal, is also fed to the path renderer 601. Thus, the same rendering that is used for rendering the audio sources in the listening room is also used for the nearby room reverberation signals located in nearby rooms.

[0161] In most cases, a reverberant signal is also fed to the reverberator 607 so that contributions to the diffuse sound in the listening room are also provided from the reverberant sound of nearby rooms.

[0162] Similarly, in some cases, the nearby room reverberation signal is further generated based on the listening room reverberation signal. For example, the reverberator 607 is configured to generate a reverberation signal that does not include any contribution from the nearby room reverberation signal (but includes, for example, contributions from sound sources within the listening room itself). Instead, the generated reverberation signal for the listening room is provided as an input to the reverberator 511 and contributes to the generated nearby room reverberation signal. Such an approach provides, in many scenarios, an improved and more accurate rendering of the natural audio of the scene.

[0163] In many embodiments, the renderer 507 is configured to render the near-room reverberation signal as a point source signal from the source position. However, in other embodiments, the renderer 507 is configured to render the near-room reverberation signal as a spatially extended audio source. Thus, in some embodiments, the near-room reverberation signal is rendered as an extended audio source.

[0164] For example, the spatial extent of the sound source is determined by the sound source circuit 513. As an example, the extent of the sound source is determined depending on the size of the transmission boundary region. As a specific example, the sound source is determined to have a spatial extent that matches the size of the transmission boundary region.

[0165] The renderer 507 then proceeds to render the nearby room reverberation signals so that they are perceived to have a spatial extent consistent with the determined extension.

[0166] Various techniques are known for rendering spatially-extended audio sources, and it will be appreciated that any suitable technique may be used.

[0167] In some particular embodiments, the renderer 507 is configured to render a spacious audio source by rendering it as multiple point sources distributed within the spaciousness of the audio source. For example, for rendering a nearby room reverberation signal, a spaciousness and a relatively large number of point sources, e.g., 10-50, distributed within the spaciousness are determined. The nearby room reverberation signals are then rendered from each point source, thereby resulting in an overall perception of a single audio source with spatial spaciousness.

[0168] Rendering each point source in the space with a signal that is uncorrelated with other signals is generally advantageous in creating a realistic perceived space. For this purpose, a decorrelator is used. Alternatively, when using a feedback delay network (FDN) reverberator, the signal extraction from the feedback loop is performed with mutually orthogonal extraction vectors, each of which is used to obtain an uncorrelated reverberant signal. The set of orthogonal vectors is derived, for example, using the Gram-Schmidt process.

[0169] Rendering an extended audio source in many embodiments provides an improved user experience, especially in large transmission boundary regions, and in particular a more realistic and natural-sounding contribution of reverberation from nearby rooms.

[0170] In many embodiments, the audio device is configured to adapt the level or gain of the reverberant signal.

[0171] Specifically, sound sources within a room contribute their total radiated energy to the room, and thereby to the reverberation, and in many embodiments, for such sources, the source energy determines the relative levels to which these sources are downmixed in the downmixer for that room.

[0172] Based on the source characteristics, normalized source energy scaling factors are calculated, which indicate the scale factors for converting the signal of the source to its corresponding total emission energy. These normalized source energy scaling factors are used in the downmixer 605 of the renderer 507 to obtain a downmix signal that represents the total emission energy of the source.

[0173] It will also be appreciated that many embodiments use coefficients based on a nominal gain (including other applicable gains such as the average of the directivity pattern, pre-gain and distance attenuation gain) at a nominal distance from the source, and furthermore the reverberation energy data (DSR) is expressed in terms of source energy corresponding to a sampling at this nominal distance from the source, rather than the total emitted energy. Those skilled in the art will be able to convert examples and embodiments based on total emitted energy to this alternative source energy representation scheme.

[0174] For a sound source outside the room under consideration, not all of its energy contributes to the reverberation of the room. The sound source emits all of its energy in a different room or area, and a small fraction of that energy leaks into the room under consideration. The fraction of that energy that leaks into the room depends on several factors, including: · Distance from the source to the room leakage. · Obscuration and diffraction of the sound source by obstacles in other rooms affecting the path from the sound source to the room leakage. · Number and size of room leaks. Strong attenuation due to room leakage.

[0175] In some reduced complexity embodiments, the energy proportions are based on (potentially frequency-dependent) gains that have already been calculated for the listener, i.e., the listener is in a room and the direct path rendering of other room sources already takes into account distance, occlusion and diffraction, and therefore provides a good approximation of the path from the source to the room leak.

[0176] Such an approach may not be completely accurate, since it may further include attenuation related to occlusions and / or attenuation related to distance traveled in the room under consideration. However, it does not require additional calculations. If the listener's gain is determined in such a way that the algorithm knows what factors are imposed by each room, it is also part of a process that gathers gain only from factors outside the room under consideration and requires little additional calculations.

[0177] In these embodiments, this gain is only part of the energy scaling. It does not take into account the size of the room leakage / transmission boundary region. They generally include the attenuation imposed by (at least one of) the transmission boundary regions. For example, if the transmission boundary region is 2 m 2 0.25m for entrances and exits 2 This allows much more energy into the room than if it were a small window.

[0178] The gain, which represents all the attenuation between the sound source and a particular position (i.e. the listener's head or just after entering the room through the transmission boundary region), is advantageously defined to correspond to the surface area of ​​the human ear. This generally coincides with the continuous rendering using HRTF pairs. The resulting gain is then squared and multiplied by the area ratio to obtain an approximation of the energy introduced into the room, which results in the downmix coefficient of the signal i associated with the sound source under consideration.

number

[0179] There may be many room leaks through which source energy reaches the room under consideration, and these are aggregated, for example using the following formula:

number

[0180] gain g i,j can be calculated in a variety of ways, simulating distance, occlusion, and diffraction for direct path rendering, as known in the art. One example of a low-complexity method focuses only on the direct path distance attenuation from the sound source to the room leakage.

number

[0181] The audio device generates a first audio component that corresponds to a localized rendering of the reverberant signal from a source position (as a point source or a diffuse source), as previously described.

[0182] In some embodiments, the audio device is further configured to generate the second audio component by rendering the nearby room reverberation signal as a reverberation signal of the listening room. Thus, in some embodiments, the nearby room reverberation is rendered in the listening room as a combination of localized and diffuse reverberation. Such an approach, for example, in many embodiments, provides a more realistic experience of the scene.

[0183] In some embodiments, the reverberation signal is therefore fed to a path renderer 601 of renderer 507 to provide a spatially localized rendering of the nearby room reverberation signal. In addition, the nearby room reverberation signal is fed to a combiner 603 where it is combined with the reverberation signal generated for the listening room itself (and the output from path renderer 601).

[0184] The renderer 507 includes a path renderer for rendering the acoustic path propagation to the listening position, the renderer configured to provide the nearby room reverberation signal to the path renderer. The renderer 507 is further configured to combine the nearby room reverberation signal with an output of the path renderer.

[0185] The renderer 507 is configured to adapt the relative levels of the two audio components in such a case.

[0186] In many embodiments, the rendering includes adapting a level of the first audio component (reflecting a localized audio source) relative to a level of the second audio component (reflecting a diffuse non-localized reverberation) depending on the listening position relative to the first transmission boundary region. Specifically, the renderer 507 is configured to increase the level of the first audio component relative to the level of the second audio component when the distance from the listening position to the transmission boundary region / neighboring room increases. Thus, the closer the listener is to the transmission boundary region and the neighboring room, the stronger the perception of the localized sound compared to the diffuse sound contribution.

[0187] In some embodiments, the renderer is configured to adapt the level of the first audio component (reflecting a localized audio source) relative to the level of the second audio component (reflecting diffuse non-localized reverberation) depending on the geometric properties of the transmission boundary region, in particular the size of the transmission boundary region.

[0188] In many embodiments, the renderer 507 is configured to reduce the level of the first audio component relative to the level of the second audio component when the size of the transmission boundary region increases, thus resulting in a weaker perception of localized sound relative to the diffuse sound contribution, as the transmission boundary region becomes larger.

[0189] Level adaptation is used, for example, to create a gradual transition between two rooms. For example, as a user moves between rooms, a smoother and more natural transition of audio from one room to the other is often achieved. For example, a transition or cross-fade region is defined relative to the listening position, and the weighting of the localized and non-localized (diffuse) components is dynamically adapted as a function of the listening position within the region.

[0190] 7-9 show examples of sound source positions 701 and cross-fade / transition regions 703 for exemplary transmission boundary regions, where the listening room is represented by B and the neighboring room is represented by A.

[0191] In Fig. 7, the sound source is a point source and the transition region is the area around the boundary opening (represented by the transmission boundary region). In the example of Fig. 8, the sound source of the nearby room reverberation signal is a diffuse source and in the example of Fig. 9, the transition region is formed only in the nearby room.

[0192] In such an example, the relative levels of the two components are changed gradually across the transition region to provide a smooth cross-fade transition.

[0193] 10, in some embodiments, an audio device includes a path renderer 1001 configured to render an acoustic path of an audio source. The path renderer 1001 specifically implements the path renderer 601 of FIG.

[0194] The audio device further comprises a number of reverberators 1003 configured to generate reverberation signals of the rooms. The reverberators 1003 specifically comprise the reverberator 511, as well as the downmixer 605 and the reverberator 607, thereby generating reverberation signals of the neighboring rooms and the listening room, respectively. The reverberators 1003 include reverberators for generating reverberation signals of other rooms.

[0195] The audio device further includes a combining circuit 1005 configured to selectively combine reverberant signals from outputs of the multiple reverberators 1003 to the path renderer 1001. Thus, the combining circuit 1005 can couple reverberant signals, such as nearby room reverberant signals, to an input of the path renderer 1001 such that the signal is rendered as a localized signal.

[0196] The audio device further includes a combining circuit 1005 configured to selectively combine the reverberation signals from the renderer 1001 with each other, with the output signal from the path renderer 1001, and with the reverberation signal directly. The result is an audio signal representative of the audio in the scene. The combining circuit 1005 includes a combiner 603.

[0197] In this example the combiner and combiner circuit 1005 is realised by a switch which switches the output of the reverberator 1003 between the input of the path renderer 1001 and the combiner function. However, it will be appreciated that in many embodiments individual gains are used which adapt the relative gain during combining to the input of the path renderer.

[0198] The audio device further includes an adapter 1007 configured to adapt the levels of the reverberant signals for the combining and splicing, for example controlling the switches of FIG. 10 or for example controlling and adapting the gains of the paths from the reverberator 1003 to the input and output of the direct path renderer 1001.

[0199] This arrangement allows the reverberant signals to be adapted and rendered as localized sources and / or diffuse reverberations, providing a highly efficient approach that is achieved with low complexity while offering high performance and substantial flexibility.

[0200] Specifically, the adapter 1007 adapts the level of the reverberation signal to the direct path renderer input and the combination, respectively, depending on one or more of the following: a) Metadata received together with the audio data of the audio source, e.g. importance functions derived by the content creator that increase or decrease the relative levels of different rooms beyond the (physically) simulated levels. b) Acoustic characteristics of the first transmission boundary region, for example a reflection coefficient of the first transmission boundary region, the higher the reflection coefficient, the higher the relative gain of the listening room reverberation signal to the input of the combiner. c) A geometric characteristic of the first transmission boundary region, for example the surface area of ​​the first transmission boundary region, the larger the surface area, the higher the relative gain of the nearby room reverberation signal to the input of the combiner. d) The listening position, e.g. the distance from the listener to the first transmission boundary region, the smaller the distance, the smaller the relative gain of the reverberation signal to the input of the combiner. e) The acoustic distance from the listening position to the transition region boundary. f) Acoustic distance from the listening position to the sound source position. g) The size of the nearby room, e.g. the dimension of the nearby room perpendicular to the first transmission boundary region, the larger the dimension, the smaller the gain of the nearby room reverberation signal relatively to the input of the combiner.

[0201] As a specific example, in some embodiments, this technique generates reverberant audio from many rooms with widely different characteristics by running multiple reverberators in parallel, typically one reverberator for each room / acoustic environment that needs to be rendered.

[0202] For processing optimization, determining which rooms need to be rendered becomes an important aspect when the number of rooms in the rendered scene increases, for example to more than three or four rooms. This is achieved in many different ways. For quality improvement, rooms are ranked based on perceptual relevance. This is achieved by ranking rooms according to the reverberant volume at the listening position. Obviously, if the listener is in an environment with reverberant characteristics, that room is likely to be the most important room to simulate.

[0203] The number of sources in a room and their volume, ideally combined with the energy of the room (DSR) and the amount of room leakage / transmission into the listening room, play an important role. For example, the room-related number for room k is derived as follows:

number

number

[0204] The effective leakage surface is determined based on its permeability. For example,

number

[0205] More advanced, and often also more complex, formulas can be derived, for example by taking into account the distance and shadowing attenuation occurring between the sound source and the listening position in room k.

[0206] Essentially, each reverberator represents a single room. The input signal from the source in the scene is downmixed at an appropriate (relative) level to represent how much influence the input signal has in the room before the reverberator creates a reverberant signal based on it. This is often a binaural signal for playback over headphones, but may also be a multi-channel signal for loudspeaker playback.

[0207] If the listener is inside the room, this is the appropriate way to render the reverberant signal of the relevant reverberator. However, the reverberant signal from other rooms should generally not be rendered as a completely diffuse signal reaching the listener from all sides. Instead, it is rendered as a localizable source near the corresponding transmission boundary region.

[0208] This rendering of the reverberant signal is the same as for rendering a normal sound source, with the addition that distance attenuation, occlusion, diffraction, and other acoustic effects play a role. In general, the room transmission area is represented by an object with a spatial extent that matches the size of the area, so that the sound appears to come from the entire room leakage (often a door or window).

[0209] Therefore, the nearby room reverberation signal is fed into an already existing direct path renderer, which generates at least one new sound source related to the nearby room reverberation signal.

[0210] In many embodiments, the routing is controlled by a cross-fade factor where both the diffuse and reverberant source representations are active at the same time, rather than a hard switch as in Figure 10. This is used to create a smooth transition when the listener is near a room leak. With 6DoF content, the listener can often move freely from one room to another, thereby benefiting from a smooth transition from the diffuse representation to the source-based representation and vice versa.

[0211] For example, the crossfade coefficient α of the reverberation in room A xf is 0.5 for listening positions at the room boundaries, 1 for listening positions at least 1 m away from the boundary of room A, and 0 for listening positions at least 1 m away from the boundary of room B. At the same time, the crossfade factor due to the reverberation of room B has an inverse relationship: if the listener is further away from the room leakage, the crossfade factor for the listener's room is 1 and the crossfade factor for all other rooms is 0, so that the reverberation in the listener's room is completely diffuse and the reverberation in all other rooms is completely directional.

[0212] In addition, the reverberant signal of one room is fed to the early reflection and / or reverberant processing of another room. In most embodiments, these routed signals do not undergo cross-fading.

[0213] An advantageous way to achieve a suitable mapping from the outputs of the reverberators to the inputs (or early reflection inputs) of other reverberators is to use a mapping matrix. As an example, the mapping matrix maps each reverberator output signal to the inputs of all other reverberators but not to itself.

number

[0214] When multiple transmission boundary regions exist, the same reverberant output signal may be processed for multiple reverberant sources, i.e. the same signal is used to render multiple reverberant sources. This is achieved by generating multiple reverberant sources with reference to the same signal.

[0215] When switching or crossfading between diffuse and directional representations of the reverberant signal, it is desirable to match these different renderings so that artifacts do not occur. Spatial crossfading helps with this, since it is often difficult to mask with a hard switch. The minimal artifact reduction technique of the embodiment with hard switching between representations has hysteresis, and there is a spatial distance between the threshold for switching from room A to room B and the threshold for switching from room B to room A.

[0216] Furthermore, level matching is advantageous. In many embodiments, it is beneficial to ensure that the signal levels of both representations are stable and similar throughout the cross-fade region. This can be achieved, for example, by adjusting the reference distance (d ref ) and the minimum listener-sound source distance (d min ) equal to the (common, average, or maximum) vertical distance from the crossfade boundary to the reverberant sound source.

[0217] In general, it is not necessary to have stable levels throughout the entire crossfade region: some embodiments match levels only in certain sub-regions.

[0218] Many other embodiments aim to dramatically fade the reverberation level to a lower volume as the listener moves outside the room.

[0219] As the size of the transmission boundary region increases, the reverberant volume of other rooms increases. That is, a large door lets through more reverberant energy than a small window. To introduce this effect, the signal rendered as a localizable source is scaled according to the size of the room leakage. The signal without the extra gain represents the reference room leakage size. For example, A ref =4m 2 Different sized room leaks are assigned gains proportional to the ratio of the room leak size to this reference room leak size.

number

[0220] Alternatively, the source spread rendering uses a level normalization mode that achieves a larger source volume for a larger spread, e.g., to compensate for the amount of point sources, the signal rendered as a point source across the spread is not attenuated, or the combined signal power represented by the point sources across the spread is reduced by the gain g from the above formula. rls Ensure that the system scales according to the

[0221] The terms audio and sound are considered equivalent and interchangeable, both referring to the physical sound pressure and / or electrical signal representation, respectively, as appropriate in the context.

[0222] It will be appreciated that the above description has described embodiments of the invention with reference to different functional circuits, units, and processors for clarity. However, it will be apparent that any suitable distribution of functionality between different functional circuits, units, or processors may be used without detracting from the invention. For example, functionality shown to be performed by separate processors or controllers may be performed by the same processor or controller. References to specific functional units or circuits should therefore be understood only as references to suitable means for providing the described functionality, rather than to any strict logical or physical structure or organization.

[0223] The invention may be implemented in any suitable form including hardware, software, firmware or any combination of these. The invention is optionally implemented at least partly as computer software running on one or more data processors and / or digital signal processors. The elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable manner. Indeed, functionality may be implemented in a single unit, in several units or as part of other functional units. Thus, the invention may be implemented in a single unit or may be physically and functionally distributed between different units, circuits and processors.

[0224] Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form described herein. Rather, the scope of the present invention is limited only by the appended claims. In addition, although certain features appear to be described in connection with specific embodiments, those skilled in the art will recognize that various features of the described embodiments may be combined in accordance with the present invention. In the claims, the term "comprises, includes, has" does not exclude the presence of other elements or steps.

[0225] Furthermore, although individually listed, a plurality of means, elements, circuits, or method steps may be implemented by, for example, a single circuit, unit, or processor. In addition, although individual features may be included in different claims, they may, possibly, be advantageously combined, and their inclusion in different claims does not imply that the combination of features is not feasible and / or advantageous. Moreover, the inclusion of a feature in one category of claims does not imply a limitation to this category, but rather indicates that the feature is equally applicable to other claim categories, as appropriate. Moreover, the order of features in the claims does not imply a particular order in which the feature must be functioned, and in particular the order of individual steps in a method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order. In addition, singular references do not exclude a plurality. Thus, "a", "first", "second", etc. do not exclude a plurality. Reference signs in the claims are merely provided as a clarification of examples and should not be construed as limiting the scope of the claims in any way.

[0226] Generally, examples of audio devices and methods therefor are illustrated by the following embodiments. Embodiment Claim 1 a first receiver (501) configured to receive audio data of an audio source of a scene including many rooms; a position circuit (505) configured to determine a listening position within the scene; a determiner (509) configured to determine a first room that includes the listening position and a second room that is a neighboring room of the first room; a second receiver (503) configured to receive spatial acoustic transmission data between the first room and the second room, the spatial acoustic transmission data describing a number of transmission boundary regions of the first room, each transmission boundary region having an acoustic transmission level of sound from the second room to the first room exceeding a threshold; a first reverberator (511) configured to determine a second room reverberation audio signal for the second room from at least one audio source in the second room and at least one characteristic of the second room; a source circuit (513) configured to determine a source position of an audio source in the second room with respect to at least a first one of the several transmission boundary regions; a renderer (507) configured to render an audio signal for a listening position, the rendering including generating a first audio component by rendering a reverberant audio signal of a second room from a sound source position; Audio equipment including: Claim 2 2. The audio device of claim 1, wherein the rendering depends on at least one of geometrical and acoustical properties of the first transmission bounding region. Claim 3 An audio device as described in claim 1 or 2, wherein the distance from the sound source position to the first transmission boundary region is at least one-tenth of the maximum distance within the first transmission boundary region. Claim 4 4. An audio device as claimed in any one of claims 1 to 3, wherein the distance from the sound source position to the first transmission boundary region is at least one tenth of the maximum distance within the second room. Claim 5 5. An audio device as claimed in any one of claims 1 to 4, wherein the distance from the sound source position to the first transmission boundary region is 20 cm or more. Claim 6 6. The audio device of any one of claims 1 to 5, wherein rendering includes rendering for an acoustic path from a sound source position through a first transmission boundary region to a listening position. Claim 7 7. The audio apparatus of any one of claims 1 to 6, wherein the rendering includes generating the second audio component by rendering the second room reverberation signal as a reverberant audio component. Claim 8 8. The audio device of claim 7, wherein the rendering includes adapting a level of the first audio component relative to a level of the second audio component depending on a listening position relative to the first transmission boundary region. Claim 9 9. The audio apparatus of claim 8, wherein the rendering includes increasing a level of the first audio component relative to a level of the second audio component for increasing distance to the second room. Claim 10 10. An audio device as claimed in any one of claims 7 to 9, wherein the rendering includes level adapting the first audio component relative to a level of the second audio component depending on a size of the first transmission boundary region. Claim 11 11. The audio device of any one of claims 1 to 10, wherein the renderer (507) is configured to render the second room reverberant audio signals from the sound source positions as spatially extended sound sources. Claim 12 Renderer (506) a path renderer (1001) for rendering audio of an acoustic path; a plurality of reverberators (1003) configured to generate a reverberation signal of a room, the plurality of reverberators including a first reverberator; a combining circuit (1005) for combining the reverberation signals from the multiple renderers into a path renderer (1001); a combination circuit (1005) for combining the reverberation signals from the plurality of renderers (1003) with the output signal from the path renderer (1001) to generate a combined audio signal; an adapter (1007) for adapting the level of the reverberation signal for combination by the combining circuit (1005) and the level of the reverberation signal for combination by the combining circuit (1005); 12. An audio device as claimed in any one of claims 1 to 11, comprising: Claim 13 The adapter (1007) Metadata received together with the audio data of the audio source; Acoustic characteristics of a first transmission boundary region; Geometric characteristics of the first transmission boundary region; Listening position, The acoustic distance from the listening position to the sound source position, and Size of the first transmission boundary region 13. The audio device of claim 12, configured to adapt a level of the reverberation signal in response to at least one of: Claim 14 14. The audio device of any one of claims 1 to 13, wherein the first reverberator (511) is further configured to generate a reverberation signal of a second room in response to the reverberation signal of the first room. Claim 15 1. A method of operating an audio device, the method comprising: receiving audio data of an audio source of a scene including many rooms; determining a listening position within a scene; determining a first room that includes the listening position and a second room that is a neighboring room of the first room; receiving spatial acoustic transmission data between a first room and a second room, the spatial acoustic transmission data describing a number of transmission boundary regions of the first room, each transmission boundary region having an acoustic transmission level of sound from the second room to the first room that exceeds a threshold; determining a second room reverberation audio signal for the second room from at least one audio source in the second room and at least one characteristic of the second room; determining a source position of an audio source in the second room for at least a first one of the several transmission boundary regions; rendering an audio signal for a listening position, the rendering including generating a first audio component by rendering a reverberant audio signal of a second room from a sound source position; The method comprising:

Claims

1. a first receiver for receiving audio data of an audio source of a scene including many rooms; a position circuit for determining a listening position within the scene; a determiner for determining a first room that includes the listening position and a second room that is a neighboring room of the first room; a second receiver for receiving spatial acoustic transmission data between the first room and the second room, the spatial acoustic transmission data describing several transmission boundary regions of the first room, each transmission boundary region having an acoustic transmission level of sound from the second room to the first room exceeding a threshold; a first reverberator for determining a second room reverberation audio signal for the second room from at least one audio source in the second room and at least one of a geometric characteristic of the second room and an acoustic characteristic of the second room; a source circuit for determining a source position of an audio source in the second room with respect to at least a first one of the several transmission boundary regions; a renderer for rendering an audio signal for the listening position to include a first audio component generated by rendering a reverberant audio signal of the second room from the sound source position; 2. An audio device, comprising:

2. The audio device of claim 1 , wherein the rendering depends on at least one of a geometric characteristic of the first transmission bounding region and an acoustic characteristic of the first transmission bounding region.

3. 3. The audio device according to claim 1, wherein the sound source circuit determines the sound source position subject to a requirement that the distance from the sound source position to the first transmission boundary region is at least one-tenth of the maximum distance within the first transmission boundary region.

4. 4. The audio device according to claim 1, wherein the sound source circuit determines the sound source position subject to the requirement that the distance from the sound source position to the first transmission boundary region is at least one-tenth of the maximum distance in the second room.

5. 5. The audio device according to claim 1, wherein the sound source circuit determines the sound source position on the premise that a distance from the sound source position to the first transmission boundary area is 20 cm or more.

6. 6. The audio device of claim 1, wherein the renderer generates the first audio component by rendering the reverberant audio signal of the second room along an acoustic path that passes through the first transmission boundary region from the sound source position to the listening position.

7. 7. The audio device of claim 1, wherein the rendering comprises generating a second audio component by rendering the second room reverberation signal as a reverberant audio component.

8. 8. The audio device of claim 7, wherein the rendering comprises adapting a level of the first audio component relative to a level of the second audio component depending on the listening position relative to a position of the first transmission boundary region.

9. 9. The audio device of claim 8, wherein the rendering comprises increasing a level of the first audio component relative to a level of the second audio component for an increasing distance from the listening position to a second room.

10. 10. An audio device according to claim 7, wherein the rendering comprises adapting a level of the first audio component relative to a level of the second audio component depending on a size of the first transmission boundary region.

11. 11. An audio device according to claim 1, wherein the renderer is adapted to render the second room reverberant audio signal from the source position as a spatially extended source.

12. The renderer: a path renderer for rendering audio for an acoustic path; a plurality of reverberators for generating a reverberant audio signal of a room, the plurality of reverberators including the first reverberator for generating a reverberant audio signal of the second room; a combining circuit for combining reverberation signals from the plurality of reverberators into the path renderer; a combining circuit for combining reverberation signals from the plurality of reverberators, including a reverberant audio signal of the second room, and an output signal from the path renderer to generate a combined audio signal; an adapter for adapting a level of the reverberation signal for the combination by the combining circuit and a level of the reverberation signal for the combination by the combining circuit; 12. An audio device according to any one of claims 1 to 11, comprising:

13. The adapter: Metadata received together with the audio data from the audio source; an acoustic characteristic of the first transmission boundary region; a geometric characteristic of the first transmitting boundary region; The listening position, The acoustic distance from the listening position to the sound source position, and The size of the first transmission boundary region 13. The audio device of claim 12, adapted to adapt a level of the reverberant signal in response to at least one of:

14. 14. The audio device of claim 1, wherein the first reverberator is further configured to generate a reverberation signal of the second room in response to a reverberation signal of the first room.

15. 15. An audio device according to claim 1, wherein the sound source circuitry generates the sound source position based on a position of the first transmission boundary region.

16. 1. A method of operating an audio device, the method comprising: receiving audio data for an audio source of a scene including many rooms; determining a listening position within the scene; determining a first room that includes the listening position and a second room that is a neighboring room of the first room; receiving spatial acoustic transmission data between the first room and the second room, the spatial acoustic transmission data describing a number of transmission boundary regions of the first room, each transmission boundary region having an acoustic transmission level of sound from the second room to the first room exceeding a threshold; determining a second room reverberation audio signal for the second room from at least one audio source in the second room and at least one of a geometric characteristic of the second room and an acoustic characteristic of the second room; determining a source position of an audio source in the second room for at least a first one of the several transmission boundary regions; rendering an audio signal for the listening position to include a first audio component generated by rendering a reverberant audio signal of the second room from the sound source position; The method comprising:

Images