Spatial audio rendering of reverberation
By leveraging active portal connections and reverb coupling in augmented and virtual reality, the problem of unrealistic reproduction of reverb characteristics in acoustic environments is solved, achieving more realistic audio rendering and environmental impressions.
Patent Information
- Application Number
- CN202380026058.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-07
- Filing Date
- 2023-02-22
- Publication Date
- 2024-11-05
AI Technical Summary
The prior art is difficult to accurately reproduce the reverberation characteristics of different acoustic environments in augmented reality and virtual reality, resulting in unreal environmental impressions.
By establishing an active portal connection between at least two acoustic environments, using reverb coupling and feedback attenuation filters, the reverb signal is dynamically adjusted to generate a more realistic audio output, ensuring feedback-free of reverb signals between different acoustic environments and real rendering.
Improves the realism and environmental impression of audio rendering, avoids acoustic feedback issues, and enhances the audio experience in virtual and real environments.
Smart Images

Figure CN118901249A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to an apparatus and method for spatial audio rendering of reverberation, but is not limited to an apparatus and method for spatial audio rendering of reverberation in an augmented reality and / or virtual reality device. Background Art
[0002] Reverberation is the persistence of sound in a space after the actual sound source has ceased. Different spaces are characterized by different reverberation properties. In order to convey the spatial impression of an environment, it is very important to reproduce the reverberation perceptually accurately. Room acoustics are usually modeled with a separately synthesized early reflection component and a statistical model for the diffuse late reverberation. Figure 1 An example of a synthesized room impulse response is depicted, where direct sound 101 is followed by discrete early reflections 103 with directions of arrival (DOA) and diffuse late reverberation 105 that can be synthesized without any specific direction of arrival. Figure 1 The delay d1(t) 102 in can be considered as indicating the direct sound arrival delay from the source to the listener, while the delay d2(t) 104 can indicate the delay from the source to the listener for one of the early reflections (in this case the first arriving reflection).
[0003] One way to reproduce reverberation is to use a set of N loudspeakers (or virtual loudspeakers reproduced in a binaural manner using a set of head-related transfer functions (HRTFs). The loudspeakers are relatively evenly located around the listener. Mutually incoherent reverberation signals are reproduced from these loudspeakers, creating the perception of ambient diffuse reverberation.
[0004] The reverberations produced by different loudspeakers must be mutually incoherent. In a simple case, reverberations can be produced using different channels of the same reverberator, where the output channels are uncorrelated but share the same acoustic properties, such as RT60 time and level (specifically, diffuse to direct ratio or reverberant to direct ratio). For example, such uncorrelated outputs with the same acoustic properties can be obtained from the output taps of a feedback delay network (FDN) reverberator by appropriately tuning the delay line lengths, or from a reverberator based on the use of decaying uncorrelated noise sequences by using different uncorrelated noise sequences in each channel. In this case, the different reverberation signals actually have the same characteristics, and the reverberations are generally considered to be similar in all directions.
[0005] The reverberation spectrum or level can be controlled using the diffuse to direct ratio (DDR), which describes the ratio of the energy (or level) of the reverberant sound energy to the direct sound energy (or the total emitted energy of the sound source). Summary of the invention
[0006] According to a first aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising means configured to: obtain a listener position; determine a first acoustic environment of the at least two acoustic environments based on the listener position; determine at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; obtain at least one second acoustic environment input audio signal; and generate at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0007] The module can be configured to: obtain at least one third acoustic environment input audio signal; determine at least one other acoustic environment of the at least two acoustic environments, and the at least one other acoustic environment is coupled to the determined at least one second acoustic environment; and wherein, the module configured to obtain the at least one second acoustic environment input audio signal can be configured to generate the at least one second acoustic environment input audio signal based at least in part on the application of processing of the at least one third acoustic environment input audio signal by at least one third acoustic environment reverberator.
[0008] The at least one third acoustic environment may be associated with at least one third acoustic environment reverberator parameter for configuring the at least one third acoustic environment reverberator.
[0009] The first acoustic environment may be associated with at least one first acoustic environment reverberator parameter for configuring the first acoustic environment reverberator, and the second acoustic environment may be associated with at least one second acoustic environment reverberator parameter for configuring the at least one second acoustic environment reverberator, wherein a module configured to generate at least one listener output audio signal based at least in part on application of processing by the at least one second acoustic environment reverberator and then the first acoustic environment reverberator to the at least one second acoustic environment input audio signal may be configured to: generate the second acoustic environment output audio signal based on application of the second acoustic environment reverberator configured by the at least one second acoustic environment reverberator parameter to the at least one second acoustic environment input audio signal; and generate an output audio signal based on application of the first acoustic environment reverberator configured by the at least one first acoustic environment reverberator parameter to the second acoustic environment output audio signal.
[0010] The active portal connection may be one of: a unidirectional connection from the second acoustic environment to the first acoustic environment; and a bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
[0011] The active portal connection may be defined relative to a distance within the first acoustic environment such that a module configured to determine the second of the at least two acoustic environments based on the first acoustic environment may be configured to determine that the listener position is less than the distance within the first acoustic environment.
[0012] The module can be further configured to: determine at least one audio element within the at least two acoustic environments; and configure the distance within the first acoustic environment based at least in part on the determination of at least one audio element within the at least two acoustic environments, so that the distance is increased when there is no audio element in the at least two acoustic environments.
[0013] The module configured to determine the at least one audio element may be configured to: obtain bitstream information defining the at least one audio element; and determine the at least one audio element based on the bitstream information.
[0014] The module may be further configured to obtain a list of active portal connections for each acoustic environment, wherein the list identifies, for a specified acoustic environment, any acoustic environments that are acoustically coupled with the specified acoustic environment.
[0015] The module configured to determine a second acoustic environment of the at least two acoustic environments based on the first acoustic environment may be configured to determine the second acoustic environment using the active portal connection list, wherein the second acoustic environment is associated with at least one second acoustic environment reverberator parameter.
[0016] The active portal connection list for each acoustic environment may include priority information associated with the active portal connection list for each acoustic environment, wherein the module configured to determine the second acoustic environment among the at least two acoustic environments based on the first acoustic environment may be configured to determine the second acoustic environment based on the priority information.
[0017] The at least one second acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0018] The at least one first acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0019] The module may be further configured to apply one of the following to the audio signal output by the second acoustic environment prior to applying the first acoustic environment reverberator: a gain control multiplier; and a feedback attenuation filter.
[0020] According to a second aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising a module configured to: select a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determine at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment through the active portal connection; and generate a bitstream, the bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled through the active portal connection.
[0021] The module may be configured to determine at least one other acoustic environment of the at least two acoustic environments, the at least one other acoustic environment being coupled to the determined at least one second acoustic environment and being associated with at least one further acoustic environment reverberator, and wherein the bitstream may further include information identifying that the at least one further acoustic environment is coupled to the at least one second acoustic environment.
[0022] The active portal connection may be one of: a unidirectional connection from the second acoustic environment to the first acoustic environment; a bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
[0023] The active portal connection may be defined relative to a distance within the first acoustic environment, wherein the bitstream further comprises the distance within the first acoustic environment.
[0024] The information may be a list of active portal connections for each acoustic environment, wherein the list identifies, for a specified acoustic environment, any acoustic environments that are acoustically coupled with the specified acoustic environment.
[0025] The information may further include at least one reverberator parameter for said configuring the acoustic environment reverberator.
[0026] The information may further include priority information associated with the active portal connection list for each acoustic environment.
[0027] The at least one second acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0028] The at least one first acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0029] The information may also include information associated with at least one of the active portal connections for configuring at least one of: a gain control multiplier; and a feedback attenuation filter.
[0030] According to a third aspect, a method for an apparatus for assisting spatial rendering in at least two acoustic environments is provided, the method comprising: obtaining a listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; obtaining at least one second acoustic environment input audio signal; and generating at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0031] The method may further include: obtaining at least one third acoustic environment input audio signal; determining at least one other acoustic environment of the at least two acoustic environments, the at least one other acoustic environment being coupled to the determined at least one second acoustic environment; and wherein obtaining the at least one second acoustic environment input audio signal may include generating the at least one second acoustic environment input audio signal based at least in part on the application of processing of the at least one third acoustic environment input audio signal by at least one third acoustic environment reverberator.
[0032] The at least one third acoustic environment may be associated with at least one third acoustic environment reverberator parameter for configuring the at least one third acoustic environment reverberator.
[0033] The first acoustic environment may be associated with at least one first acoustic environment reverberator parameter for configuring the first acoustic environment reverberator, and the second acoustic environment may be associated with at least one second acoustic environment reverberator parameter for configuring the at least one second acoustic environment reverberator, wherein generating at least one listener output audio signal based at least in part on application of processing by the at least one second acoustic environment reverberator and then the first acoustic environment reverberator to the at least one second acoustic environment input audio signal may include: generating a second acoustic environment output audio signal based on application of the second acoustic environment reverberator configured by the at least one second acoustic environment reverberator parameter to the at least one second acoustic environment input audio signal; and generating an output audio signal based on application of the first acoustic environment reverberator configured by the at least one first acoustic environment reverberator parameter to the second acoustic environment output audio signal.
[0034] The active portal connection may be one of: a unidirectional connection from the second acoustic environment to the first acoustic environment; and a bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
[0035] The active portal connection may be defined relative to a distance within the first acoustic environment such that determining the second of the at least two acoustic environments based on the first acoustic environment may include determining that the listener position is less than the distance within the first acoustic environment.
[0036] The method may further include: determining at least one audio element within the at least two acoustic environments; and configuring the distance within the first acoustic environment based at least in part on the determination of at least one audio element within the at least two acoustic environments, so that the distance is increased when there is no audio element in the at least two acoustic environments.
[0037] Determining the at least one audio element may include: acquiring bitstream information defining the at least one audio element; and determining the at least one audio element based on the bitstream information.
[0038] The method may also include obtaining a list of the active portal connections for each acoustic environment, wherein the list identifies, for a specified acoustic environment, any acoustic environments that are acoustically coupled with the specified acoustic environment.
[0039] Determining a second acoustic environment of the at least two acoustic environments based on the first acoustic environment may include determining the second acoustic environment using the active portal connection list, wherein the second acoustic environment is associated with at least one second acoustic environment reverberator parameter.
[0040] The active portal connection list for each acoustic environment may include priority information associated with the active portal connection list for each acoustic environment, wherein determining a second acoustic environment among the at least two acoustic environments based on the first acoustic environment may include determining the second acoustic environment based on the priority information.
[0041] The at least one second acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0042] The at least one first acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0043] The method may further comprise applying one of the following to the audio signal output by the second acoustic environment prior to applying the first acoustic environment reverberator: a gain control multiplier; and a feedback attenuation filter.
[0044] According to a fourth aspect, a method for an apparatus for assisting spatial rendering in at least two acoustic environments is provided, the method comprising: selecting a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment through the active portal connection; and generating a bitstream, the bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled through the active portal connection.
[0045] At least one other acoustic environment of the at least two acoustic environments is determined, the at least one other acoustic environment being coupled to the determined at least one second acoustic environment and being associated with at least one other acoustic environment reverberator, and wherein the bitstream may also include information identifying the at least one other acoustic environment and being coupled to the at least one second acoustic environment.
[0046] The active portal connection may be one of: a unidirectional connection from the second acoustic environment to the first acoustic environment; a bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
[0047] The active portal connection may be defined relative to a distance within the first acoustic environment, wherein the bitstream further comprises the distance within the first acoustic environment.
[0048] The information may be a list of the active portal connections for each acoustic environment, wherein the list identifies, for a given acoustic environment, any acoustic environments that are acoustically coupled with the given acoustic environment.
[0049] The information may also include at least one reverberator parameter for configuring the acoustic environment reverberator.
[0050] The information may also include priority information associated with the list of active portal connections for each acoustic environment.
[0051] The at least one second acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0052] The at least one first acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0053] The information may also include information associated with at least one of the active portal connections for configuring at least one of: a gain control multiplier; and a feedback attenuation filter.
[0054] According to a fifth aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, enable the apparatus to at least: obtain a listener position; determine a first acoustic environment of the at least two acoustic environments based on the listener position; determine at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; obtain at least one second acoustic environment input audio signal; and generate at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then by the first acoustic environment reverberator.
[0055] The device may be caused to: acquire at least one third acoustic environment input audio signal; determine at least one other acoustic environment of the at least two acoustic environments, the at least one other acoustic environment being coupled to the determined at least one second acoustic environment; and wherein the device caused to acquire the at least one second acoustic environment input audio signal may be caused to generate the at least one second acoustic environment input audio signal based at least in part on the application of processing of the at least one third acoustic environment input audio signal by at least one third acoustic environment reverberator.
[0056] The at least one third acoustic environment may be associated with at least one third acoustic environment reverberator parameter for configuring the at least one third acoustic environment reverberator.
[0057] The first acoustic environment may be associated with at least one first acoustic environment reverberator parameter for configuring the first acoustic environment reverberator, and the second acoustic environment may be associated with at least one second acoustic environment reverberator parameter for configuring the at least one second acoustic environment reverberator, wherein the apparatus caused to generate at least one listener output audio signal based at least in part on application of processing by the at least one second acoustic environment reverberator and then the first acoustic environment reverberator to the at least one second acoustic environment input audio signal may be caused to: generate the second acoustic environment output audio signal based on application of the second acoustic environment reverberator configured by the at least one second acoustic environment reverberator parameter to the at least one second acoustic environment input audio signal; and generate the output audio signal based on application of the first acoustic environment reverberator configured by the at least one first acoustic environment reverberator parameter to the second acoustic environment output audio signal.
[0058] The active portal connection may be one of: a unidirectional connection from the second acoustic environment to the first acoustic environment; and a bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
[0059] The active portal connection may be defined relative to a distance within the first acoustic environment, and the apparatus caused to determine the second of the at least two acoustic environments based on the first acoustic environment may be caused to determine that the listener position is less than the distance within the first acoustic environment.
[0060] The device may be further caused to: determine at least one audio element within the at least two acoustic environments; and configure the distance within the first acoustic environment based at least in part on the determination of at least one audio element within the at least two acoustic environments, so that the distance is increased when there is no audio element in the at least two acoustic environments.
[0061] The apparatus caused to determine the at least one audio element may be caused to: obtain bitstream information defining at least one audio element; and determine the at least one audio element based on the bitstream information.
[0062] The apparatus may be further caused to obtain a list of active portal connections for each acoustic environment, wherein the list identifies, for a specified acoustic environment, any acoustic environments that are acoustically coupled with the specified acoustic environment.
[0063] The apparatus caused to determine a second acoustic environment of the at least two acoustic environments based on the first acoustic environment may be caused to determine the second acoustic environment using the active portal connection list, wherein the second acoustic environment is associated with at least one second acoustic environment reverberator parameter.
[0064] The active portal connection list for each acoustic environment may include priority information associated with the active portal connection list for each acoustic environment, wherein the device that is caused to determine the second acoustic environment among the at least two acoustic environments based on the first acoustic environment may be caused to determine the second acoustic environment based on the priority information.
[0065] The at least one second acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0066] The at least one first acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0067] The apparatus may be further caused to apply one of the following to the audio signal output by the second acoustic environment prior to applying the first acoustic environment reverberator: a gain control multiplier; and a feedback attenuation filter.
[0068] According to a sixth aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least: select a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determine at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment via the active portal connection; and generate a bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled via the active portal connection.
[0069] The apparatus may be caused to determine at least one other acoustic environment of the at least two acoustic environments, the at least one other acoustic environment being coupled to the determined at least one second acoustic environment and being associated with at least one other acoustic environment reverberator, and wherein the bitstream may also include information identifying that the at least one other acoustic environment is coupled to the at least one second acoustic environment.
[0070] The active portal connection may be one of: a unidirectional connection from the second acoustic environment to the first acoustic environment; a bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
[0071] The active portal connection may be defined relative to a distance within the first acoustic environment, wherein the bitstream also includes the distance within the first acoustic environment.
[0072] The information may be a list of active portal connections for each acoustic environment, wherein the list identifies, for a specified acoustic environment, any acoustic environments that are acoustically coupled with the specified acoustic environment.
[0073] The information may also include at least one reverberator parameter for configuring the acoustic environment reverberator.
[0074] The information may also include priority information associated with the active portal connection list for each acoustic environment.
[0075] The at least one second acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0076] The at least one first acoustic environment reverberator may include at least one filter configured to attenuate at least one frequency.
[0077] The information may also include information associated with at least one of the active portal connections for configuring at least one of: a gain control multiplier; and a feedback attenuation filter.
[0078] According to a seventh aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising: an acquisition circuit configured to acquire a listener position; a determination circuit configured to determine a first acoustic environment of the at least two acoustic environments based on the listener position; a determination circuit configured to determine at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; an acquisition circuit configured to acquire at least one second acoustic environment input audio signal; and a generation circuit configured to generate at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0079] According to an eighth aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising: a selection circuit configured to select a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; a determination circuit configured to determine at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment via an active portal connection; and a generation circuit configured to generate a bitstream, the bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled via the active portal connection.
[0080] According to a ninth aspect, there is provided a computer program [or a computer-readable medium comprising program instructions] comprising instructions for causing an apparatus for assisting spatial rendering in at least two acoustic environments to perform at least the following operations: obtaining a listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; obtaining at least one second acoustic environment input audio signal; and generating at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0081] According to a tenth aspect, there is provided a computer program [or a computer-readable medium including program instructions] comprising instructions for causing an apparatus for assisting spatial rendering in at least two acoustic environments to perform at least the following operations: selecting a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment via an active portal connection; and generating a bitstream, the bitstream including information identifying the first acoustic environment and the at least one second acoustic environment coupled via the active portal connection.
[0082] According to an eleventh aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing an apparatus for assisting spatial rendering in at least two acoustic environments to perform at least the following operations: obtaining a listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; obtaining at least one second acoustic environment input audio signal; and generating at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0083] According to a twelfth aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing an apparatus for assisting spatial rendering in at least two acoustic environments to perform at least the following operations: selecting a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment via an active portal connection; and generating a bitstream, the bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled via the active portal connection.
[0084] According to a thirteenth aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising: a module configured to obtain a listener position; a module configured to determine a first acoustic environment of the at least two acoustic environments based on the listener position; a module configured to determine at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; a module configured to obtain at least one second acoustic environment input audio signal; and a module configured to generate at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0085] According to a fourteenth aspect, there is provided an apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising: a module configured to select a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; a module configured to determine at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment via an active portal connection; and a module configured to generate a bitstream, the bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled via the active portal connection.
[0086] According to a fifteenth aspect, a computer-readable medium is provided, comprising program instructions for causing an apparatus for assisting spatial rendering in at least two acoustic environments to perform at least the following operations: obtaining a listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment via an active portal connection; obtaining at least one second acoustic environment input audio signal; and generating at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by at least one second acoustic environment reverberator and then the first acoustic environment reverberator.
[0087] According to a sixteenth aspect, a computer-readable medium is provided, comprising program instructions for causing an apparatus for assisting spatial rendering in at least two acoustic environments to perform at least the following operations: selecting a first acoustic environment among the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment among the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment via an active portal connection; and generating a bitstream, the bitstream comprising information identifying the first acoustic environment and the at least one second acoustic environment coupled via the active portal connection.
[0088] A device includes modules for performing the actions of the method described above.
[0089] A device is configured to perform the actions of the method described above.
[0090] A computer program comprises program instructions for causing a computer to execute the method as described above.
[0091] A computer program product stored on a medium can cause an apparatus to execute the method described herein.
[0092] An electronic device may include the apparatus described herein.
[0093] A chipset may include the device described herein.
[0094] The embodiments of the present application are intended to solve the problems associated with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0095] For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings, in which:
[0096] Figure 1 Models of room acoustics and room impulse response are shown;
[0097] Figure 2 An example environment in which embodiments may be implemented is shown, wherein an audio scene with an audio portal or acoustic coupling is shown;
[0098] Figure 3 schematically illustrates example apparatus in which some embodiments may be implemented;
[0099] Figure 4 It shows that Figure 3 A flow chart of the operation of the illustrated example apparatus;
[0100] Figure 5 Schematically illustrates how Figure 3 Example reverb controller shown;
[0101] Figure 6 It shows that Figure 5 A flow chart of the operation of an example reverberator controller is shown;
[0102] Figure 7 Schematically illustrates how Figure 5 The example active portal connection determiner shown;
[0103] Figure 8 It shows that Figure 7 A flow chart of the operation of an example active portal connection determiner is shown;
[0104] Figures 9a to 9c An example of portal connection is schematically shown;
[0105] Fig.10 Schematically illustrates how Figure 3 Example reverb shown;
[0106] Fig.11It shows that Fig.10 A flow chart of the operation of an example reverberator is shown;
[0107] Fig.12 Schematically illustrates how Figure 3 The example reverberator output signal spatialization controller shown;
[0108] Fig.13 It shows that Fig.12 A flow chart of the operation of an example reverberator output signal spatialization controller is shown;
[0109] Fig.14 Schematically illustrates how Figure 3 The example reverberator output signal shown is a spatializer;
[0110] Fig.15 It shows that Fig.14 A flow chart of the operation of an example reverberator output signal spatializer is shown;
[0111] Fig.16 Schematically illustrates how Fig.11 Example FDN reverberator shown;
[0112] Fig.17 Schematically illustrates a method for Fig.11 Example feedback filter designer for the FDN reverberator shown;
[0113] Fig.18 It shows that Fig.18 A flow chart of the operation of an example feedback filter designer is shown;
[0114] Fig.19 schematically illustrates an example apparatus with transmission and / or storage in which some embodiments may be implemented; and
[0115] Fig. 20 An example apparatus suitable for implementing the means shown in the preceding figures is shown. DETAILED DESCRIPTION
[0116] Suitable means and possible mechanisms for parameterizing and rendering an audio scene with reverberation are described in further detail below.Thus, in some embodiments there is a means for rendering spatial audio in at least two acoustic environments or for controlling a reverberator in a spatial audio rendering.
[0117] As mentioned above, reverberation can be rendered using, for example, a Feedback Delay Network (FDN) reverberator with appropriate delay line length tuning. FDN allows for individual control of the reverberation time (RT60) and the energy of different frequency bands. It can therefore be used to render reverberation based on the characteristics of a room or modeled space. The reverberation time and the energy of different frequencies are affected by the frequency-dependent absorption characteristics of the room.
[0118] As mentioned above, the reverberation spectrum or level can be controlled using the diffuse to direct ratio, which describes the ratio of the energy (or level) of the reverberant sound energy to the direct sound energy (or the total emitted energy of the sound source). In ISO / IEC JTC1 / SC29 / WG6 N00054 MPEG-I Immersive Audio Encoder Input Format, the input to the encoder is provided in the form of a DDR value, which indicates the ratio of the diffuse (reverberant) sound energy to the total emitted energy of the sound source. Another well-known measurement is RDR, which refers to Reverberation to Direct Ratio, and can be measured from the impulse response. The relationship between these two is described in ISO / IEC JTC1 / SC29 / WG6N0083 MPEG-I Immersive Audio CfP Supplementary Information, Recommendations and Specifications, Version 1:
[0119] 10*log10(DDR)=10*log10(RDR)–41dB.
[0120] refer to Figure 1 , the RDR can be calculated by
[0121] - Add the squares of the sample values of the diffuse late reverberation part 105
[0122] - Add the squares of the sample values of the direct sound part 101
[0123] - Calculate the ratio of these two sums to derive the RDR.
[0124] The logarithmic RDR can be obtained as 10*log10(RDR).
[0125] In a virtual environment for virtual reality (VR) or a real physical environment for augmented reality (AR), there can be multiple acoustic environments, each with its own reverberation parameters that can be different in different acoustic environments. Such an environment can be rendered by multiple reverberators running in parallel so that a reverberator instance runs in each acoustic environment. As the listener moves in the environment, the current environment reverberation is rendered as an enveloping spatial sound around the user, while the reverberation from nearby acoustic spaces is rendered through so-called acoustic portals. Acoustic portals reproduce the reverberation of nearby acoustic environments as spatially extended sound sources.
[0126] Figure 2An example of such an environment is shown. In this example, an audio scene is shown, which includes a first acoustic environment AE1 203, a second acoustic environment AE2 205 and an outdoor 201. An acoustic coupling AC1 207 between the first acoustic environment AE1 203 and the second acoustic environment AE2 205 is shown.
[0127] In this example, the sound or audio sources 210 are located within the second acoustic environment AE2 205. In this example, the audio sources 210 include a first audio source, drummer S1 2103, and a second audio source, guitarist S2 2102. The listener 202 is further shown moving in the audio scene and at a position P1 2001 in the first acoustic environment AE1 203, a position P2 2002 in the second acoustic environment AE2 205, and a position P3 2003 in the outdoor 201.
[0128] Several virtual (for VR) or physical (for AR) acoustic environments can be rendered by multiple digital reverberators running in parallel, each reverberator reproducing reverberation according to the characteristics of the acoustic environment. In addition, the environments can provide input to each other through so-called portals. This means that when an acoustic environment AE1 203 is connected to a second acoustic environment AE2 205 through a portal or acoustic coupling AC1 207, the reverberation output of AE1 203 should also be reverberated with the reverberator associated with AE2 205. This will increase the realism of the audio reproduction, because in a real connected enclosure, the reverberated sound from the enclosure will leak into the connected enclosure and reverberate there. The reverberated sound of the adjacent environment emitted through the portal may sound too dry if the reverberated sound from the adjacent environment is not reverberated.
[0129] Since portals or acoustic couplings (e.g. AC1) are usually bidirectional (similar to real acoustic enclosures connected by openings), the reverberation output of AE2 205 should also be fed into the reverberator associated with AE1 203, because when the listener is in AE1 203, it is better to let the acoustic characteristics of AE1 203 also affect the reverberation sound of AE2 205 and prevent the reverberation signal of AE2 205 emitted through the portal from sounding too dry. However, this will create acoustic feedback problems, where certain audible frequencies are recycled between the acoustic environments and amplified uncontrollably, resulting in annoying artifacts. This can also be called a positive feedback-loop. Typical artifacts are over-amplified frequency components of the source signal that are reverberated in the reverberator. Such artifacts will produce a howling sound that sounds very annoying.
[0130] GB2200043.4 describes the control of activating / prioritizing reverberators, which specifically discusses a mechanism for prioritizing reverberators and activating only a subset of reverberators based on the prioritization. GB2200335.4 also describes a method for adjusting the reverberation level, especially in augmented reality (AR) rendering. WO2021186107 describes the use of FDN for late reverberation modeling from acoustic environment information, and specifically describes the design of DDR filters to adjust the late reverberation level based on input DDR data. GB2020673.6 describes a method and apparatus for fusing a virtual scene description and a listener space description in a bitstream for 6DoF rendering and specifically for late reverberation modeling of immersive audio scenes, wherein the acoustic environment is a combination of a virtual scene specified by the content creator and listening space parameters affected by the listener consumption space. Therefore, this background describes a method for rendering in an AR audio scene that includes virtual scene description acoustic parameters and real-world listening space acoustic parameters. GB2101657.1 describes how to derive post reverberation rendering filter parameters for low latency renderer applications. GB2116093.2 discusses the reproduction of diffuse reverberation, wherein a method is proposed that enables the reproduction of a rotatable diffuse reverberation using a plurality of processing paths (at least 3, typically 6-20 paths) (virtual) multi-channel signal, wherein the characteristics of the reverberation may be direction-dependent (i.e., having different reverberation characteristics in different directions) by the following operations: by determining at least two panning gains based on a target direction and a position of a (virtual) speaker in a (virtual) speaker group (e.g., using VBAP), obtaining a mutually incoherent reverberation signal for each of the determined gains (e.g., using the output of two reverberators tuned to produce mutually incoherent outputs, or using a decorrelator), applying the determined gains to the corresponding obtained reverberation signals to obtain a reverberant multi-channel signal, combining the reverberant multi-channel signals from different processing paths, and reproducing the combined reverberant multi-channel signal from the corresponding (virtual) speakers. GB2115533.8 discusses a method for seamless listener transfer between acoustic environments.
[0131] Concepts discussed in further detail in the embodiments below relate to reproducing late reverberation in a 6DoF audio rendering system when at least two acoustic environments are present, wherein second-order reverberation (or reverberation of connected acoustic environments) is enabled by rendering the reverberation of a first acoustic environment having a reverberation signal from a second acoustic environment as input without causing acoustic feedback.
[0132] In some embodiments, this may be accomplished by apparatus and methods configured to:
[0133] Get listener position;
[0134] Acquiring a first acoustic environment (listener AE) based on a listener position;
[0135] acquiring at least one second acoustic environment connected to the current acoustic environment based on the first acoustic environment;
[0136] obtaining reverberators corresponding to the first and second acoustic environments;
[0137] receiving an output signal of a reverberator corresponding to a second acoustic environment;
[0138] providing the output signal (of the digital reverberator corresponding to the second acoustic environment) as an input signal to the reverberator corresponding to the first acoustic environment; and
[0139] Using the input signal, a reverberator corresponding to the first acoustic environment is used for post reverberation.
[0140] In some embodiments, the apparatus and method are configured to create an active portal connection list for each acoustic environment corresponding to a listener being in the acoustic environment.
[0141] Furthermore, in some embodiments, the method and apparatus are configured to determine the acoustic environment to which the current listener acoustic environment is connected by processing an active portal connection list associated with the acoustic environment to which the current listener acoustic environment is connected.
[0142] In some embodiments, the portal connections in the list are one-way connections.
[0143] Furthermore, in some embodiments, the apparatus and method are configured to dynamically determine the depth to which the acoustic environment needs to be added to the active portal connection list.
[0144] In some embodiments, the depth may be based at least in part on the presence of an audio element in the acoustic environment, such that the depth is increased when there are no audio elements in the acoustic environment.
[0145] Furthermore, in some embodiments, the determination of whether an audio element is present in the acoustic environment is implemented based on the bitstream information.
[0146] Thus, in some embodiments, there is priority information associated with a plurality of active portal connection lists that is obtained or determined, and the apparatus or method selects an active portal connection list to apply based on the priority information.
[0147] In some embodiments, a filter is designed to attenuate frequencies that may be over-amplified due to feedback. The parameters of such a filter may be associated with the virtual scene or with an opening between two acoustic enclosures in the scene.
[0148] In some embodiments, there is at least one of a gain control multiplier or a feedback reduction filter applied to the output of a reverberator before it is input to another reverberator to control (reduce) the amount of feedback.
[0149] It is understood that ISO / IEC 23090-4MPEG-I Audio Phase2 will standardize bitstream and renderer processing. There will also be an encoder reference implementation, but as long as the output bitstream follows the specification, it can be modified later. This allows the quality of the codec to be improved through new encoder implementations even after the standard is finalized.
[0150] For the embodiments described herein, the sections that go over the different parts of the ISO / IEC 23090-4 standard may be as follows: The specification bitstream may contain an active portal connection structure corresponding to each acoustic environment that a listener can reach. The specification bitstream may also contain information about the depth of the connected acoustic environments that need to be rendered, the number of audio elements in the acoustic environments, and the gain coefficients or feedback attenuation filters used for the acoustic environment connections; and the specification renderer may decode the bitstream to obtain scene and reverberator parameters, use the reverberator parameters to initialize the reverberator for rendering, determine the connection information between the acoustic environments, and use the reverberator and active portal connection information to render the reverberation signal.
[0151] about Figure 3 , showing an example apparatus system suitable for implementing some embodiments.
[0152] The inputs to the apparatus system are scene and reverberator parameters 300, listener gesture parameters 302, and an audio signal 306. The apparatus system generates a reverberation signal 314 as an output (e.g., binauralized using head-related transfer function (HRTF) filtering for reproduction to headphones, or panned using vector basis amplitude panning (VBAP) for reproduction to loudspeakers).
[0153] In some embodiments, the apparatus includes a reverberator controller 301. The reverberator controller 301 is configured to obtain or receive scene and reverberator parameters 300. In some embodiments, the reverberator parameters are in the form of enclosure room geometry and parameters of a digital feedback delay network (FDN) reverberator. In some embodiments, the scene and reverberator parameters also include the location of the enclosure room geometry (or acoustic environment) so that the apparatus or method can determine the distance and orientation between the listener and each acoustic environment or between the acoustic environments. In addition, in some embodiments, the scene and reverberator parameters 300 are configured to include the location and geometry of portals so that sound can be transferred between acoustic environments.
[0154] The reverberator controller 301 is also configured to determine and generate reverberator connection information indicating which (several) reverberators should be connected to each other. This active portal / reverberator connection information 304 may change over time and is passed to the reverberator 305 and the reverberator output signal spatialization controller 303. To determine the active portal / reverberator connection information 304, in some embodiments, the reverberator controller 301 is configured to utilize any active portal connection information from the scene and reverberator parameters 300 and the listener posture parameters 302, from which the controller is configured to determine the listener's current position in the virtual scene and which acoustic environments are connected to the current acoustic environment in the active portal connection information.
[0155] Thus, the active portal / reverberator connection 304 signal or information may be forwarded to the reverberator 305, which is also configured to receive the audio signal s in (t) (where t is time) 306 and the reverberator parameters from the scene and the reverberator parameters 300 are used as input to initialize the FDN reverberator to reproduce the reverberation according to the reverberator parameters.
[0156] Each reverberator within the reverberator is configured to reproduce reverberation according to the characteristics (reverberation time and level) of the acoustic environment from which the corresponding reverberator parameters are derived.
[0157] In an example embodiment, reverberator parameters are derived by an encoder based on acoustic environment parameters and written into the bitstream. Figure 3 An example embodiment receives the bitstream.
[0158] Therefore, in these embodiments, the reverberator 305 is configured to reverberate the audio signal 306 based on the reverberator parameters and the reverberator connection 304. Details of the reverberation process will be further described below.
[0159] The resulting reverberator output signal s rev,r (j, t) 310 (where j is the output audio channel index and r is the reverberator index) is the output of the reverberator. There are multiple reverberators, each of which generates several output audio signals.
[0160] Furthermore, the apparatus includes a reverberator output signal spatialization controller 303 configured to generate a reverberator output channel position 312. The reverberator output signal spatialization controller 303 is configured to receive the listener posture parameter 302, the active portal / reverberator connection 304, and the scene and reverberator parameters 300, and generate the reverberator output channel position 312. In some embodiments, the reverberator output channel position 312 indicates the position of the reverberator output channel when rendering the s rev,r Each signal in (j, t) is the Cartesian coordinate to be used. In some other embodiments, other representations (or other coordinate systems) may be used, such as polar coordinates.
[0161] The reverberator output signal 310 is input to the reverberator output signal spatializer 307 which is configured to generate an output signal suitable for reproduction through headphones or loudspeakers. The reverberator output signal spatializer 307 is also configured to also receive reverberator output channel position 312 information from the reverberator output signal spatialization controller 303 .
[0162] In some embodiments, the reverberator output channel position 312 indicates that the reverberator output channel position 312 is not rev,r Each signal in (j, t) is the Cartesian coordinate to be used. In alternative embodiments, other coordinates or representations may be used, such as polar coordinates.
[0163] The reverberator output signal spatializer 307 is configured to render each reverberator output signal into a desired output format (e.g., binaural) and then add the signals to produce an output reverberation signal 314. For example, for binaural reproduction, the reverberator output signal spatializer 307 may be configured to use HRTF filtering to render the reverberator output signals 310 in their desired positions indicated by the reverberator output channel positions 312.
[0164] In some embodiments, the reverberation in the reverberation signal 314 is based on the desired scene and reverberator parameters 310 , and takes into account the listener posture parameters 302 .
[0165] about Figure 4 , showing that according to some embodiments Figure 3 Flowchart of the operation of the example apparatus shown in .
[0166] Thus, for example, the method may include obtaining scene and reverberator parameters and obtaining listener posture parameters, such as Figure 4 As shown in step 401.
[0167] In addition, obtaining audio signals such as Figure 4 As shown in step 403.
[0168] Then, the active portal / reverberator connection control is determined based on the acquired scene and reverberator parameters and the listener posture parameters, such as Figure 4 As shown in step 405.
[0169] The reverberator controlled by the active portal / reverberator connection control is then applied to the audio signal, e.g. Figure 4 As shown in step 407.
[0170] Furthermore, a reverberator output signal spatialization control is determined based on the acquired scene and reverberator parameters, listener posture parameters, and active portal / reverberator connection controls, such as Figure 4 As shown in step 409.
[0171] Then, the reverberator spatialization based on the reverberator output signal spatialization control can be applied to the reverberant audio signal from the reverberator to generate an output reverberant audio signal such as Figure 4 As shown in step 411.
[0172] Then, the output reverberation audio signal is as follows Figure 4 As shown in step 413, it is output.
[0173] about Figure 5 , further details an example reverberator controller 301. In some embodiments, the reverberator controller 301 includes an active portal connection determiner 501. The active portal connection determiner 501 is configured to receive scene parameters 500 from the scene and reverberator parameters 300, and determine active portal connection information 506 (for each acoustic environment) that can be passed to the reverberator connection determiner 503.
[0174] Furthermore, the reverberator controller 301 comprises a reverberator connection determiner 503. The reverberator connection determiner 503 is configured to receive active portal connection information 506 (for each acoustic environment) and the listener gesture parameter 302, and to generate active portal / reverberator connection 304 information.
[0175] about Figure 6 , showing that according to some embodiments Figure 5 A flow chart of the operation of an example reverberator controller 301 is shown in FIG.
[0176] Therefore, in some embodiments, Figure 6 As shown in step 601, scene parameters are obtained.
[0177] Then, based on the scenario parameters, the method is configured to determine the active portal connection information (for each acoustic environment), such as Figure 6 As shown in step 603.
[0178] Furthermore, in some embodiments, Figure 6 As shown in step 605, listener posture parameters or information are obtained.
[0179] After acquiring the listener posture parameters and the active portal connection information (for each acoustic environment) 506, the method may be configured to determine or generate active portal / reverberator connection information, such as Figure 6 As shown in step 607.
[0180] The method then includes outputting the generated active portal / reverberator connection information, such as Figure 6 As shown in step 609.
[0181] Figure 7 The schematic diagram further shows in detail Figure 5 The active portal connection determiner 501 is shown. The active portal connection determiner 501 is configured to determine a list of identifiers of active portal connections when a listener is in a defined acoustic environment (which can be identified by a starting acoustic environment identifier value - startAEId).
[0182] In some embodiments, the active portal connection determiner 501 includes a portal connection determiner 701 configured to identify a portal and an acoustic environment to which the portal is connected or coupled from scene description geometry information in the bitstream and encoder generated connection information (which may be a subset of scene geometry related information).
[0183] In addition, the active portal connection determiner 501 includes a portal connection list generator 703, which is configured to generate an identifier list from the output of the portal connection determiner 701, the identifier list being configured to identify active portal connections when the listener is in the acoustic environment identified by the identifier-startAEId. The identifier list of active portal connections when the listener is in startAEId506 can then be output.
[0184] about Figure 8 , showing Figure 7 A flowchart of the operation of an example active portal connection determiner 501 is shown in FIG.
[0185] In some embodiments, the method may include obtaining scene description geometry information in the bitstream and connection information generated by the encoder (a subset of scene geometry related information), such as Figure 8 As shown in step 801.
[0186] Then determine the portal connection based on the information obtained, such as Figure 8 As shown in step 803.
[0187] The operation of looping through the portal connections to determine the active set of portals for each acoustic environment (when the listener is in it) is as follows Figure 8 As shown in step 805.
[0188] Then output the set of active portals listed, such as Figure 8 As shown in step 807.
[0189] In some embodiments, the determined active portal connections may thus be a portal connection list to indicate connections between acoustic environments. In some embodiments, even if the acoustic connections are two-way or bidirectional, the list may be generated as a unidirectional connection to facilitate feedback-free reverberation rendering.
[0190] For example, Figure 9aA series of acoustic environments, {AE1 901, AE2 903, AE3 905, AE4 907}, are schematically shown, wherein there is a two-way connection 902 between AE1 901 and AE2 903, a two-way connection 904 between AE2 903 and AE3 905, and a two-way connection 906 between AE2 903 and AE4 907. If reverberation rendering is implemented based on these two-way connections, there may be acoustic feedback problems, for example, the reverberation from AE1 901 and AE2 903 will return to the reverberator of AE1 901 again, and so on, which may lead to undesired amplification of certain frequencies and may cause, for example, howling artifacts.
[0191] In contrast, in the following embodiments, a series of unidirectional active connection data structures are created for possible scenarios of a listener being in each acoustic environment.
[0192] Figure 9b The active acoustic environment connections are depicted when the listener is in AE1 901. In this example, there is a one-way connection or coupling 912 from AE2 903 to AE1 901, a one-way connection 914 from AE3 905 to AE2 903, and a one-way connection 916 from AE4 907 to AE2 903. In this example, there is no one-way connection 922 from AE1 901 to AE2 903 because this would result in reciprocal feedback between AE1 901 and AE2 903.
[0193] Fig.9c Active acoustic environment connections are depicted when the listener is in AE4 907. In this example, there is a unidirectional connection or coupling 922 from AE1 901 to AE2 903, a unidirectional connection 914 from AE3 905 to AE2 903, and a unidirectional connection 926 from AE4 907 to AE2 903. Correspondingly, two additional sets of active connections will be created in the four acoustic environments, corresponding to the situations when the listener is in AE2 903 and AE3 905 (not shown).
[0194] In some embodiments, a list data structure corresponding to each different acoustic environment that a listener can be in can be created as follows. As input, the method receives a data structure indicating pairs of portal connections between acoustic environments. The data structure is obtained from the bitstream and is derived by the encoder by analyzing the scene geometry. The method first iterates over all portal connections and obtains the first acoustic environment in each connection as startAEId.
[0195] For each acoustic environment startAEId{
[0196] For each portal p connected to startAEId{
[0197] If p's source is not startAEId{
[0198] Add p to the activity portal for startAEId
[0199] Add the source AE of portal p to the adjacent AE set
[0200] }
[0201] }
[0202] For each acoustic environment AE_a in the adjacent AE,
[0203] For each portal p connected to AE_a
[0204] If the source of p is not startAEId and the source of p is not another AE in the adjacent AEs {
[0205] Add p to the activity portal for startAEId
[0206] Add the source AE of portal p to the adjacent AE set
[0207] }
[0208] }
[0209] }
[0210] }
[0211] Thus, the above process creates a tuple of (startAEId, set {identifiers of active portals when the listener is in acoustic environment startAEId}).
[0212] Fig.10 Schematically further details are shown Figure 3 The reverberator 305 includes a reverberator initializer 1001 configured to receive reverberator parameters and active portal connection information 304 / 506. The reverberator initializer 1001 is configured to configure or initialize the reverberator and control the processing of the reverberator based on activation, and the parameters of the reverberator are provided in the reverberator parameters.
[0213] In some embodiments, as described in further detail below, the reverberator parameters are parameters for an FDN reverberator.
[0214] In some embodiments, the reverberator includes a reverberator processor 1003. In some embodiments, the reverberator processor includes FDN reverberators (as shown later), each of which is configured by a reverberator initializer 1001. In such embodiments, after the parameters of the FDN are provided, the audio signal 306 is input to the reverberator processor 1003 to produce a reverberator output signal 310 having the desired reverberation characteristics. In some embodiments, there are multiple FDN reverberators running in parallel. For example, in some embodiments, an FDN may be used for each acoustic environment. In such embodiments, each FDN may then be configured with its own parameters and its own output signal.
[0215] According to the determined active portal connection information, the reverberator outputs the signal s rev,r (j, t) 310 is provided as at least part of the input to the other reverberators. If the acoustic environment corresponding to reverberator r is connected to the acoustic environment corresponding to reverberator k through a portal, and the corresponding portal connection is currently active based on the active portal connection information, then s rev,r (j, t) will be provided as the input signal to the reverberator k. In the embodiment, when the reverberator accepts mono input, the output signal s rev,r (j, t) is the sum for output channel j.
[0216] Providing the output of reverberator r as input to reverberator k has the desired effect, namely, the reverberated output s rev,r (j,t) is reverberated again with reverberator k.
[0217] For example, consider a virtual scene that includes a main hall (with a reverberator r) and an entrance room (with a reverberator k). In this case, it is desired that the reverberant sound of the main hall is also reverberated in the entrance room.
[0218] Thus, in some embodiments, the routing of the reverberator output signal may be activated based on active portal connection information that is different for each listener acoustic environment. Based on the listener position, the reverberator controller 301 determines the listener acoustic environment identifier, which is used as startAEId to look up the active portal connection from the tuple (startAEId, set {identifiers of active portals when the listener is in the acoustic environment startAEId}). The determination of the listener acoustic environment identifier may be performed by determining whether the listener position is at least partially located within a geometric element (e.g., a box or grid enclosing the acoustic environment) and then obtaining an identifier associated with the acoustic environment. For such a determination whether the position is within a defined geometric shape, for example, a known geometric containment check may be used. Alternatively or additionally, the determination whether the listener is in the acoustic environment may be performed based at least in part on the distance between the listener position and the center of the acoustic environment enclosing the geometric shape or one or more boundaries thereof. The active portal connection for this acoustic environment may then be used to look up the identifier of the connected acoustic environment and its reverberator identifier. The outputs of the reverberators whose identifiers are found based on the active connection information are added and fed into the reverberator currently being processed by the reverberator 305 .
[0219] The summed output signal can be multiplied by the coefficient connectedReverbGainMultiplier or ConnectedReverbMultiplier (depending on which is available, see later for details). This multiplier can be used to adjust the reverb to a suitable level. The multiplier value can be adjusted by the encoder and can be adjusted so that no feedback occurs.
[0220] The summed output signal can optionally be filtered using a feedback attenuation filter. The coefficients for the feedback attenuation filter can be taken from the bitstream (see FeedbackAttenuationFilterStruct later). The design process of such a filter will be described later. Such a filter can be applied if some feedback sound artifacts are present despite the presence of a unidirectionally connected second-order reverberation rendering.
[0221] The resulting reverberator output signal s rev,r (j,t) 310 (where j is the output audio channel index and r is the reverberator index) is the output of the reverberator 305 .
[0222] about Fig.11 , a flow chart showing the operation of the reverberator 305 according to some embodiments.
[0223] For example, a reverberator is configured to obtain reverberator parameters and active portal connections, such as Fig.11 As shown in step 1101.
[0224] Then, the reverberator whose parameters are provided in the reverberator parameters is initialized, such as Fig.11 As shown in step 1103.
[0225] The method can then be configured to obtain an audio signal such as Fig.11 As shown in step 1105.
[0226] Then, the reverberators are looped through, and if this reverberator has a connected reverberator, the output of the connected reverberator is taken as the input signal and processed to produce the output signal of this reverberator, such as Fig.11 As shown in step 1107 in .
[0227] The output of the reverberator output signal (reverberated audio signal) is as follows Fig.12 As shown in step 1109 in .
[0228] about Fig.12 , further details are shown in the reverberator output signal spatialization controller 303, such as Figure 3 shown.
[0229] The reverberator output signal spatialization controller 303 is configured to receive the scene and reverberator parameters 300 and the listener posture parameters 302. The reverberator output signal spatialization controller 303 is configured to use the listener posture parameters 302 and the scene and reverberator parameters 300 to determine the acoustic environment in which the listener is currently located and to provide the reverberator output channels to surround the listener's position. This means that when the reverberator is located within an acoustic enclosure, the reverberation caused by the acoustic enclosure is rendered as a diffuse signal that wraps around the listener.
[0230] In some embodiments, the reverberator output signal spatialization controller 303 comprises a listener acoustic environment determiner 1201 configured to obtain scene and reverberator parameters 300 and listener gesture parameters 302 and to determine the listener acoustic environment.
[0231] In some embodiments, the reverberator output signal spatialization controller 303 includes a listener reverberator corresponding to the listener acoustic environment determiner 1203, which is further configured to determine the listener reverberator corresponding to the listener acoustic environment information.
[0232] In some embodiments, the reverberator output signal spatialization controller 303 includes a head tracking output position for a listener reverberator provider 1205 which is configured to provide or determine a head tracking output position for a listener.
[0233] In some embodiments, the reverberator output signal spatialization controller 303 includes an acoustic portal determiner (directly connected from active portal to listener acoustic environment) 1207 configured to obtain active portal connection 316 information and determine the portal connected to any acoustic environment connected to the listener acoustic environment.
[0234] In some embodiments, the reverberator output signal spatialization controller 303 comprises a geometry determiner and output channel position generator 1209 configured to obtain the geometry (for each portal) and provide channel positions for the connected acoustic environment reverberators based on the geometry.
[0235] Therefore, the output of the reverberator output signal spatialization controller 303 is the reverberator output channel position 312 .
[0236] about Fig.13 , shows an example reverberator output signal spatialization controller 303 (e.g. Fig.12 ) operation.
[0237] Thus, for example, Fig.13 Step 1303 shows obtaining scene and reverberator parameters, and Fig.13 Step 1301 shows obtaining listener posture parameters.
[0238] The method then includes determining the listener acoustic environment, such as Fig.13 As shown in step 1305.
[0239] After confirming, Fig.13 As shown in step 1307 , a listener reverberator corresponding to the listener's acoustic environment is determined.
[0240] Furthermore, the method includes providing a head-tracked output position of the reverberator for a listener, such as Fig.13 As shown in step 1309.
[0241] The method includes obtaining an active portal connection, such as Fig.13 As shown in step 1311.
[0242] The method includes determining an active portal directly connected to the listener's acoustic environment from the active portal, such as Fig.13 As shown in step 1313.
[0243] The method then includes obtaining the geometry for each portal found and providing channel positions for the connected acoustic environment reverberator based on the geometry, such as Fig.13 As shown in step 1315.
[0244] Then, if Fig.13As shown in step 1317, the output reverberator outputs the channel position.
[0245] In some embodiments, the reverberator corresponding to the acoustic environment in which the user is currently located is rendered by the reverberator output signal spatializer 307 as an immersive audio signal surrounding the user. rev,r The signal in (j,t) is rendered as a point source around the listener.
[0246] Other reverberators may be audible in the current environment through the acoustic portal. The reverberator output signal spatialization controller 303 uses the portal position information carried in the scene parameters to provide appropriate positions in the reverberator output channel positions for the reverberator outputs corresponding to the portals. In order to obtain a spatially extended perception of the portal sound, the output channels corresponding to the reverberators to be rendered at the portals are provided at positions along the portal geometry dividing the two acoustic spaces.
[0247] The Reverberator Output Signal Spatialization Controller uses the listener pose and scene as well as the reverberator parameters to determine the acoustic environment the listener is currently in and provides the reverberator output channel positions around the listener. This means that when the reverberator is located within an acoustic enclosure, the reverberation caused by that acoustic enclosure is rendered as a diffuse signal that wraps around the listener.
[0248] Other reverberators can be made audible in the current environment in two ways. The first is to combine Fig.10 The output of the connected room reverberator is fed as an input signal into the current environment reverberator, and after reverberation with the current room reverberator, the signal is audible in the immersive output signal of the current room reverberator.
[0249] A second way to make adjacent acoustic environments audible in the current environment is through directional portal outputs. In some embodiments, the reverberator output signal spatialization controller uses the portal position information carried in the scene parameters to provide appropriate positions in the reverberator output channel positions for the reverberator outputs corresponding to the portals. In order to obtain a spatially extended perception of the portal sound, the output channels corresponding to the reverberator to be rendered at the portal are provided with positions along the portal geometry that divides the two acoustic spaces. The reverberator controller may provide active portal connection information to the reverberator output signal spatialization controller, and based thereon may determine the currently active portal for the listener's acoustic environment.
[0250] refer to Fig.14 , wherein the reverberator output signal spatializer 307 is shown in further detail. The reverberator output signal spatializer 307 is configured to receive the position 312 from the reverberator output signal spatialization controller 303. Furthermore, the reverberator output signal 310 from the reverberator 305 is also received.
[0251] In some embodiments, the reverberator output signal spatializer includes a head-related transfer function (HRTF) filter 1401 configured to render each reverberator output into a desired output format (eg, binaural).
[0252] Furthermore, in some embodiments, the reverberator output signal spatializer includes an output channel combiner 1403 configured to combine (or sum) the signals to generate the output reverberation signal 314 .
[0253] Thus, for example for binaural reproduction, the reverberator output signal spatializer 307 may use HRTF filtering to render the reverberator output signal at the desired position of the reverberator output signal indicated by the reverberator output channel positions.
[0254] about Fig.15 , a flow chart showing the operation of a reverberator output signal spatializer according to some embodiments.
[0255] Thus, the method may include Fig.15 Step 1500 shows obtaining the reverberator output signal, and Fig.15 Step 1501 shows obtaining the output channel position of the reverberator.
[0256] The method may then include applying a HRTF filter configured by the reverberator output channel position to the reverberator output signal, such as Fig.15 As shown in step 1503.
[0257] The method may then include summing or combining the output channels, such as Fig.15 As shown in step 1505.
[0258] Then, the reverberated audio signal can be output as Fig.15 As shown in step 1507.
[0259] refer to Fig.16, shows an example reverberator (or reverberator processor 1003) according to some embodiments. The reverberator 1003 is enabled or configured to produce a reverberation whose characteristics match the room parameters. There may be multiple such reverberators, each of which is parameterized based on the reverberation characteristics of the acoustic environment. An example reverberator implementation includes a feedback delay network (FDN) reverberator and a DDR control filter, which enables the reproduction of reverberation with a desired frequency-dependent RT60 time and level. The room parameters are used to adjust the FDN reverberator parameters so that it produces the desired RT60 time and level. An example of a level parameter may be a direct scattering ratio (DDR) (or scattering to total energy ratio used in ISO / IEC 23090-4). The output of the FDN reverberator is a reverberated audio signal, which is then reproduced as two output signals for binaural headphone reproduction, and for speaker output, it means typically more than two output audio signals. For example, multiple outputs (e.g., 15 FDN delay line outputs) can be reproduced as binaural outputs by HRTF filtering.
[0260] Fig.16 An example FDN reverberator is shown in detail, which can be used to generate D uncorrelated output audio signals. In this example, each output signal can be rendered at a certain spatial position around the listener to obtain an enveloping reverberation perception.
[0261] The example FDN reverberator is configured such that the reverberation parameters are processed to generate the coefficients GEQ of each attenuation filter 1661 d (GEQ1, GEQ2, ...GEQ D ), the coefficient A of the feedback matrix 1657, and the length m of the D delay lines 1659 d (m1, m2, ...m D ), and DDR energy ratio control filter coefficient GEQ ddr The example FDN reverberator 1003 thus illustrates a D channel output by providing the output from each FDN delay line as a separate output.
[0262] In some embodiments, each attenuation filter GEQ d 1661 is implemented as a graphic EQ filter using M biquad IIR band filters. Since the octave band M=10, the parameters of each graphic EQ include feedforward and feedback coefficients for the biquad IIR filter, the gain for the biquad band filter, and the total gain. In some embodiments, any suitable method can be implemented to determine the FDN reverberator parameters, for example, the method described in GB patent application GB2101657.1 can be implemented to derive the FDN reverberator parameters so that the RT60 time required for the virtual / physical scene can be reproduced.
[0263] The reverberator uses a time delay network 1659, feedback elements (shown as attenuation filter 1761, feedback matrix 1657 and combiner 1655, and output gain 1663) to generate a very dense impulse response for the later part. Input samples 1651 are input to the reverberator to produce a reverberant audio signal component, which can then be output.
[0264] The FDN reverberator includes multiple recirculating delay lines. A unitary matrix A 1657 is used to control the recirculation in the network. The attenuation filters 1661 can be implemented as graphic EQ filters in some embodiments. These filters are implemented as cascades of second-order partial IIR filters, which can facilitate controlling the energy decay rate of different frequencies. The filters 1661 are designed so that they attenuate the required decibel amount when each pulse passes through the delay line, and so that the required RT60 time is obtained. Therefore, the input of the encoder can provide the RT60 time required for each specified frequency f, denoted as RT60(f). For frequency f, the required attenuation per signal sample is calculated as attenuationPerSample(f) = -60 / (samplingRate*RT60(f)). The length is m d The decibel attenuation of the delay line is attenuationDb(f)=m d *attenuationPerSample(f).
[0265] For each delay line, the attenuation filter is designed as a cascade of graphic equalizer filters, e.g. and J. Liski, “Accurate cascade graphic equalizer” (IEEE Signal Processing Letters, Vol. 24, No. 2, pp. 176-180, February 2017). The design process outlined in the paper cited above takes as input a set of octave band command gains. There are also methods for a similar graphic EQ structure that can support one-third octave bands, increase the number of biquad filters to 31, and provide better matching to match detailed target responses, such as J. Liski, “Third-octave and Bark graphic-equalizer design with symmetric band filters,” in Applied Sciences (Switzerland), Vol. 10, No. 4, 1222. https: / / doi.org / 10.3390 / app10041222.
[0266] Furthermore, in some embodiments, The design process of J. Liski, “Accurate cascade graphic equalizer” (IEEE Signal Processing Letters, Vol. 24, No. 2, pp. 176-180, February 2017) is also used to design the filter GEQ for reverberation DDR control DDR The input to the design process is the reverberation gain (in decibels).
[0267] The parameters of the FDN reverberator can be adjusted to produce a reverberation with characteristics that match the input room parameters. For this reverberator, the parameters include each decay filter GEQ d , coefficients of 1661, feedback matrix coefficients A 1657, length m for D delay lines 1659 d , and the spatial position of the delay line d.
[0268] The number of delay lines D can be adjusted according to the quality requirements and the desired trade-off between reverberation quality and computational complexity. In one embodiment, an efficient implementation with D=15 delay lines is used. This enables the definition of the feedback matrix coefficients A in the form of Galois sequences that facilitate efficient implementations, as proposed by Rocchesso in "Maximally Dispersive but Efficient Feedback Delay Networks for Artificial Reverberation" (IEEE Signal Processing Letters, Vol. 4, No. 9, September 1997).
[0269] The length of the delay line d is m d It can be determined based on the virtual indoor dimensions. For example, a shoebox (or cuboid) shaped indoor can be defined with dimensions xDim, yDim, zDim. If the indoor is not cuboid shaped (or shaped like a shoebox), then the shoebox or cuboid can be installed into the indoor, and the dimensions of the installed shoebox can be used for the delay line length. Alternatively, the dimensions can be obtained as the three longest dimensions in a non-shoebox shaped indoor, or other suitable methods can be used.
[0270] In some embodiments, the delay can be set to be proportional to the resonant frequency of the standing wave in the virtual room or the physical room. In some embodiments, the delay line length m d Can be further configured to be mutually prime.
[0271] The parameters of the FDN reverberator are adjusted so that the reverberation it produces has the characteristics of RT60 and DDR that match the acoustic environment that the FDN reverberator is to be associated with. The adjustment of the parameters is done by the encoder in our current implementation of the VR scene and written to the bitstream, as well as in the renderer of the AR scene.
[0272] The reverberation ratio parameter may be referred to as the diffuse to total energy ratio (DDR) or the reverberation to direct ratio (RDR) or other equivalent representations. The ratio parameter may be equivalently represented on a linear scale or a logarithmic scale.
[0273] Fig.17 An example feedback attenuation filter determiner configured to determine an attenuation filter for a connection between acoustic environments is schematically illustrated.
[0274] In some embodiments, the filter determiner comprises reverberation parameters for two acoustic environments determiner 1701. In some embodiments, the reverberation parameters for two acoustic environments determiner 1701 is configured to take the input signal 306 and the scene and reverberation parameters 300 and determine reverberation parameters for two acoustic environments.
[0275] In some embodiments, the filter determiner further includes a second order reverberation signal renderer (a reverberator using the input signal and two acoustic environments) 1703 configured to render a second order reverberation signal.
[0276] The filter determiner also includes a reference reverberation signal determiner (using at least one of the reverberators of the two acoustic environments) 1705 configured to determine a reference reverberation signal.
[0277] The filter determiner further comprises a difference determiner 1707 configured to determine a difference between the second order reverberation signal and a reference reverberation signal caused by feedback.
[0278] Furthermore, the filter determiner comprises a coefficient determiner 1709 configured to generate or determine coefficients of a filter that can be used to attenuate the differences.These feedback attenuation filter parameters may then be output.
[0279] about Fig.18 , showing a schematic diagram of a method according to some embodiments Fig.17 Example operation of the filter determiner shown in .
[0280] The method includes obtaining an input signal such as Fig.18 As shown in step 1801.
[0281] Then get the scene and reverberator parameters like Fig.18 As shown in step 1802.
[0282] After obtaining the parameters and input signals, the reverberator parameters for the two acoustic environments are determined, such as Fig.18 As shown in step 1803.
[0283] The method may then include rendering a second-order reverberation signal (using the input signal and the reverberators of the two acoustic environments), such as Fig.18As shown in step 1805.
[0284] Then, the method comprises determining a reference reverberation signal (using at least one reverberator of the two acoustic environments), such as Fig.18 As shown in step 1807.
[0285] Furthermore, the method may include determining a difference (a difference between the second order reverberation signal and a reference reverberation signal caused by feedback), such as Fig.18 As shown in step 1809.
[0286] The method may then include determining coefficients (of a filter that may be used to attenuate the difference), such as Fig.18 As shown in step 1811.
[0287] Then, the feedback attenuation filter parameters can be output as Fig.18 As shown in step 1813.
[0288] The method may be performed in an encoder device and the coefficients of the feedback attenuation filter may be signaled in the bitstream (see FeedbackAttenuationFilterStruct). The method for feedback attenuation filter design is as shown above, first determining the reverberator parameters for the reverberators of the two connected acoustic environments. This may be the same steps as those used to obtain the reverberator parameters for these acoustic environments described above. The method then renders a reverberation signal including second-order reverberation from the two acoustic environments using the above method. The input signal may be a suitable wideband signal, such as an impulse or white noise. The method then determines a reference reverberation signal using at least one of the two reverberators of the two acoustic environments. The reference signal does not have second-order reverberation (or its possible feedback artifacts). The method then proceeds to determine a difference signal between the second-order reverberation signal and the reference signal without second-order reverberation. The difference signal may be, for example, a logarithmic spectral difference over an octave band or a third octave band. The method then proceeds to design a filter to attenuate the difference signal. For example, if the difference signal contains high spectral peaks due to feedback from a second order reverberation, the designed filter can be a notch filter with the frequency of the notch corresponding to the amplified peak in the difference signal. The same design process and graphic EQ filter structure as previously used for the attenuation filter or DDR control filter design of a digital reverberator can be used. Alternatively, the feedback attenuation filter can be a dedicated notch filter.
[0289] about Fig.19 , a system is shown in which an embodiment is implemented in an encoder device 1901, which performs part of the functions; writes data to a bitstream 1921 and sends it to a renderer device 1941, which decodes the bitstream, performs reverberator processing according to the embodiment and outputs audio for listening to headphones.
[0290] Fig.19 The encoder side 1901 of can be executed on a content creator computer and / or a network server computer. The output of the encoder is a bitstream 1921, which can be used for downloading or streaming. The decoder / renderer 1941 function runs on the end-user device, which can be a mobile device, personal computer, sound bar, tablet, car media system, home HiFi or theater system, head mounted display for AR or VR, smart watch or any system suitable for audio consumption.
[0291] The encoder 1901 is configured to receive a virtual scene description 1900 and an audio signal 1904. The virtual scene description 1900 may be provided in an MPEG-I encoder input format (EIF) or other suitable format. Typically, the virtual scene description contains an acoustically relevant description of the content of the virtual scene, and includes, for example, scene geometry as a grid, acoustic materials, an acoustic environment with reverberation parameters, the location of sound sources, and other audio element related parameters (e.g., whether reverberation is to be rendered for the audio element). In some embodiments, the encoder 1901 includes a reverberation parameter determiner 1911, which is configured to receive the virtual scene description 1900 and is configured to obtain reverberation parameters. In one embodiment, the reverberation parameters may be obtained from RT60, DDR, pre-delay, and area / enclosure parameters of the acoustic environment.
[0292] In some embodiments, the encoder 1901 further comprises a reverberation payload encoder 1913 configured to obtain the determined reverberation parameters and reverberation ratio processing parameters and to generate the reverberation parameters.
[0293] The encoder 1901 also includes an MPEG-H 3D audio encoder 1914 configured to take the audio signal 1904 and MPEG-H encode it and pass it to a bitstream encoder 1915 .
[0294] In some embodiments, the encoder 1901 also includes a scene and portal connection parameter acquirer 1915, which is configured to acquire scene and portal connection parameters from the virtual scene description (in the manner described above). These scene and portal connection parameters can be passed to the scene and portal connection payload encoder 1917, which is configured to generate encoded scene and portal connection parameters to be passed to the bitstream encoder 1915.
[0295] In some embodiments, encoder 1901 further includes a bitstream encoder 1915 configured to receive the output of reverberation payload encoder 1913, scene and portal connection payload encoder 1917, and the encoded audio signal from MPEG-H encoder 1914, and generate a bitstream 1921 that can be passed to a bitstream decoder 1941. In some embodiments, bitstream 1921 can be streamed to an end-user device or made available for download or storage.
[0296] In some embodiments, decoder 1941 includes a bitstream decoder 1951 configured to decode a bitstream.
[0297] The decoder 1941 may also include a reverb payload decoder 1953 configured to obtain the encoded reverb parameters and decode these parameters in an opposite or inverse operation to the reverb payload encoder 1913 .
[0298] The listening space description LSDF generator 1971 is configured to generate LSDF information and pass the LSDF information to the reverberator controller 1955 and the reverberator output signal spatialization controller 1959 .
[0299] In addition, the head posture generator 1957 receives information from a head mounted device or similar device and generates head posture information or parameters, which can be passed to the reverberator controller 1955, the reverberator output signal spatialization controller 1959 and the HRTF processor 1963.
[0300] In some embodiments, decoder 1941 includes a reverberator controller 1955 , which also receives the output of scene, portal, and reverberation payload decoders 1953 and generates reverberation parameters for configuring the reverberator and passes them to reverberator 1961 .
[0301] In some embodiments, the decoder 1941 includes a reverberator output signal spatialization controller 1959 configured to configure the reverberator output signal spatializer 1962 .
[0302] The decoder 1941 includes an MPEG-H 3D audio decoder 1954 configured to decode the audio signal and pass it to a (FDN) reverberator 1961 and a direct sound processing 1955 .
[0303] The decoder 1941 also includes a (FDN) reverberator 1961, which is configured by a reverberator controller 1955 and is configured to achieve appropriate reverberation of the audio signal. In some embodiments, the reverberators may be configured by the reverberation controller 1955 to prioritize and activate subsets thereof.
[0304] The output of the (FDN) reverberator 1961 is configured to be output to a reverberator output signal spatializer 1962 .
[0305] In some embodiments, the decoder 1941 includes a reverberator output signal spatializer 1962 configured to apply spatialization and output to a binaural combiner 1967 .
[0306] In addition, the decoder / renderer 1941 includes a direct sound processor 1965, which is configured to receive the decoded audio signal and is configured to implement any direct sound processing, such as air absorption and distance gain reduction, and can pass it to the HRTF processor 1963, which, through head orientation determination (from a suitable sensor 1991), can generate a direct sound component, which is passed to the binaural signal combiner 1967 together with the reverberant component from the HRTF processor 1963. The binaural signal combiner 1967 is configured to combine the direct and reverberant parts to generate a suitable output (e.g. for headphone reproduction).
[0307] Furthermore, in some embodiments, the decoder includes a head orientation determiner 1991 which passes the head orientation information to the HRTF processor 1963 .
[0308] Although not shown, various other audio processing methods may be applied, such as early reflection rendering in combination with the proposed method.
[0309] As mentioned before, ISO / IEC 23090-4 MPEG-I Audio Phase 2 will normatively standardize the bitstream and renderer processing. There will also be an encoder reference implementation, but it can be modified later as long as the output bitstream follows the specification specifications. This allows codec quality to be improved through new encoder implementations also after the standard is finalized.
[0310] The parts that go into different parts of the MPEG-I standard can be:
[0311] The encoder reference implementation will include
[0312] o Export reverberator parameters for each acoustic environment based on its RT60 and DDR
[0313] oGet the scene parameters from the encoder input and write them to the bitstream. This contains at least the position and geometry of each acoustic enclosure and the position and geometry of the acoustic portals connecting the acoustic enclosures.
[0314] o Obtain information about the active portals for each acoustic environment when the listener is in each acoustic environment.
[0315] oGet information about the presence of audio elements in the acoustic environment.
[0316] o Writes a bitstream description containing (optional) reverberator parameters and scene parameters, active portals and the presence of audio elements. If there is at least one virtual enclosure with reverberation parameters in the virtual scene description, the parameters of the corresponding reverberator are written to the reverb payload.
[0317] • A standard bitstream may contain (optional) reverberator parameters as well as second order reverberation related extensions using the syntax described below. The bitstream may be streamed to an end user device or made available for download or storage.
[0318] The canonical renderer can decode the bitstream to obtain the scene and reverberator parameters, and perform the reverberation rendering described in the present invention.
[0319] oFor VR rendering, reverberator and scene parameters are derived in the encoder and sent in the bitstream.
[0320] o For AR rendering, reverberator parameters and scenes are exported in the renderer based on the Listening Space Description Format (LSDF) file or corresponding representation.
[0321] A full specification renderer will also take other parameters related to room acoustics and sound source properties from the bitstream and use them to render direct sound, early reflections, diffraction, sound source spatial extent or width, and other acoustic effects in addition to diffuse late reverberation. The invention presented here focuses on rendering the diffuse late reverberation part, specifically how to implement second order reverberation.
[0322] Additionally, an example bitstream syntax and semantics description is shown here:
[0323] In a first example, a bitstream syntax for concatenated reverb bitstream information is shown, which carries information about the depth of concatenated reverb creation, concatenated reverb gain multiplier, presence or absence of active audio source.
[0324] grammar:
[0325] reverbPayloadStruct(){
[0326] unsigned int(2)numberOfSpatialPositions;
[0327] unsigned int(1)hasConnectedReverbDepthLimit;
[0328] unsigned int(1)hasConnectedReverbGainMultiplier;
[0329] unsigned int(8)numberOfAcousticEnvironments;
[0330] for(int i=0;i<numberOfSpatialPositions;i++){
[0331] signed int(32)azimuth;
[0332] signed int(32)elevation;
[0333] }
[0334] if(hasConnectedReverbDepthLimit){
[0335] unsigned int(3)connectedReverbDepthLimit;
[0336] reserved bits(5)=0;
[0337] }
[0338] if(hasConnectedReverbGainMultiplier)
[0339] unsigned int(8)connectedReverbGainMultiplier;
[0340] reserved bits(4)=0;
[0341] for(int i=0;i<numberOfAcousticEnvironments;i++){
[0342] unsigned int(16)environmentsId;
[0343] filterParamsStruct();
[0344] unsigned int(1)hasActiveAudioSource;
[0345] if(hasActiveAudioSource){
[0346] unsigned int(1)animated_audio_element_present;
[0347] unsigned int(1)interactive_element_present;
[0348] unsigned int(1)dynamic_update_element_present;
[0349] reserved bits(5)=0;
[0350] }
[0351] unsigned int(1)hasAESpecificConnectedReverbGainMultiplier;
[0352] if(hasAESpecificConnectedReverbGainMultiplier){
[0353] unsigned int(8)AEConnectedReverbGainMultiplier;
[0354] }
[0355] reserved bits(6)=0;
[0356] for(int j=0;j<numberOfSpatialPositions;j++){
[0357] unsigned int(32)delayLineLength;
[0358] filterParamsStruct();
[0359] }
[0360] }
[0361] }
[0362] aligned(8)filterParamsStruct(){
[0363] SOSLength;
[0364] If(SOSLength>0){
[0365] for(i=0;i<SOSLength;i++){
[0366] signed int(32)b1;
[0367] }
[0368] for(i=0;i <SOSLength;i++){
[0369] signed int(32)b2;
[0370] }
[0371] for(i=0;i <SOSLength;i++){
[0372] signed int(32)a1;
[0373] }
[0374] for(i=0;i <SOSLength;i++){
[0375] signed int(32)a2;
[0376] }
[0377] signed int(32)globalGain;
[0378] signed int(32)levelDb;
[0379] }
[0380] }
[0381] Where numberOfSpatialPositions defines the number of delay line positions used for the late reverberation payload. This value is defined using an index corresponding to the specific number of delay lines. A value of the bit string "0b00" signals to the renderer that the spatial orientation value used for the delay line is 15. The other three values "0b01", "0b10", and "0b11" are reserved.
[0382] azimuth defines the azimuth of the delay line relative to the listener. The range is between -180 and 180 degrees.
[0383] elevation defines the elevation of the delay line relative to the listener. The range is between -90 and 90 degrees.
[0384] hasConnectedReverbDepthLimit equal to 1 indicates that there is information about the number of acoustic environments available for connected reverb rendering. A value equal to 0 indicates that the renderer can use its own defaults and there is no information about this in the bitstream.
[0385] connectedReverbDepthLimit defines an upper limit on the number of acoustic environments that can be daisy-chained to feed reverb in the listener's current acoustic environment. This is an upper limit, so different renderers will dynamically choose their own criteria (depending on computational load) to choose a smaller number.
[0386] hasConnectedReverbGainMultiplier equal to 1 indicates that there is information about the reverb output gain multiplier before it is fed as input to the next reverb for connected reverb rendering. A value equal to 0 indicates that there is no such information at a global level for all acoustic environments.
[0387] ConnectedReverbGainMultiplier indicates the gain multiplier value that is applied to the total output of the reverb, which is then fed as input to the next reverb for a connected reverb rendering.
[0388] hasActiveAudioSources equal to 1 indicates that there is information about the presence of active audio sources in the specific acoustic environment. Additionally, active audio sources may be due to the presence of interactively related active audio sources in AE, animations, or static audio sources, or due to dynamic updates arriving while the render is running. A value equal to 0 indicates that there are no active audio sources in the specific acoustic environment.
[0389] hasAESpecificConnectedReverbGainMultiplier equal to 1 indicates that a gain multiplier value exists that overrides the ConnectedReverbGainMultiplier value. The override value is specified by AEConnectedReverbGainMultiplier.
[0390] numberOfAcousticEnvironments defines the number of acoustic environments in the audio scene. reverbPayloadStruct() carries information about the acoustic environment or environments present in the audio scene at the time.
[0391] environmentId This value defines a unique identifier for the acoustic environment.
[0392] delayLineLength defines the length in samples of the graphic equalizer (GEQ) filter used to configure the attenuation filter. The lengths of different delay lines corresponding to the same acoustic environment are mutually prime.
[0393] filterParamsStruct()This structure describes the graphic equalizer cascade filter and is used to configure the attenuation filter for the delay line. The same structure is then used to configure the filter for the diffuse to direct reverberation ratio. The following table describes the details of this structure.
[0394] The semantics of filterParamsStruct() are as follows
[0395] SOSLength is the length of each second-order partial filter coefficient.
[0396] The b1, b2, a1, a2 filter is configured with coefficients b1, b2, a1, and a2.
[0397] globalGain specifies the gain factor of the GEQ in decibels.
[0398] levelDB specifies the sound level (in decibels) to use for each delay line.
[0399] Another example of connection reverb bitstream information is also shown, which carries information about selected connection alternatives to be considered by the renderer. In addition, the bitstream also carries information about the priority of each alternative. The priority indicates which connections are more important than other connections. In an embodiment of this embodiment, if the renderer cannot render a connection reverb with all necessary connections, an error indication "Insufficient resources" can be indicated to the user.
[0400] reverbPayloadStruct(){
[0401] unsigned int(2)numberOfSpatialPositions;
[0402] unsigned int(1)hasConnectedReverbDepthLimit;
[0403] unsigned int(1)hasConnectedReverbGainMultiplier;
[0404] unsigned int(1)hasConnectedReverbActiveConnectionsInfoPresent;
[0405] unsigned int(8)numberOfAcousticEnvironments;
[0406] for(int i=0;i<numberOfSpatialPositions;i++){
[0407] signed int(32)azimuth;
[0408] signed int(32)elevation;
[0409] }
[0410] if(hasConnectedReverbDepthLimit){
[0411] unsigned int(3)connectedReverbDepthLimit;
[0412] reserved bits(5)=0;
[0413] }
[0414] if(hasConnectedReverbGainMultiplier)
[0415] unsigned int(8)connectedReverbGainMultiplier;
[0416] reserved bits(3)=0;
[0417] for(int i=0;i<numberOfAcousticEnvironments;i++){
[0418] unsigned int(16)environmentsId;
[0419] filterParamsStruct();
[0420] unsigned int(1)hasActiveAudioSource;
[0421] if(hasActiveAudioSource){
[0422] unsigned int(1)animated_audio_element_present;
[0423] unsigned int(1)interactive_element_present;
[0424] unsigned int(1)dynamic_update_element_present;
[0425] reserved bits(5)=0;
[0426] }
[0427] unsigned int(1)hasAESpecificConnectedReverbGainMultiplier;
[0428] if(hasAESpecificConnectedReverbGainMultiplier){
[0429] unsigned int(8)AEConnectedReverbGainMultiplier;
[0430] }
[0431] if(hasConnectedReverbActiveConnectionsInfoPresent){
[0432] ConnectedReverbActiveConnectionsInfoStruct();
[0433] }
[0434] reserved bits(5)=0;
[0435] for(int j=0;j<numberOfSpatialPositions;j++){
[0436] unsigned int(32)delayLineLength;
[0437] filterParamsStruct();
[0438] }
[0439] }
[0440] }
[0441] aligned(8)filterParamsStruct(){
[0442] SOSLength;
[0443] If(SOSLength>0){
[0444] for(i=0;i<SOSLength;i++){
[0445] signed int(32)b1;
[0446] }
[0447] for(i=0;i<SOSLength;i++){
[0448] signed int(32)b2;
[0449] }
[0450] for(i=0;i<SOSLength;i++){
[0451] signed int(32)a1;
[0452] }
[0453] for(i=0;i<SOSLength;i++){
[0454] signed int(32)a2;
[0455] }
[0456] signed int(32)globalGain;
[0457] signed int(32)levelDb;
[0458] }
[0459] }
[0460] aligned(8)ConnectedReverbActiveConnectionsInfoStruct(){
[0461] unsigned int(8)numberOfActiveConnectionsAcousticEnvironments;
[0462] for(int i=0;i<numberOfActiveConnectionsAcousticEnvironments;i++){
[0463] unsigned int(16)activeConnectionEnvironmentId;
[0464] unsigned int(8)num_acoustic_coupling;
[0465] for(int i=0; i <num_acoustic_coupling;i++){
[0466] unsigned int(8)active_connection_priority;
[0467] unsigned int(1)hasActiveConnectionSpecificMultiplier;
[0468] if(hasActiveConnectionSpecificMultiplier){
[0469] unsigned int(8)ConnectedReverbMultiplier;
[0470] unsigned int(1)hasFeedbackAttenuationFilter;
[0471] if(hasFeedbackAttenuationFilter){
[0472] FeedbackAttenuationFilterStruct();
[0473] }
[0474] }
[0475] }
[0476] aligned(8)FeedbackAttenuationFilterStruct(){
[0477] filterParamsStruct();
[0478] }
[0479] The difference between this example and the previous one is that hasConnectedReverbActiveConnectionsInfoPresent equal to 1 indicates that there is information about active connections between acoustic environments in the bitstream for performing connected reverberation rendering. A value equal to 0 indicates that the renderer can use the scene description to derive active connections between acoustic environments.
[0480] ConnectedReverbActiveConnectionsInfoStruct()
[0481] numberOfActiveConnectionsAcousticEnvironments defines the number of acoustic environments to be considered for connected reverb rendering.
[0482] hasConnectedReverbGainMultiplier equal to 1 indicates that there is information about the reverb output gain multiplier before the reverb rendering that feeds it as input to the next reverb for connection. A value equal to 0 indicates that there is no such information at a global level for all acoustic environments.
[0483] The activeConnectionEnvironmentId indicates the acoustic environment that has an active connection with the current acoustic environment.
[0484] active_connection_priority indicates the priority of the connection to be considered when performing connected reverb rendering. A value of 255 is considered required, while values less than 255 are considered optional. This provides the renderer with flexibility in controlling the computational complexity of connected reverb rendering.
[0485] ConnectedReverbMultiplier defines the multiplier used for the reverb output before it is fed to the current AE reverb. This value is present if hasActiveConnectionSpecificMultiplier is equal to 1. ConnectedReverbMultiplier overrides the value of the multiplier defined by AEConnectedReverbGainMultiplier as well as connectedReverbGainMultiplier.
[0486] FeedbackAttenuationFilterStruct defines the coefficients of the feedback attenuation filter used for the connection between acoustic environments (if any).
[0487] about Fig. 20 , the example electronic device can be used as any device part of the above system. The device can be any suitable electronic device or device. For example, in some embodiments, the device 2000 is a mobile device, a user device, a tablet computer, a computer, an audio playback device, etc. The device can be configured to implement an encoder or a renderer or any functional block as described above.
[0488] In some embodiments, the device 2000 includes at least one processor or central processing unit 2007. The processor 2007 may be configured to execute various program codes, such as the methods described herein.
[0489] In some embodiments, the device 2000 includes a memory 2011. In some embodiments, at least one processor 2007 is coupled to the memory 2011. The memory 2011 can be any suitable storage module. In some embodiments, the memory 2011 includes a program code portion for storing program codes that can be implemented on the processor 2007. In addition, in some embodiments, the memory 2011 can also include a storage data portion for storing data (e.g., data processed or to be processed according to the embodiments described herein). Whenever necessary, the implemented program code stored in the program code portion and the data stored in the storage data portion can be retrieved by the processor 2007 through the memory-processor coupling.
[0490] In some embodiments, the device 2000 includes a user interface 2005. In some embodiments, the user interface 2005 can be coupled to a processor 2007. In some embodiments, the processor 2007 can control the operation of the user interface 2005 and receive input from the user interface 2005. In some embodiments, the user interface 2005 can enable a user to input commands to the device 2000, for example, via a keypad. In some embodiments, the user interface 2005 can enable a user to obtain information from the device 2000. For example, the user interface 2005 can include a display configured to display information from the device 2000 to the user. In some embodiments, the user interface 2005 can include a touch screen or touch interface that can enable information to be input to the device 2000 and can display information to the user of the device 2000. In some embodiments, the user interface 2005 can be a user interface for communication.
[0491] In some embodiments, the device 2000 includes an input / output port 2009. In some embodiments, the input / output port 2009 includes a transceiver. In such embodiments, the transceiver can be coupled to the processor 2007 and configured to enable communication with other devices or electronic devices, for example, via a wireless communication network. In some embodiments, the transceiver or any suitable transceiver or transmitter and / or receiver module can be configured to communicate with other electronic devices or devices via a wire or wired coupling.
[0492] The transceiver can communicate with another device via any suitable known communication protocol. For example, in some embodiments, the transceiver can use a suitable Universal Mobile Telecommunications System (UMTS) protocol, a wireless local area network (WLAN) protocol such as, for example, IEEE 802.X, a suitable short-range radio frequency communication protocol such as Bluetooth, or an infrared data communication path (IRDA).
[0493] Input / output port 2009 may be configured to receive signals.
[0494] In some embodiments, device 2000 may function as at least part of a renderer.Input / output port 2009 may be coupled to a headset (which may be a head-tracked headset or a non-tracked headset) or the like.
[0495] In general, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described using block diagrams, flow charts, or other graphical representations, it is fully understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.
[0496] Embodiments of the present invention can be implemented by computer software that can be executed by a data processor of a mobile device (e.g., in a processor entity), or by hardware, or by a combination of software and hardware. In addition, in this regard, it should be noted that any block of the logic flow in the figure can represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored on a physical medium such as a memory chip or a memory block implemented in a processor, a magnetic medium such as a hard disk or a floppy disk, and an optical medium such as a DVD and its data variant CD.
[0497] The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technical environment and may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a gate-level circuit, and a processor based on a multi-core processor architecture, as non-limiting examples.
[0498] Embodiments of the present invention may be practiced in various components such as integrated circuit modules. The design of integrated circuits is essentially a highly automated process. Complex and powerful software tools are available to convert a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.
[0499] Programs, such as those offered by Synopsys Inc. of Mountain View, Calif., and Cadence Design Inc. of San Jose, Calif., use well-established design rules along with a library of pre-stored design blocks to automatically route conductors and locate components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) can be transferred to a semiconductor fabrication facility or "fab" for manufacturing.
[0500] The foregoing description provides a complete and informative description of exemplary embodiments of the present invention by way of exemplary and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the relevant arts in view of the foregoing description when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications to the teachings of the present invention will still fall within the scope of the present invention as defined in the appended claims.
Claims
1. An apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising a module configured to perform the following operations: Get listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment through an active portal connection; Acquiring at least one second acoustic environment input audio signal; as well as At least one listener output audio signal is generated based at least in part on application of processing by at least one second acoustic environment reverberator and then the first acoustic environment reverberator to the at least one second acoustic environment input audio signal.
2. The device according to claim 1, wherein: The modules are configured as: Acquiring at least one third acoustic environment input audio signal; at least one other acoustic environment of the at least two acoustic environments is determined, the at least one other acoustic environment being coupled to the determined at least one second acoustic environment; and wherein the module configured to obtain the at least one second acoustic environment input audio signal is configured to generate the at least one second acoustic environment input audio signal based at least in part on the application of processing of the at least one third acoustic environment input audio signal by at least one third acoustic environment reverberator.
3. The device according to claim 2, wherein: The at least one third acoustic environment is associated with at least one third acoustic environment reverberator parameter for configuring the at least one third acoustic environment reverberator.
4. The device according to any one of claims 1 to 3, wherein: The first acoustic environment is associated with at least one first acoustic environment reverberator parameter for configuring the first acoustic environment reverberator, and the second acoustic environment is associated with at least one second acoustic environment reverberator parameter for configuring the at least one second acoustic environment reverberator, wherein the module configured to generate at least one listener output audio signal based at least in part on application of processing of the at least one second acoustic environment input audio signal by the at least one second acoustic environment reverberator and then the first acoustic environment reverberator is configured to: generating a second acoustic environment output audio signal based on application of a second acoustic environment reverberator configured by the at least one second acoustic environment reverberator parameters to the at least one second acoustic environment input audio signal; and An output audio signal is generated based on application of the first acoustic environment reverberator configured by the at least one first acoustic environment reverberator parameter to the second acoustic environment output audio signal.
5. The device according to any one of claims 1 to 4, wherein: The active portal connection is one of the following: a one-way connection from the second acoustic environment to the first acoustic environment; and A bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
6. The device according to any one of claims 1 to 5, wherein: The active portal connection is defined relative to a distance within the first acoustic environment such that a module configured to determine the second of the at least two acoustic environments based on the first acoustic environment is configured to determine that the listener position is less than the distance within the first acoustic environment.
7. The device according to claim 6, wherein: The module is further configured to: determining at least one audio element within the at least two acoustic environments; and The distance within the first acoustic environment is configured based at least in part on a determination of at least one audio element within the at least two acoustic environments such that the distance is increased when there are no audio elements within the at least two acoustic environments.
8. The device according to claim 7, wherein: The module configured to determine the at least one audio element is configured to: obtaining bitstream information defining at least one audio element; and The at least one audio element is determined based on the bitstream information.
9. The device according to any one of claims 1 to 8, wherein: The module is further configured to obtain a list of active portal connections for each acoustic environment, wherein the list identifies, for a specified acoustic environment, any acoustic environments that are acoustically coupled with the specified acoustic environment.
10. The device according to claim 9, wherein: The module configured to determine a second acoustic environment of the at least two acoustic environments based on the first acoustic environment is configured to determine the second acoustic environment using the active portal connection list, wherein the second acoustic environment is associated with at least one second acoustic environment reverberator parameter.
11. The device according to claim 9 or 10, wherein: The active portal connection list for each acoustic environment includes priority information associated with the active portal connection list for each acoustic environment, wherein the module configured to determine a second acoustic environment among the at least two acoustic environments based on the first acoustic environment is configured to determine the second acoustic environment based on the priority information.
12. The apparatus according to any one of claims 1 to 11, the at least one second acoustic environment reverberator comprising at least one filter configured to attenuate at least one frequency.
13. The apparatus according to any one of claims 1 to 12, the at least one first acoustic environment reverberator comprising at least one filter configured to attenuate at least one frequency.
14. The device according to any one of claims 1 to 13, wherein: The module is further configured to apply one of the following to the audio signal output by the second acoustic environment prior to applying the first acoustic environment reverberator: a gain control multiplier; and a feedback attenuation filter.
15. An apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising means configured to: selecting a first acoustic environment of the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment of the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator and the second acoustic environment is coupled to the first acoustic environment through an active portal connection; and A bitstream is generated, the bitstream including information identifying the first acoustic environment and the at least one second acoustic environment coupled through an active portal connection.
16. The device according to claim 15, wherein: The active portal connection is one of the following: a one-way connection from the second acoustic environment to the first acoustic environment; and A bidirectional connection from the second acoustic environment to the first acoustic environment and from the first acoustic environment to the second acoustic environment.
17. The device according to any one of claims 15 or 16, wherein: The information also includes at least one reverberator parameter for configuring the acoustic environment reverberator.
18. The device according to claims 15 to 17, wherein: The information also includes priority information associated with the list of active portal connections for each acoustic environment.
19. A method for an apparatus for assisting spatial rendering in at least two acoustic environments, the method comprising: Get listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment through an active portal connection; Acquiring at least one second acoustic environment input audio signal; as well as At least one listener output audio signal is generated based at least in part on application of processing by at least one second acoustic environment reverberator and then the first acoustic environment reverberator to the at least one second acoustic environment input audio signal.
20. A method for an apparatus for assisting spatial rendering in at least two acoustic environments, the method comprising: selecting a first acoustic environment of the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment of the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator, and the second acoustic environment is coupled to the first acoustic environment through an active portal connection; as well as A bitstream is generated, the bitstream including information identifying the first acoustic environment and the at least one second acoustic environment coupled through an active portal connection.
21. An apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to at least: Get listener position; determining a first acoustic environment of the at least two acoustic environments based on the listener position; determining at least one second acoustic environment of the at least two acoustic environments based on the first acoustic environment, wherein the second acoustic environment is coupled to the first acoustic environment through an active portal connection; Acquiring at least one second acoustic environment input audio signal; as well as At least one listener output audio signal is generated based at least in part on application of processing by at least one second acoustic environment reverberator and then the first acoustic environment reverberator to the at least one second acoustic environment input audio signal.
22. An apparatus for assisting spatial rendering in at least two acoustic environments, the apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to at least: selecting a first acoustic environment of the at least two acoustic environments, wherein the first acoustic environment is associated with at least one first acoustic environment reverberator; determining at least one second acoustic environment of the at least two acoustic environments, wherein the at least one second acoustic environment is associated with at least one second acoustic environment reverberator and the second acoustic environment is coupled to the first acoustic environment through an active portal connection; and A bitstream is generated, the bitstream including information identifying the first acoustic environment and the at least one second acoustic environment coupled through an active portal connection.
