Late reverberation distance decay
By adjusting late reverberation levels based on sound-source listener distance, the renderer improves the realism of virtual audio scenes, addressing the unrealistic nature of constant reverberation levels in existing technologies.
Patent Information
- Application Number
- JP2024527300
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-09
- Filing Date
- 2022-11-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure 2024541313000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to late reverberation distance decay. In particular, the present invention relates to providing improved perceptual authenticity of a simulated acoustic environment. The concept is illustrated within a binaural reproduction system, but can also be extended to other forms of audio reproduction. [Background technology]
[0002] A key aspect of simulated experiences such as Virtual Reality (VR) or Augmented Reality (AR) is the ability to create physical spaces and environments in which subjects can perceive complex acoustic phenomena. This is especially true in the case of so-called "six degrees of freedom" (6DoF) rendering, where subjects can move freely in rooms with specific physical properties and thus experience various acoustic phenomena. The rendered sound generally consists of a direct sound, an early reflection part (ER), and a late reverberation part (LR).
[0003] Figure 3 shows the theoretical level of sound versus distance dependence for a point source in a closed room and corresponds to figure 1.13 of [1]. In particular, figure 1 visualizes the sound level dependence between a point source and a receiver (listener) over distance in a closed room. Near the source there are free-field conditions, where the level drops by a factor of 2 or 6 dB per doubling of distance. Far from the source, in a reverberant field, assumed to be completely diffuse, the level remains constant. The boundary between these two regions is determined by the critical distance. The critical distance for omnidirectional sources and receivers is calculated by: TIFF2024541313000002.tif1649A is the equivalent absorption area [m 2 ] and V is the volume of the room [m 3 ] and R.T. 60 is the reverberation time [s] (see https: / / en.wikipedia.org / wiki / Critical_distance).
[0004] Modeling sound sources and receivers in a room typically involves three different stages in the auralization of a virtual environment: direct sound, early reflections, and late reverberation processing.
[0005] FIG. 4 shows a standard implementation of a sound source in a room with three stages, direct sound, early reflections, and late reverberation processing.
[0006] As can be seen in Figure 4, the first two stages have distance-dependent level adjustments, with both levels decreasing as the source-receiver distance increases. The level of the late reverberation stage is usually assumed to be constant inside the room. At the critical distance mentioned above, the amounts of direct sound level and reverberation level are equal. The reproduction stage finally renders the output to either binaural headphones or loudspeaker reproduction. [Prior art documents] [Non-patent literature]
[0007] [Non-Patent Document 1] https: / / en.wikipedia.org / wiki / Critical_distance Summary of the Invention
[0008] The object of the present invention is to provide an improved concept for rendering virtual audio scenes. This object is solved by a renderer according to claim 1, a bitstream according to claim 21, an encoder according to claim 23, a method according to claim 27, a method according to claim 28 and a computer program according to claim 29.
[0009] A renderer according to an embodiment is provided. The renderer is configured to render the virtual audio scene according to one or more audio channels of each of one or more sound sources emitting sound into the virtual audio scene, and processes the one or more audio channels of said sound sources. The renderer comprises a late reverberation module configured to generate one or more late reverberation channels according to the one or more audio channels of the sound sources, the one or more late reverberation channels representing late reverberant parts of the sound emitted by the sound sources into the virtual audio scene. Furthermore, the renderer comprises an acoustic scene generator for generating one or more audio output channels for playing the virtual audio scene using the one or more late reverberation channels. The late reverberation module is configured to generate one or more late reverberation channels according to the one or more audio channels of the sound sources according to a distance between the sound sources of the virtual audio scene and the listener.
[0010] Further, a bitstream according to an embodiment is provided, the bitstream comprising an encoding of one or more audio channels of one or more sound sources emitting sound into the virtual audio scene, the bitstream further comprising one or more data fields comprising one or more information parameters including an indication regarding the strength of distance decay of late reverberation.
[0011] Further, an encoder configured to generate a bitstream according to an embodiment is provided. The encoder is configured to generate the bitstream such that the bitstream includes an encoding of one or more audio channels of each of one or more sound sources emitting sound into the virtual audio scene. Furthermore, the encoder is configured to generate the bitstream such that the bitstream further includes one or more data fields including one or more information parameters including an indication regarding the strength of distance attenuation of late reverberation.
[0012] Further, a method according to an embodiment is provided, which is configured to render a virtual audio scene as a function of one or more audio channels of each of one or more sound sources emitting sound into the virtual audio scene, and which processes one or more audio channels of said sound sources. - generating one or more late reverberation channels in response to one or more audio channels of a sound source, the one or more late reverberation channels representing a late reverberation part of a sound emitted by the sound source into the virtual audio scene; and - generating one or more audio output channels for playing the virtual audio scene using one or more late reverberation channels.
[0013] Generating one or more late reverberation channels as a function of one or more audio channels of a sound source is performed as a function of a distance between an object sound source of the virtual audio scene and a listener.
[0014] Further, a method for generating a bitstream according to an embodiment is provided, the method comprising: - generating a bitstream such that the bitstream includes an encoding of one or more audio channels of each of one or more sound sources emitting sound into a virtual audio scene; and - generating the bitstream such that the bitstream further comprises one or more data fields including one or more information parameters including an indication regarding the strength of the distance decay of the late reverberation.
[0015] Furthermore, a computer program according to an embodiment is provided for performing one of the above mentioned methods when executed on a computer or signal processor.
[0016] In the following, embodiments of the invention are explained in more detail with reference to the following figures: [Brief description of the drawings]
[0017] [Figure 1] FIG. 2 illustrates a renderer for rendering a virtual audio scene according to an embodiment. [Diagram 2] 2 shows an apparatus according to an embodiment comprising a decoder and a renderer of the embodiment of FIG. 1; [Diagram 3] This shows the theoretical sound level versus distance dependence for a point sound source in a closed room. [Figure 4] A standard implementation of a sound source in a room with three stages: direct sound, early reflections, and late reverberation processing is shown. [Diagram 5] 13 illustrates a novel behavior of level dependence in a reverberant field according to an embodiment. [Figure 6] 1 illustrates a room simulation having three stages of direct sound, early reflections, and late reverberation processing with distance-dependent level adjustment according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0018] FIG. 1 illustrates a renderer 100 for rendering a virtual audio scene according to an embodiment.
[0019] According to an embodiment, a renderer 100 is provided, which is configured to render a virtual audio scene according to one or more audio channels of each of one or more sound sources emitting sound into the virtual audio scene and processes the one or more audio channels of said sound sources.
[0020] The renderer 100 comprises a late reverberation module 110 configured to generate one or more late reverberation channels in response to one or more audio channels of a sound source, the one or more late reverberation channels representing a late reverberant part of a sound emitted by the sound source into the virtual audio scene.
[0021] Furthermore, the renderer 100 comprises an audio scene generator 120 for generating one or more audio output channels for playing the virtual audio scene using one or more late reverberation channels.
[0022] The late reverberation module 110 is configured to generate one or more late reverberation channels in response to one or more audio channels of a sound source in the virtual audio scene in response to a distance between the sound source and the listener.
[0023] In an embodiment, the late reverberation module 110 may be configured to generate one or more late reverberation channels in response to one or more audio channels of a sound source, such that, for example, the sound pressure level and / or amplitude and / or magnitude and / or energy of the one or more late reverberation channels may be adapted in response to, for example, a distance between a sound source of the virtual audio scene and a listener.
[0024] According to an embodiment, the late reverberation module 110 may be configured to render the sound pressure level and / or amplitude and / or magnitude and / or energy of one or more late reverberation channels such that, for example, a greater distance between a sound source of the virtual audio scene and the listener results in a stronger attenuation of the level and / or amplitude and / or energy of the one or more late reverberation channels compared to a smaller distance between a sound source of the virtual audio scene and the listener.
[0025] In an embodiment, the late reverberation module 110 may be configured to render the sound pressure level and / or amplitude and / or magnitude and / or energy of the one or more late reverberation channels as a function of a first distance between the sound source and the listener such that, for example, when the distance between the sound source and the listener is half the first distance, the sound pressure level of the one or more late reverberation channels may be reduced by a value of, for example, 1 dB to 2 dB compared to the attenuation of the one or more audio channels.
[0026] According to an embodiment, the renderer 100 may further comprise a direct sound module configured to generate one or more direct sound channels in response to one or more audio channels of a sound source, such that a larger distance between a sound source of the virtual audio scene and the listener results in a stronger attenuation of the level and / or amplitude and / or energy of the one or more direct sound channels compared to a smaller distance between the sound source of the virtual audio scene and the listener, The sound scene generator 120 may be configured to generate one or more audio output channels for reproducing a virtual audio scene, for example, using one or more direct sound channels.
[0027] In an embodiment, when the distance between a sound source of the virtual audio scene and the listener is a greater distance rather than a smaller distance, the late reverberation module 110 may be configured to render the sound pressure level and / or amplitude and / or magnitude and / or energy of the one or more late reverberation channels such that, for example, a greater distance results in a relatively smaller attenuation of the sound pressure level and / or amplitude and / or magnitude and / or energy of the one or more late reverberation channels compared to the attenuation of the level and / or amplitude and / or energy of the one or more direct sound channels conducted by the direct sound module in response to the greater distance.
[0028] According to an embodiment, when the distance between the sound source of the virtual audio scene and the listener is the current distance compared to when the distance between the sound source of the virtual audio scene and the listener is half the current distance, the late direct sound module may be configured to render the sound pressure level and / or amplitude and / or magnitude and / or energy of the one or more direct sound channels such that, for example, the sound pressure level of the one or more direct sound channels is reduced by a value between 5 dB and 7 dB and the late reverberation module 110 may be configured to render the sound pressure level and / or amplitude and / or magnitude and / or energy of the one or more late reverberation channels such that, for example, the sound pressure level of the one or more late reverberation channels is reduced by a value between 1 dB and 2 dB.
[0029] In an embodiment, the renderer 100 may be configured to receive one or more information parameters including, for example, an instruction regarding the strength of the distance attenuation of the late reverberation. The late reverberation module 110 may be configured to adapt the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of one or more late reverberation channels depending on, for example, the distance between a sound source of the virtual audio scene and the listener and depending on the instruction of the strength of the distance attenuation of the late reverberation.
[0030] According to an embodiment, the bitstream may, for example, include one or more information parameters, The renderer 100 may, for example, be configured to receive a bitstream and, for example, to obtain one or more information parameters from the bitstream, or the renderer 100 may, for example, be configured to receive the one or more information parameters from another unit that receives the bitstream and obtains the one or more information parameters from the bitstream.
[0031] In an embodiment, the one or more information parameters include a distance drop decibel factor and a reference distance. The late reverberation module 110 can be configured to adapt the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of one or more late reverberation channels depending on, for example, the distance between a sound source of the virtual audio scene and the listener, depending on the distance drop decibel factor and depending on the reference distance.
[0032] According to an embodiment, the late reverberation module 110 may, for example, distanceGainDbFactor * log10(refDistance / distance); and distanceGainDbFactor = distanceGainDropDb / log10(2.0); to adapt the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of one or more late reverberation channels in response to a gain dbGain which depends on distanceGainDropDb denotes the distance drop decibel factor, refDistance denotes the reference distance, where distance denotes the distance between the sound source of the virtual audio scene and the listener.
[0033] In an embodiment, the reference distance may be, for example, a reference distance of an audio element according to the MPEG-I 6DoF Audio Encoder Input Format (EIF), and the audio element may be, for example, a sound source.
[0034] According to an embodiment, the late reverberation module 110 may be configured to generate one or more late reverberation channels using, for example, a feedback delay network reverberator.
[0035] In an embodiment, the renderer 100 may further comprise an early reflection module configured to generate one or more early reflection channels, for example in response to one or more audio channels of a sound source. The sound scene generator 120 may be configured to generate one or more audio output channels for playing the virtual audio scene, for example with the one or more early reflection channels.
[0036] In an embodiment, the renderer 100 may be configured to determine a distance between a sound source of the virtual audio scene and a listener, for example depending on the position of the sound source and the position of the listener. The position of the sound source and the position of the listener may be defined in three dimensions, and / or the position of the sound source and the position of the listener may be defined in two dimensions, and / or the position of the sound source may be defined in, for example, three dimensions and the position and orientation of the listener may be defined in, for example, six degrees of freedom, so that the position of the listener may be defined in, for example, three dimensions and the orientation of the listener's head may be defined using, for example, three rotation angles.
[0037] According to an embodiment, one or more audio channels of one or more sound sources may be represented in the Ambisonics domain and the sound scene generator 120 may be configured to reproduce the virtual audio scene according to, for example, the characteristics of one of a plurality of spherical harmonic functions associated with one of the one or more audio channels of said sound source. Or, one or more audio channels of said sound source may be represented in a different domain, different from the Ambisonics domain, said one or more audio channels of said sound source may be derived from one or more other channels of said sound source represented in the Ambisonics domain, each audio channel of the one or more audio channels may be derived from and associated with one of the one or more other channels according to, for example, the characteristics of one of a plurality of spherical harmonic functions.
[0038] In an embodiment, the renderer 100 may comprise a binauralizer configured to generate two audio output channels for reproducing a virtual audio scene, for example according to one or more late reverberation channels.
[0039] According to an embodiment, the bitstream may, for example, include one or more audio channels for each of the one or more audio sources. The renderer 100 may, for example, be configured to receive the bitstream and, for example, to obtain from the bitstream the one or more audio channels for each of the one or more audio sources, or the renderer 100 may, for example, be configured to receive the one or more audio channels for each of the one or more audio sources from another unit that receives the bitstream and obtains from the bitstream the one or more audio channels for each of the one or more audio sources.
[0040] FIG. 2 shows an apparatus according to an embodiment comprising the decoder 50 and renderer 100 of the embodiment of FIG.
[0041] The decoder 50 is configured to decode the bitstream to obtain one or more audio channels for each of the one or more audio sources.
[0042] The renderer 100 is configured to render a virtual audio scene according to one or more audio channels of each of the one or more sound sources.
[0043] According to an embodiment, the bitstream may, for example, include one or more information parameters. The decoder 50 may, for example, be configured to obtain the one or more information parameters from the bitstream. Furthermore, the renderer 100 may, for example, be configured to receive the one or more information parameters from the decoder 50.
[0044] Further, a bitstream according to an embodiment is provided, the bitstream comprising an encoding of one or more audio channels of one or more sound sources emitting sound into the virtual audio scene, the bitstream further comprising one or more data fields comprising one or more information parameters including an indication regarding the strength of distance decay of late reverberation.
[0045] According to an embodiment, the one or more information parameters may include, for example, a distance drop decibel factor and, optionally, a reference distance.
[0046] Further, an encoder configured to generate a bitstream according to an embodiment is provided. The encoder is configured to generate the bitstream such that the bitstream includes an encoding of one or more audio channels of each of one or more sound sources emitting sound into the virtual audio scene. Furthermore, the encoder is configured to generate the bitstream such that the bitstream further includes one or more data fields including one or more information parameters including an indication regarding the strength of distance attenuation of late reverberation.
[0047] According to an embodiment, the encoder may be configured to generate a bitstream such that the one or more information parameters include a distance drop decibel factor and a reference distance, for example.
[0048] In an embodiment, the encoder may comprise an input interface arranged to receive instructions, for example from a content creator, regarding the strength of the distance attenuation of the late reverberation.
[0049] According to an embodiment, the encoder may comprise a decision module configured to determine instructions regarding the strength of the distance attenuation of late reverberation from a content creator, for example by an automatic process depending on one or more characteristics of the virtual environment.
[0050] The following describes certain embodiments and considerations on which embodiments of the present invention are based.
[0051] As mentioned above, in state-of-the-art implementations the late reverb level is constant, i.e. independent of the distance from the sound source to the listener, and follows the theoretical behavior shown in Figure 3. When rendering a large reverberant space (e.g. a cathedral with the sound source at the far end of the room), this leads to unrealistic behavior, since the overall level never decreases when moving away from the sound source, outside the critical distance, to any longer distance. Even after an additional distance of 1 km, the level of the late reverb does not decay (if the simulated room is large enough).
[0052] However, measurements of real room acoustics show that the level of the diffuse sound field is not perfectly constant beyond the critical distance in physical reality. In particular, in large rooms that are not perfectly diffuse, there is a falloff in late reverberation (less than 6 dB per doubling of distance). As a rule of thumb, the level falls off beyond the critical distance by 1-2 dB per doubling of distance, depending on the absorption properties of the wall materials.
[0053] Embodiments of the present invention provide highly immersive rendering by including this discovery in an interactive room simulation of the real experience.
[0054] The embodiment described here achieves this by adding a source-listener distance dependent level variation to the late reverb stage, see Figures 5 and 6.
[0055] Figure 5 shows the new behavior of the level dependence in the reverberant field according to the embodiment. The new behavior is shown by the dashed line (blue line) in Figure 5, and shows a reduction in the level dependence in the reverberant field of about 1-2 dB per doubling of distance.
[0056] Further specific embodiments are described below.
[0057] FIG. 6 shows a room simulation having three stages of direct sound, early reflections, and late reverberation processing with distance-dependent level adjustment according to an embodiment.
[0058] The method for source-listener dependent level attenuation can be implemented before, during, or after the late reverb processing of Figure 6 as shown in Figure 6. In our preferred embodiment, the method is applied to the input of the signal that goes to the late reverb processing.
[0059] The level adjustment method of the present invention begins by obtaining the position in Cartesian coordinates (x,y,z) of the item to be rendered. sourceLocation = item->position.location; Next, the method obtains the absolute distance dist between the sourceLocation and the listenerLocation (also in Cartesian coordinates). dist = (sourceLocation - listenerLocation).abs(); The method then takes the maximum value of dist and the value of minimumDistance. This is done to prevent excessive level rise of late reverb when very close to the sound source. Currently, minimumDistance is defined as 1 meter. In a preferred embodiment, the late reverb minimumDistance can be signaled in the bitstream from the scene encoder to the scene decoder / renderer. dist = max(minimumDistance, dist); The value of distanceGain to be applied to the reverb input signal is calculated by the method calculateDistanceGain based on the values of dist and refDistance of the rendered item. refDistance is the reference distance in meters for the rendered item, as defined by the content creator in the encoder input format file and signaled as a bitstream parameter. Reference distance is the distance at which the calculated attenuation of this input signal is 0 dB, as defined in the MPEG-I encoder input format [2]. distanceGain = calculateDistanceGain(dist, item->refDistance); itemGain = item->gain * distanceGain; Then itemGain contains the gain to be applied to the reverb input signal for this rendering item, combining any static gain defined in the bitstream by the content creator for this rendering item in item->gain and the calculated distanceGain.
[0060] The method implemented in calculateDistanceGain is as follows: dbGain = distanceGainDbFactor * log10(refDistance / distance); distanceGain = pow(10.0, dbGain / 20.0); Here, distanceGainDbFactor is calculated as follows: distanceGainDbFactor = distanceGainDropDb / log10(2.0); In an embodiment, distanceGaindDropDb is signaled in the bitstream and typically has a value between 1 dB and 2 dB to implement a level drop of between 1 dB and 2 dB per doubling of distance.
[0061] The above formula is merely an example, in other embodiments the linear gain can be calculated directly to achieve the desired attenuation (distanceGaindDropDb per distance doubled).
[0062] The input signal after gain has been applied is fed to a digital reverberator. In a preferred embodiment, the digital reverberator is a Feedback Delay Network (FDN) reverberator. Other suitable reverberator implementations may be used as well.
[0063] Specific embodiments will be described below. In one possible embodiment, distanceGaindDropDb can be determined by a content creator by experimenting with different values, listening to the output, and adjusting the value so that the output sounds perceptually believable at all locations in the virtual scene, taking into account that person's experience and artistic intent.
[0064] In another embodiment, distanceGaindDropDb may be determined by an auto-encoder process that performs the following steps. - obtaining a virtual environment comprising a geometric shape and one or more acoustic materials having at least sound absorption parameters; selecting a location for the sound source in the virtual environment that is not too close to any of the boundaries of the virtual environment; - selecting a first receiver position at a reference distance from the position of the sound source; - selecting at least one second receiver position having a distance from the position of the sound source that is greater than a reference distance; performing acoustic modeling, e.g., using geometric acoustic modeling, wave-based acoustic modeling, or a combination thereof, to obtain a first impulse response at a first receiver position and a second impulse response at a second receiver position; - obtaining a first level value from the first impulse response, the first level value corresponding to a time interval of the diffuse late reverberation; obtaining from the second impulse response a second level value corresponding to a time interval of the diffuse late reverberation; performing a line fitting on the first level value and the second level value in decibels to obtain a slope of the line; - signaling the slope of the line as distanceGainDropDb to a rendering device of the bitstream.
[0065] The above method is applicable to rendering of virtual reality (VR) scenes, where the virtual scene is provided to an encoder device that is capable of determining and signaling appropriate parameters (such as distance dependent level attenuation) to a rendering device.
[0066] In some embodiments, the rendering takes place in an augmented reality (AR) scenario, where no data about the playback room is available to the encoder device, but information about the user listening space and its acoustic characteristics (such as dimensions, materials and reverberation time) is provided only at rendering time, for example as a listening space description file.
[0067] In large indoor spaces, implementing a distance-dependent level of attenuation in the late reverb processing can be useful, enhancing the realism of the audio reproduction.
[0068] In one embodiment of the present invention, a similar method of acoustic simulation as described above is applied by a rendering device upon receiving listening space description file parameters. This procedure generates distanceGainDropDb parameters that can be used to render reverberation and generate source-listener dependent distance gain attenuation when the listener is inside the space defined by the listening space description file.
[0069] However, because the AR processing is performed at renderer startup, which should not take too long, it is desirable for the procedures performed in the renderer to be computationally simpler than those performed in the encoder.
[0070] In an embodiment of the invention, instead of performing an acoustic simulation using a listening space description file, the procedure calculates the volume of the space described in the listener space description file and / or the average of the material absorption coefficients of the listener space description file, and performs a mapping from the listening space volume and its average absorption coefficient to a value suitable for the distance-dependent level attenuation. For example, a small space with low average absorption may receive a small value for distanceGainDropDb, meaning that there is little source-listener dependent distance attenuation of late reverberation, whereas a larger space with more absorption may receive a larger value for distanceGainDropDb, meaning that there is some distance-dependent level attenuation in such a space.
[0071] Certain aspects of the embodiments are described below. First, aspects of rendering according to some specific embodiments will be described.
[0072] According to an embodiment, a renderer is provided that is equipped to render a virtual audio scene comprising one or more sound sources and that comprises a stage for rendering late reverb, characterized in that the late reverb rendering depends on one or more reverb control parameters including a reverb time (e.g. RT60), and the late reverb level is rendered depending on the distance between the sound source and the listener and depending on a measure of the intensity of the distance attenuation.
[0073] In a preferred embodiment, this measure of the strength of late reverberation distance attenuation indicates the relative attenuation increase expressed in decibels per doubling of distance.
[0074] In a more preferred embodiment, a value of 1 to 2 dB per twice the distance is applied.
[0075] In a further preferred embodiment, the degree of strength of the late reverb distance decay is read from the bitstream.
[0076] Next, aspects of the bitstream according to some specific embodiments will be described.
[0077] 1. A bitstream for rendering an acoustic scene by a renderer, characterized in that for at least one description of late reverberation in a particular part of a scene, a bitstream field is included which indicates an intensity of distance attenuation to be applied to the rendering of the late reverberation in this part of the scene.
[0078] In a preferred embodiment, this field indicating the strength of reverberation distance attenuation represents the relative attenuation increase expressed in decibels per doubling of distance.
[0079] A field of application of particular embodiments may for example be the field of real-time auditory virtual environments or the field of real-time virtual and augmented reality.
[0080] It should be mentioned herein that all alternatives or aspects described above, and all aspects defined by the independent claims in the following claims, can be used individually, i.e. without alternatives or purposes other than the contemplated alternatives, purposes or independent claims. However, in other embodiments, two or more alternatives or aspects or independent claims can be combined with each other, and in other embodiments, all aspects or alternatives and all independent claims can be combined with each other.
[0081] The encoded or processed signals of the present invention may be stored in a digital or non-transitory storage medium or may be transmitted over a wireless or wired transmission medium, such as the Internet.
[0082] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or feature of a method step, and similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or function of a corresponding apparatus.
[0083] Depending on particular implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using digital storage media, such as floppy disks, DVDs, CDs, ROMs, PROMs, EPROMs, EEPROMs, flash memories, etc., having electronically readable control signals stored thereon, which cooperate (or can cooperate) with a programmable computer system to carry out the respective methods.
[0084] Some embodiments according to the invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.
[0085] In general, embodiments of the present invention can be implemented as a computer program product with program code that operates to perform one of the methods when the computer program product is run on a computer. The program code may for example be stored on a machine readable carrier.
[0086] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.
[0087] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0088] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer readable medium) comprising the computer program for performing one of the methods described herein recorded thereon.
[0089] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, the data stream or the sequence of signals being adapted to be transferred via a data communication connection, such as for example the Internet.
[0090] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
[0091] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0092] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.
[0093] The above-described embodiments merely illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the immediate patent claims, and not by the specific details presented as descriptions and explanations of the embodiments herein.
[0094] literature [1] Ginn, K.B., Architectual Acoustics. 1978. Available from: https: / / www.bksv.com / media / doc / bn1329.pdf. ISBN: 87 87355 24 8.
[0095] [2] ISO / IEC JTC1 / SC29 / WG6 (MPEG Audio): N0054 - MPEG-I Immersive Audio Encoder Input Format. 30 April 2021
Claims
1. A renderer (100) for rendering a virtual audio scene according to one or more audio channels of each of one or more sound sources emitting sound into said virtual audio scene, said renderer (100) comprising: a late reverberation module (110) configured to generate one or more late reverberation channels in response to the one or more audio channels of the sound source, the one or more late reverberation channels representing a late reverberant part of the sound emitted by the sound source into the virtual audio scene; an acoustic scene generator (120) for generating one or more audio output channels for reproducing the virtual audio scene using the one or more late reverberation channels; The renderer (100), wherein the late reverberation module (110) is configured to generate the one or more late reverberation channels in response to the one or more audio channels of the sound source in the virtual audio scene in response to a distance between the sound source and a listener.
2. the late reverberation module (110) is configured to generate the one or more late reverberation channels as a function of the one or more audio channels of the sound source such that a sound pressure level and / or an amplitude and / or a magnitude and / or an energy of the one or more late reverberation channels are adapted as a function of the distance between the sound source of the virtual audio scene and the listener, The renderer (100) of claim 1.
3. the late reverberation module (110) is configured to render the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels in such a way that a greater distance between the sound source of the virtual audio scene and the listener results in a stronger attenuation of the level and / or the amplitude and / or the energy of the one or more late reverberation channels compared to a smaller distance between the sound source of the virtual audio scene and the listener, The renderer (100) of claim 2.
4. the late reverberation module (110) is configured to render the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels as a function of the first distance between the sound source and the listener such that, when the distance between the sound source and the listener is half the first distance, the sound pressure level of the one or more late reverberation channels may be reduced by a value between 1 dB and 2 dB compared to the attenuation of the one or more audio channels, A renderer (100) according to claim 2 or 3.
5. the renderer (100) further comprises a direct sound module configured to generate one or more direct sound channels in response to the one or more audio channels of the sound source, wherein a greater distance between the sound source in the virtual audio scene and the listener results in a stronger attenuation of the level and / or the amplitude and / or the energy of the one or more direct sound channels compared to a smaller distance between the sound source in the virtual audio scene and the listener, the sound scene generator (120) is configured to generate the one or more audio output channels for reproducing the virtual audio scene using the one or more direct sound channels. A renderer (100) according to one of the claims 1 to 4.
6. when the distance between the sound source of the virtual audio scene and the listener is the greater distance rather than the smaller distance, the late reverberation module (110) is configured to render the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels such that the greater distance results in an attenuation of the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels that is relatively smaller compared to the attenuation of the level and / or the amplitude and / or the energy of the one or more direct sound channels conducted by the direct sound module in response to the greater distance, The renderer (100) of claim 5.
7. when the distance between the sound source of the virtual audio scene and the listener is the current distance compared to when the distance between the sound source of the virtual audio scene and the listener is half the current distance; said late direct sound module is configured to render the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of said one or more direct sound channels such that the sound pressure level of said one or more direct sound channels is reduced by a value between 5 dB and 7 dB, said late reverberation module (110) is configured to render the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of said one or more late reverberation channels such that the sound pressure level of said one or more late reverberation channels is reduced by a value between 1 dB and 2 dB, The renderer (100) of claim 6.
8. The renderer (100) is configured to receive one or more information parameters including an indication regarding the strength of the distance decay of late reverberation; the late reverberation module (110) is configured to adapt the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels depending on the distance between the sound source of the virtual audio scene and the listener and depending on the indication of the strength of the distance attenuation of late reverberation. A renderer (100) according to one of claims 1 to 7, further dependent on claim 2.
9. the bitstream includes said one or more information parameters; The renderer (100) is configured to receive the bitstream and is configured to obtain the one or more information parameters from the bitstream, or the renderer (100) is configured to receive the one or more information parameters from another unit that receives the bitstream and obtains the one or more information parameters from the bitstream. The renderer (100) of claim 8.
10. The one or more information parameters include a distance drop decibel factor and a reference distance; the late reverberation module (110) is configured to adapt the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels in response to the distance between the sound source of the virtual audio scene and the listener, in response to the distance drop decibel factor and in response to the reference distance. A renderer (100) according to claim 8 or 9.
11. The late reverberation module (110) is configured to adapt the sound pressure level and / or the amplitude and / or the magnitude and / or the energy of the one or more late reverberation channels according to a gain dbGain which depends on distanceGainDbFactor * log10(refDistance / distance); and distanceGainDbFactor=distanceGainDropDb / log10(2.0); distanceGainDropDb indicates the distance drop decibel coefficient, refDistance indicates the reference distance, Distance indicates the distance between the sound source of the virtual audio scene and the listener. The renderer (100) of claim 10.
12. The reference distance is a reference distance of an audio element according to MPEG-I 6DoF Audio Encoder Input Format (EIF), and the audio element is the sound source. A renderer (100) according to claim 10 or 11.
13. the late reverberation module (110) is configured to generate the one or more late reverberation channels using a feedback delay network reverberator; A renderer (100) according to one of the preceding claims.
14. The renderer (100) further comprises an early reflection module configured to generate one or more early reflection channels in response to the one or more audio channels of the sound source; the sound scene generator (120) is configured to generate the one or more audio output channels for playing the virtual audio scene using the one or more early reflection channels. A renderer (100) according to one of the preceding claims.
15. the renderer (100) is configured to determine the distance between the sound source of the virtual audio scene and a listener depending on a position of the sound source and depending on a position of the listener, the position of the sound source and the position of the listener are defined in three dimensions; and / or the positions of the sound source and the positions of the listener are defined in two dimensions; and / or the position of the sound source is defined in three dimensions, the position and orientation of the listener is defined in six degrees of freedom, the position of the listener is defined in three dimensions and the orientation of the listener's head is defined using three rotation angles; A renderer (100) according to one of the preceding claims.
16. the one or more audio channels of a source of the one or more sound sources are represented in the Ambisonics domain, and the sound scene generator (120) is configured to reproduce the virtual audio scene according to characteristics of one of a plurality of spherical harmonic functions associated with one of the one or more audio channels of the sound source, or the one or more audio channels of the sound source are represented in a different domain, different from the Ambisonics domain, the one or more audio channels of the sound source are derived from one or more other channels of the sound source represented in the Ambisonics domain, and each audio channel of the one or more audio channels is derived from one of the one or more other channels in accordance with characteristics of one of a plurality of spherical harmonic functions associated with the other channels; A renderer (100) according to one of the preceding claims.
17. the renderer (100) comprises a binauralizer configured to generate two audio output channels for reproducing the virtual audio scene as a function of the one or more late reverberation channels. A renderer (100) according to one of the preceding claims.
18. the bitstream comprises the one or more audio channels of each of the one or more audio sources; the renderer (100) is configured to receive the bitstream and to obtain from the bitstream the one or more audio channels for each of the one or more audio sources, or the renderer (100) is configured to receive the one or more audio channels for each of the one or more audio sources from another unit that receives the bitstream and obtains from the bitstream the one or more audio channels for each of the one or more audio sources, A renderer (100) according to one of the preceding claims.
19. An apparatus comprising: a decoder (50) configured to decode the bitstream to obtain the one or more audio channels of each of the one or more audio sources; A renderer (100) according to one of claims 1 to 18 for rendering a virtual audio scene according to said one or more audio channels of each of said one or more sound sources; 13. An apparatus comprising:
20. The renderer (100) is a renderer (100) according to claim 9, the bitstream includes the one or more information parameters; the decoder (50) is configured to obtain the one or more information parameters from the bitstream; the renderer (100) is configured to receive the one or more information parameters from the decoder (50); 20. The apparatus of claim 19.
21. A bitstream comprising: - encoding one or more audio channels of each of one or more sound sources emitting sound into the virtual audio scene; one or more data fields containing one or more information parameters including an indication regarding the strength of the distance decay of late reverberation; A bitstream including:
22. the one or more information parameters include a distance drop decibel factor and, optionally, a reference distance; 22. A bitstream as claimed in claim 21.
23. 1. An encoder configured to generate a bitstream, comprising: the encoder is configured to generate the bitstream such that the bitstream includes an encoding of one or more audio channels for each of one or more sound sources emitting sound into a virtual audio scene; 11. The encoder configured for generating the bitstream such that the bitstream further comprises one or more data fields comprising one or more information parameters including an indication regarding a strength of distance decay of late reverberation.
24. the encoder is configured to generate the bitstream such that the one or more information parameters include a distance drop decibel factor and a reference distance.
24. The encoder of claim 23.
25. the encoder comprises an input interface configured to receive the instruction regarding the intensity of the distance attenuation of late reverberation from a content creator. An encoder according to claim 23 or 24.
26. the encoder comprises a determining module configured to determine the instruction from a content creator regarding the intensity of the distance attenuation of late reverberation by an automatic process dependent on one or more characteristics of a virtual environment; An encoder according to claim 23 or 24.
27. 1. A method for rendering a virtual audio scene as a function of one or more audio channels of each of one or more sound sources emitting sound into said virtual audio scene, said method comprising: generating one or more late reverberation channels in response to the one or more audio channels of the sound source, the one or more late reverberation channels representing a late reverberation portion of the sound emitted by the sound source into the virtual audio scene; and using the one or more late reverberation channels to generate one or more audio output channels for playing the virtual audio scene; 13. The method of claim 12, wherein generating the one or more late reverberation channels as a function of the one or more audio channels of the sound source is performed as a function of a distance between the object sound source of the virtual audio scene and a listener.
28. 1. A method for generating a bitstream, comprising the steps of: generating a bitstream such that the bitstream includes an encoding of one or more audio channels for each of one or more sound sources emitting sound into a virtual audio scene; and generating said bitstream such that said bitstream further comprises one or more data fields containing one or more information parameters including an indication regarding the strength of distance decay of late reverberation; A method comprising:
29. A computer program for carrying out the method of claim 27 or 28 when executed on a computer or signal processor.
