Early reflection pattern generation concepts for sonification.

A simplified early reflection pattern generation method using fundamental physical parameters addresses computational inefficiencies in VR/AR by adapting to room characteristics, ensuring perceptual accuracy and reducing signaling costs.

JP2024543362AInactive Publication Date: 2024-11-21FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2024527264
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-09
Filing Date
2022-11-08
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing methods for early reflection rendering in virtual and augmented reality applications require significant computational resources due to precise geometric calculations, which are not necessary for perceptual accuracy and are sensitive to changes in sound source and listener position, especially in complex environments.

Method used

A simplified early reflection pattern generation method using fundamental physical parameters such as RT60, pre-delay time, room dimensions, and frequency dependence of RT60, independent of exact geometry, allowing for efficient rendering across varying acoustic environments.

Benefits of technology

Reduces computational complexity while maintaining perceptual accuracy by using a fixed spatial pattern that adapts to room characteristics, suitable for both indoor and outdoor environments, and reduces signaling costs by transmitting minimal side information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543362000001_ABST
    Figure 2024543362000001_ABST
Patent Text Reader

Abstract

The present application relates to an early reflection processing concept for auralization. Embodiments relate to an apparatus and method for sound rendering taking into account early reflections, as well as an apparatus and method for determining early reflection patterns.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to an early reflection processing concept for auralization.

[0002] Room impulse response (RIR) describes the relationship between a sound source in an acoustic environment (room) and a receiver (i.e., a listener). It specifies the response of a room to a unit impulse in the time domain and corresponds to the room transfer function in the frequency domain. The room impulse response consists of the direct sound path, early reflections (ERs), and diffuse late reverberation.

[0003] In binaural (or loudspeaker) rendering for virtual and augmented reality (VR / AR) applications, the room impulse response from a particular sound source and listener position can vary considerably. In six degree of freedom (6DOF) VR / AR applications, the listener is typically free to move throughout the scene, causing the room impulse response to constantly change. As a result, a significant amount of computation must be expended to determine each reflection from the sound source to the listener, taking into account wall geometry, occlusions, and other effects to calculate a physically accurate reflection pattern.

[0004] It is the discovery of the present invention that an exact acoustic reproduction of the early reflection (ER) pattern in a room is not necessary for a perceptually convincing rendering, and that this rendering can be done in a way that is largely abstracted from the exact geometric details of the room. In this way, many calculations can be saved. If the reflection pattern has to be transmitted from the encoder to the renderer, a significant part of the side information associated with efficiently calculating reflections as a function of the listener position can be omitted compared to the prior art in regular geometry-based rendering. [Background technology]

[0005] Non-patent literature [1] concerns replacing the precisely calculated "real" ER with more general and simple ER patterns. The idea was to find, describe and simulate (non-patent literature [2] and [3]) perceptually orthogonal parameters describing small or large sound sources (e.g. an orchestra) on a stage in a large room (e.g. a concert hall) and to play them in a loudspeaker setup (e.g. stereo) or binaurally over headphones. A composer or sound engineer could use these parameters (source presence, source warmth, source brilliance, room presence, dynamic reverberation, envelopment, reverberation, etc.) to set the scene. SPAT software has been used for a long time for such productions (non-patent literature [4]). This approach was also adopted by MPEG-4 (ISO standard) (non-patent literature [5]).

[0006] In a dynamic 6DOF environment, the acoustic description of the room (dimensions, RT60, ...) can vary a significant amount. The positions of the sound sources and receivers are completely free and have to be calculated in real time for the auralization. The perceptual parameters are highly dependent on these changing physical settings and cannot be defined as constants and are therefore not adequate for this task.

[0007] Herein, the invention has a new approach that requires only a few basic physical parameters of the environment to select and tune a simple basic ER pattern. This approach has the advantage that no specific sound engineering background is required to define the parameters. The parameters are obtained directly from the physical model. The simple ER patterns used are adaptable to different room sizes and different RT60 values. Even for outdoor environments, simple ER patterns are defined, which was not the case in SPAT. Since the human auditory system cannot analyze the fine structure of early reflections (e.g., non-patent document [6]), the perceptual degradation of this approach to a complete physically correct simulation is limited.

[0008] The following newly invented simple ER pattern uses room acoustic parameters such as RT60, predelay time, room volume or room dimensions, and frequency dependence of RT60. The ER pattern is specifically defined to create a smooth transition between direct sound and late reverberation. It should be frequency neutral and based on the proximity between walls, source opening, and receiver.

[0009] The aim is to match the acoustic parameters of the entire room and produce a plausible and convincing perception for the listener, which is sufficient in most cases since the listener cannot directly compare it to a "real" physically accurate ER.

[0010] It is possible to avoid spending precise geometric calculations of ER, especially in applications such as real-time auditory virtual environments and augmented reality, especially with visibility checks. A precise calculation of the "real" ER can be difficult and sensitive to generating artifacts by making the ER appear and disappear depending on the exact (and time-varying) positions of the sound source and the listener. This can be avoided by using a constant ER pattern calculated once when entering a scene by moving from one acoustic environment to another defined by different acoustic parameters.

[0011] The present invention utilizes an encoder-bitstream-renderer scenario. In one case (a), a default simple ER pattern can be calculated using room acoustic parameters available to the renderer alone. These parameters are adjusted in real time by the source-listener distance and the source-listener azimuth angle. In case (b), the scene geometry is pre-analyzed in a more sophisticated way in the encoder. A simple ER pattern of a small number of ERs is then pre-computed in the encoder and sent in the bitstream to the renderer, where it is adjusted by the listener distance and angle (or other information available at rendering time) in a similar way to case (a). These two cases give full flexibility for a wide range of future approaches where further analytical knowledge can be incorporated in the encoder later.

[0012] (motivation) The room impulse response (RIR) describes the relationship between sound sources and receivers (listeners) in an acoustic environment (room) and specifies the room's response to a unit impulse (see, for example, Fig. 21). The room impulse response consists of the direct sound path, early reflections (ER) and a diffuse late sound part. Fig. 21 shows an example of a monaural RIR with second-order ER, generated using the acoustic room simulation program RAVEN (Non-Patent Document [7]).

[0013] Especially in complex physical environments / rooms defined by many surfaces, a geometrically correct ER calculation with the necessary visibility checks ("Is this sound source in the direct line of sight to the listener?") is very time consuming. On the other hand, it is known that human auditory perception suppresses many details of the ER for direct sounds (Law of the First Wave Front, Precedence Effects, Scene Analysis (Non-Patent Documents [8] and [9]) and therefore in many cases an accurate modeling of the ER part in the impulse response is not necessary to achieve a convincing rendering quality (e.g., Non-Patent Document [6]). The auditory system uses the ER to determine or refine several perceptual attributes, among which are: Position of the sound source relative to the receiver Source-to-receiver distance Apparent source width (ASW) Level and frequency dependent absorption of the boundary (non-patent literature

[10] ) Proximity to a closed boundary

[0014] BACKGROUND OF THEINVENTION Several approaches are known to simplify the ER calculation. The first approach is to just avoid the calculation of ER altogether, i.e. to render a sound without simulated ER, i.e. only the direct sound and the late reverberation (see Fig. 22). The late reverberation starts at the so-called pre-delay time. Fig. 22 shows an RIR with a direct sound and a late reverberation starting at the pre-delay time 0.13s, and there is no ER.

[0015] The next possibility is to calculate only the geometrically strict first reflections (see Figure 23). For a shoebox room, this reduces the number of ERs from about 27 to 6. Figure 23 shows an RIR with first reflections, late reverberation (left) and a plan view (right). The square (red) is the source, the circle (blue) is the receiver, the line connecting the circle and the square (red) is the direct sound and the further lines coming out of the circle (blue) are the reverberations, with lengths proportional to the logarithmic level.

[0016] The next possibility is just two ERs aligned with the direct sound (see Fig. 24). The effect of lateral reflections on ASW is known from concert hall acoustics (non-patent literature

[11] ). Note that this is very simple to compute compared to true geometric simulations. Fig. 24 shows a RIR with two reflections aligned with the direct sound (left) and a top view (right).

[0017] In the next pattern, the two side reflections are replaced by four reflections to each side of the direct sound and four fixed source position independent reflection arrays, each consisting of four reflections (at ±45° and ±135°) (see Fig. 25). This pattern is inspired by the SPAT algorithm (non-patent literature [1] and [5]), but does not implement all the details, especially the effect of all the input parameters. The parameters for this pattern are specifically defined to generate ASW-like perceptual receiver attributes. Besides RT60, no room acoustic properties are used. Fig. 25 shows the RIR with the "SPAT" pattern (left) and the top view (right). The cross parts (green and blue) are the ER.

[0018] The aforementioned approach is designed such that the input parameters that define the ER pattern are perceptual parameters. The perceptual parameters should describe the listener's perception caused by the ER. The drawback is that it only vaguely adapts to room-related parameters. Setting perceptually defined parameters such as source presence, source warmth, source brilliance, room presence, dynamic reverberation, envelopment, and reverberation requires knowledge and experience of acoustic engineering. This is an obvious drawback for designers who define the physical characteristics of real-time VR / AR systems and do not have experience in perceptual sound engineering. Especially for VR applications, the geometry of the virtual physical space is often well known as a by-product of the visualization process. There are also no ER patterns for outdoor environments known for the SPAT algorithm. Summary of the Invention [Problem to be solved by the invention]

[0019] The aim of the present invention is to avoid the drawbacks of the state of the art by explicitly using room acoustic and physical parameters to define ER patterns. Moreover, different patterns are defined depending on the room characteristics, and these patterns are also suitable for outdoor environments (where precise description of the geometry is difficult). The patterns have different numbers of ER depending on the room size or other physical parameters.

[0020] The new ER pattern is Perceptually more plausible rendering compared to "real" ER -Reduction in the amount of calculation compared to "real" ER calculations -Adaption of ER patterns according to the physical characteristics of the room - Setting the required parameters does not require specific audio engineering skills and experience ·Separate ER patterns for indoor and outdoor use (For encoder / bitstream / renderer scenarios involving bitstream transmission) If the given pattern is computed within the renderer, no additional side information is required. (For encoder / bitstream / renderer scenarios involving bitstream transmission) if the predefined patterns are computed from the scene geometry within the encoder, very little additional lateral information is required; It is characterized by:

[0021] This is achieved by using parameterizable, yet fixed, spatial ER patterns that do not depend on the exact geometry of the room. In a preferred embodiment of the invention, the patterns are also independent of the listener position within the room. Instead, only one (or a few) global characteristic parameters are used to configure the ER pattern. In this way, the patterns can be rendered very efficiently.

[0022] The following newly invented ER patterns use room acoustic parameters, specifically RT60, pre-delay time, room dimensions or room volume, and frequency dependence of RT60 for pattern configuration. The ER patterns are defined to create a smooth transition (in time) between the direct sound and the late reverberation. They should be timbrally neutral. The ER patterns depend on the room volume and surfaces. The ER patterns are independent of the source and receiver positions in the room.

[0023] The goal of the present invention is to adapt to the overall room acoustic parameters and create a plausible and convincing perception for the listener, which is sufficient for most use cases, especially since the listener has no possibility to directly compare with a "real" physically accurate ER rendering. [Means for solving the problem]

[0024] According to a first aspect of the present invention, the inventors of the present application have recognized that one problem encountered when trying to utilize early reflection (ER) rendering of an audio signal is due to the fact that the early reflections depend on the relationship between the sound source position and the listener position. The inventors have found that it is possible to consider a source position independent ER pattern, e.g. without floor reflections, so that the rendering result is still very good and at the same time ER rendering becomes easier. The early reflection part of the room impulse response used for rendering is determined only by the early reflection pattern. The spatial relationship between the sound source and the listener is not considered for the early reflection part of the room impulse response. Furthermore, the early reflection position in the early reflection pattern is invariant with respect to changes in the listener's head orientation. This is based on the discovery that the same ER pattern can be used to determine the early reflection part of the room impulse response regardless of whether the listener looks at the sound source or in any other direction.

[0025] Thus, according to a first aspect of the present application, an apparatus for sound rendering is configured to receive information about a listener position and a sound source position. The apparatus is configured to render an audio signal of a sound source using a room impulse response, the early reflection part of the room impulse response being determined only by an early reflection pattern. The early reflection pattern refers to a constellation of early reflection positions, e.g. a constellation defined in terms of the angle between the lines connecting these positions and referring to a set of positions, a synonymous term being "pattern". The early reflection pattern is arranged at the listener position such that the early reflection positions are located around and in an angular direction from the listener position, the angular direction being invariant to changes in the listener head orientation, i.e. the constellation is translationally arranged at the listener position.

[0026] According to a second aspect of the present invention, the inventors of the present application have recognised that one problem encountered when attempting to use early reflection (ER) rendering of an audio signal stems from the fact that early reflection patterns for outdoor environments are highly individual and dependent on the physical setup of the scene. The inventors have found that ER patterns generated using a moderate analysis of the environment can yield acoustically convincing, yet computationally moderate ER rendering results.

[0027] Thus, according to a second aspect of the present application, an apparatus for determining an early reflection pattern for sound rendering is configured to perform a geometric analysis of an acoustic environment, the geometric analysis being performed by determining, at each of one or more analysis locations, a function indicative of a value representative of the early reflection contribution for each of different distances from each analysis location, and testing the function or a further function derived from the function for one or more maxima to derive one or more control parameters. Additionally, the apparatus is configured to determine an early reflection pattern indicative of a constellation of early reflection locations by positioning the early reflection locations using the one or more control parameters.

[0028] According to a third aspect of the present invention, the inventors of the present application have realised that one problem encountered when trying to exploit early reflection (ER) rendering of an audio signal stems from the fact that transmitting early reflection patterns of an audio scene for rendering can result in high signalling costs. The inventors have found that by using bitstream hints, ER patterns can be generated, resulting in acoustically convincing, but computationally moderate ER rendering results. By using only hints in the bitstream, the signalling costs can be reduced, as there is no need to transmit the complete ER pattern.

[0029] Thus, according to a third aspect of the present application, an apparatus for sound rendering is configured to receive first information regarding a listener position and a sound source position. The apparatus is configured to receive a bitstream comprising a representation of an audio signal of a sound source located at the sound source position and one or more early reflection pattern parameters, and is for example configured to read the representation of the audio signal and the early reflection pattern parameters from the bitstream. For example, the bitstream is an audio bitstream with the early reflection parameters in a header or metadata field of the bitstream, or a file format stream comprising an audio bitstream representing an audio signal and with the early reflection parameters in packets of the file format stream or tracks of the file format stream. Additionally, the apparatus is configured to determine an early reflection pattern indicative of a constellation of early reflection positions as a function of the one or more early reflection pattern parameters. Furthermore, the apparatus is configured to render the audio signal of the sound source using a room impulse response, the early reflection part of the room impulse response being determined by the early reflection pattern. The early reflection pattern indicative of a constellation of early reflection positions, for example a constellation defining a mutual arrangement of positions in terms of angles between lines connecting these positions and indicative of a set of positions, a synonymous term being "pattern". The early reflection pattern is positioned at the listener position such that the early reflection positions are located around and in an angular direction from the listener position, and the angular direction is invariant to changes in listener head orientation, i.e. the constellation is translationally positioned at the listener position.

[0030] According to a fourth aspect of the invention, the inventors of the present application have recognized that one problem encountered when attempting to utilize early reflection (ER) rendering of an audio signal is due to the fact that a huge amount of computation must be expended to determine each reflection from the sound source to the listener, taking into account wall geometry, occlusions, and other effects to calculate a physically accurate reflection pattern. The inventors have found that simple room acoustic parameters such as room dimensions, room volume, or pre-delay can be utilized to determine the number of early reflection positions in the early reflection pattern. There is no need to analyze the real early reflections of the scene, since the early reflections can be approximated depending on the room acoustic parameters. The inventors have found that the generation of the ER pattern by the dependence of the ER number on the room acoustic parameters results in acoustically convincing, but computationally moderate ER rendering results.

[0031] Thus, according to a fourth aspect of the present application, an apparatus for determining an early reflection pattern for sound rendering is configured to receive at least one room acoustic parameter representative of an acoustic characteristic of an acoustic environment, the apparatus being configured to determine an early reflection pattern indicative of a constellation of early reflection locations such that a number of the early reflection locations depends on the at least one room acoustic parameter.

[0032] According to a fifth aspect of the present invention, the inventors of the present application have recognized that one problem encountered when attempting to utilize early reflection (ER) rendering of an audio signal stems from the fact that each sound source is associated with a different early reflection pattern. The inventors have found that it is not necessary to use different ER patterns for signals of different sound sources. This is based on the idea that it is possible to weight and sum signals based on a source-listener relationship, such that a weighted sum is rendered based on the ER pattern. The inventors have found that ER rendering by using ER patterns for two or more sound sources results in acoustically convincing, but computationally moderate ER rendering results.

[0033] Thus, according to a fifth aspect of the present application, an apparatus for sound rendering is configured to receive information about a listener position, a first sound source position and a second sound source position. The apparatus is configured to render the audio signals of the two sound sources using a room impulse response, the early reflection part of the room impulse response being determined by an early reflection pattern. The early reflection pattern denotes a constellation of early reflection positions, e.g. a constellation defines the mutual arrangement of the positions in terms of the angle between the lines connecting these positions and denotes a set of positions, a synonymous word being "pattern". The early reflection pattern is arranged at the listener position such that the early reflection positions are located around the listener position and in an angular direction from the listener position, the angular direction being invariant to changes in the orientation of the listener's head, i.e. the constellation is translationally arranged at the listener position. The apparatus is configured to render the audio signals of the two sound sources by forming a weighted sum of a first audio signal of the first sound source arranged at the first sound source position and a second audio signal of the second sound source arranged at the second sound source position. The weighted sum weights the second audio signal more than the second audio signal if a first distance between the first sound source position and the listener position is less than a second distance between the second sound source position and the listener position, and weights the second audio signal more than the first audio signal if the first distance is greater than the second distance. Further, the apparatus is configured to render the audio signals of the two sound sources by generating an early reflection contributing loudspeaker signal associated with an early reflection portion of the room impulse response by rendering the weighted sum from the early reflection positions.

[0034] According to a sixth aspect of the invention, the inventors of the present application have recognized that one problem encountered when attempting to utilize early reflection (ER) rendering of an audio signal stems from the fact that in order to compute a physically accurate reflection pattern, a huge amount of computation must be expended in determining each reflection from the sound source to the listener, taking into account wall geometry, occlusions and other effects. The inventors have found that simple room acoustic parameters such as room dimensions, room volume or pre-delay can be utilized to parameterize a function that defines the location of the early reflections. There is no need to analyze the real early reflections of the scene, since the early reflections can be approximated according to the room acoustic parameters. Furthermore, it has been found that spiral functions provide a good distribution of early reflection locations. The inventors have found that ER pattern generation using one or more spiral functions results in perceptually convincing, yet computationally moderate ER rendering results.

[0035] Thus, according to a sixth aspect of the present application, an apparatus for determining an early reflection pattern for sound rendering is configured to receive at least one room acoustic parameter representative of an acoustic characteristic of an acoustic environment, and to determine an early reflection pattern indicating a constellation of early reflection positions by parameterizing one or more helical functions centered on a listener position, and to position the early reflection positions using the one or more helical functions.

[0036] The drawings are not necessarily to scale, emphasis instead being placed generally upon illustrating the principles of the invention.In the following description, various embodiments of the invention are described with reference to the following drawings, in which: [Brief description of the drawings]

[0037] [Figure 1] 1 illustrates an embodiment of an early reflection pattern. [Diagram 2] 1 illustrates an embodiment of an early reflection pattern determined using a helical function. [Diagram 3]Figure 3a shows an embodiment of an early reflection pattern versus time, Figure 3b shows an embodiment of an early reflection pattern versus a top-down spatial view, and Figure 3c shows an embodiment of an early reflection pattern versus frequency dependence. [Figure 4] It shows the level relationship between the listener, the direct source and the reflections. [Diagram 5] 1 shows an implementation of a simple ER algorithm in an encoder / decoder / renderer. [Figure 6] SUMMARY OF THE DISCLOSURE An apparatus for determining early reflection patterns by analyzing an environment is presented. [Figure 7] FIG. 1 shows a top-down aerial view of an embodiment of an ER pattern with four initial reflection locations. [Figure 8] 1 shows a geometric analysis of an outdoor scene. [Figure 9] The mesh of analysis points is shown. [Figure 10] The distribution of reflecting surfaces versus distance is shown, averaged over several analysis locations. [Figure 11a] 1 shows a first embodiment of an outdoor ER pattern. [Figure 11b] 1 shows a second embodiment of an outdoor ER pattern. [Figure 12] The amplitude falloff versus distance for a point sound source with different values ​​of distAlpha. [Figure 13] FIG. 1 shows a block diagram illustrating the summing of different sound sources into one source signal using distance weighting method. [Figure 14] It shows the level relationships between the listener, the two direct sources, and the summed reflections. [Figure 15] 1 shows an example of the overall rendering process. [Figure 16] 1 illustrates an embodiment of an apparatus for sound rendering. [Figure 17] 1 illustrates an embodiment of an apparatus for sound rendering using ER pattern parameters. [Figure 18] 1 illustrates an embodiment of an apparatus for determining an ER pattern depending on indoor acoustic parameters. [Figure 19] 1 illustrates an embodiment of an apparatus for rendering a weighted sum of two or more source signals. [Figure 20] 1 illustrates an embodiment of an apparatus for determining an ER pattern using a helical function. [Figure 21] An example of a monaural second-order RIR generated using the room acoustics simulation program RAVEN is shown below. [Figure 22] It shows a RIP with direct sound and late reverberation starting at a pre-delay time of 0.13 s, and no ER. [Figure 23] A RIP with first reflections and late reverberation (left) and a top view of the RIP (right) are shown. [Figure 24] Shown is an RIR with two reflections aligned to the direct sound (left) and a top view of the RIP (right). [Diagram 25] Shown is an RIR with a "SPAT" pattern (left) and a top view of a RIP (right). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] In the following description, identical or equivalent elements, or elements having identical or equivalent functions, are designated by the same or equivalent reference numbers even if they are present in different drawings.

[0039] In the following description, numerous details are set forth to more thoroughly explain the embodiments of the present invention. However, it will be apparent to one skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the embodiments of the present invention. Furthermore, features of different embodiments described below in this specification may be combined with each other unless otherwise specified.

[0040] In the following, various examples are described that may help achieve reduced complexity in audio rendering when using early reflection processing concepts. The simplified early reflection processing concepts described herein may, for example, be provided in addition to other heuristically designed early reflection processing concepts, or exclusively.

[0041] To facilitate understanding of the following embodiments of the present application, we begin with a general presentation of an early reflection pattern 1 according to an embodiment of the present invention. The features described with respect to the early reflection pattern 1 of FIG. 1 can also be applied to any other early reflection pattern 1 described herein.

[0042] The early reflection pattern 1 denotes a constellation of early reflection positions ERP (see ERP1 and ERP2). For example, a constellation may denote a set of positions ERP and define their mutual arrangement, for example in terms of the angle α between the lines connecting these positions with the center 2 of the pattern 1. A synonym for constellation is "pattern".

[0043] The early reflection position ERP, i.e., the position of the early reflection, may indicate or identify a position within the environment 5, e.g., an indoor room or an outdoor area, where early reflections of an audio signal may occur. For example, a listener located at the center 2 of the early reflection pattern 1 may perceive early reflections coming from the early reflection position ERP. That is, the early reflection position ERP may indicate the position from which a listener located at the center of the early reflection pattern 1 will receive early reflections from that position.

[0044] The early reflection pattern 1 is, for example, arranged at the listener position 10 such that the early reflection positions ERP are located around the listener position 10 and in an angular direction from the listener position 10, the angular direction being invariant to changes in the listener head orientation, i.e. the constellation is translationally arranged at the listener position 10. For example, the early reflection positions ERP may be determined to be approximately uniformly angularly distributed around the listener position 10.

[0045] According to one embodiment, the early reflection pattern 1, i.e. the early reflection positions ERP, may be determined such that the lines connecting each early reflection position ERP1 / ERP2 and the listener position 10 (see 7 and 8 in Fig. 1) do not overlap each other, i.e. are separate from each other. This allows for an even distribution and prevents stacking of early reflection positions in the environment 5.

[0046] 1, the center 2 of the early reflection pattern 1 may be located at the listener position 10. The center 2 of the early reflection pattern 1 may be linked to the listener position 10 and the early reflection pattern 1 may translate together with the listener. However, if the listener rotates, the early reflection position ERP does not change, i.e. the early reflection pattern 1 does not follow the rotational movement of the listener.

[0047] According to one embodiment, the early reflection position ERP, together with the listener position 10, lies in a horizontal plane.

[0048] According to an embodiment, an apparatus for audio rendering or for generating an early reflection pattern 1 may be configured to determine the early reflection positions ERP by adjusting the azimuth rotation of the constellation according to a pattern orientation parameter in a bitstream containing a representation of the audio signal to be rendered. In other words, the complete early reflection pattern 1 may be rotated to more adequately approximate the real early reflections, for example in a specific environment 5. This azimuth rotation is not performed in response to a movement, for example a rotational movement of the listener. The adjustment of the azimuth rotation of the constellation may be performed at the initial determination of the early reflection pattern 1. Once the early reflection pattern 1 is determined, all early reflection positions ERP may only undergo the same translation in response to a translation of the listener position 10. The positioning of the early reflection positions ERP with respect to the center 2 of the pattern 1 may be determined with the aid of the adjustment of the azimuth rotation of the constellation. Once determined, the pattern 1 may not be further adjusted, i.e. a movement of the listener position does not change the relative positioning between the early reflection positions ERP and the center 2 of the pattern 1.

[0049] According to an embodiment, at least one room acoustic parameter representative of an acoustic characteristic of the acoustic environment may be taken into account when determining the early reflection pattern. The at least one room acoustic parameter comprises one or more of the following: room dimensions, room volume, and pre-delay time until late reverberation. Preferably, the at least one room acoustic parameter comprises only one of these acoustic characteristics of the acoustic environment. The at least one room acoustic parameter may be received or read from a bitstream, for example a bitstream comprising a representation of an audio signal to be rendered using early reflection pattern 1.

[0050] According to one embodiment, the early reflection pattern 1 can be determined such that the number of early reflection positions depends on at least one room acoustic parameter and / or the mutual spacing of the early reflection positions is changed / adapted in response to at least one room acoustic parameter, e.g. the mutual spacing of the early reflection positions varies with a central extension around the listener position.

[0051] According to one embodiment, the number of early reflex positions ERP in Pattern 1 is: the larger the room dimensions, the greater the number of early reflection locations and / or the furthest early reflection location from the listener position is spaced farther apart, or the larger the volume of the room, the greater the number of early reflection positions and / or the furthest early reflection positions from the listener position are spaced farther apart, or The greater the pre-delay time before the late reverberation, the greater the number of early reflection positions and / or the farthest early reflection positions are spaced farther apart from the listener position. It can be determined.

[0052] The "early reflection position farthest from the listener position" is understood to be the "distance from the farthest of the early reflection positions to the listener position." According to one embodiment, the early reflection positions ERP are located near the center 2 of the pattern 1, and the more early reflection positions ERP are included in the pattern 1, the farther away the farthest early reflection position is from the center 2.

[0053] According to an embodiment, the mutual spacing of the early reflection positions ERP can be modified / adapted depending on at least one room acoustic parameter by uniformly increasing the distance from each early reflection position ERP to the center 2 as the room size, the room volume or the pre-delay time to late reverberation increases. Optionally, the mutual spacing of the early reflection positions ERP can be modified / adapted depending on at least one room acoustic parameter such that the larger the room size, or the larger the room volume or the larger the pre-delay time to late reverberation, the larger the distance from the most distant one of the early reflection positions ERP to the listener position 10 and the distance is smaller than the pre-delay time. This allows a uniform distribution of the early reflection positions ERP and therefore an acoustically convincing ER rendering result. It can be advantageous if the distance from the most distant one of the early reflection positions ERP to the listener position 10 increases more than the distance from the closest one of the early reflection positions ERP to the listener position 10 as the room size, the room volume or the pre-delay time to late reverberation increases.

[0054] FIG. 2 shows an embodiment of an early reflection pattern 1 that can be used for early reflection processing of an audio signal. The early reflection pattern 1 includes early reflection positions ERP (see ERP11 to ERP15 (ERP1) and ERP21 to ERP25 (ERP2) in FIG. 2). FIG. 2 shows ten early reflection positions ERP as an example. However, it is clear that the early reflection pattern 1 can include a different number of early reflection positions ERP. The early reflection pattern 1 may include two or more early reflection positions ERP, for example, only early reflection positions ERP11 and ERP21.

[0055] As shown in Fig. 2, two helical functions 3 and 4 centered on a listener position, i.e. a center 2, can define the positions of early reflections, i.e. early reflection positions ERP, for example early reflection positions ERP in an environment 5. However, it is clear that alternatively the positions of early reflections can be defined by only one helical function 3 or 4, or by two or more helical functions. An apparatus for rendering audio or for generating an early reflection pattern 1 may be configured to position the early reflection positions ERP using one or more helical functions 3, 4 in order to determine an early reflection pattern 1 in an environment 5. For example, each apparatus may be configured to position a first set of early reflection positions ERP1 (see ERP11 to ERP15) using a first helical function 3 and to position a second set of early reflection positions ERP2 (ERP21 to ERP25) using a second helical function 4.

[0056] Each of the first set of early reflection positions ERP1 is associated with a corresponding second set of early reflection positions ERP2. For example, the early reflection position ERP11 may be associated with a corresponding early reflection position ERP21, the early reflection position ERP12 may be associated with a corresponding early reflection position ERP22, the early reflection position ERP13 may be associated with a corresponding early reflection position ERP23, the early reflection position ERP14 may be associated with a corresponding early reflection position ERP24, and the early reflection position ERP15 may be associated with a corresponding early reflection position ERP25. For each of the first set of early reflection positions ERP1, each of the early reflection positions ERP1 is located on the opposite side of a line perpendicularly intersecting a connection line between each of the early reflection positions ERP1 and the corresponding early reflection position ERP2 of the second set of early reflection positions ERP2. This ensures that the listener receives early reflections from different directions and also prevents early reflection positions from piling up in one area. This arrangement using a helical function allows for an even distribution of early reflection positions within the environment 5, resulting in an acoustically convincing, yet computationally moderate, early reflection rendering of the audio signal.

[0057] Figure 2 shows an example in which, for each initial reflection position ERP1 of the first set, the corresponding initial reflection position ERP2 of the second set of initial reflection positions ERP2 is angularly offset relative to the connecting line and offset in a common angular direction for all initial reflection positions ERP1 of the first set of initial reflection positions ERP1.

[0058] According to one embodiment, a device for audio rendering or for generating an early reflection pattern 1 uses two helical functions 3 and 4: Each of the first set of early reflection positions ERP1 is associated with a corresponding one of the second set of early reflections ERP2; and For each of the first set of initial reflection positions ERP1, each initial reflection position ERP1 is disposed on one side of a line that perpendicularly intersects an axis passing through the pattern center 2 and each of the first set of reflection positions ERP1 at the pattern center 2, and each corresponding initial reflection position ERP2 of the second set of initial reflections ERP2 is disposed on the opposite side of each line; and such that each corresponding initial reflection position ERP2 of the second set of initial reflection positions ERP2 is angularly offset (see γ for the corresponding initial reflection positions ERP11 and ERP21) with respect to the respective axis in an angular direction that is common for all initial reflection positions ERP1 of the first set of initial reflection positions ERP1 and / or common for all initial reflection positions ERP2 of the second set of initial reflection positions ERP2, It may be configured to position the initial reflection positions ERP1 and ERP2.

[0059] The one or more helical functions 3, 4 can define the initial reflection positions ERP in polar coordinates (r, β). 1~5 , β1 1~5 ) for defining the first reflection position ERP2 of the second set of first reflection positions ERP2, and (r2 1~5 , β2 1~5 ) for more information.

[0060] As will be explained in more detail below, in particular with reference to section 1 "Indoor ER Parameter Calculation", the one or more helical functions 3, 4 may be parameterized depending on at least one room acoustic parameter, i.e. each helical function 3, 4 defines a respective early reflection position ERP that depends on at least one room acoustic parameter. The at least one room acoustic parameter may include one or more of the following: room dimensions, room volume, and pre-delay time to late reverberation. The at least one room acoustic parameter may represent an acoustic characteristic of the acoustic environment 5.

[0061] For example, one or more of the spiral functions 3, 4 may be The larger the room dimensions or the larger the room volume or the longer the pre-delay time to late reverberation, the larger the number of early reflection positions ERPs, and / or For each early reflection position ERP, the larger the room dimensions, the larger the room volume, or the longer the pre-delay time until the late reverberation, the greater the distance from each early reflection position ERP to the center 2 of the early reflection pattern 1. It can be parameterized according to at least one room acoustic parameter.

[0062] According to one embodiment, an apparatus for audio rendering or for generating an early reflection pattern 1 may be configured to parameterize one or more spiral functions and determine the number of early reflection positions ERP such that the larger the room dimensions or the larger the room volume or the larger the pre-delay time until late reverberation, the greater the distance from the most distant one of the early reflection positions to the listener position, which distance is smaller than the pre-delay time.

[0063] According to an embodiment, the device for audio rendering or for generating the early reflection pattern 1 may be configured to support different determinations of the early reflection pattern. The device for audio rendering or for generating the early reflection pattern 1 may be configured to select the type of determination depending on the environment 5. For example, determining the early reflection pattern 1 using one or more helical functions 3, 4, e.g. the first determination, and / or determining the early reflection pattern 1 such that the number of early reflection positions depends on at least one room acoustic parameter, e.g. the first determination, may be associated with an indoor environment such as a room (see in particular section 1 "Indoor ER Parameter Calculation"). Such a determination, e.g. the first determination, may be selected if the acoustic environment 5 is an indoor environment or if a pattern type index in a bitstream containing a representation of the audio signal to be rendered assumes a predefined state. Another determination, e.g. the second determination, is described in more detail in section 3 "Outdoor ER Patterns".

[0064] As already mentioned above, one of the newly invented ER patterns 1 for indoor use consists of two spirals (see Fig. 3). This pattern 1 has the advantage of covering all directions around the listener 10 while providing a uniform distribution over time without clustering. The number of early reflections (ER) can be adapted to the size of the room, which can also be derived from the pre-delay of the late reverberation. The frequency dependence of the RT60 can also define the frequency dependence of the ER. The RT60, or average absorption coefficient, defines an additional amplification in addition to the usual distance effect. From the frequency dependence of the RT60, a simple shelving filter is calculated to adapt the frequency response of the early reflections to the overall absorption behavior described by the RT60. Fig. 3 shows the new ER pattern 1 over a) time, b) spatial view from above, and c) frequency dependence.

[0065] (1. Indoor ER parameter calculation) The following description of the indoor ER parameter calculations refers to FIGS.

[0066] The variable parameters of the spiral patterns, i.e. the first spiral function 3 and the second spiral function 4, are mainly set by the pre-delay time, for example, the pre-delay time until the late reverberation is used, which is for example as follows:

number

[0067] The parameter is set to depend on the room's pre-delay, which defines the onset time of the late reverberation and is calculated using equation (1) below.

number

[0068] A first set of early reflection positions ERP1 can be determined in polar coordinates as (r1;β1) and a second set of early reflection positions ERP2 can be determined in polar coordinates as (r2;β2) using a first helical function 3 and a second helical function 4. Calculation of the azimuth and radius of the ER positions using the two helical patterns is as follows:

number

[0069] The constant distfactor may correspond to the constant distFac described above. According to one embodiment, the distfactor may be determined based on at least room acoustic parameters, for example, the distfactor may be determined such that the greater the pre-delay time before late reverberation, the greater the distfactor.

[0070] As can be seen in FIG. 2, a polar axis 6 passes through the center 2 of the initial reflection pattern 1. The origin, i.e. center 2, of the initial reflection pattern 1 represents the pole. A half line passes from the pole in a reference direction, i.e. representing the polar axis 6, and the initial reflection position ERB1 of the first set of initial reflection positions ERB1(1~5) The azimuth angle β1 defines the angular coordinate of (1~5) , and the initial reflection position ERB2 of the second set of initial reflection positions ERB2 (1~5) The azimuth angle β2 defines the angular coordinate of (1~5) represents the angle from the polar axis 6. The radial coordinate of the initial reflection position ERP1 is oriented in the reference direction, and the radial coordinate of the initial reflection position ERP is oriented in the direction opposite to the reference direction (see FIG. 2 and Equations 4 and 5).

[0071] The apparatus for sound rendering may be configured to generate early reflection contribution loudspeaker signals related to the early reflection parts of the room impulse response by performing a rendering of the audio signals of one or more sound sources from early reflection positions ERP, for example by adjusting the level depending on the distance from each early reflection position to the listener position (see, for example, the determination of amp1 and amp2 above). For example, for each of a first set of early reflection positions ERB1, the audio signal of the sound source is rendered from each early reflection position ERB1 at a level amp1, and for each of a second set of early reflection positions ERB2, the audio signal of the sound source is rendered from each early reflection position ERB2 at a level amp2.

[0072] The amplitude of the reflection is influenced by several of the following parameters: a) Standard Distance Method (degrades by a factor of 2 as distance doubles) b) Correction according to the following formula (6): Depends on.

number

[0073] As can be seen in Figure 4, the level relationship between reflections and direct source levels is fixed. The levels of the five sources shown (one direct source and four early reflections) move up and down with respect to the source / listener distance (sldistance). Figure 4 shows the level relationships between the listener, the direct source and the reflections.

[0074] Rendering the audio signal of the sound source coming from each early reflection location at a level adjusted according to the distance from each early reflection location to the listener's position offsetting (20) the level at which the audio signal of the sound source is rendered from each early reflection location using a level offset or amplifying said level by a level factor, the level offset or level factor being common for all early reflection locations; and Set the level offset or level factor according to the amplitude correction factor (see Equation 6) It can be executed by:

[0075] For example, for each of the first set of early reflection positions ERB1, the level amp1 at which the audio signal of the sound source is rendered from each early reflection position ERB1 is offset by ampCorrection (see Equation 6), and for each of the second set of early reflection positions ERB2, the level amp2 at which the audio signal of the sound source is rendered from each early reflection position ERB2 is offset by ampCorrection (see Equation 6). The amplitude correction factor, i.e. ampCorrection in Equation 6, may be included in a bitstream containing a representation of the audio signal. According to one embodiment, the amplitude correction factor is included in one or more early reflection pattern parameters.

[0076] According to one embodiment, rendering the audio signal of the sound source from each early reflection position in a manner that adjusts the level as a function of the distance from each early reflection position to the listener position can be performed by modifying the level adjustment as a function of the distance from each early reflection position to the listener position, relative to the level adjustment used by the device for rendering the audio signal from the sound source position, according to a distance attenuation (amp1 and amp2). The distance attenuation may be included in a bitstream that includes a representation of the audio signal. According to one embodiment, this attenuation is included in one or more early reflection pattern parameters.

[0077] As can be seen from figure 4, when rendering, the level at which the audio signal of the sound source is rendered from each early reflection position is offset by 20, and the same offset is applied to all early reflection positions ERP of early reflection pattern 1. Additionally, when rendering, the level at which the audio signal of the sound source is rendered from each early reflection position may be attenuated depending on the distance between each early reflection position and the listener, for example using a compensation distance method.

[0078] As described above for an audio signal of a single source, the rendering technique can also be applied to two or more audio signals of two or more sources, with special rendering being applied to a weighted sum of two or more audio signals, the calculation of which is described in more detail in Section 5.

[0079] (2. Implementation in VR system) Figure 5 shows a structural diagram of a simple ER software algorithm in an encoder / decoder environment. Figure 5 shows the implementation of a simple ER algorithm in the encoder and decoder / renderer. First, a decision is made whether to use a predefined ER pattern or not. The next decision is whether the ER pattern is for indoor or outdoor. For indoor patterns, no further parameters need to be sent. The ER pattern is calculated from the already existing acoustic scene parameters. For outdoor patterns, the scene geometry is analyzed, these parameters are sent, and the ER outdoor pattern is calculated in the decoder. For details, see section 3. For transitions from one acoustic environment to the next, see section 4. For handling multiple sound sources in one scene, see section 5.

[0080] (3. Outdoor ER Pattern) The embodiment shown in Fig. 6 relates to an apparatus 100 for determining an early reflection pattern 1 for sound rendering, which is configured to perform a geometric analysis 110 of an acoustic environment 5 by determining, at each of one or more analysis positions 50 (see 501 to 505), a function 112 indicative of a value 116 representative of an early reflection contribution for each of different distances 114 from each analysis position 50. The function 112 or a further function derived from the function 112 is analyzed with respect to one or more maxima 118 to derive one or more control parameters 120. Furthermore, the apparatus 100 is configured to determine an early reflection pattern 1 indicative of a constellation of early reflection positions ERP (see ERP1 to ERP4) by positioning the early reflection positions using the one or more control parameters. Features of the apparatus 100 are explained in more detail below.

[0081] Specifically, for an outdoor scene (although not limited to an outdoor scene), a new pattern 1 is designed with four ERs arranged in a nearly intersecting manner (see FIG. 7). FIG. 7 shows a top-down spatial view of the new ER pattern 1 with four early reflection positions ERP1-ERP4. The different distances, i.e., the individual distances between each early reflection position and the center 2, can be defined by pre-delay times and compression factors derived from a geometric analysis 110 of the scene, i.e., environment 5.

[0082] The use of ER patterns for outdoor environments is highly specific and depends on the physical setting of the scene. The geometric analysis 110 described below captures perceptually significant features of the outdoor scene, i.e., the environment 5, that are relevant for the perception of ER.

[0083] FIG. 8 shows a geometric analysis of an outdoor scene. a) Top view of a ring around an analysis point; b) Side view of the ring around the analysis point with increasing height. From a central listening point, e.g. analysis point 50, concentric rings are placed. The area of ​​the ring defined by the radius and height represents the maximum possible reflected energy at this distance (see FIG. 8). There is a spacing d (e.g. 3 m) between the rings. A half-line with an angular spacing α (e.g. 6°) is sent out from the analysis point 50. The first surface that is hit is counted into the existing reflecting surfaces at this distance and summed over the ring. With this approach, it is possible to determine a function 112 that indicates a value representing the early reflection contribution for each of the different distances from each analysis position 50. This function may be determined for each analysis point 50.

[0084] In other words, the acoustic environment 5 is radially sampled with respect to the nearest reflecting surface distance to obtain radial sampling results. Additionally, a radial integration over the radial sampling results and a weighting of the radial sampling results may be performed to obtain the function 112. The weighting may be performed according to the radial distance such that the early reflection contribution decreases with increasing distance.

[0085] Figure 9 shows a mesh of analysis points 50 in (a) top view and (b) side view. The dashed-dotted line indicates the user reachable area of ​​the scene, i.e. the environment 5. Inside the user reachable area, several (e.g., nine) analysis points are located (see Figure 9). The mesh is a 3D mesh, since some points are inside the geometric mesh of the scene and need to be excluded.

[0086] Alternatively, to analyze each function 112 for each analysis point, it is advantageous from an efficiency point of view to sum, for example average, the functions 112 determined at one or more analysis locations to obtain a further function 112' as shown in Fig. 10. The data over all mesh points can be averaged and the distribution analyzed. This represents the outdoor reflected energy over space and distance (see Fig. 10). Fig. 10 shows the distribution of reflective surface area versus distance, averaged over several analysis points 50.

[0087] As can be seen from figure 10, further functions 112' derived from the functions associated with the individual analysis points are examined with respect to the two maxima to derive, as one or more control parameters 120, a first amplitude a1 and a first distance p1 to the closest of the two maxima 1181 and a second amplitude a2 and a second distance p2 to the furthest of the two maxima 1182. Alternatively, it is possible to derive one or more control parameters 120 from each of the functions associated with the individual analysis points.

[0088] The amplitudes a1 and a2, together with their distances p1 and p2, are for example input values ​​for calculating outdoor ER pattern 1. Outdoor ER pattern 1 includes four ERs (see FIG. 11a).

[0089] According to the embodiment shown in FIG. 11a, the ER pattern 1 is Setting the distance from the listener position 10 to the first early reflection position ERP1 and the third early reflection position ERP3 in accordance with p2; and Setting a ratio (see compFactor) between the distance from the listener position to the first early reflection position ERP1 and the third early reflection position ERP3 and the distance from the listener position to the second early reflection position ERP2 and the fourth early reflection position ERP4 based on the difference or quotient between a first term that depends on a1 and a second term that depends on a2. is determined by.

[0090] Figure 11a shows an outdoor ER pattern 1 of four reflections (see circle (blue) and cross (red) around the listener). The distance p2 to the maximum value 1182 of the second distribution defines the distance to the two more distant reflections (see early reflection positions ERP1 and ERP3). The compression factor compFactor can define the distance to the two closer reflections (see early reflection positions ERP2 and ERP4). The compression factor can be defined by the relationship between the amplitudes, for example:

number

[0091] Four early reflex position ERPs i can be arranged to be located at polar coordinates (r(i), β(i)) (i=1 to 4).

[0092] The angular coordinates may be: β(1) ≈ 5° - 15°, β(2) ≈ 90° - 110°, β(3) ≈ 180° - 200°, β(4) ≈ 270° - 290°. According to one embodiment:

number

[0093] The radial coordinate may be determined according to Equations 7 and 8 below, and may be allowed to deviate from the calculated radius value by up to 40%.

number

[0094] As can be seen, the radial coordinates of the early reflection positions ERP1 and ERP3 are determined using Equation 7, and for the early reflection positions ERP2 and ERP4, Equation 7 is modified to become Equation 8.

[0095] According to an embodiment shown in Fig. 11b, the four early reflection positions ERP1 to ERP4 can be arranged such that the first early reflection position ERP1 and the second early reflection position ERP2 are located opposite each other with respect to a first line 1000 that crosses the listener position 10, and the third early reflection position ERP3 and the fourth early reflection position ERP4 are located opposite each other with respect to a second line 2000 that is perpendicular to the first line 1000 and crosses the listener position 10. According to one embodiment, the ER pattern 1 is: Setting the distance from the listener position 10 to the first early reflection position ERP1 and the second early reflection position ERP2 to depend on p2; and Setting a ratio between the distances to the first and third early reflection positions ERP1 and ERP3 and the distances to the second and fourth early reflection positions ERP2 and ERP4 based on a difference or quotient between a first term that depends on a1 and a second term that depends on a2. is determined by.

[0096] The level falloff of an acoustic point source in free-field conditions follows the 1 / r law, which corresponds to a falloff in amplitude by a factor of 2 for every doubling of distance (

[13] ). If the contributions of different reflecting regions are aggregated to a small number of ERs, this falloff with distance should be reduced by an exponential factor.

number

[0097] The value of distAlpha [0.5..1] can be estimated from the area distribution, e.g.

number

[0098] A deviation of approximately 20% from the calculated distAlpha value may be tolerated.

[0099] According to one embodiment, distAlpha can be set as follows:

number

[0100] FIG. 12 shows the amplitude reduction versus distance of a point source for different values ​​of distAlpha.

[0101] When the geometric analysis is performed in the encoder, the only parameters that need to be passed to the renderer are the algorithm parameters: predelay, compFactor, and distAlpha.

[0102] If a more detailed geometric analysis yields an ER pattern that cannot be derived by the equations defined above, then every single reflection position and relative amplitude can be transmitted individually to represent the desired pattern.

[0103] Example values ​​from geometric analysis of different outdoor scenarios to calculate the ER patterns are as follows: [preDelay, compFac, ampFac, distAlpha] Outdoor location surrounded by rocks [144,0.47,2.2,1] Town Street [109,0.44,1,0,65] Park in town [57,0.58,1,0,58]

[0104] As already mentioned above with respect to Fig. 2, according to an embodiment, the device for audio rendering or for the generation of the early reflection pattern 1 may be configured to support different determinations of the early reflection pattern. The device for audio rendering or for the generation of the early reflection pattern 1 may be configured to select a type of determination depending on the environment 5. According to an embodiment, a first determination may be performed that includes positioning the early reflection position ERP using one or more control parameters 120 as described in this section. The first determination may be selected if the acoustic environment is an outdoor environment or if a pattern type index in a bitstream that includes a representation of the audio signal to be rendered assumes a predefined state. Optionally, a second determination may be performed using one or more helical functions, as described above. However, it is clear that other types of determination may also be available for selection.

[0105] (4. Portal Operation) Portals represent the boundary from one acoustic environment to the next, from one room to the next, or from a room to a free-field environment. To smooth the transition through such portals, a cross-fading process between related simple ER patterns is useful. For example, within a region of d = 5m, the level of contribution from one acoustic environment fades out.

[0106] According to one embodiment, an apparatus for rendering may be configured to support a first determination scheme of the initial reflection pattern 1 and a second determination scheme of the initial reflection pattern 1, the first determination scheme being different from the second determination scheme (e.g., see the description of Section 1 and FIG. 2 for the first determination scheme, and Section 3 for the second determination scheme). The apparatus may be configured to use the first determination scheme or the second determination scheme when determining the initial reflection pattern 1 depending on a pattern type index. This index may be included in one or more of the initial reflection pattern parameters.

[0107] (5. Combining multiple sound sources into one ER pattern) In a real environment, every sound source has an individual ER pattern that depends on the location of the sound source and the receiver. In a simplified simulation, all sound sources in an environment have the same ER pattern arranged around the listener. When the sound source or the listener moves, the source-listener distance changes and therefore the important level relationship with the direct sound changes. This level relationship must be maintained.

[0108] In a preferred embodiment of the present invention, this can be accommodated in a computationally efficient way, as explained in FIG. 13. FIG. 13 shows a block diagram illustrating the summing of different sound sources (AS1, AS2, ...) into one sound source signal using a distance weighting method. First, the level relationship between the different sound sources AS is considered based on the value of the distance between the sound source and the listener. Then, using an appropriate distance weighting method, the different sound sources AS can be summed into a single sound source signal. Therefore, only one ER pattern 1 that covers all sound sources AS in the simulated environment has to be auralized. This pattern 1 follows the lateral movement of the listener (i.e., the translation in the x, y, z directions, but not the listener's head orientation). Specifically, when the listener moves in a certain direction, the position ERP of the ER in the ER pattern 1 moves with the listener. However, the ERP remains in a constant predefined spatial orientation, regardless of the listener's head orientation.

[0109] According to an embodiment, an audio rendering device for generating an early reflection pattern 1 may be configured to render signals of two or more sound sources using a room impulse response, the early reflection part of the room impulse response being determined by the early reflection pattern by forming a weighted sum of a first audio signal of a first sound source arranged at a first sound source position and a second audio signal of a second sound source arranged at a second sound source position, and by generating an early reflection contribution loudspeaker signal associated with the early reflection position of the room impulse response by rendering the weighted sum from the early reflection positions. The weighted sum may for example weight the first audio signal more than the second audio signal if a first distance between the first sound source position and the listener position is smaller than a second distance between the second sound source position and the listener position, and weight the second audio signal more than the first audio signal if the first distance is larger than the second distance.

[0110] According to one embodiment, the early reflection contribution loudspeaker signals for the early reflection portion of the room impulse response may be generated by rendering a weighted sum from each early reflection location with a methodology level adjusted according to the distance from each early reflection location to the listener position.

[0111] In Fig. 14 the level relationship between the listener, two direct sound sources and the reflections of the two direct sound sources is visualized. The level of each direct sound source depends on the individual source-listener distances. These can vary individually. The common level of the direct sound sources is calculated by summing the individual levels. From this level the associated reflections are calculated according to their respective distances.

[0112] FIG. 14 shows the level relationships between the listener, the two direct sources and the summed reflections.

[0113] The degradation caused by source-listener distance is source specific. There is an additional ampCorrection for the complete ER pattern.

number

[0114] (6. Brief Overview) (6.1. Rendering Aspects) 1. A renderer equipped to render early reflection patterns in a virtual auditory environment, the early reflection patterns comprising: Does not depend on a detailed geometric description of the room, e.g. may only consider the room dimensions and / or the room volume and / or the pre-delay until late reverberation. It does not depend on the individual sound source and listener positions (all sound sources in an environment share the same ER pattern), but only on the sound source-listener distance. For example, in early reflection position ERP, they are rendered at a fixed position relative to the user (rather than a position in space depending on the source and listener positions). · Rendered to the user at a fixed position, e.g. the early reflection position ERP (rather than a position in space that depends on the sound source and the listener's position). In the preferred embodiment, the position of the ER in the pattern, i.e., the early reflection position ERP, follows the lateral movement of the listener (i.e., translation in x, y, z directions, but not the listener's head orientation). Specifically, as the listener moves in a given direction, the position of the ER in the ER pattern moves with the listener. However, the ERP remains at a constant, given spatial orientation, regardless of the listener's head orientation.

[0115] An example of an overall rendering process is shown in Fig. 15. One or more of the features described with respect to Fig. 15 may be included in an apparatus for sound rendering as described herein.

[0116] 15 shows an apparatus 200 for sound rendering. The apparatus 200 is configured to render one or more audio signals 2121 / 2122 of one or more sound sources 2101 / 2102. The audio signals 212 (see 2121 and 2122) can be rendered by taking into account the direct sound (see 2201 and 2202), the early reflections (see 230) and / or the late reverberations (see 240).

[0117] In the direct path 2201 / 2202, one or more audio signals 2121 / 2122 can be rendered to obtain a direct sound contribution loudspeaker signal 2221 / 2222 for each of the one or more audio signals 2121 / 212. For example, for each of the audio signals 2121 and 2122 to be rendered, a respective direct sound contribution loudspeaker signal 2221 / 2222 can be determined taking into account the distance d1 / d2 between each associated sound source 2101 / 2102 and the listener position 10, as well as the angle α1 / α2 between each sound source 2101 / 2102 and the listener's orientation. The direct sound contribution loudspeaker signal 2221 / 2222 relates to the direct sound source part of the room impulse response.

[0118] According to an embodiment, the device 200 may be configured to mix (260) one or more audio signals 2121 / 2122 of one or more sound sources 2101 / 2102 to obtain a mixed audio signal 262. In the mixing 260, the signals 2121 / 2122 may be panned depending on the position of each associated sound source 2101 / 2102. For example, for each of the audio signals 2121 / 2122, the distance d1 / d2 between each associated sound source 2101 / 2102 and the listener position 10 is taken into account in the panning / mixing 260. Optionally or additionally, the mixing may be performed as described in section 5.

[0119] The apparatus 200 may be configured to render an audio signal of one or more sound sources 2101 / 2102, e.g. a mixed audio signal 262, e.g. a weighted sum of audio signals 2121 and 2122, using a room impulse response, the early reflection part of which is determined, e.g. by an early reflection pattern 1 in the ER path 230, to obtain an early reflection contributing loudspeaker signal 232 related to the early reflection part of the room impulse response. The early reflection contributing loudspeaker signal 232 may be generated by performing a rendering of the audio signal from early reflection positions ERP (see ERP1 to ERP6).

[0120] Optionally, the device 200 may comprise an ER pattern determiner 270, e.g., a device for generating an early reflection pattern 1. The determination of the early reflection pattern 1 may be performed as described in one of the above-mentioned embodiments (see, e.g., FIG. 2 and sections 1, 3 and 5). The ER pattern determiner 270 may obtain ER pattern information 310 for generating the early reflection pattern 1. The ER pattern information 310 may include one or more of the following: an ER pattern type (indoor / outdoor), a predelay, compfactor and / or distAlpha (e.g., outdoor), and a room dimension, room volume and / or predelay time (e.g., indoor). For example, depending on the determination to be used by the ER pattern determiner 270, the ER pattern determiner 270 receives or reads from the bitstream 300 an environment description 310, e.g., one or more room acoustic parameters or one or more control parameters, or a bitstream hint 320, e.g., one or more early reflection pattern parameters.

[0121] The bitstream 300 may include a representation 2141 of an audio signal 2121 associated with a first audio source 2101 and a representation 2142 of an audio signal 2122 associated with a second audio source 2102 .

[0122] According to an embodiment, the bitstream 300 may include / comprise one or more of the parameters mentioned herein. The bitstream 300 may include a representation 2141 / 2142 of an audio signal of a sound source 2101 / 2102 located at a sound source position and including one or more early reflection pattern parameters. For example, the bitstream 300 is an audio bitstream having early reflection parameters in a header or metadata field of the bitstream, or a file format stream including an audio bitstream representing an audio signal and having early reflection parameters in packets of the file format stream and tracks of the file format stream. The one or more early reflection pattern parameters include one or more of a pattern type index, a pre-delay time to late reverberation, a compression factor, an amplitude correction factor, a distance decay exponent, a pattern orientation parameter, and one or more frequency response parameters.

[0123] In the ER path 230, i.e. when generating the early reflection contribution loudspeaker signal 232, the device 200 is optionally configured to render the audio signal of one or more sound sources 2101 / 2102 from each early reflection position ERP in a manner that is spectrally shaped according to one or more frequency response parameters (see FIG. 3c). In FIG. 3c, the circle (blue) indicates the frequency dependence of the RT60. For all early reflections, the same frequency dependence can be applied. Another frequency dependence can be applied by a bass boost for source or receiver wall proximity (<2m). The one or more frequency response parameters may be included in a bitstream, which may also include a representation of the audio signal of the sound source 2101 / 2102 or the individual signals 212 and 212. The one or more frequency response parameters may be included in one or more early reflection pattern parameters.

[0124] The device 200 may be configured to use HRTFs specific to the listener's head orientation when performing the rendering of the audio signals of one or more sound sources 2101 / 2102 from the early reflection positions ERP. HRTFs stand for Head Related Transfer Functions.

[0125] In any diffusion path 240, one or more audio signals 2121 / 2122 can be rendered to obtain a diffuse late reverberant loudspeaker signal 242. The apparatus 200 may be configured to generate a diffuse late reverberant part of a room impulse response and, for example, to use this room impulse response to render one or more audio signals 2121 / 2122 in the diffusion path 240. The diffuse late reverberant loudspeaker signal 242 is related to the diffuse late reverberant part of the room impulse response.

[0126] The apparatus 200 may be configured, when rendering one or more audio signals 2121 / 2122, to generate a set of loudspeaker signals 252 by forming a sum 250 over direct sound contribution loudspeaker signals 2221 / 2222 associated with the direct source portion of the room impulse response, early reflection contribution loudspeaker signals 232 associated with the early reflection portion of the room impulse response, and optionally a diffuse late reverberation loudspeaker signal 242 associated with the diffuse late reverberation portion of the room impulse response.

[0127] (Interior Rendering) a) ER pattern covering the gap between the direct sound and the onset of late reverberation. b) ER patterns distributed in the horizontal plane. c) ER patterns controlled by room acoustic parameters such as room dimensions, room volume, pre-delay time to late reverberation, RT60 to set the number of ERs, spacing of ERs, and amplitude behavior of ERs with respect to distance. d) ER pattern, which can have 2-20 ERs. e) ER, whose position is determined by the helix. f) The ER, whose position is determined by two spiral arms. g) The position is as follows:

number

[0128] (Outdoor Rendering) k) Sparse ER patterns, e.g., with 2-6 reflections, especially outdoor scenes. l) A geometric analysis of reflective surfaces throughout the scene is used to derive the levels and pre-delay of the ER outdoor pattern. m) The distribution summed over the distance is used to derive the ER pattern parameters. n) Perform this analysis for a mesh of possible listening locations within the area reachable by the user. o) Use the first two peaks of such a distribution along with the corresponding distances. p) Calculate predelay, compFactor, and distAlpha from this distribution value.

[0129] (whole) q) Applying fade-in and fade-out of ER pattern levels when changing from one sound scene and / or room to another.

[0130] 6.2. Transmission, Bitstream and Signalling Aspects a) Indoor scenes can be completely computed within the decoder / renderer using the room acoustic parameters given by the scene. b) In particular, outdoor scenes can benefit from geometric analysis in the encoder. Only the control parameters of the patterns have to be transmitted. In the preferred embodiment, the parameters include: number of algorithms / patterns, pre-delay to late reverberation, compression factor of the patterns compared to the pre-delay, amplitude correction factor, distance decay exponent, pattern orientation parameters, frequency response description. c) If new ER patterns are to be used, they can be fully calculated in the encoder and then sent to the decoder. These ER patterns are defined by the time position and relative level of reflections (with respect to normal distance attenuation) (number of ERs for each azimuth, elevation, radius, amplitude correction factor, distance attenuation exponent, frequency response description). d) The decoder / renderer can be pre-provisioned with several ER patterns. In this case, the bitstream signaling contains a field indicating which pre-provisioned ER pattern should be used. Furthermore, the parameters of this pattern are signaled as described in b.1.

[0131] (7. Application Fields) Time-consuming exact geometric calculation of ER can be avoided especially in the following applications: Real-time auditory virtual environment Real-time Augmented Reality

[0132] 8. Further embodiments FIG. 16 shows the position of the listener 10 and the position of the sound source pos s15 shows an embodiment of an apparatus 200 for rendering sound, configured to receive information about the distance d between the listener and the sound source. This information can be used to determine the distance d between the listener and the sound source. Optionally, the apparatus 200 may be configured to use the distance as described with respect to the apparatus 200 of FIG. 15. The apparatus 200 is configured to render 202 an audio signal 212 of a sound source using a room impulse response 400, the early reflection part 410 of which is exclusively determined by an early reflection pattern 1. The early reflection pattern 1 shows a constellation of early reflection positions ERP (see ERP1 to ERP4) and is arranged at the listener position 10 such that the early reflection positions ERP are located around the listener position 10 and in an angular direction from the listener position 10, the angular direction being invariant to changes in the listener head orientation.

[0133] The apparatus 200 may comprise any of the features described above. For example, the apparatus 200 may comprise the apparatus 100 of Figure 6, 18 or 20 for determining an early reflection pattern for sound rendering. Alternatively, the apparatus 200 may comprise a different apparatus for determining an early reflection pattern for sound rendering, for example an apparatus configured to perform a determination as described with respect to Figure 2 and / or as described in sections 1, 3 and 5.

[0134] FIG. 17 shows the position of the listener 10 and the position of the sound source pos s 15 shows an embodiment of an apparatus 200 for rendering sound, configured to receive first information relating to a sound source position pos . This information may be used to determine a distance d between the listener and the sound source. Optionally, the apparatus 200 may be configured to use the distance as described with respect to the apparatus 200 of FIG. 15. The apparatus 200 may, for example, receive a sound source position pos sand one or more early reflection pattern parameters 310, and is configured to, for example, read the representation of the audio signal and the early reflection pattern parameters from the bitstream 300. The bitstream 300 is, for example, an audio bitstream having the early reflection parameters 310 in a header or metadata field of the bitstream 300, or a file format stream including an audio bitstream representing an audio signal and having the early reflection parameters 310 in packets of the file format stream and in tracks of the file format stream.

[0135] The one or more early reflection pattern parameters 310 may comprise one or more of a pattern type index, a pre-delay time to late reverberation, a compression factor, an amplitude correction factor, a distance decay exponent, a pattern orientation parameter, and one or more frequency response parameters.

[0136] Additionally, the device 200 is configured to determine 270 an early reflection pattern 1 in response to one or more early reflection pattern parameters 310, for example as described with respect to FIG. 2 and / or as described in sections 1, 3 and 5. The early reflection pattern 1 indicates a constellation of early reflection positions ERP (see ERP1 to ERP4). For example, the device 300 may be configured to perform the determination 270 of the early reflection pattern 1 such that the number of early reflection positions ERP is greater the greater the pre-delay time until the late reverberation. Additionally or alternatively, the device 200 is configured to perform the determination 270 of the early reflection pattern 1 such that the farthest early reflection position ERP from the listener position 10 is spaced farther away the greater the pre-delay time until the late reverberation. The spaced distance may be less than the pre-delay time.

[0137] Furthermore, the apparatus 200 is configured to render 202 the audio signal of the sound source using a room impulse response 400, an early reflection portion 410 of the room impulse response 400 being determined by an early reflection pattern 1. The early reflection pattern 1 describes a constellation of early reflection positions ERP (see ERP1 to ERP4) and is arranged at the listener position 10 such that the early reflection positions ERP are located around the listener position 10 and in an angular direction from the listener position 10, which is invariant to changes in the listener head orientation.

[0138] According to one embodiment, apparatus 200 is configured to read from bitstream 300, if the pattern type index indicates a determination method that is parameterized by the encoder, e.g., as described in Section 1, as part of one or more early reflection pattern parameters 310, the number of early reflections in the early reflection pattern, the azimuth angle, elevation angle, radius, e.g., distance to the listener position, for each early reflection, an amplitude correction factor for each early reflection, a distance attenuation exponent for each early reflection, as well as a frequency response description for each early reflection.

[0139] The device 200 may have any of the features described above.

[0140] Fig. 18 shows an embodiment of an apparatus 100 for determining an early reflection pattern 1 for sound rendering, configured to receive at least one room acoustic parameter 310 representative of an acoustic property of an acoustic environment 5. The apparatus 100 is configured to determine 270 the early reflection pattern 1 such that a number 272 of early reflection positions ERP (see ERP1 to ERP6) depends on the at least one room acoustic parameter 310. The early reflection pattern 1 describes a constellation of early reflection positions. The apparatus 100 may in particular comprise the features described above with reference to Fig. 2 and sections 1 and 5.

[0141] FIG. 19 shows the position of the listener at 10 and the position of the first sound source at pos S1 , and the second sound source position pos S22 shows an embodiment of an apparatus 200 for rendering sound, configured to receive information about a first sound source position pos 1 and a second sound source position pos 2 . The apparatus 200 is configured to render (202) audio signals 2121 and 2122 of two sound sources 2101 and 2102 using a room impulse response 400, the early reflection part 410 of which is determined by an early reflection pattern 1. The early reflection pattern 1 describes a constellation of early reflection positions ERP (see ERP1 to ERP4) and is arranged at the listener position 10 such that the early reflection positions ERP are located around the listener position 10 and in an angular direction from the listener position 10, the angular direction being invariant to changes in the listener head orientation. The rendering (202) further comprises a first sound source position pos S1 and a first audio signal 2121 of a first sound source 2101 located at a second sound source position pos S2 The weighted sum 204 is performed by forming a weighted sum 204 with a second audio signal 2122 of a second sound source 2102 located at the first sound source position pos s1 and the listener position 10, the first distance d1 is s2 and the listener position 10 is smaller than a second distance d2, the first audio signal 2121 is weighted with a larger w1 than the second audio signal 2122, and the second audio signal 2122 is weighted with a larger w2 than the first audio signal 2121, if the first distance d1 is larger than the second distance d2. Additionally, the rendering is performed by generating an early reflection contribution loudspeaker signal 232 related to the early reflection part 410 of the room impulse response 400 by rendering a weighted sum 204 from the early reflection positions ERP. The device 200 may in particular include the features described in section 5. However, it is clear that the device 200 may also comprise a device for determining an ER pattern 1 as described in any of the above embodiments.

[0142] Fig. 20 shows an embodiment of an apparatus 100 for determining (270) an early reflection pattern 1 for sound rendering, configured to receive at least one room acoustic parameter 310 describing an acoustic characteristic of an acoustic environment 5. The apparatus 100 is configured to determine (270) an early reflection pattern 1 by parameterizing one or more helical functions 3 and 4 centered on the listener position 10 and by positioning early reflection positions ERP (see ERP11 to ERP14 and ERP21 to ERP24) using the one or more helical functions 3 and 4. The early reflection pattern 1 represents a constellation of early reflection positions ERP. The apparatus 100 may in particular comprise the features as described with respect to Fig. 2 and section 1, although it is clear that the apparatus may also comprise other features described herein.

[0143] 9. ALTERNATIVE EMBODIMENTS Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, with a block or apparatus corresponding to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0144] The inventive rendered audio signal or the inventive early reflection pattern information can be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium, such as the Internet.

[0145] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or in software. Implementations can be implemented using digital storage media, such as floppy disks, DVDs, CDs, ROMs, PROMs, EPROMs, EEPROMs or flash memories, on which electronically readable control signals are stored that cooperate (or can cooperate) with a programmable computer system so that the respective methods are performed.

[0146] Some embodiments according to the invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0147] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0148] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0149] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0150] A further embodiment of the inventive method is therefore a data carrier (or a digital storage medium or a computer readable medium) having recorded thereon a computer program for performing one of the methods described herein.

[0151] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, the data stream or the sequence of signals being for example arranged to be transferred via a data communication connection, for example via the Internet.

[0152] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0153] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0154] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to implement some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0155] The above-described embodiments merely illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended that the present invention be limited only by the scope of the appended claims, and not by the specific details presented by the description and illustration of the embodiments herein.

[0156] (10. Cited non-patent literature) [1] Jot, J.-M., Real-time spatial processing of sounds for music, multimedia and interactive human-computer interfaces. Audio and Multimedia, 1997(ACM Multimedia Systems Journal, February 1997). (Available at http: / / citeseerx.ist.psu.edu / viewdoc / download?doi=10.1.1.54.6319&rep=rep1&type=pdf) [2] Jullien, J.P., E. Kahle, S. Winsberg, and O. Warusfel, Some Results on Objective Characterisation of Room Acoustical Quality in Both Laboratory and Real Environments, 1992, IRCAM, France. (Available from https: / / kahle.be / articles / IRCAM_Room_Acoustical_Quality_1992.pdf) [3] Jot, J.-M., O. Warusfel, E. Kahle, and M. Mein. Binaural Concert Hall Simulation in Real Time. IEEE 93. 1993. Mohonk (USA). [4] Carpentier, T. A New Implementation of Spat in Max 15th Sound and Music Computing Conference (SMC2018) 2018. Limassol, Cyprus. (<https: / / hal.archives-ouvertes.fr / hal-02094499 / document>.) [5] Vaananen, R. and J. Huopaniemi, Advanced AudioBIFS: Virtual Acoustics Modeling in MPEG-4 Scene Description. IEEE Transactions on Multimedia, 2004. 6(5): p. 661-675. [6] Brinkmann, F., H. Gamper, N. Raghuvanshi, and I. Tashev. Towards Encoding Perceptually Salient Early Reflections for Parametric Spatial Audio Rendering. 148th AES Convention. 2020. Vienna, Austria. [7] Brinkmann, F., et al., A Round Robin on Room Acoustical Simulation and Auralization. J. Acoust. Soc. Am., 2019. 145(4): p. 2746..2760 DOI: https: / / doi.org / 10.1121 / 1.5096178. [8] Bregman, A.S., Auditory Scene Analysis (The Perceptual Organization of Sound). 1990, MIT Press. ISBN: 9780262022972. [9] Blauert, J., Spatial Hearing, The Psychophysics of Human Sound Localization. 2nd ed. 1997, Cambrigde Massachusetts: MIT Press. ISBN: 0-262-02413-6.

[10] Angus, J.A.S., The Effects of Specular Versus Diffuse Reflections on the Frequency Response at the Listener. J. Audio Eng. Soc., 2001. 49(3): p. 125-133.

[11] Barron, M. and A.H. Marshall, Spatial Impression due to Early Lateral Reflections in Concert Halls: The Derivation of a Physical Measure. Journal of Sound and Vibration, 1981. 77(2): p. 211-232.

[12] Bech, S. Perception of Reproduced Sound: Audibility of Individual Reflections in a Complete Sound Field. 96th AES Convention. 1994. Amsterdam, The Netherlands.

[13] Kuttruff, H., Room Acoustics (fourth edition). 2000: Spon Press. ISBN: 0-419-24580-4.

Claims

1. An apparatus (100) for determining an early reflection pattern (1) for sound rendering, comprising: receiving at least one room acoustic parameter (310) representative of an acoustic characteristic of the acoustic environment (5); and determining an early reflection pattern (1) indicative of a constellation of early reflection locations, the number of said early reflection locations being dependent on said at least one room acoustic parameter (310); The apparatus (100) is configured as follows.

2. 2. The device (100) of claim 1, wherein the initial reflection pattern (1) is intended to be positioned at the listener position (10) such that the initial reflection positions are located around the listener position (10) and in an angular direction from the listener position (10), the angular direction being invariant to changes in listener head orientation.

3. 3. The apparatus (100) of claim 1 or 2, wherein the at least one room acoustic parameter (310) is Room dimensions, The volume of the room, and Pre-delay time before late reverberation An apparatus (100) comprising one or more of:

4. In the apparatus (100) according to any one of claims 1 to 3, the at least one room acoustic parameter (310) is Room dimensions, Room volume, and Pre-delay time before late reverberation The apparatus (100) includes only one parameter selected from.

5. 5. The apparatus (100) according to claim 1, configured to vary the spacing between the early reflection locations and the number of the early reflection locations depending on the at least one room acoustic parameter (310).

6. The device (100) according to any one of claims 1 to 5, configured to parameterize one or more helical functions (3, 4) centred on a listener position (10) depending on the at least one room acoustic parameter (310) and to position the early reflection positions using the one or more helical functions (3, 4).

7. The apparatus (100) according to any one of claims 1 to 6, configured to read the at least one room acoustic parameter (310) from a bitstream (300) comprising a representation of an audio signal to be rendered using the early reflection pattern (1).

8. An apparatus (100) according to any one of claims 1 to 7, comprising: assisting a first determination of the initial reflection pattern (1) and a second determination of the initial reflection pattern (1), the first determination being different from the second determination and involving positioning an initial reflection position using the one or more control parameters (120); selecting said first decision if said acoustic environment (5) is an outdoor environment or if a pattern type index in a bitstream (300) containing a representation of an audio signal to be rendered assumes a predetermined state. An apparatus (100) configured to:

9. An apparatus (100) according to any one of claims 1 to 8, comprising: the larger the size of the room, the greater the number of early reflection locations; or The larger the volume of the room, the greater the number of early reflection positions; or The greater the pre-delay time until the late reverberation, the greater the number of early reflection positions. An apparatus (100) configured to determine the number of early reflection locations.

10. An apparatus (100) according to any one of claims 1 to 9, comprising: The larger the room dimensions, the further away the furthest early reflection locations are from the listener position (10), or The larger the volume of the room, the farther away the furthest early reflection location is from the listener position (10), or The larger the pre-delay time to the late reverberation, the further away the location of the furthest early reflection is from the listener position (10). An apparatus (100) configured to determine the number of early reflection locations.

11. An apparatus (100) according to any one of claims 1 to 10, configured to determine the early reflection positions such that the early reflection positions are approximately uniformly angularly distributed around a listener position (10).

12. The apparatus (100) according to any one of claims 1 to 11, configured to determine the initial reflection position such that connection lines between the initial reflection position and a listener position (10) do not overlap each other.

13. The apparatus (100) according to any one of the preceding claims, configured to determine the position of the early reflection such that the position of the early reflection lies in a horizontal plane together with a listener position (10).

14. The apparatus (100) of any one of claims 1 to 13, configured to determine the early reflection positions by adjusting an azimuth rotation of the constellation according to a pattern orientation parameter in a bitstream (300) comprising a representation of an audio signal to be rendered.

15. An apparatus (100) according to any one of claims 1 to 14, comprising: - parameterizing one or more spiral functions (3, 4) centered on said listener position (10); and positioning said initial reflection locations using said one or more helical functions (3, 4); An apparatus (100) configured to determine the initial reflection pattern (1) by:

16. 16. The apparatus (100) of claim 15, wherein the one or more helical functions (3, 4) include a first helical function (3) and a second helical function (4), and the apparatus (100) is configured to position a first set of initial reflection positions using the first helical function (3) and to position a second set of initial reflection positions using the second helical function (4) such that each of the first set of initial reflection positions is associated with a corresponding early reflection position of a second set of initial reflections and is located on opposite sides of a line that intersects perpendicularly with a connection line between each initial reflection position and the corresponding early reflection position.

17. 17. The apparatus (100) of claim 16, wherein for each initial reflection location of the first set, a corresponding initial reflection location of the second set of initial reflections is angularly offset relative to the connecting line in a common angular direction for all initial reflection locations of the first set of initial reflection locations.

18. 18. The apparatus (100) according to claim 16 or claim 17, wherein the one or more helical functions (3, 4) comprise a first helical function (3) and a second helical function (4), the apparatus (100) being configured to position a first set of initial reflection positions using the first helical function (3) and to position a second set of initial reflection positions using the second helical function (4) such that the first set of initial reflection positions are determined as (r1, β1) in polar coordinates and the second set of initial reflection positions are determined as (r2, β2) in polar coordinates, wherein: [0010] where nER is the number of early reflection positions and distfactor is a constant.

19. Receiving information about a listener position (10), a first sound source position and a second sound source position; and Rendering the audio signals of these two sources using the room impulse response (400) An apparatus (200) for sound rendering configured to: The early reflection position (410) of the room impulse response (400) is determined by an early reflection pattern (1), which is Shows the constellation of early reflection locations, and the early reflection locations are arranged at the listener position (10) such that they are located around the listener position and in an angular direction from the listener position, the angular direction being invariant to changes in listener head orientation; The rendering step includes: A first audio signal of a first sound source located at the first sound source position (212 1 ), and a second audio signal of a second sound source located at the second sound source position (212). 2 ) and forming a weighted sum (204) of the first audio signal (212) if a first distance between the first sound source position and the listener position (10) is smaller than a second distance between a second sound source position and the listener position (10). 1 ) to the second audio signal (212 2 ) and if the first distance is greater than the second distance, the second audio signal (212 2 ) to the first audio signal (212 1 ) and Rendering the weighted sum (204) from the early reflection locations to generate an early reflection contributed loudspeaker signal (232) associated with the early reflection portion (410) of the room impulse response (400). The apparatus (200) is configured to:

20. 20. The apparatus (200) of claim 19, further configured to generate a diffuse post-reverberant part of the room impulse response.

21. 21. The apparatus (200) of claim 19 or 20, further configured to, when rendering the audio signal, generate a set of loudspeaker signals (252) by forming a sum over direct sound contribution loudspeaker signals (222) associated with a direct sound source part of the room impulse response (400) and early reflection contribution loudspeaker signals (232) associated with an early reflection part (410) of the room impulse response (400).

22. The apparatus (200) of any one of claims 19 to 21, further configured to render the weighted sum (204) from each early reflection location in a manner that adjusts its level depending on the distance from each early reflection location to the listener position (10) when generating an early reflection contributed loudspeaker signal (232) associated with an early reflection portion (410) of the room impulse response (400).

23. 23. The apparatus (200) of claim 22, further comprising the step of: rendering the weighted sum (204) from each early reflection location in a manner that adjusts the level depending on the distance from each early reflection location to the listener location (10): - offsetting (20) the level at which the audio signal of the sound source is rendered from each early reflection location using a level offset or amplifying said level by a level factor, said level offset or level factor being common for all early reflection locations; and Setting the level offset or level coefficient according to an amplitude correction coefficient. The apparatus (200) is configured to:

24. 24. The apparatus (200) of claim 22 or 23, further configured to modify a level adjustment depending on the distance from each early reflection position to the listener position (10) relative to a level adjustment used by the apparatus (200) for rendering the audio signal from the sound source position according to a distance decay exponent when rendering the weighted sum (204) from each early reflection position in a manner that adjusts a level depending on the distance from each early reflection position to the listener position (10).

25. 25. The apparatus (200) of any one of claims 19 to 24, further configured to render the weighted sum (204) of the sound sources from each early reflection location in a manner that is spectrally shaped in accordance with one or more frequency response parameters when generating an early reflection contributed loudspeaker signal (232) associated with an early reflection portion (410) of the room impulse response (400).

26. The apparatus (200) of any one of claims 19 to 25, further configured to use HRTFs specific to a listener's head orientation when performing the rendering of the weighted sum (204) from the early reflection positions.

27. An apparatus (200) for sound rendering, comprising: Receiving first information relating to a listener position (10) and a sound source position; and Rendering an audio signal of said sound source using a room impulse response (400), an early reflection portion (410) of said room impulse response (400) being determined by an early reflection pattern (1), said early reflection pattern (1) being: Shows the constellation of early reflection locations, and The early reflection positions are arranged at the listener position (10) such that they are located around the listener position (10) and in an angular direction from the listener position (10), the angular direction being invariant to changes in listener head orientation. said rendering; and configured to: The apparatus (200) comprises an apparatus (100) for determining an initial reflection pattern (1) according to any one of claims 1 to 18.

28. 28. The apparatus (200) of claim 27, further configured to generate a diffuse post-reverberant portion of the room impulse response (400).

29. 29. The apparatus (200) of claim 27 or 28, further configured to, when rendering the audio signal, generate a set of loudspeaker signals (252) by forming a sum over direct sound contribution loudspeaker signals (222) associated with a direct sound source part of the room impulse response (400) and early reflection contribution loudspeaker signals (232) associated with an early reflection part (410) of the room impulse response (400).

30. The apparatus (200) of any one of claims 27 to 29, further configured to generate an early reflection contribution loudspeaker signal (232) related to an early reflection portion (410) of the room impulse response (400) by performing a rendering of the audio signal of the sound source from the early reflection locations.

31. 31. The apparatus (200) of claim 30, further configured to render the audio signal of the sound source from each early reflection position in a manner that adjusts its level depending on the distance from each early reflection position to the listener position (10) when generating an early reflection contributed loudspeaker signal (232) associated with the early reflection portion (410) of the room impulse response (400) by performing rendering of the audio signal of the sound source from the early reflection positions in a manner that adjusts its level depending on the distance from each early reflection position to the listener position (10).

32. 32. The apparatus (200) of claim 31, further comprising the steps of: rendering the audio signal of the sound source from each early reflection location in a manner that adjusts the level depending on the distance from each early reflection location to the listener location (10); - offsetting (20) the level at which the audio signal of the sound source is rendered from each early reflection location using a level offset or amplifying said level by a level factor, said level offset or level factor being common for all early reflection locations; and Setting the level offset or level coefficient according to an amplitude correction coefficient. The apparatus (200) is configured to:

33. 33. The apparatus (200) of claim 31 or 32, further configured to modify, when rendering the audio signal of the sound source from each early reflection position in a manner that adjusts the level depending on the distance from each early reflection position to the listener position, the level adjustment depending on the distance from each early reflection position to the listener position in accordance with a distance attenuation exponent for a level adjustment used by the apparatus (200) for rendering the audio signal from the sound source position.

34. The apparatus (200) of any one of claims 30 to 33, further configured to render the audio signal of the sound source from each early reflection location in a spectrally shaped manner according to one or more frequency response parameters when performing the rendering of the audio signal of the sound source from the early reflection locations to generate an early reflection contributed loudspeaker signal (232) associated with an early reflection portion (410) of the room impulse response (400).

35. The apparatus (200) of any one of claims 27 to 34, further configured to use HRTFs specific to a listener's head orientation when performing the rendering of the audio signal of the sound source from the early reflection positions.

36. A bitstream (300) subject to sound rendering according to any one of claims 27 to 35.

37. A digital storage medium storing a bitstream (300) subject to sound rendering according to claim 36.

38. A method for determining an early reflection pattern (1) for sound rendering, comprising the steps of: receiving at least one room acoustic parameter (310) indicative of an acoustic characteristic of an acoustic environment (5); determining an early reflection pattern (1) indicative of a constellation of early reflection locations such that the number of early reflection locations depends on the at least one room acoustic parameter; A method comprising:

39. Receiving information about a listener position (10), a first sound source position and a second sound source position; Rendering the audio signals of these two sources using the room impulse response (400). A method of sound rendering comprising: The early reflection position (410) of the room impulse response (400) is determined by an early reflection pattern (1), which is Shows the constellation of early reflection locations, and the early reflection locations are arranged at the listener position (10) such that they are located around the listener position and in an angular direction from the listener position, the angular direction being invariant to changes in listener head orientation; The rendering step includes: A first audio signal of a first sound source located at the first sound source position (212 1 ), and a second audio signal of a second sound source located at the second sound source position (212). 2 ) and forming a weighted sum (204) of the first audio signal (212) if a first distance between the first sound source position and the listener position (10) is smaller than a second distance between a second sound source position and the listener position (10). 1 ) to the second audio signal (212 2 ) and if the first distance is greater than the second distance, the second audio signal (212 2 ) to the first audio signal (212 1 forming a weighted sum (204) that weights the sum more heavily than the sum of the first and second inputs; Rendering the weighted sum (204) from the early reflection positions to generate an early reflection contributed loudspeaker signal (232) associated with the early reflection portion (410) of the room impulse response (400). The method is carried out by

40. 1. A method of sound rendering comprising the steps of: Receiving first information relating to a listener position (10) and a sound source position; Rendering the audio signal of the sound source using a room impulse response (400), an early reflection portion (410) of the room impulse response (400) being determined by an early reflection pattern (1), the early reflection pattern (1) being: Shows the constellation of early reflection locations, and The early reflection positions are arranged at the listener position (10) such that they are located around the listener position (10) and in an angular direction from the listener position (10), the angular direction being invariant to changes in listener head orientation. said rendering step; Equipped with The method comprises a method for determining an initial reflection pattern (1) according to claim 38.

41. A computer program causing a computer to carry out the method according to any one of claims 38 to 40 when the computer program is executed.

Citation Information

Patent Citations

  • System to move a virtual sound away from a listener using a crosstalk canceler

    US10524080B1

  • Early Reflection Method for Enhanced Externalization

    US20080273708A1

  • Signal processing device, method, and program

    WO2019078035A1

  • Rendering reverberation

    WO2021186102A1