Diffraction modelling based on grid pathfinding

Abstract

This document describes a method for processing audio content for presentation in a three-dimensional audio scene, wherein the audio content includes a sound source at a source location. The method includes: obtaining a voxelized representation of the three-dimensional audio scene, wherein the voxelized representation indicates volume elements in which sound can propagate and volume elements in which sound is occluded; generating a two-dimensional projection map of the audio scene based on the voxelized representation by applying a projection operation to the voxelized representation projected onto a horizontal plane; and determining parameters indicating the location of a virtual sound source based on the source location, a listener location, and the projection map, so as to simulate the effect of sound diffraction generated by the three-dimensional audio scene on the source signal of the sound source at the source location by presenting a virtual source signal from the virtual source location. A corresponding device and a corresponding computer program product are also described.

Description

Technical Field

[0001] The present invention generally relates to a method for processing audio content for audio presentation in a three-dimensional audio scene, taking into account diffraction effects caused by elements of the three-dimensional audio scene. Specifically, the present invention relates to a method for (acoustic) diffraction modeling based on grid pathfinding. The present invention further relates to corresponding apparatus and computer program products.

[0002] Although some embodiments will be described herein with specific reference to the present invention, it will be appreciated that the invention is not limited to this field of use and may be applied in a wider context. Prior Art

[0003] Any discussion of the prior art throughout this disclosure should not be considered as an admission that this art is widely known or forms part of the common general knowledge in the field.

[0004] Acoustic diffraction refers to the various phenomena that occur when a wave encounters an obstacle or a hole in a barrier. The psychoacoustic perception of acoustic diffraction refers to the "bending" of sound around an obstacle or the "stretching" of sound through a small opening in an acoustic barrier. This explains why we can perceive sound perfectly well in a concert hall even when it is obstructed by a pillar or rod, or why we can hear sounds from a room with the door slightly open while walking down a hallway.

[0005] Acoustic diffraction is a naturally occurring phenomenon that plays an important role in conveying a correct interpretation of the acoustic environment. It is particularly perceptually relevant in the presence of obstructing acoustic barriers and / or small openings therein. Failure to simulate acoustic diffraction effects would, of course, compromise the realistic reproduction of the sound scene.

[0006] Despite the importance of acoustic diffraction in the physical world, diffraction modeling in virtual environments (e.g., virtual reality or gaming worlds) is often neglected. Modeling acoustic diffraction effects is often abandoned entirely or replaced by a direct signal propagation approach. Even state-of-the-art audio rendering solutions have yet to achieve a state where they can accurately reproduce real-world acoustic effects in real time within a three-dimensional virtual environment. One reason for this is that properly modeling physical diffraction effects is a computationally intensive task. This is due to the complexity of representing the geometry of occlusion / diffraction-related objects (e.g., walls and holes), the dimensionality of the space used for audio rendering (e.g., 3D virtual reality), and the requirements for realism and content creator intent for the modeled effects (e.g., audibility).

[0007] Therefore, there is a need for realistic and computationally feasible sound diffraction modeling. In other words, there is a need for an improved method and apparatus for processing audio content for presentation in a (virtual) three-dimensional audio scene. Summary of the Invention

[0008] According to a first aspect of the present invention, a method for processing audio content for presentation in a three-dimensional audio scene is provided. The three-dimensional audio scene may be a virtual audio scene. The audio content may include a sound source at a source location. The method may include obtaining a voxelized representation of the three-dimensional audio scene. The voxelized representation may indicate volume elements (e.g., voxels) within which sound can propagate and volume elements (e.g., voxels) within which sound is occluded. Thus, the voxelized representation may be said to include a description of the occlusion geometry of the three-dimensional audio scene. In some embodiments, the volume elements (e.g., voxels) in the voxelized representation may be cubes with a predefined edge length (e.g., approximately 10 centimeters). In other embodiments, the volume elements (e.g., voxels) may have an adaptive size, i.e., an adaptive edge length. Volume elements within which sound can propagate (e.g., freely propagate) may be considered to be filled with air. In some embodiments, the volume element may have an associated occlusion coefficient ranging from a coefficient value indicating free sound propagation (e.g., 0) to complete occlusion (e.g., 1). The method may further include obtaining a two-dimensional projection map of the audio scene. The two-dimensional projection map may be associated with the voxelized representation via a projection operation onto a horizontal plane. The projection map may include a grid, each element of which represents a horizontal position in the audio scene and indicates whether at least one volume element at that horizontal position (e.g., above or below) in the voxelized representation is a volume element in which sound can propagate. The projection map may also be referred to as an occlusion map or a propagation map. The method may further include determining parameters indicating a virtual source position of a virtual sound source based on the source position, a listener position, and the projection map, to simulate an effect of acoustic diffraction of the three-dimensional audio scene on a source signal of the sound source at the source position by rendering a virtual source signal from the virtual source position.

[0009] By considering a voxelized representation of the 3D audio scene, the complexity of the representation can be significantly reduced. This can be achieved without perceptual degradation of the listening experience by appropriately selecting the voxel size (e.g., an edge length of approximately 10 centimeters, or a few centimeters). Complexity can be further reduced by projecting onto the 2D projection map, thereby allowing the use of a 2D pathfinding algorithm. The path output by the pathfinding algorithm contains sufficient information to generate a virtual sound source at a virtual source location that realistically simulates the effects of sound diffraction in the original 3D audio scene. The complexity reduction achieved by the proposed method allows for providing a realistic listening experience in a 3D audio scene with reasonable computational effort. Specifically, this enables realistic sound rendering in 3D audio scenes even for real-time applications, such as virtual reality applications or computer / console games.

[0010] In some embodiments, obtaining the two-dimensional projection map may include generating the two-dimensional projection map based on the voxelized representation by applying the projection operation onto the horizontal plane to the voxelized representation. If the volume elements have associated occlusion coefficients, generating the projection map may involve a truncated step of mapping any of these coefficient values ​​to one of two extreme values ​​(e.g., to 0 or 1). Depending on the situation (e.g., the frequency of expected geometric changes), the voxelized representation and the two-dimensional projection map may be pre-computed (e.g., for a plurality of sizes of volume elements) and accessed when needed. For example, pre-computation may be performed on the encoder side, and the voxelized representation(s) and the two-dimensional projection map(s) may be provided to the decoder side as part of a bitstream. This may further reduce the computational load on the decoder side.

[0011] In some embodiments, determining the parameters indicative of the virtual source position may include applying a path-finding algorithm to the projection map to determine a path in the projection map between a two-dimensional projection of the source position (e.g., a grid element containing the source position) and a two-dimensional projection of the listener position (e.g., a grid element containing the listener position), the path along grid elements indicating that at least one volume element at a respective horizontal position in the voxelized representation is a volume element in which sound may propagate. Such grid elements may be referred to as non-occluding grid elements.

[0012] Depending on the specific requirements of the rendering environment, a wide range of reliable and efficient path-finding algorithms are readily available and can be selected. Thus, by appropriately selecting the path-finding algorithm, the proposed method can be specifically adjusted to the computing power available on the rendering side.

[0013] In some embodiments, determining the parameters indicative of the virtual source position may include calculating a line of sight between the source position and the listener position in the three-dimensional audio scene. The determination may further include, if the calculated line of sight intersects a volume element in the voxelized representation where the sound is occluded, and if a two-dimensional projection of the line of sight on the horizontal plane intersects a grid element in the projection map indicating that none of the volume elements at the horizontal position in the voxelized representation is a volume element in which sound can propagate, applying a pathfinding algorithm to determine a path in the projection map between a two-dimensional projection of the source position and a two-dimensional projection of the listener position, the path along grid elements indicating that at least one volume element at a respective horizontal position in the voxelized representation is a volume element in which sound can propagate. Under the above conditions, the determination may further include determining the parameters indicative of the virtual source position based on the determined path. It should be understood that the above sub-steps of the determination step may imply respective determination steps regarding whether an intersection exists in the voxelized representation and / or the projection map.

[0014] By performing a check for intersection of the direct line of sight with an occluding voxel in the voxelized representation or with an occluding grid element in the projection map, special cases that are easily handled can be identified for special processing, thereby further reducing the overall computational load.

[0015] In some embodiments, the method may further include the step of obtaining a refined voxelized representation and a refined two-dimensional projection map if the pathfinding algorithm cannot identify a path in the projection map between the two-dimensional projection of the source position and the two-dimensional projection of the listener position. Under the above conditions, the method may further include applying the pathfinding algorithm to determine a path in the refined projection map between the two-dimensional projection of the source position and the two-dimensional projection of the listener position, the path along grid elements indicating that at least one volume element at a respective horizontal position in the refined voxelized representation is a volume element in which sound can propagate. The refined two-dimensional projection map may be associated with the refined voxelized representation by the projection operation onto the horizontal plane. Furthermore, the volume elements of the refined voxelized representation may have an edge length that is smaller than the volume elements of the voxelized representation. For example, the edge length of the volume elements of the refined voxelized representation may be half the edge length of the volume elements of the (initial / previous) voxelized representation. Whether or not to obtain the refined voxelized representation and the refined two-dimensional projection map may depend on whether time (e.g., CPU time) is available within a current cycle (e.g., time period) of a rendering process. For example, the cycle length may be related to a desired (e.g., predetermined) refresh rate (update rate) for the audio presentation. If CPU time is available, the representation may be refined; otherwise, if a path has not yet been found, the sound from the sound source may not be presented.

[0016] This allows for continuous improvement in the accuracy of the representation. If a "coarse" representation yields acceptable results in terms of finding a path and determining that the virtual source position of the virtual sound source is probable, then the coarse representation can be maintained. On the other hand, if no such path is found (and CPU time is available), the granularity of the voxelized representation is refined to check for openings in the occlusion geometry that were not identified at the coarse granularity but would allow the sound from the sound source to be perceived at the listener's position. Thus, the proposed conditional refinement allows for an optimal compromise between representation accuracy and reduced computational load.

[0017] In some embodiments, determining the parameters indicative of the virtual source position may include determining a distance and an azimuth between the listener position and the virtual source position based on the determined path. The distance may be determined based on a path length of the determined path. For example, the distance may be determined as the path length of the (complete) determined path (e.g., the actual length of the path or an estimate / approximation thereof, such as the Manhattan distance or a weighted number of voxels intersecting the path). Alternatively, the path length of the determined path may be adjusted based on a difference in the vertical coordinates of the listener position and the source position, or based on an extrapolation into the voxelized representation. Determining the azimuth may include identifying a nearest directional change in the determined path starting from the two-dimensional projection of the listener position. Determining the azimuth may further include determining the azimuth as an azimuth between the two-dimensional projection of the listener position and the identified nearest directional change. It should be understood that the (nearest) directional change is caused by a volume element of the sound being occluded.

[0018] Therefore, the fact is exploited that the decision path in the projection mapping contains sufficient information for effectively determining a distance and an azimuth angle of the virtual source position that will produce a realistic listening experience after rendering.

[0019] In some embodiments, determining the parameters indicative of the virtual source position may include determining an elevation angle between the listener position and the virtual source position based on the determined path and the voxelized representation. Determining the elevation angle may include identifying a nearest directional change in the determined path starting from the two-dimensional projection of the listener position. Determining the elevation angle may further include determining a volume element in the voxelized representation within which sound can propagate at the horizontal location of the identified directional change. Here, the volume element within which sound can propagate at the horizontal location of the determined directional change may be determined as the volume element having the nearest vertical coordinate to the vertical coordinate of the listener position, having the smallest distance to the direct line of sight between the listener position and the sound source position, or being within a largest contiguous subset of voxels within which sound can propagate. Determining the elevation angle may further include determining the elevation angle as the elevation angle between the listener position and the determined volume element.

[0020] Thereby, use is made of the fact that the determined path in the projection mapping and the voxelized representation contain sufficient information for efficiently determining an elevation angle of the virtual source position that will produce a realistic listening experience after rendering.

[0021] In some embodiments, determining the parameters indicative of the virtual source position may include calculating a line of sight between the source position in the three-dimensional audio scene and the listener position. Determining the parameters indicative of the virtual source position may further include, if the calculated line of sight intersects a volume element in the voxelized representation where the sound is occluded, and if a two-dimensional projection of the line of sight on the horizontal plane does not intersect a grid element in the projection map indicating that none of the volume elements at the horizontal position in the voxelized representation is a volume element in which sound can propagate, determining a distance and an azimuth between the listener position and the virtual source position based on the two-dimensional projection of the line of sight on the horizontal plane. It should be understood that if the calculated line of sight does not intersect a volume element in the voxelized representation that occludes the sound (in which case the original source position can be used), or if no path is found in the projection map (in which case the sound need not be rendered at all), then a virtual source position need not be determined.

[0022] Thereby, a specific trivial case can be identified and handled in a simple manner to further reduce the overall computational load caused by the proposed method while still producing a realistic listening experience.

[0023] In some embodiments, determining the parameters indicative of the virtual source position may further include determining an elevation angle between the listener position and the virtual source position based on the line of sight and the voxelized representation. Determining the elevation angle may include identifying a nearest volume element from the listener position that intersects the calculated line of sight and occludes the sound. Determining the elevation angle may further include determining a volume element in the voxelized representation within which the sound can propagate, at the horizontal position of the identified volume element. Determining the elevation angle may further include determining the elevation angle as an elevation angle between the listener position and the determined volume element.

[0024] Thereby, exploitation is made of the fact that, in the special case of identification, the direct line of sight and the voxelized representation contain sufficient information for efficiently determining an elevation angle of the virtual source position that will produce a realistic listening experience.

[0025] According to another aspect of the present invention, a device is provided, comprising a processor coupled to a memory storing instructions for the processor. The processor may be adapted to cause the device to execute a method according to any of the above aspects and embodiments thereof.

[0026] According to another aspect of the present invention, a computer program is provided, which includes instructions for causing a processor that executes the instructions to perform the method according to any one of the above-mentioned first aspect and its embodiments.

[0027] According to another aspect of the present invention, a computer-readable storage medium storing the computer program according to the aforementioned aspect is provided. Simple diagram description

[0028] Example embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0029] 1A and 1B illustrate an example of an acoustic diffraction scene caused by a three-dimensional audio scene and occluding elements in a two-dimensional horizontal projection of the scene on a two-dimensional voxelized grid, respectively.

[0030] FIG. 2 illustrates a flow chart of an example of a method for processing audio content for presentation in a three-dimensional audio scene.

[0031] 3A and 3B illustrate examples of paths in two-dimensional projection mapping.

[0032] FIG. 4 is a flow chart illustrating another example of a method for processing audio content for presentation in a three-dimensional audio scene.

[0033] FIG. 5 schematically illustrates an example of an apparatus implementing a method for processing audio content for presentation in a three-dimensional audio scene. Implementation Method

[0034] FIG1A provides an example of a three-dimensional audio scene 100, and FIG1B depicts its horizontal projection onto a two-dimensional projection plane. This audio scene illustrates the effects of acoustic diffraction on a sound emitted at a source location (source localization) 110, as perceived at a listener location (listener location) 120. A direct path 130 between source location 110 and listener location 120 is blocked by an occluding element 140 in the audio scene, such as a wall or other extended element between source location 110 and listener location 120. However, the sound emitted at source location 110 will be perceived at listener location 120. This is due to acoustic diffraction, which allows the sound to reach listener location 120 along all paths around occluding element 140. From a psychoacoustic perspective, the shortest "bending" path 150 can be considered the dominant path. This effect needs to be properly modeled to allow realistic audio rendering in (virtual) three-dimensional audio scenes containing occluding elements.

[0035] Broadly speaking, the present invention proposes a computationally efficient method for acoustic diffraction modeling based on a path-finding algorithm using an equidistant grid of voxelized occlusion geometry for representing a three-dimensional (audio) scene. To achieve this, the present invention utilizes a simplified (but sufficiently accurate) geometry representation using a voxelization method (e.g., an adaptive voxelization method). Furthermore, the present invention utilizes a two-dimensional space for diffraction modeling by shrinking the associated geometry representation, potentially in conjunction with components for controlling the sound effects of approximating acoustic occlusion / diffraction phenomena for content creators and encoder operators.

[0036] Thus, the method and apparatus according to the present invention allow for perceptually realistic simulation of acoustic occlusion / diffraction effects in dynamic and interactive three-dimensional virtual environments, aiming to enhance the overall user experience and facilitate wider deployment of virtual reality (VR) applications. Here and hereinafter, "simulation of occlusion / diffraction" is understood in the sense of modeling the perceptual effects of occlusion / diffraction, and does not necessarily imply an exact simulation of physical effects.

[0037] As a starting point, the present invention assumes a (virtual) three-dimensional space (e.g., a 3D audio scene) containing an occlusion geometry description (i.e., information about occlusion elements in the audio scene, for example, represented by meshes or geometric primitives and possible occlusion coefficients obtained for or assigned to their surfaces). The present invention further assumes a position (location) of the listener(s) / user(s) and the audio source(s), as well as the audio signal(s) (e.g., waveforms) emitted by the audio source(s). These elements can be considered as input to the methods proposed by embodiments of the present invention. However, it should be noted that these methods can also receive a processed version of the occlusion geometry description. For example, these methods can receive a voxelized representation of the three-dimensional space (e.g., a 3D audio scene) as input.

[0038] Based on the above assumptions (i.e., the above inputs), methods and apparatus according to embodiments of the present invention seek to simulate the perceived effects of acoustic occlusion / diffraction by providing (e.g., determining or generating) a virtual audio source (virtual sound source) at a virtual source location, possibly along with a virtual source signal. The virtual source location is defined by its coordinates, including a distance (e.g., radius) and a direction (e.g., azimuth and elevation). However, it should be noted that the virtual source location can also be defined using other coordinate representations (e.g., in a Cartesian coordinate system), and the present invention should not be limited to spherical coordinates. If the position / location is expressed in a coordinate system different from the coordinate system used for sound presentation, an appropriate transformation between coordinate systems may be performed, for example, during audio presentation. The virtual source signal can be determined based on the original waveform of the source signal, and modifications (e.g., gain, filtering, etc.) may be applied to it.

[0039] FIG2 is a flowchart illustrating an example of a method 200 for processing audio content for presentation in a three-dimensional audio scene. The audio content includes a sound source at a source position that emits a (sound) source signal, i.e., emits sound based on the source signal (e.g., a source waveform).

[0040] At 202, a scene geometry is received as input. The scene geometry may be a mesh representation of a three-dimensional audio scene (three-dimensional audio environment). The mesh representation includes a mesh or a set of meshes that represent, for example, the surfaces of objects (elements of the scene) in the three-dimensional audio scene, such as local occluders (e.g., obstacles) or global occluders (e.g., walls). In some implementations, each mesh or mesh surface may have an associated occlusion coefficient.

[0041] At 204 , a determination is made as to whether an update of the scene geometry and / or corresponding occlusion properties has occurred. If so (Yes), the method proceeds to 206 to determine a voxelized representation of the 3D audio scene, or to update at least a sub-portion of the voxelized representation based on an updated sub-portion of the scene geometry (e.g., if the voxelized representation is already available and the change in scene geometry is relatively small), followed by generating a 2D projection map of the voxelized representation at 208 . Otherwise (No), the method proceeds to 214 .

[0042] At 206 , a selection of acoustically relevant geometry is performed. Thus, a subset of acoustically relevant data representing objects (of a size and type that can / should cause occlusion / diffraction effects) is selected from all available scene geometry information. This step may be optional.

[0043] At 208, voxelization (i.e., a voxelization method / algorithm) is applied to the scene geometry or acoustic-related geometry (e.g., occlusion / diffraction-related geometry) to generate a voxelized representation of the 3D audio scene. The voxelized representation includes a plurality of voxels (volume elements) of regular size arranged in a regular 3D grid. Thus, a voxel can be a cube with a predefined edge length (e.g., approximately 10 centimeters).

[0044] In one example embodiment, a voxelization algorithm can be applied to a meshed representation of a three-dimensional audio scene to produce a voxelized representation. This step involves reducing the representation complexity of the geometry by a 3D voxelization method / algorithm (e.g., an adaptive 3D voxelization method / algorithm) that divides the three-dimensional space into two (abstract) types of voxels (volume elements): occluder object material (OM) voxels such as concrete or wood, and sound propagation medium (SM) voxels such as air or water. Thus, the set of voxels VOX is given by the set of OM voxels and SM voxels. (1)

[0045] At 210, voxels for diffraction modeling are extracted / selected. To this end, the OM voxels are divided into two groups of voxels, namely, local occlusion (LO) voxels (corresponding to local occlusion obstacles, such as (for example) furniture) and global occlusion (GO) voxels (corresponding to global occlusion obstacles, such as (for example) walls). It is assumed that the local occlusion obstacles do not have an impact on the large-scale directionality of sound propagation, so that the sound from a sound source behind a local occlusion obstacle propagates in the same direction as the original sound at least within a sufficient distance from the local occlusion obstacle, possibly with some attenuation. Therefore, the set of OM voxels is separated by the following (2)

[0046] Subsequent steps in method 200 may then only consider GO voxels. Therefore, in other words, step 210 involves removing any voxels corresponding to local occluders from the generated voxelized representation, where local occluders represent objects in the 3D audio scene that are not expected to affect the large-scale directionality of sound propagation. For example, the boundaries of the 3D audio scene (e.g., walls) are global occluders that should be retained at this time. Local occluders can be identified by applying image processing techniques (e.g., including filtering) to the generated voxelized representation, for example, assuming that they will be removed by appropriate spatial filtering. Local occluders can also be identified based on the spatial properties of the OM voxels (e.g., the number and / or size of voxels representing an occluding obstacle) and / or their relationship to surrounding SM voxels (e.g., the free volume for sound propagation). Similarly, local occluders can be directly specified by the corresponding metadata and reflect the intent of the audio scene creator. It should be noted that step 210 is optional.

[0047] Steps 206 , 208 and 210 may be said to aim at controllably reducing the complexity of the geometric representation of the 3D audio scene by determining an importance of geometric elements relative to a strength of the occlusion / diffraction effects they cause.

[0048] In some implementations, steps 206, 208, and 210 may be replaced by obtaining a voxelized representation of the 3D audio scene in a different manner, such as by receiving the voxelized representation from an external source. For example, the voxelized representation may be read from a storage memory or extracted (e.g., decoded) from a bitstream. In these cases, step 202 is also omitted. Generally speaking, method 200 may be said to include a step of obtaining a voxelized representation of the 3D audio scene, where the voxelized representation indicates volume elements in which sound can propagate (e.g., SM voxels) and volume elements in which sound is occluded (e.g., OM voxels). Voxels in which sound can propagate (e.g., freely propagating) may be considered to be filled with air. In some implementations, as described above, this step may be implemented by steps 206, 208, and 210. Obtaining the voxelized representation of the 3D audio scene may depend on whether an update of the 3D audio scene has occurred.

[0049] In optional step 236, the effect of any voxels removed from the voxelized representation in step 210 can be taken into account by applying local attenuation modeling to simulate the attenuation of the sound by the removed voxels corresponding to local occluders, depending on the listener position. Any attenuation gain or filter determined in this step can be applied to the virtual source signal determined in 230 and described below.

[0050] At 212, a two-dimensional projection map (or projection matrix) for diffraction modeling is generated (e.g., calculated) from the voxelized representation (e.g., from GO voxels). For example, a two-dimensional projection map of the audio scene can be generated based on the voxelized representation by applying a projection operation to the voxelized representation. The projection operation projects onto a horizontal plane (e.g., horizontal from a listener's perspective). Thus, the projection map can include a grid, each element of which represents a horizontal position in the audio scene and indicates whether at least one voxel (volume element) at that horizontal position in the voxelized representation (e.g., above, within, or below the horizontal plane) is a voxel in which sound can propagate (e.g., an SM voxel or, equivalently, not a GO voxel). Thus, the projection map can also be referred to as a propagation map or an occlusion map.

[0051] In some embodiments, the following shrinkage method may be used to obtain a two-dimensional projection map (projection matrix) from a three-dimensional voxelized representation (three-dimensional voxel matrix) or a portion thereof representing global occlusion barriers: (3)

[0052] Here, the value and can each belong to a range between extreme values ​​indicating free sound propagation (no obstruction; e.g., value 0) and complete obstruction (e.g., value 1). For example, the values ​​of PM and GO can belong to the range [0, 1] and correspond to 0 - No occlusion 1- Completely blocked.

[0053] Needless to say, the reverse assignment of values ​​is also possible, in which case equation (3) has to be adapted by replacing the minimum value with the maximum value.

[0054] For values ​​of PM and GO in the range [0, 1], an optional pruning step may be performed in the context of generating a projection map to render the items of the projection map as Boolean types, as follows: (4)

[0055] in is the occlusion threshold. Generally speaking, generating a projection map may involve a pruning step that maps any of the occlusion coefficient values ​​of the voxels of the voxelized representation to one of two extreme values ​​(e.g., to 0 or 1). In other words, generating a two-dimensional projection map may involve a pruning operation such that the grid elements of the projection map indicate either free sound propagation (no occlusion) or complete occlusion (occlusion).

[0056] On the other hand, it is also possible to have the occlusion coefficient values ​​of the voxels in the voxelized representation (e.g., the values ​​of GO) already be of Boolean type. In this case, the PM matrix can be obtained as follows: If for ,

[0057] Step 212 is to introduce The concept of "walls and holes" is represented by a matrix to reduce the dimensionality of the space used for diffraction modeling. In fact, if sound propagation is possible for any voxel at a given horizontal position (i.e., in a "row" of voxels at that horizontal position), then the projection map will indicate that sound propagation is possible for that horizontal position (i.e., for that grid element), and therefore indicates at least one "hole" at that horizontal position.

[0058] Generating the 2D projection mapping may depend on whether an update of the 3D audio scene has occurred.

[0059] In some embodiments, step 212 can be replaced by obtaining a two-dimensional projection map in a different manner, for example, by receiving the two-dimensional projection map from an external source. For example, the two-dimensional projection map can be read from a storage memory or extracted (e.g., decoded) from a bitstream. Generally speaking, method 200 can be said to include a step of obtaining a two-dimensional projection map of the audio scene, wherein the two-dimensional projection map is associated with the voxelized representation via a projection operation onto a horizontal plane. The projection map includes a grid, each element of which represents a horizontal position in the audio scene and indicates whether at least one volume element at that horizontal position in the voxelized representation is a volume element in which sound can propagate. In some embodiments, as described above, this step can be implemented by step 212, for example, in the sense that obtaining the two-dimensional projection map includes applying a projection operation to the voxelized representation. Obtaining the two-dimensional projection map can still depend on whether an update of the three-dimensional audio scene has occurred.

[0060] At 214, a determination is made as to whether an update of the listener position and / or source position has occurred. If so (Yes), the method proceeds to 216 to determine parameters indicating a virtual source position of a virtual sound source. Otherwise (No), the method proceeds to 230 to use a previously determined virtual source position.

[0061] Generally speaking, determining a parameter indicating a virtual source position of a virtual sound source is performed based on the (original) source position, the listener's position, and the projection mapping. In some cases, the determination may be further based on a voxelized representation. The purpose of determining the virtual source position indicating the virtual sound source is to simulate an effect of acoustic diffraction of a three-dimensional audio scene on a source signal of a sound source at the (original) source position by rendering a virtual source signal from the virtual source position.

[0062] Determining parameters indicative of the virtual source position may be implemented by some or all of steps 216, 218, 220, 222, 224, 226, 228, and 234 discussed below. Generally speaking, this step may be said to involve applying a path-finding algorithm to the projection map to determine a path in the projection map between a two-dimensional projection of the source position (e.g., a grid element including the position) and a two-dimensional projection of the listener position (e.g., a grid element including the position), the path along grid elements (e.g., non-occlusion grid elements) indicating that at least one volume element at a respective horizontal position in the voxelized representation is a volume element in which sound can propagate.

[0063] As mentioned above, determining the parameters indicating the virtual source position of the virtual sound source may depend on whether an update of the listener position and / or the source position occurs.

[0064] At 216, a line of sight is calculated between the source position and the listener position in the three-dimensional audio scene. This line of sight may also be referred to as a listener-to-source (user-to-source) direct line of sight. Its calculation may involve conventional geometric calculations. In some implementations, the listener position and / or source position may be quantized to the center of their respective located voxels. The line of sight may be a line of sight in three-dimensional space.

[0065] At 218, a check is performed to see if the calculated line of sight intersects a voxel in the voxelized representation of the sound being occluded (e.g., a GO voxel). If so (Yes), the method proceeds to 220. Otherwise (No), the method proceeds to 238. In the latter case, diffraction modeling is unnecessary because there is a direct line of sight between the sound source and the listener.

[0066] At 220, a check is made whether a two-dimensional projection of the line of sight onto the horizontal plane (of the projection map) and any of the volume elements indicating the horizontal position in the voxelized representation are not a volume element in which sound can propagate (e.g., a PM element indicating an occlusion, such as 1). If yes, the method proceeds to step 222. Otherwise, the method proceeds to step 234. In the latter case, pathfinding in the projection mapping is unnecessary because a direct (straight) line exists between the 2D projections of the source position and the listener position.

[0067] Steps 218 and 220 are intended to reduce the amount of computation required to determine the parameters indicating the virtual source location. This is achieved by performing special processing for simple or trivial cases. Steps 218 and 220 may be optional in some implementations; however, the general processing will produce reasonable results even for simple or trivial cases.

[0068] At 222, a pathfinding algorithm is applied to determine a path in the projective mapping between a two-dimensional projection of the source position and a two-dimensional projection of the listener position, the path along a grid element (e.g., a non-occluded grid element) indicating that at least one volume element at a respective horizontal position in the voxelized representation is a volume element within which sound can propagate. In some implementations, the listener position and / or source position may be quantized to the center of the projection (e.g., grid element) of their respective located voxels. Any pathfinding algorithm may be applied at this step, including, for example, A* and JPS. In this sense, the proposed model is modular with respect to the pathfinding algorithm.

[0069] In one example embodiment, step 222 may include the following sub-steps: Calculation of voxel indices (voxels) corresponding to listener (user) and source (object) positions: and . · Check as appropriate , (i.e., whether the listener and source locations are within the sound propagation material), = (In this case, spatial audio presentation can be skipped), (In this case, spatial audio presentation can also be skipped, assuming that the source is too far away from the listener to be perceived), etc., and the corresponding rules are applied. Run a path finding algorithm (e.g., A*, JPS, etc.) to obtain a path (e.g., approximately the shortest trajectory from the source to the listener). The following parameters can be applied to its control (e.g., acceleration and search loop termination): -Heuristic parameters of the path finding algorithm - a previously detected path (e.g., for a dynamic scene) -(several) thresholds on computing time and / or amount of resources - A threshold value for maximum path length expressed in, for example, a number of 2D grid elements (2D projection of voxels), distance units, sound attenuation, resulting source object sound level, etc.

[0070] As described above, if the calculated line of sight intersects a volume element in the voxelized representation where the sound is occluded, and if the two-dimensional projection of the line of sight on the horizontal plane intersects a grid element (e.g., an occlusion grid element) in the projection mapping indicating that any of the volume elements at that horizontal position in the voxelized representation is not a volume element in which sound can propagate, step 222 is performed.

[0071] At 224, a check is performed to determine whether a path was found in step 222. If a path was found (Yes), the method proceeds to 226. If a path was not found (No), the method proceeds to 238 and may not determine parameters indicating a virtual source position of a virtual sound source. The method may then completely skip diffraction modeling and / or audio rendering for the current listener position and current source position. An alternative process for the case where a path was not found will be described below with reference to FIG. 4.

[0072] Steps 226 and 228 generally relate to determining parameters indicative of the virtual source location based on the path determined in step 222 .

[0073] More specifically, at 226 , a distance (ie, a radius) and an azimuth (ie, a horizontal component) between the listener position and the virtual source position are determined based on the determined path.

[0074] The distance may be determined based on the length of one of the determined paths. For example, the distance may be determined as - The length of the complete path on the projection map determined in step 222; - the length of the complete path on the projection map, adjusted according to the difference between the listener and source positions in their vertical coordinates (e.g., Z dimension) (i.e., their height relative to the horizontal plane); or - The length of the complete path between the listener and the source extrapolated into a voxelized representation of the 3D audio scene (e.g., GO voxels).

[0075] Incidentally, if the determined distance between the virtual source position and the listener position exceeds a maximum render distance of the renderer intended to be used, the virtual source position may be set to the maximum rendering distance, and any remaining distance not yet accounted for may be encapsulated as an additional attenuation gain for the virtual sound source.

[0076] The azimuth angle can be determined by first identifying a most recent change in direction in the determined path, starting from a two-dimensional projection of the listener's position. The azimuth angle can then be determined as the azimuth angle between the two-dimensional projection of the listener's position (into the projection map) and the most recent identified change in direction. For example, the azimuth angle can be determined based on the relative position of the two-dimensional projection of the listener's position and the respective grid elements of the projection map in which the most recent identified change in direction falls. In other words, when starting from the listener's position, the azimuth angle can be determined based on a direction of the determined path. In other words, the azimuth angle can be determined based on a direction in the projection map from the listener's position to the first occluding grid element of the determined path, after which the path changes direction. It should be understood that the azimuth angle can be determined with reference to a predetermined reference direction, such as, for example, the x- or y-dimension in the coordinate system of the three-dimensional audio scene.

[0077] FIG3A schematically illustrates an example of determining the azimuth angle of a virtual source position. The projection map 30 shown therein includes occlusion grid elements 50 (shaded; i.e., grid elements representing respective horizontal positions in the audio scene and indicating that none of the voxels at that horizontal position in the voxelized representation is a voxel through which sound can propagate) and non-occlusion grid elements 40 (i.e., grid elements representing respective horizontal positions in the audio scene and indicating that at least one of the voxels at that horizontal position in the voxelized representation is a voxel through which sound can propagate). For illustrative purposes, horizontal projections of the occlusion elements 140 of the audio scene are also shown, but these projections are not part of the projection map 30. The two-dimensional projection 60 of the line of sight between (the two-dimensional projection of) source position 10 and (the two-dimensional projection of) listener position 20 in the projection map 30 is occluded by at least one of the occlusion grid elements 50 of the projection map 30. Therefore, a path 70 between source position 10 and listener position 20 is determined by a path-finding algorithm. Path 70 changes its direction after occluding grid element 90. Therefore, the location of the most recent change in direction 80 of path 70 is determined. An azimuth is then determined to indicate the direction 95 to the most recent identified change in direction 80 of path 70. This direction 95 is typically different from the direction 60 of (a two-dimensional projection of) the direct line of sight.

[0078] At 228, an elevation angle (i.e., a vertical component) between the listener position 120 and the virtual source position is determined based on the determined path and the voxelized representation. Specifically, the elevation angle can be determined by first identifying the nearest directional change in the determined path starting from the two-dimensional projection of the listener position. This can be performed similarly to step 226, or the nearest directional change identified in step 226 can be reused. Next, a volume element in the voxelized representation is determined at the horizontal position of the identified directional change, within which the sound can propagate. For example, such a (non-occluded) voxel can be determined in the voxel row at the horizontal position of the grid element containing the nearest identified directional change. Specifically, such a (non-occluded) voxel can be determined at the horizontal position of the nearest identified directional change having a vertical coordinate closest to the vertical coordinate of the listener position. Finally, the elevation angle can be determined as the elevation angle between the listener position and the determined voxel. In other words, when extrapolating into a voxelized representation of a three-dimensional audio scene, an elevation angle can be determined from the listener's position based on a direction of the determined path. It should be understood that the elevation angle can be determined with reference to a predetermined reference plane, such as, for example, a horizontal plane onto which the voxelized representation is projected to generate a projection map.

[0079] FIG3B schematically illustrates an example of determining the elevation angle of a virtual source position. Specifically, FIG3B illustrates a (vertical) section through the voxelized representation along a vertical section plane aligned with one of two directions defined by the grid of the two-dimensional projection map and intersecting the location (e.g., voxel) of the nearest direction change 80 of the determined path 70 between listener position 20 and source position 10 (closest to the two-dimensional projection of listener position 20). Unless source position 10 and listener position have a common horizontal voxel index (i.e., are arranged in the same horizontal row of the voxelized representation), any choice of direction defined by the grid of the two-dimensional projection map is appropriate. If the two positions share a common horizontal index, the vertical section plane should be selected so that it intersects the direct line of sight between source position 10 and listener position 20. The vertical section includes occluding vertical grid elements 55 and non-occluding vertical grid elements 45.

[0080] In this example, the vertical section plane is a plane orthogonal to the two-dimensional projection map of FIG3A and includes the bottom horizontal row of voxels in FIG3A (i.e., the horizontal row of voxels containing the voxel with the nearest direction change 80). The source position 10, the listener position 20, and the direct line of sight are projected onto the vertical section plane as vertical projections 15, 25, and 85, respectively. In this example, the vertical section plane includes four vertical occlusion grid elements 55, arranged in a vertical row and corresponding to the voxels representing the support of occlusion element 140 to the right in FIG1A.

[0081] To determine the elevation angle, an unobstructed vertical grid element 45 in the vertical cross section is identified that is at the same horizontal position as the nearest change in direction 80. In this example, the unobstructed vertical grid element directly below the row of obstructed vertical grid elements 55 is identified. Generally speaking, the unobstructed vertical grid element can be identified that is at the horizontal position of the nearest change in direction 80 and that is, for example, a) closest to the vertical projection 85 of the line of sight, b) closest to the vertical coordinate of the listener position 20, or c) corresponds to a voxel that is part of a maximal contiguous subset of unobstructed voxels. The identified unobstructed vertical grid element is used to determine the elevation angle. For example, a vertical projection of the elevation angle between the vertical projection 5 of the virtual source position and the vertical projection 25 of the listener position can be determined based on the horizontal and vertical distances between the identified unobstructed vertical grid element (e.g., its center) and the vertical projection 25 of the listener position. The actual elevation angle may then be determined based on the azimuth angle and the vertical projection of the elevation angle, for example by applying a trigonometric transformation to the vertical projection of the elevation angle based on a difference angle between the azimuth direction and the direction of the vertical section plane.

[0082] Note that the determination of the elevation angle as shown in FIG3B may use the actual position coordinates of the vertical projections 15, 25 of the source position and the listener position (as shown by the shaded and dashed circles), whereas the path finding as shown in FIG3A may refer only to the respective voxels (voxel indices) containing these positions.

[0083] Note further that FIG. 3B shows the vertical projection 5 of the virtual source position at a greater distance from the vertical projection 25 of the listener position than from the vertical projection 15 of the source position, which is attributable to the fact that the length of the determination path 70 exceeds the length of the direct line of sight 60 .

[0084] In summary, a non-occluded (i.e., non-GO) voxel is identified at the horizontal position of the identified most recent change in direction 80. This non-occluded voxel can be, for example, the non-occluded voxel closest to the listener position 20 in a vertical coordinate (e.g., height), the non-occluded voxel closest to the direct line of sight, or the non-occluded voxel that is part of a largest contiguous subset of non-occluded voxels (e.g., belonging to the largest opening in a wall). The elevation angle is then determined based on the identified non-occluded voxel at the horizontal position of the identified most recent change in direction 80.

[0085] Importantly, the above determination of the azimuth and elevation angles requires only knowledge of the last segment of the determined path between the most recent change in direction 80 and the listener position 20. Knowledge of any other profiles of the path is not required (and is irrelevant) for determining these angles. If present, these further profiles are referenced for determining the radius (or distance) to the virtual source position.

[0086] As mentioned above, if the calculated line of sight intersects a voxel in the voxelized representation where the sound is occluded (e.g., a GO voxel), but the two-dimensional projection of the line of sight onto the horizontal plane does not intersect a grid element in the projection map indicating that none of the volume elements at that horizontal position in the voxelized representation is a volume element in which sound can propagate, the method proceeds to 234. At 234, a distance (i.e., radius) and an azimuth (i.e., horizontal component) between the listener position and the virtual source position are determined based on the two-dimensional projection of the line of sight onto the horizontal plane. The distance can be determined in the same manner as in step 226, but with the two-dimensional projection of the line of sight replacing the determined path. When viewed from the listener position, the azimuth can be determined based on the direction of the two-dimensional projection of the line of sight. The method then proceeds to 228, where the elevation angle is determined. In this case, the elevation angle is determined based on the calculated line of sight and the voxelized representation. This can be accomplished in the following manner. First, an occluding voxel (e.g., a GO voxel) of the voxelized representation that the line of sight intersects is identified. If there is more than one intersecting occluding voxel, the one closest to the listener's position is selected. In other words, this is equivalent to identifying the nearest voxel (volume element) from the listener's position that intersects the calculated line of sight and occludes the sound. Next, for that occluding voxel, the nearest non-occluding voxel at the same horizontal position (e.g., above or below the occluding voxel) is determined. This can be equivalent to determining a volume element in the voxelized representation at the horizontal position of the identified voxel in which the sound can propagate. Finally, the elevation angle is determined as the elevation angle between the listener's position and the determined non-occluding volume element.

[0087] At 230, a virtual source signal (e.g., waveform) of the virtual source at the determined virtual source position is determined based on the source signal (e.g., waveform) of the sound source. For example, the virtual source signal may be the original waveform (i.e., source signal) with applied modifications (e.g., gain, filter, etc.). These modifications may be selected from the following options: -none - Gains / filters affected by the local environment of the virtual source (e.g. late reverberation) -Gain / Filters Affected by All Environments

[0088] In other words, the original source signal can be used as the virtual source signal, or the virtual source signal can be generated by applying gain and / or filters to the source signal. Applying the filter to the source signal can include, for example, applying reverberation to a region corresponding to the (original) source position, applying reverberation to a region corresponding to the virtual source position, or generally applying reverberation to one or more regions along the decision path.

[0089] At 232, rendering (e.g., 3DoF rendering) is applied to the virtual source at the virtual source location. Generally speaking, this step can be said to involve rendering the virtual source signal from the virtual source location to the listener location. Furthermore, any conventional processing steps of the audio source, including any post-processing steps, can be performed at this time.

[0090] At 238, if the calculated line of sight does not intersect a voxel (e.g., a GO voxel) in the voxelized representation of the occluded sound, then this step is performed, or if no path is found in the projection mapping, then diffraction modeling is skipped. If there is a direct line of sight between the source position and the listener position that is not blocked by any occluding voxels (e.g., GO voxels), then diffraction of the sound by the 3D audio scene need not be considered when seeking to provide a realistic listening experience and a lifelike perception of sound propagation in the 3D audio scene. If no path is found, then the sound from the sound source may not be rendered at all.

[0091] It should be understood that the processing flow of steps 214 to 232 of method 200, together with steps 234, 236, and 238, may be performed for each of a plurality of source positions and / or listener positions (eg, for each of a plurality of pairs of a source position and a listener position).

[0092] In the above-described method 200, if a path is not found (No at step 224), parameters indicating a virtual source position of a virtual sound source may not be determined. Subsequently, method 200 may completely skip diffraction modeling and / or audio rendering for the current listener position and current source position. A method 400 that provides an alternative handling for the case where a path is not found will be described below with reference to FIG.

[0093] Steps 402, 404, 406, 408, 410, 412, 414, 416, 418, 420, 422, and 434 of method 400 may be performed in the same manner as steps 202, 204, 206, 208, 210, 212, 214, 216, 218, 220, 222, and 234 of method 200 described above, respectively, and the respective statements made above may also apply hereto.

[0094] At 424, a check is made to see if a path has been found in step 422. If a path has been found (Yes), the method proceeds to 426. If no path has been found (No), the method proceeds to 440.

[0095] Steps 426, 428, 430, and 432 of method 400 may then be performed in the same manner as steps 226, 228, 230, and 232 of method 200 described above, and the respective statements made above also apply here. Similarly, step 436 may be performed in the same manner as step 236 described above.

[0096] At 440, a check is made to see if CPU time is available. This step can be performed, for example, by a real-time resource management application (real-time resource manager). It can be assumed that a specific period (time period) is available for determining the virtual source position and performing the actual rendering. This period or time period can be related to (e.g., derived from) an update rate or refresh rate of the rendering. Next, a check can be made in each period to see if there is sufficient time (CPU time) available to perform another iteration of voxelization and subsequent steps. This can be based on an estimate of the time required for this iteration, for example, based on the characteristics of the 3D audio scene and / or the previous amount of time required at a given voxelization granularity. If no CPU time is found to be available (No), the method proceeds to 438, which can be performed in the same manner as step 238 described above. If CPU time is found to be available (Yes), the method proceeds to 442.

[0097] At 442, the voxelization granularity is refined. This corresponds to reducing the size (e.g., edge length) of the voxels. For example, the edge length of the voxels may be halved at this point, so that each voxel is divided into eight smaller voxels for the next iteration. Generally speaking, the size (e.g., edge length) may be reduced according to a predefined ratio. The method then returns to 408 to perform voxelization of the three-dimensional audio scene with the refined granularity. Similar to the above, this step can be replaced by obtaining a voxelized representation of the three-dimensional audio scene with the refined granularity. For example, this refined voxelized representation can be read from a storage memory or extracted from a bitstream (possibly after requesting the refined voxelized representation at the appropriate refined granularity from a source device (such as, for example, an encoder)).

[0098] The remaining steps are then performed as described above. It should be understood that the refined 2D projection map can also be obtained, for example, by reading the refined 2D projection map from a storage memory or extracting the refined 2D projection map from a bitstream (possibly after requesting the refined voxelized representation at an appropriate refinement granularity from a source device (such as, for example, an encoder).

[0099] It should be noted that as long as it is determined that no path has been found at 424 and CPU time is found to be available at 440, the refinement of the voxelization granularity may be repeated (possibly multiple times). A further criterion that may be applied at this stage is to check whether a predetermined minimum granularity of the voxelization has been reached. If the minimum granularity has been reached, the method may continue as if no CPU time was found to be available at step 440.

[0100] In yet another embodiment, the refinement of the granularity of the voxelized representation can be repeated (i.e., iteratively performed) as long as CPU time is available, regardless of whether a path has been found. This can be implemented by changing the order of steps 440 and 424, i.e., first checking whether CPU time is available at 440, and only after CPU time is found to be unavailable (Yes), checking whether a path has been found at 424. It should be understood that in this case, if CPU time is found to be unavailable (No) at step 440, the method will proceed to step 424. Then, if a path has not been found (No) at step 424, the method will proceed to step 438. A further criterion that can be applied at step 440 is to check whether a predetermined minimum granularity of the voxelization has been reached.

[0101] In addition to refining the granularity of the voxelization at step 442 , the present embodiment may modify parameters of the path-finding algorithm (eg, start and / or target positions) based on paths previously discovered at a coarser granularity of the voxelization.

[0102] In the above-described embodiments, the granularity of the voxelization (e.g., voxel size or edge length) may be reset to a predefined (e.g., default) value whenever an update to the geometry is detected in step 404. In some embodiments, this may be the only instance in which the granularity of the voxelization is reset (or, generally, coarsened).

[0103] Although a method for processing audio content has been described above, it should be understood that the present invention also relates to apparatus and devices (e.g., encoders, decoders, renderers, playback devices, etc.) suitable for performing the described method, including computer programs containing instructions suitable for causing a device with processing capabilities to perform the described method, and computer-readable storage media storing such computer programs.

[0104] FIG5 shows an example of an apparatus 500 suitable for performing the described methods. Apparatus 500 includes a processor 510 and a memory 520 coupled to processor 510 and storing instructions for processor 510. Processor 510 is suitable for performing the method(s) described above. Apparatus 500 may receive input 530 including, for example, audio content (including source signals and indications of source locations) and an indication of a three-dimensional audio scene, and may generate output 540 including, for example, a virtual source signal and indications of virtual source locations, or a rendered audio signal.

[0105] Note that the proposed method can be performed by either an encoder or a decoder, or can be distributed between the encoder and decoder. In the first case, the encoder can generate a representation of the rendered (virtual) sound signal as an output. In the second case, the decoder can generate the voxelized representation and the two-dimensional projection map itself, if its computational power allows. Alternatively, in the third case, the voxelized representation and the two-dimensional projection map can be generated by the encoder (possibly at different voxelization granularities) and provided to the decoder, for example, as part of a bitstream. The decoder then obtains the voxelized representation and the two-dimensional projection map, for example by extracting them from the bitstream, and proceeds with the further steps of the method(s) described above. Furthermore, hybrid scenarios are also possible, in which the voxelized representation is obtained (e.g., from the encoder) and the decoder generates the two-dimensional projection map based on the obtained voxelized representation. explain

[0106] Unless otherwise specifically stated, as will be apparent from the following discussion, it should be understood that throughout the discussion of this disclosure, terms such as "processing," "computing," "calculating," "determining," "analyzing," or the like are used to refer to the actions and / or procedures of a computer or computing system or similar electronic computing device that manipulate and / or transform data represented as physical (such as electrical) quantities into other data similarly represented as physical quantities.

[0107] In a similar manner, the term "processor" may refer to any device or portion of a device that processes electronic data (e.g., from register and / or memory) to transform that electronic data into other electronic data that can be stored in register and / or memory, for example. A "computer" or a "computing machine" or a "computing platform" may include one or more processors.

[0108] An example embodiment of each of the methods described herein takes the form of a computer-readable carrier medium carrying a set of instructions (e.g., a computer program for execution on one or more processors (e.g., one or more processors that are part of a website server configuration)). Thus, as one skilled in the art will appreciate, example embodiments of the present invention may be embodied as a method, an apparatus (such as a dedicated device), an apparatus (such as a data processing system), or a computer-readable carrier medium (e.g., a computer program product). The computer-readable carrier medium carries computer-readable code comprising a set of instructions that, when executed on one or more processors, causes the processor or processors to perform a method. Thus, aspects of the present invention may take the form of a method, an entirely hardware example embodiment, an entirely software example embodiment, or an example embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of carrier medium (e.g., a computer program product on a computer-readable storage medium) carrying computer-readable program code embodied in the medium.

[0109] References throughout this specification to "an example embodiment," "some example embodiments," or "an example embodiment" mean that a particular feature, structure, or characteristic described in connection with the example embodiment is included in at least one example embodiment of the present invention. Thus, appearances of the phrases "in an example embodiment," "in some example embodiments," or "in an example embodiment" in various places throughout this specification are not necessarily all referring to the same example embodiment. Furthermore, one of ordinary skill in the art will appreciate from this disclosure that the particular features, structures, or characteristics may be combined in any suitable manner in one or more example embodiments.

[0110] It should be understood that in the above description of example embodiments of the present invention, various features of the present invention may sometimes be grouped together in a single example embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding understanding of one or more of the various inventive aspects. However, this approach to the present invention is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single previously disclosed example embodiment. Accordingly, the claims immediately following the embodiment are hereby expressly incorporated into this embodiment, with each claim standing on its own as a separate example embodiment of the present invention.

[0111] Furthermore, while some of the exemplary embodiments described herein include some features and not other features included in other exemplary embodiments, combinations of features from different exemplary embodiments are intended to be within the scope of the present invention and form different exemplary embodiments, as will be understood by those skilled in the art. For example, in the following invention claims, exemplary embodiments from any application may be used in any combination.

[0112] While what is believed to be the best mode of the present invention has been described, those skilled in the art will recognize that other and further modifications may be made to the present invention without departing from the spirit of the present invention, and it is intended that all such changes and modifications fall within the scope of the present invention be claimed. For example, any formulas given above are merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams, and operations may be interchanged between functional blocks. Steps may be added to or deleted from the described methods within the scope of the present invention.

[0113] 5: Vertical projection 10: Source location 15: Vertical projection 20: Listener Position 25: Vertical Projection 30: Projection Mapping 40: Non-occluding grid elements 45: Non-blocking vertical grid elements 50: Occlude grid elements 55: Block vertical grid elements 60: 2D projection 70: Path 80: Recent direction changes 85: Vertical Projection 90: Occlude grid elements 95: Direction 100: 3D audio scene 110: Source location 120: Listener position 130: Direct Path 140: Occluding elements 150: Shortest "Curved" Path 200:Method 202: Steps 204: Steps 206: Steps 208: Steps 210: Steps 212: Steps 214: Steps 216: Steps 218: Steps 220: Steps 222: Steps 224: Steps 226: Steps 228: Steps 230: Steps 232: Steps 234: Steps 236: Steps 238: Steps 400:Method 402: Steps 404: Steps 406: Steps 408: Steps 410: Steps 412: Steps 414: Steps 416: Steps 418: Steps 420: Steps 422: Steps 424: Steps 426: Steps 428: Steps 430: Steps 432: Steps 434: Steps 436: Steps 438: Steps 440: Steps 442: Steps 500: Equipment 510: Processor 520: Memory 530: Input 540: Output

Claims

1. A method for processing audio content for presentation in a three-dimensional audio scene, wherein the audio content includes a sound source at a source location, the method comprising: Obtain a voxelized representation of the three-dimensional audio scene, wherein the voxelized representation indicates volume elements in which sound can propagate and volume elements in which sound is occluded; obtain a two-dimensional projection map of the audio scene, wherein the two-dimensional projection map is associated with the voxelized representation by a projection operation on a horizontal plane; and determine parameters indicating the virtual source position of a virtual sound source based on the source position, a listener position, and the projection map, so as to simulate the effect of sound diffraction of the three-dimensional audio scene on the source signal of the sound source at the source position by presenting a virtual source signal from the virtual source position, wherein the parameters indicating the virtual source position include a distance and an azimuth angle between the listener position and the virtual source position determined by a determination path.

2. An apparatus for processing audio content for presentation in a three-dimensional audio scene, wherein the audio content includes a sound source at a source location, the apparatus comprising: A first processor is configured to obtain a voxelized representation of the three-dimensional audio scene, wherein the voxelized representation indicates volume elements in which sound can propagate and volume elements in which sound is occluded; a second processor is configured to obtain a two-dimensional projection map of the audio scene, wherein the two-dimensional projection map is associated with the voxelized representation by a projection operation on a horizontal plane; and a third processor is configured to determine, based on the source location, a listener location, and the projection map, parameters indicating the location of a virtual source of a virtual sound source, so as to simulate the effect of sound diffraction of the three-dimensional audio scene on the source signal of the sound source at the source location by presenting a virtual source signal from the virtual source location.

Images