Apparatus and method for rendering an audio scene using effective intermediate diffraction paths

Abstract

An apparatus for rendering an audio scene comprising an audio source located at an audio source position and a plurality of diffraction objects, the apparatus comprising: a diffraction path provider for providing a plurality of intermediate diffraction paths through the plurality of diffraction objects, the intermediate diffraction paths having a plurality of diffraction objects start point and an output edge and associated filter information of the intermediate diffraction path; a renderer for rendering the audio source at a listener position, wherein the renderer is configured for: determining one or more valid intermediate diffraction paths from the audio source position to the listener position based on the output edge of the intermediate diffraction paths and the listener position; determining a filter representation of a complete diffraction path for each of the one or more valid intermediate diffraction paths; and computing an audio output signal of the audio scene using an audio signal associated with the audio source and the filter representation of each complete diffraction path.

Description

TECHNICAL FIELD

[0001] The present invention relates to audio signal processing, and in particular, to audio signal processing in the context of geometrical acoustics, which can be used for e.g. virtual reality or augmented reality applications. BACKGROUND

[0002] The term "virtual acoustics" is commonly applied when sound signals are processed to contain features of an emulated acoustic space and reproduced spatially using binaural or using multichannel techniques. Thus, virtual acoustics includes spatial sound reproduction and room acoustics modeling [1].

[0003] In terms of room modeling techniques, the most accurate propagation modeling method is to solve the theoretical wave equation under a set of boundary conditions. However, due to computational complexity, most of the methods based on numerical solvers are limited to pre-compute relevant acoustic features (such as parametric models) to approximate the impulse responses: it becomes a mess when the frequency of interest and / or the size of the scene space (volume / surface) increases, even with the presence of dynamically moving objects. Given the fact that recent virtual scenes are becoming larger and more complex to enable very detailed and sensitive interactions between players and objects within the scene or between players, current numerical methods are not sufficient to handle interactive, dynamic and large-scale virtual scenes. Some algorithms have demonstrated their rendering capabilities by pre-computing relevant acoustic features using parametric directional coding for pre-computed sound propagation [2, 3] and efficient GPU-based time-domain solvers for the wave equation [4]. However, these methods require high quality system resources, such as graphics cards, multi-core computing systems.

[0004] For interactive sound propagation environments, geometrical acoustics (GA) techniques are a practical and reliable approach. Commonly used GA techniques include the image source method (ISM) and the ray tracing method (RTM) [5, 6], and improved methods using beam tracing and view frustum tracing have been developed for interactive environments [7, 8]. For diffracted sound modeling, Kouryoumjian [9] proposed the uniform theory of diffraction (UTD), while Svensson

[10] proposed the Biot-Tolstoy-Medwin (BTM) model to better approximate diffracted sound in a numerical sense. However, current interactive algorithms are limited to static scenes

[11] or first-order diffraction in dynamic scenes

[12] .

[0005] By combining the two categories below, it is possible to implement a hybrid method

[13] : numerical methods for low frequencies and GA methods for high frequencies.

[0006] Especially in complex sound scenes with multiple diffraction objects, the requirements for modeling the sound diffraction around edges become very high. Therefore, very powerful computing resources are needed to fully model the diffraction effects of sound in audio scenes with multiple diffraction objects. Summary of the Invention

[0007] The purpose of this invention is to provide an improved concept for rendering audio scenes.

[0008] This objective is achieved by means of a device or method for rendering an audio scene or by a computer program for rendering an audio scene.

[0009] This invention is based on the discovery that the processing of sound diffraction can be significantly enhanced by using intermediate diffraction paths between the starting or input edge and the final or output edge of a sound scene that already has associated filter information. This associated filter information covers the entire path between the starting and final edges, regardless of whether there is a single diffraction or several diffractions between them. The process relies on the fact that the path between the starting and final edges (i.e., the route the sound wave must take due to diffraction) does not depend on the typically variable listener position, nor on the audio source position. Even when the audio source has a variable position, only the variable source position or the variable listener position can change over time, but any intermediate diffraction path between the starting and final edges of the diffracting objects depends only on the geometry, and nothing else. This diffraction path is constant because it is defined solely by the diffracting objects provided by the audio scene geometry. This path can only change over time when the shape of one of the multiple diffracting objects changes, and this means that for movable rigid geometry, this path will not change. Furthermore, many objects in the audio scene are static, meaning they are immovable. Providing complete filter information for the entire intermediate diffraction path can improve processing efficiency, especially at runtime. Even filter information for intermediate diffraction paths that ultimately are not used due to lack of validation must be calculated, but this calculation can be performed during the initialization / encoding steps rather than at runtime. In other words, any runtime processing regarding filter information or intermediate diffraction paths is only necessary for dynamic objects that rarely occur, but for static objects that typically appear, the filter information associated with a particular intermediate diffraction path remains unchanged, regardless of any moving listener or moving audio source.

[0010] An apparatus for rendering an audio scene comprising an audio source located at an audio source location and a plurality of diffraction objects includes: a diffraction path provider for providing multiple intermediate diffraction paths through the plurality of diffraction objects, wherein the intermediate diffraction paths have start points or start edges and output edges or end edges of the plurality of diffraction objects, and associated filter information of the intermediate diffraction paths, the information describing the entire sound propagation due to diffraction from the start point or start edge to the output edge or output or end point. Typically, the multiple intermediate diffraction paths are provided by a preprocessor in an initialization step or a pre-computation step, which occur before actual runtime processing, for example, in a virtual reality environment. The diffraction path provider does not necessarily need to compute all this information at runtime, but can provide this information, for example, as a list of intermediate diffraction paths, which the renderer can access during runtime processing.

[0011] A renderer is configured to render an audio source at the listener's location, wherein the renderer is configured to determine one or more valid intermediate diffraction paths from the audio source location to the listener location based on the output edge of the intermediate diffraction path and the listener's location. The renderer is configured to: for each of the one or more valid intermediate diffraction paths, use the associated filter information of the valid intermediate diffraction path combined with filter information describing the propagation of the audio signal from the output edge of that valid intermediate diffraction path or from the final edge to the listener's location, to determine a filter representation of the complete diffraction path from the audio source location to the listener's location corresponding to the valid intermediate diffraction path. The audio output signal of the audio scene can be calculated using the audio signal associated with the audio source and the complete filter representation of each complete diffraction path.

[0012] Depending on the application, the audio source location is fixed, so the diffraction path provider determines each valid intermediate diffraction path such that the starting point of each valid intermediate diffraction path corresponds to the fixed audio source location. Alternatively, when the audio source location is variable, the diffraction path provider determines the input or starting edge of multiple diffraction objects as the starting point of the intermediate diffraction path. The renderer is configured to additionally determine one or more valid intermediate diffraction paths based on the input edges of one or more intermediate diffraction paths and the audio source location, i.e., to determine paths that can belong to a specific audio source location, so as to additionally determine the final filter representation of the complete diffraction path based on additional filter information from the source to the input edge, such that in this case, the complete filter representation is determined by three parts. The first part is the filter information of sound propagation from the sound source location to the input edge. The second part is the association information belonging to the valid intermediate diffraction paths, and the third part is the sound propagation from the output or final edge to the actual listener location.

[0013] This invention is advantageous because it provides an efficient method and system for simulating diffracted sound in complex virtual reality scenes. It is also advantageous because it allows for the modeling of sound propagation via both static and dynamic geometric objects. In particular, the invention is advantageous because it provides a method and system for calculating and storing diffraction path information based on a set of prior-known geometric primitives. Specifically, the diffracted sound path includes a set of attributes, such as a set of geometric primitives for potential diffraction edges, diffraction angles, and diffraction edges in between.

[0014] This invention is advantageous because it allows for the analysis of geometric information of given primitives and the extraction of useful databases via a preprocessor to improve the speed of real-time sound rendering. Specifically, processes such as those disclosed in U.S. Application 2015 / 0378019 A1, or others described later, can pre-compute a visibility graph between edges whose structure minimizes the number of diffraction edges that need to be considered at runtime. The visibility between two edges does not necessarily imply specifying the exact path from the source to the listener, as the locations of the source and listener are often unknown during the pre-computation phase. Instead, the visibility graph between all possible edge pairs is a map navigating from a set of visible edges from the source to a set of visible edges from the listener. Attached Figure Description

[0015] Preferred embodiments of the invention will then be discussed with reference to the accompanying drawings, in which:

[0016] Figure 1 This is a top view of an example scene with four static objects;

[0017] Figure 2a It is a top view of an example scene with four static objects and a single dynamic object;

[0018] Figure 2b It is a list used to describe the diffraction paths in the case of no dynamic object (DO) and with dynamic object (DO);

[0019] Figure 3 This is a top view of an example scene with six static objects;

[0020] Figure 4 This is a top view of an example scene with six static objects, used to illustrate how to calculate higher-order diffraction paths based on the first or input edge;

[0021] Figure 5 A block diagram of an algorithm is shown that pre-calculates intermediate diffraction paths (including higher-order paths) and renders diffraction sounds in real time.

[0022] Figure 6A block diagram of an algorithm for pre-calculating intermediate diffraction paths (including higher-order paths) and rendering diffracted sound considering dynamic objects in real time, according to a preferred third embodiment, is shown.

[0023] Figure 7 An apparatus for rendering a sound scene according to a preferred embodiment is shown;

[0024] Figure 8 It shows having Figure 4 and Figure 3 An example path list of the two intermediate diffraction paths shown;

[0025] Figure 9 The process for calculating the filter representation of the complete diffraction path is shown;

[0026] Figure 10 A preferred embodiment for retrieving filter information associated with an effective intermediate diffraction path is shown;

[0027] Figure 11 The process for verifying one or more potentially valid intermediate diffraction paths in order to obtain a valid intermediate diffraction path is illustrated; and

[0028] Figure 12 This demonstrates rotations performed on the unrotated or original source position to improve the audio quality of the rendered audio scene. Detailed Implementation

[0029] Figure 7 An apparatus for rendering an audio scene comprising an audio source with an audio source signal located at an audio source location and multiple diffraction objects is shown. The diffraction path provider 100 includes, for example, a memory filled by a preprocessor that has already performed calculations of intermediate diffraction paths during an initialization step (i.e., prior to runtime processing operations performed by the renderer 200). Depending on information about the list of intermediate diffraction paths obtained by the diffraction path provider 100, the renderer is configured to calculate an audio output signal in the desired output format (e.g., binaural, stereo, 5.1, or any other output format) for use with headphones' speakers or loudspeakers, or simply for storage or transmission. For this purpose, the renderer 200 receives not only the list of intermediate diffraction paths, but also, on the one hand, the listener's location and the audio source signal, and on the other hand, the audio source location.

[0030] Specifically, renderer 200 is configured to render an audio source at the listener's location, such that a sound signal reaching the listener's location is calculated. This sound signal exists because the audio source is placed at the audio source location. To this end, the renderer is configured to determine one or more valid intermediate diffraction paths from the audio source location to the listener location based on the output edge of the intermediate diffraction path and the actual listener location. The renderer is also configured to: for each of the one or more valid intermediate diffraction paths, use the associated filter information of the valid intermediate diffraction path combined with filter information describing the propagation of the audio signal from the output edge of that valid intermediate diffraction path to the listener location to determine a filter representation of the complete diffraction path from the audio source location to the listener location corresponding to the valid intermediate diffraction path among the one or more valid intermediate diffraction paths.

[0031] The renderer uses the audio signal associated with the audio source and a filter representation of each complete diffraction path to calculate the audio output signal of the audio scene. Depending on the implementation, the renderer can also be configured to additionally calculate first-order, second-order, or higher-order reflections in addition to diffraction calculations, and furthermore, the renderer can be configured to calculate contributions from one or more additional audio sources, as well as contributions from direct sound propagation from sources with direct sound propagation paths not obscured by diffraction objects, if present in the sound scene.

[0032] Preferred embodiments of the invention will then be described in more detail. Specifically, any first-order diffraction path can be calculated in real time if necessary, but this is particularly problematic in complex scenarios with several diffraction objects.

[0033] Especially for high-order diffraction paths, calculating them in real time is problematic due to the large amount of redundant information in the visibility map, as shown in US 2015 / 0378019 A1. For example, in conjunction with... Figure 1 In scenarios similar to the one described above, when the source is to the right of the first edge and the listener is to the left of the fifth edge, the diffracted sound can be imagined traveling from the source to the listener via both the first and fifth edges. However, constructing diffraction paths edge-by-edge in real-time based on the visibility graph becomes computationally complex, especially as the average number of visible edges increases and the diffraction order becomes higher. Furthermore, the lack of runtime edge-to-edge visibility checks limits the diffraction effects between dynamic objects and between a static object and a dynamic object. Only the diffraction effects of static objects or individual dynamic objects can be considered. The only way to combine the diffraction effects of dynamic objects with those associated with static objects is to update all visibility graphs using the primitives of the repositioned dynamic objects. However, this is nearly impossible at runtime.

[0034] The method of this invention aims to reduce the runtime computation required to specify possible (first-order / higher-order) diffraction paths from the source to the listener through edges of static and dynamic objects. As a result, a set of multiple diffracted sound / audio streams is rendered with appropriate latency. An embodiment of the preferred concept is applied using a UTD model to multiple visible and correctly oriented edges with a newly designed system hierarchy. As a result, the embodiment can render higher-order diffraction effects through static geometry, through dynamic objects, through a combination of static geometry and dynamic objects, or also through a combination of multiple dynamic objects. Further details regarding the preferred concept are described in the following subsections.

[0035] The main idea behind this embodiment stems from the question: "Do we need to keep calculating intermediate diffraction paths?" For example, as Figure 3 As shown, from the source to the listener, we can say that there are three possible diffraction paths: (source) – (1) – (5) – (listener), (source) – (9) – (13) – (listener), and (source) – (1) – (7) – (11) – (listener). For simplicity of illustration, the last path via the three intermediate edges including (1), (7), and (11) is a good example. In an interactive environment, the source can move and the listener can also move. However, in any case, the intermediate paths including (1) – (7), (7) – (11), and (11) – (13) will not change unless there are dynamic objects that occlude these intermediate paths. (Note that how to handle / combine the diffraction effects of dynamic objects will be introduced at the end of this section.) Therefore, once the intermediate paths from the first order to the allowed higher orders can be pre-computed via the edges within the intermediate paths, adjacent polygons (e.g., triangular meshes), and diffraction angles, it will minimize the computation required at runtime.

[0036] For example, Figure 4 An example scenario with six static objects is shown to illustrate how to compute higher-order diffraction paths based on edges. In this example, it starts with the first edge, and the edge may contain several pieces of relevant information, such as:

[0037] struct DiffractionEdge {

[0038] const EAR::Geomtery* parentGeometry;

[0039] int meshID;

[0040] int edgeID;

[0041] std::pair < ​​EAR::Vector3, EAR::Vector3 > vtxCoords;

[0042] std::pair < ​​EAR::Mesh::Triangle*, EAR::Mesh::Triangle* > adjTris;

[0043] float internalAngle;

[0044] std::vector < DiffractionEdge* > visibleEdgeList;

[0045] };

[0046] For example, `parentGeometry` or `meshID` indicates the geometry to which the selected edge belongs. Furthermore, an edge can be physically defined as a line formed by two vertices (through their coordinates or vertex IDs), and adjacent triangles will help calculate angles based on the edge, source, or listener. `internalAngle` is the angle between two adjacent triangles, indicating the maximum possible diffraction angle around that edge. It is also an indicator that can determine whether the edge is a potential diffraction edge.

[0047] From the selected edge (in this example, such as Figure 4 Starting from the first side shown, we can imagine two possible diffraction directions entering the open space from one of the triangular meshes and from another of the triangular meshes. These directions are visualized by the normal vectors of the adjacent triangles (shown as red and blue arrows). For example, along the red surface normal (in a counter-clockwise direction), by checking for the existence of edge regions (i.e., dark regions) of the wave to be diffracted, edges 2, 4, 5, and 7 will be the next few edges used for diffraction. For example, a sound wave cannot diffract from edge 1 to edge 6 because at edge 6, both sides of edge 6 are visible from edge 1, meaning that there is no dark region at edge 6 for a sound wave originating from edge 1. And as a next step, the next possible diffraction edges, namely edges 10 and 11, can be found from edge 7. For example, if we navigate to edge 11, the intermediate angles from edges 1, 7, to edge 11 can then be calculated. The intermediate angle is defined as the angle between the inward and outward waves. In this example, the inward wave to side 7 is the vector from side 1 to side 7, while the outward wave is the vector from side 7 to side 11. This can be represented as... 1-7-11However, there is no intermediate angle at the beginning or end of the path. Instead, a maximum allowed angle (MAAS) for the source and a minimum allowed angle (MAAL) for the listener can be assigned. This means that if the source angle relative to the associated surface normal (in this case, the red one at edge 1) is greater than MAAS, the source can see the second edge (e.g., edge 7). Under the same concept, if the listener has an angle less than a given MAAL, the listener can see the edge before the last edge in the path. Based on the MAAL and MAAS values, the angles of the source and listener relative to the associated surface normal can be calculated in real time, thus verifying the path. Therefore, Figure 4 In the scenario, a pre-computed fourth-order path 400 can be defined as follows: Figure 8 The vectors of the sides, triangles, and angles shown at the top.

[0048] The overall process of the preferred pre-calculation of intermediate paths and related real-time rendering algorithms is as follows: Figure 5 As shown. Once all possible diffraction paths within the scene have been pre-calculated, it is only necessary to find a list of edges visible from the source position (the starting point of diffraction) and a list of edges visible from the listener position (the ending point) if the direct path between the source and the listener is occluded. Then, the source angle relative to the associated triangle (e.g., 1-R in Table 1) and the listener angle relative to the triangle (e.g., 12-B in Table 1) need to be calculated. If the source angle is less than MAAS and the listener angle is greater than MAAL, it will be the valid path along which the sound source signal will propagate. The source position and diffraction filter can then be updated using the edge vertex information and the associated angle. The binaural rendering module will synthesize (rotated and filtered) diffraction source information using appropriate directional filters (e.g., head-related transfer function (HRTF)). It is possible to add more features to the diffraction frame, such as directionality or distance effects.

[0049] Figure 1 An audio scene with four diffraction objects is shown, where a first-order diffraction path exists between edge 1 and edge 5. Figure 2b The upper part shows the first-order diffraction path from edge 1 to edge 5, where the angle criterion is that the source angle relative to the starting or input edge 1 must be less than the maximum permissible angle of the source. This is Figure 1 The same applies to the minimum permissible angle (MAAL) for the listener. Specifically, relative to the situation described above. Figure 1 The listener position angle calculated for the edge between vertex 5 and vertex 6 is greater than the listener's minimum allowable angle (MAAL). For Figure 1 The current source location and Figure 1 The current listener position in the current listener position. Figure 2bThe diffraction path shown at the top is valid, and the diffraction characteristics between edge 1 and edge 5 (which would be associated filter information) can be pre-stored for diffraction paths without dynamic objects, or can be used... Figure 2b The edge list can be used to calculate it simply.

[0050] Figure 4 The intermediate diffraction path 400 from the input or starting edge to the output or final edge 12 is shown. Figure 3 Another intermediate diffraction path 300 from the starting edge 1 to the final edge 13 is shown. Figure 3 and Figure 4 The audio scene in the image has additional diffraction paths, which either go from the source to edge 9, then to edge 13, and then to the listener, or from the source to edge 1 and from there to edge 5 and then from that edge to the listener. However, Figure 3 Path 300 in the code is only for... Figure 3 The listener position indicated in the diagram (which satisfies the listener's minimum permissible angle, i.e., the angular standard MAAL) is determined to be the valid intermediate diffraction path. However, Figure 4 The listener position shown will not meet this MAAL standard for path 300.

[0051] on the other hand, Figure 3 The listener's position in the middle will not be satisfied Figure 4 The path 400 shown is the MAAL standard. Therefore, the diffraction path provider 100 or preprocessor will forward... Figure 8 The multiple intermediate diffraction paths shown include those listed as the first entry in the list. Figure 4 The intermediate diffraction path 400 is shown, and provides Figure 3 Other intermediate diffraction paths 300 are shown. This list of intermediate diffraction paths will be provided to the renderer, and the renderer will determine one or more valid intermediate diffraction paths from the audio source location to the listener location. Figure 3 In the case of the listener's position shown, only Figure 8 Path 300 in the list will be determined as a valid intermediate diffraction path, while for Figure 4 The listener position shown will only be determined as a valid intermediate diffraction path for path 400.

[0052] refer to Figure 3 or Figure 4 In the scenario described, any intermediate diffraction path with a final edge or output edge of 15, 10, 7, or 2 will not be determined as a valid intermediate diffraction path because these edges are not visible to the listener. Figure 7 The renderer 200 in the middle will theoretically be able to pass through Figure 4In the audio scene, only the intermediate diffraction paths with a final or output edge visible to the listener (i.e., edges 4, 5, 12, and 13 in this example) are selected from all intermediate diffraction paths. This will correspond to edgelist(lis).

[0053] Similarly, relative to the source location, in the case of... Figure 7 Any pre-calculated intermediate diffraction paths provided in the intermediate diffraction path provider 100 with starting edges such as 3, 6, 11, or 14 will not be selected at all. Only those diffraction paths with starting edges 1, 8, 9, or 16 will be selected for specific verification using the MAAS angle standard. These edges will be in the edgelist (src).

[0054] In summary, the determination of the actual effective intermediate diffraction paths is selected in a three-stage process, which are used to ultimately determine the filter representation of sound propagation from the source to the listener. In the first stage, only pre-stored diffraction paths with a starting edge matching the source location are selected. In the second stage, only those intermediate diffraction paths with an output edge matching the listener location are selected. In the third stage, each of those selected paths is validated using both angular standards for the source and angular standards for the listener. The renderer then uses only the intermediate diffraction paths that have successfully passed through all three stages to calculate the audio output signal.

[0055] Figure 10 A preferred implementation of the selection information is shown. In step 102, potential starting edges for a specific source location are determined using geometric information of the audio scene, the source location, and specifically multiple intermediate diffraction paths pre-calculated by a preprocessor. In step 104, potential final edges for a specific listener location are determined. In step 106, potential intermediate diffraction paths are determined based on the results of steps 102 and 104, where steps 102 and 104 correspond to the first and second stages described above. In step 108, potential intermediate diffraction paths are verified by using angular conditions MAAS or MAAL, or generally by determining visibility, whether an edge is a diffraction edge. Step 108 illustrates the third stage described above. Input data MAAS and MAALS are obtained from, for example... Figure 8 The list of intermediate diffraction paths shown is obtained.

[0056] Figure 11 Another process is shown for the set of steps performed in block 108 to verify potential intermediate diffraction paths. In step 112, the source position angle is calculated relative to the starting edge. This corresponds to, for example... Figure 1The calculation of angle 113. In step 114, the listener's position angle is calculated relative to the final edge. This corresponds to the calculation of angle 113. Figure 1 The calculation of angle 115. In step 116, the source position angle is compared with the maximum permissible angle MAAS of the source, and if it is determined that the comparison result belongs to the case where the angle is greater than MAAS, the test has failed, as shown in box 120. However, when it is determined that angle 113 is less than MAAS, the first validity test has passed.

[0057] However, this intermediate diffraction path is only valid if the second verification also passes. This is obtained through the result of box 118 (i.e., comparing MAAL with the listener position angle 115). When angle 115 is greater than MAAL, the second contribution that passes the validity test is obtained as shown in box 122, and filter information is retrieved from the list of intermediate diffraction paths as shown in step 126, or, in the case of parameterized representation, based on data in the list, for example, according to... Figure 8 The middle angle indicated in the list is used to calculate filter information.

[0058] Once the filter information associated with the effective intermediate diffraction path is obtained (such as...) Figure 11 (The situation after step 126 in the text). Figure 7 The audio renderer 200 must calculate the final filter information, such as Figure 9 shown. Specifically, Figure 9 Step 126 in the middle corresponds to Figure 11 Step 126. In step 128, the starting filter information for the starting edge of the effective intermediate diffraction path from the source location is determined. Specifically, this describes, for example, Figure 1 The source in the image is the filter information for the audio propagation up to the vertex (1) of the edge. This propagation information refers not only to the attenuation due to distance but also to the angle. As is known from geometric diffraction theory (GTD) or uniform diffraction theory (UTD) or any other model of sound diffraction that may be used in this invention, the frequency characteristics of the diffracted sound depend on the diffraction angle. When the source angle 113 is very small, the source angle is closer to 113. Figure 1 Compared to the case where the source angle is close to 0 or very small, typically only the low-frequency components of the sound are diffracted, while the high-frequency components are attenuated more significantly. In this case, the high-frequency attenuation is reduced compared to the case where the source angle is close to 0 or very small.

[0059] Similarly, the final filter information from the final or output edge 5 to the listener position is determined again based on the listener angle 115 relative to MAAL. Thus, once these three filter information items or filter contributions are determined, they are combined in step 132 to obtain a filter representation of the complete diffraction path, which includes the path from the source to the starting edge, the intermediate diffraction path, and the path from the output or final edge to the listener position. This combination can be performed in several ways, and one effective method is to transform each of the three filter representations obtained in steps 128, 126, and 130 into a spectral representation to obtain the corresponding transfer function, and then multiply these three transfer functions in the spectral domain to obtain the final filter representation, which can be used if the audio renderer operates in the frequency domain. Alternatively, if the audio renderer operates in the time domain, the frequency domain filter information can be transformed into time domain filter information. Alternatively, the three filter items can be convolved using the time domain filter impulse responses representing the respective filter contributions, and then the audio renderer can use the resulting time domain filter impulse responses for rendering. In this scenario, the renderer will perform a convolution operation between the audio source signal on one side and the complete filter representation on the other.

[0060] Subsequently, it was shown Figure 5 A flowchart is provided to offer a preferred implementation for rendering static diffraction objects. The process begins at box 202. Then, pre-calculation steps are provided to generate, as shown by... Figure 7 The diffraction path provider 100 provides a list. In step 204, the tracker is set up with a mesh for occlusion testing. This step determines all distinct diffraction paths between edges, where, by definition, a diffraction path only occurs when a direct path between two non-adjacent edges is occluded. When considering, for example... Figure 3 At this time, the path between edge 1 and edge 11 is blocked by edge 7, thus diffraction occurs. This situation is determined, for example, by box 204. In box 206, a list of potential diffraction edges is calculated using the interior angles between two adjacent triangles. This process determines the midpoint or interior angle of such a diffraction path portion (i.e., the portion between edge 1 and edge 11). Step 206 also determines another diffraction path portion between edge 7 via edge 11 to edge 12 or between edge 7 via edge 11 to edge 13. The corresponding diffraction paths are pre-calculated (e.g., as shown in box 204). Figure 8 As shown), such that, for example Figure 3 Path 300 or for example Figure 4 Path 400 describes the entire sound propagation (for path 300, from edge 1 to edge 13, or for...). Figure 4The associated filter information for path 400 (from edge 1 to edge 12) is pre-computed. The pre-computed process is completed, and the runtime steps are executed. In step 210, the renderer 200 obtains the source location data and the listener location data. In step 212, a directional path occlusion test between the source and the listener is performed. The process continues only if the test result in box 212 indicates that the directional path is occluded. If the direct path is not occluded, direct propagation occurs, and any diffraction is not a problem for that path.

[0061] In step 214, a list of visible edges from the source is determined on one hand, and a list of visible edges from the listener is determined on the other. This process corresponds to... Figure 6 Steps 102, 104, and 106. In step 216, the path that starts from the input edge of the edge list and ends at the output edge of the listener's edge list is verified. This corresponds to in Figure 10 The process is performed in box 108. In step 218, a filter representation is determined such that the source position can be updated by rotation relative to the associated edge, and the diffraction filter, for example, from a UTD model database, can be updated. However, in general, the invention is not limited to UTD model database applications, but can be applied using any specific calculation and application of filter information from the diffraction path. In step 220, the audio output signal of the audio scene is calculated, for example, by means of binaural rendering using an associated delay line module, the presence of which is for rendering distance effects when they are not included in the corresponding binaural rendering directional filter (e.g., certain HRTF filters).

[0062] Figure 12 This illustrates a rotation performed on an unrotated source position to improve the audio quality of a rendered audio scene. This rotation is preferably applied to... Figure 5 Step 218 or Figure 6 Step 218. Rotating the source position for rendering or spatialization purposes helps improve spatial perception of the original source position. Therefore, regarding Figure 12 The sound source is rendered at a new source position 142, which is obtained by rotating the original source position 143 around edge 9 by an angle DA_9 to an intermediate position 141. This angle is determined by the line connecting edge 13 and edge 9, thus obtaining a straight line. The intermediate position 141 is then rotated around edge 13 by an angle DA_13 to have a straight line from the listener to the final rotated new source position 142. Therefore, not only is the equalization or attenuation value spatialized depending on the frequency, but the perceived direction of the original source (at the rotated new source position 142) is also spatialized. Since the sound diffraction effect changes the angle of sound propagation in each diffraction process, this final rotated new source position is the perceived source position.

[0063] Refer to an exemplary diffraction path from source to listener: “Source – (9) – (13) – Listener”. Use the rotated new source position 142 to generate additional φ (phi) and θ (theta) information for reproducing spatial sound.

[0064] Complete, correlated filter information considering the exact source / listener location already provides precise EQ information for each frequency, i.e., the attenuation effect caused by diffraction. A low-level implementation is already constructed using the original source location and the distance to the original source. This low-level implementation is enhanced by additionally creating the information needed to select an appropriate HRTF filter. For this purpose, the original sound source is rotated relative to the relevant edge by a certain number of diffraction angles to generate the location of the diffracted source. The azimuth and elevation angles can then be derived from this location relative to the listener, and the total propagation distance along this path can be obtained.

[0065] Figure 12 The calculation of the distance between the new source position 142 and the listener position obtained through the rotation process in the final render is also shown. Preferably, this distance is additionally used to determine the delay of the distance-related attenuation between the two for rendering the source.

[0066] Subsequently, further explanation is given regarding the use and determination of the new source position 142 after the rotation of the original source position 143. Each step in the calculation for obtaining the effective path processes the original source position 143. However, to achieve binaural rendering that allows users equipped with headphones to experience better immersive sound in VR space, it is preferable to provide the sound source position to the binaural units such that they can apply appropriate spatial filtering (H_L and H_R) to the original audio signal, where H_L / H_R is referred to as the head-dependent transfer function (HRTF), as described, for example, at https: / / www.ece.ucdavis.edu / cipic / spatial-sound / tutorial / hrtf / .

[0067] Filtered_S_L=H_L(φ,θ,w)*S(w)

[0068] Filtered_S_R=H_R(φ,θ,w)*S(w)

[0069] A mono signal S(w) does not provide any locational cue for generating spatial sound. However, sound filtered by HRTF can reproduce a spatial impression. Therefore, φ and θ (i.e., the relative azimuth and elevation angles of the diffraction source) should be provided through this process. This is why the original sound source is rotated. Therefore, in addition to receiving filter information, the renderer also receives... Figure 12The information of the new source position 142. Although the orientation of the original source position 143 may typically be used for low-level implementations to avoid the complexity of source rotation, this process is subject to... Figure 12 The influence of orientation error is visible. However, for low-level implementations, this error is acceptable. The same applies to the influence of distance. Figure 12 As shown, the distance from the new source position 142 after rotation to the listener is slightly longer than the distance from the original source position 143 to the listener. This distance error is acceptable for low-level implementations to reduce complexity. However, for advanced applications, this error can be avoided.

[0070] Therefore, the position of the diffracted sound source can also be provided by rotating the original source relative to the relevant edge to generate the propagation distance from the source to the listener, where this distance is used for attenuation due to distance.

[0071] The process used to generate additional information about φ, θ, and distance is also useful for multi-channel playback systems. The only difference is that for multi-channel playback systems, a different set of spatial filters is applied to S(w) to feed Filtered_S_i to the i-th speaker, such as "Filtered_S_i = H_i(φ, θ, w, other parameters) * S(w)".

[0072] A preferred embodiment relates to the operation of a renderer configured to: calculate a rotated audio source position depending on an effective intermediate diffraction path or depending on a complete diffraction path, the rotated audio source position being different from the audio source position due to diffraction effects produced by the effective intermediate diffraction path or depending on the complete diffraction path, and use the rotated audio source position when calculating the audio output signal of the audio scene (220), or the renderer is configured to: calculate the audio output signal of the audio scene using, in addition to using a filter representation, a sequence of edges associated with the complete diffraction path and a sequence of diffraction angles associated with the complete diffraction path.

[0073] In another embodiment, the renderer is configured to determine the distance from the listener's position to the rotated source position and use that distance when calculating the audio output signal of the audio scene.

[0074] In another embodiment, the renderer is configured to select one or more directional filters depending on the rotated source position and a predetermined output format for the audio output signal, and to apply the one or more directional filters and filter representations to the audio signal when calculating the audio output signal.

[0075] In another embodiment, the renderer is configured to determine an attenuation value based on the distance between the rotated source position and the listener's position; and to apply the attenuation value to the audio signal in addition to one or more directional filters that represent or depend on the audio source position or the rotated audio source position.

[0076] In another embodiment, the renderer is configured to determine the rotated source position in a series of rotation operations that include at least one rotation operation.

[0077] In the first step of this sequence, starting from the first diffraction edge of the complete diffraction path, the path portion from the first diffraction edge to the source position is rotated in the first rotation operation to obtain a straight line from the second diffraction edge (or, in the case where the complete diffraction path has only the first diffraction edge, from the listener position) to the first intermediate rotated source position, where, when the complete diffraction path has only the first diffraction edge, the first intermediate rotated source position is the rotated source position. This sequence will be performed for a single diffraction edge. In the case of two diffraction edges, the first edge will be... Figure 12 In the middle, edge 9 is the first middle position, and the middle position is 141.

[0078] In the case of more than one diffraction edge, the result of the first rotation operation is rotated around the second diffraction edge in the second rotation operation to obtain a straight line from the third diffraction edge (or, in the case where the complete diffraction path has only the first and second diffraction edges, from the listener position) to the second intermediate rotated source position, where, when the complete diffraction path has only the first and second diffraction edges, the second intermediate rotated source position is the rotated source position. This sequence will be performed for two diffraction edges. In the case of two diffraction edges, the first edge will be... Figure 12 The first edge is 9, and the second edge is 13.

[0079] In paths with more than two diffraction edges (e.g., Figure 3 In the case of path 300), continue the process, whereby utilizing Figure 3 The third diffraction edge 11 in the middle, and then using Figure 3 edge 13 or using Figure 4 Edge 12 in the middle, additionally performs one or more rotation operations, and usually until the complete diffraction path is processed and a straight line from the listener position to the rotated source position obtained at that time is obtained.

[0080] A preferred embodiment for handling dynamic objects (DOs) is then shown. For this purpose, refer to... Figure 2b , Figure 2b It shows: from one moment to another, relative to Figure 1 The situation in the audio scene has been placed in the center of the dynamic object DO. This means...Figure 2b The diffraction path from edge 1 to edge 5 shown in the top row is interrupted by the diffraction object DO, and two new diffraction paths have been generated, one from edge 1 to edge 7 and then to edge 5, and the other from edge 1 to edge 3 and then to edge 5. The listener is placed... Figure 2a In the case on the left, these diffraction paths are relevant. Because a dynamic object has been placed into the sound scene, the MAAS and MAAL conditions have also changed relative to the case without a dynamic object. Figure 2b The lower part shows two additional intermediate diffraction paths to enhance... Figure 1 A list of intermediate diffraction paths, such as... Figure 6 As shown in project 226. Specifically, when assuming Figure 1 When the original path from edge 1 to edge 5 is only part of a larger reflection case (in which the source is not placed close to edge 1, but rather there is one or more other reflection paths between objects, and the situation is similar for the listener), it can be easily updated at runtime for the target object in the following way: Figure 1 Pre-computed diffraction paths exist in the absence of dynamic objects: replace the diffraction path from edge 1 to edge 5 with only two additional paths, while keeping the earlier portions of the diffraction path from edge 1 to any starting edge as is, and also keeping the portion of the path from edge 5 to any output edge or final edge as is.

[0081] Figure 6 This illustrates the process performed with dynamic objects. In step 222, it is determined whether the dynamic object DO has changed its position, for example, through translation and rotation. The edges attached to the dynamic object are updated. Figure 2a In the example, step 222 will determine: with Figure 1 Compared to the earlier time point shown, there exists a dynamic object with specific edges 70, 60, 20, and 30. In step 214, an intermediate diffraction path including edges 1 and 5 will be found. In step 224, it will be determined that an interruption exists because the dynamic object is placed in the path between edges 1 and 5. In step 224, the following will be found: Figure 2b The additional paths shown at the bottom of the diagram go from edge 1 through dynamic object edge 30 to edge 5 on one hand, and from edge 1 through dynamic object edge 60 to edge 5 on the other. In step 226, the uninterrupted path will be enhanced through the two additional paths. This means that it will be enhanced by using... Figure 2bThe two additional path portions shown replace the path portion between edge 1 and edge 5 to modify the path from any (not shown) input edge to edge 1 and from edge 5 to any (not shown) output edge (not shown in the figure). The first path portion from the input edge to edge 1 is concatenated with these two path portions to obtain two additional intermediate diffraction paths, and the output portion of the original path extending from edge 5 to the output edge is also concatenated with two corresponding enhanced intermediate diffraction paths, so that due to the dynamic object, two new enhanced early diffraction paths have been generated from an early intermediate diffraction path (without the dynamic object). Figure 6 The other steps shown are the same as Figure 5 Similar to what has already been shown.

[0082] Rendering the diffraction effect of dynamic objects (at runtime) is one of the best ways to utilize immersive media to create interactive impressions for entertainment. Preferred strategies for considering the diffraction of dynamic objects are as follows:

[0083] 1) In the pre-calculation step:

[0084] A. If dynamic objects / geometry exist, pre-calculate the possible (intermediate) diffraction paths around the given dynamic object.

[0085] B. If there are multiple dynamic objects / geometry, pre-calculate possible (intermediate) diffraction paths around a single object based on the assumption that diffraction is not allowed between different dynamic objects.

[0086] C. If dynamic / geometry and static objects / geometry exist, pre-calculate possible paths around the dynamic or static object based on the assumption that diffraction is not allowed between static and dynamic objects.

[0087] 2) During runtime steps:

[0088] A. Potential edges belonging to a repositioned dynamic mesh are updated only when the dynamic mesh is repositioned (in terms of translation and rotation).

[0089] B. Find the list of visible edges of the source and the list of visible edges of the listener.

[0090] C. Verify the path that starts at the edge list of the source and ends at the edge list of the listener.

[0091] D. Test the visibility between intermediate edge pairs, and if there is an intrusion of a disruptive object (which may be a dynamic or static object), enhance the path within the verified path with edges, triangles, and angles.

[0092] Figure 6 The diagram shows an extended algorithm for handling dynamic objects / geometry, which has additional steps compared to a static scene.

[0093] Given that the preferred method pre-calculates (intermediate) diffraction path information, which does not need to be revisited except in special cases, this presents many practical advantages compared to existing techniques that do not allow updating pre-calculated data. Furthermore, the flexible feature of combining multiple diffraction paths to generate an enhanced path makes it possible to consider both static and dynamic objects together.

[0094] (1) Lower computational complexity: The preferred method does not require constructing the complete path from the given source location to the listener's location at runtime. Instead, it only needs to find a valid intermediate path between the two points.

[0095] (2) Ability to render the diffraction effect of a combination of static and dynamic objects or multiple dynamic objects: Existing techniques require updating the entire visibility graph between (static or dynamic) edges at runtime to account for the diffraction effect of both static and dynamic objects. Preferred methods require an efficient splicing process for two valid paths / path portions.

[0096] On the other hand, pre-calculating (intermediate) diffraction paths requires more time compared to existing technologies. However, it is possible to control the size of the pre-calculated path data by applying reasonable constraints (e.g., the maximum allowable attenuation level in a complete path, the maximum propagation distance, the maximum order of diffraction, etc.).

[0097] 1) [A Geometric Acoustics-Based Method] has invented a preferred method for applying a UTD model to multiple visible / correctly oriented edges based on pre-computed (intermediate) path information. Except in rare cases (e.g., interruptions caused by dynamic objects), real-time monitoring of this pre-computed data is (most of the time) unnecessary. Therefore, this invention minimizes real-time computation.

[0098] 2) [Modular] Each pre-calculated path is treated as a module.

[0099] A. For static scenes, in the real-time step, we only need to find the effective module between two spatial points.

[0100] B. In dynamic scenarios, even if there is an interruption caused by another object (A) in the valid path of object (B), we still need to use the valid path through A to enhance the path through B. (Imagine stitching together two different images).

[0101] 3) [Supports full dynamic interaction] Real-time rendering diffraction effects can be achieved, including combinations of static and dynamic objects or multiple dynamic objects.

[0102] It should be noted that all the alternatives or aspects discussed above, as well as all aspects defined by the independent claims in the appended claims, can be used individually, i.e., they can be used in the absence of any other alternatives or objectives besides the contemplated alternatives, objectives, or independent claims. However, in other embodiments, two or more alternatives or aspects or independent claims can be combined with each other, and in other embodiments, all aspects or alternatives and all independent claims can be combined with each other.

[0103] The encoded signal of the present invention can be stored on a digital storage medium or a non-transitory storage medium, or it can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium (e.g., the Internet).

[0104] Although some aspects have been described in the context of the apparatus, it will be clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also indicate a description of the features of the corresponding block or item or the corresponding apparatus.

[0105] Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. Implementations may be carried out using digital storage media (e.g., floppy disks, DVDs, CDs, ROMs, PROMs, EPROMs, EEPROMs, or FLASH memories) that store electronically readable control signals thereon, in cooperation with (or capable of cooperating with) a programmable computer system, such that the corresponding methods are executed.

[0106] Some embodiments of the invention include a data carrier having electronically readable control signals, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0107] Typically, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. The program code may, for example, be stored on a machine-readable medium.

[0108] Other embodiments include a computer program stored on a machine-readable carrier or non-transitory storage medium for performing one of the methods described herein.

[0109] In other words, embodiments of the method of the present invention are therefore computer programs having program code for performing one of the methods described herein when the computer program is run on a computer.

[0110] Therefore, another embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) on which a computer program is recorded, the computer program being used to perform one of the methods described herein.

[0111] Therefore, another embodiment of the method of the present invention represents a data stream or signal sequence of a computer program used to perform one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0112] Another embodiment includes a processing means, such as a computer or a programmable logic device, which is configured or adapted to perform one of the methods described herein.

[0113] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0114] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0115] The above embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and variations of the arrangements and details described herein will be readily apparent to those skilled in the art. Therefore, the invention is intended to be limited only by the scope of the appended claims and not by the specific details given by way of the description and explanation of the embodiments herein.

[0116] References

[0117] [1] L. Savioja and V. Välimäki. Interpolated rectangular 3-D digitalwaveguide mesh algorithms with frequency warping. IEEE Trans. Speech AudioProcess., 11(6) 783–790, 2003

[0118] [2] Mehra, R., Raghuvanshi, N., Antani, L., Chandak, A., Curtis, S.,And Manocha, D. Wave-based sound propagation in large open scenes using anequivalent source formulation, ACM Trans. on Graphics 32(2) 19:1–19:13, 2013

[0119] [3] Mehra, R., Antani, L., Kim, S., and Manocha, D. Source andlistener directivity for interactive wave-based sound propagation, IEEETransactions on Visualization and Computer Graphics, 20(4) 495–503, 2014

[0120] [4] Nikunj Raghuvanshi and John M. Snyder, Parametric directionalcoding for precomputed sound propagation, ACM Trans. on Graphics 37(4) 108:1-108:14, 2018

[0121] [5] J. B. Allen, and D. A. Berkley, Image method for efficientlysimulating small-room acoustics. The Journal of the Acoustical Society ofAmerica 65(4) 943–950, 1979

[0122] [6] M. Vorländer, Simulation of the transient and steady-state soundpropagation in rooms using a new combined raytracing / image-source algorithm,The Journal of the Acoustical Society of America 86(1) 172–178, 1989

[0123] [7] T. Funkhouser, I. Carlbom, G. Elko, G. Pingali, M. Sondhi, and J.West, A beam tracing approach to acoustic modeling for interactive virtualenvironments, In Proc. of ACM SIGGRAPH, 21–32, 1998

[0124] [8] M. Taylor, A. Chandak, L. Antani, and D. Manocha, Resound:interactive sound rendering for dynamic virtual environments, In Proc. of theseventeen ACM international conference on Multimedia, 271–280, 2009

[0125] [9] R. G. Kouyoumjian and P. H. Pathak, A uniform geometrical theoryof diffraction for an edge in a perfectly conducting surface, In Proc. of theIEEE 62, 11, 1448–1461, 1974

[0126]

[10] U. P. Svensson, R. I. Fred, and J. Vanderkooy, An analyticsecondary source model of edge diffraction impulse responses, AcousticalSociety of America Journal 106 2331–2344, 1999

[0127]

[11] N. Tsingos, T. Funkhouser, A. Ngan, and I. Carlbom, Modelingacoustics in virtual environments using the uniform theory of diffraction, InProc. of the SIGGRAPH, 545–552, 2001

[0128]

[12] M. Taylor, A. Chandak, Q. Mo, C. Lauterbach, C. Schissler, andD. Manocha, Guided multiview ray tracing for fast auralization. IEEETransactions on Visualization and Computer Graphics 18, 1797–1810, 2012

[0129]

[13] H. Yeh, R. Mehra, Z. Ren, L. Antani, D. Manocha, and M. Lin,Wave-ray coupling for interactive sound propagation in large complex scenes,ACM Trans. Graph. 32, 6, 165:1–165:11, 2013。

Claims

1. An apparatus for rendering an audio scene including an audio source located at an audio source location and a plurality of diffraction objects, the apparatus comprising: A diffraction path provider is used to provide multiple intermediate diffraction paths through the plurality of diffraction objects, wherein the intermediate diffraction paths have the start points and output edges of the plurality of diffraction objects and the associated filter information of the intermediate diffraction paths; and A renderer is configured to render the audio source at the listener's location, wherein the renderer is configured to: Based on the output edge of the intermediate diffraction path and the listener's position, one or more valid intermediate diffraction paths are determined from the audio source position to the listener's position. For each of the one or more effective intermediate diffraction paths, a filter representation of the complete diffraction path from the audio source location to the listener location is determined using a combination of the associated filter information of the effective intermediate diffraction path and filter information describing the propagation of the audio signal from the output edge of the effective intermediate diffraction path to the listener location. The audio output signal of the audio scene is calculated using the audio signal associated with the audio source and the filter representation of each complete diffraction path.

2. The apparatus according to claim 1, wherein, The audio source location is fixed, and the diffraction path provider is configured as a preprocessor and configured to: determine each valid intermediate diffraction path such that the starting point of each valid intermediate diffraction path corresponds to the audio source location, and wherein the renderer is configured to: determine the one or more valid intermediate diffraction paths based on the input edges of the one or more intermediate diffraction paths and the audio source location, and determine a filter representation of the complete diffraction path based on another filter information describing the propagation of the audio signal from the audio source location to the input edges of the valid intermediate diffraction paths associated with the complete diffraction path, or Wherein, the audio source location is variable, and wherein the diffraction path provider is configured as a preprocessor and configured to determine the input edges of the plurality of diffraction objects as the starting points of intermediate diffraction paths, and wherein the renderer is configured to: determine the one or more valid intermediate diffraction paths based on the input edges of the one or more intermediate diffraction paths and the audio source location, and determine a filter representation of the complete diffraction path based on another filter information describing the propagation of the audio signal from the audio source location to the input edges of the valid intermediate diffraction paths associated with the complete diffraction path.

3. The apparatus according to claim 1, wherein, The renderer is configured to perform an occlusion test on the direct path from the audio source location to the listener location, and to determine one or more valid intermediate diffraction paths only if the occlusion test indicates that the direct path is occluded.

4. The apparatus according to claim 1, in, The renderer is configured to determine the filter representation of the complete diffraction path by multiplying the frequency domain representation of the associated filter information with the frequency domain representation of the filter information propagating from the output edge of the effective intermediate diffraction path to the listener's position, or the frequency domain representation of another filter information describing the propagation of the audio signal from the audio source position to the input edge of the effective intermediate diffraction path.

5. The apparatus according to claim 1, wherein, The renderer is configured as follows: The starting group of potential input edges is determined based on the location of the audio source, or the final group of potential output edges is determined based on the location of the listener. Using the starting group or the final group, one or more potentially valid intermediate diffraction paths are retrieved from a pre-stored list of intermediate diffraction paths, and The one or more potentially valid intermediate diffraction paths can be verified using the source angle standard and the source angle between the source position and the corresponding input edge, or by using the final angle standard and the listener angle between the listener position and the corresponding output edge.

6. The apparatus according to claim 5, wherein, The renderer is configured to: calculate the source angle, compare the source angle with the maximum permissible angle (MAAS) of the source as a standard for the source angle, and verify that a potential intermediate diffraction path is a valid intermediate diffraction path when the source angle is less than the maximum permissible angle of the source. The renderer is configured to: calculate the listener angle, compare the listener angle with the minimum permissible angle MAAL of the listener as the standard for the listener angle, and verify that a potential intermediate diffraction path becomes a valid intermediate diffraction path when the listener angle is greater than the minimum permissible angle of the listener.

7. The apparatus according to claim 1, in, The diffraction path provider is configured to access a memory having a pre-stored list of entries for the plurality of intermediate diffraction paths, wherein each intermediate diffraction path entry includes a sequence of sides extending from the input side to the output side, or a sequence of triangles extending from the input triangle to the output triangle, including one or more intermediate angles and including a listener angle standard.

8. The apparatus according to claim 7, in, The pre-stored list of intermediate diffraction path entries includes associated filter information or references to said associated filter information, or The renderer is configured to derive the associated filter information based on data from intermediate diffraction path entries in the pre-stored list.

9. The apparatus according to claim 1, in, The plurality of diffraction objects in the audio scene include dynamic objects, and wherein the diffraction path provider is configured to provide at least one intermediate diffraction path around the dynamic object.

10. The apparatus according to claim 1, in, The plurality of diffraction objects in the audio scene include two or more dynamic diffraction objects, and wherein the diffraction path provider is configured to provide an intermediate diffraction path around a single dynamic diffraction object based on the assumption that diffraction is not allowed between two different dynamic objects.

11. The apparatus according to claim 1, in, The plurality of diffraction objects in the audio scene include one or more dynamic objects and one or more static objects, wherein the diffraction path provider is configured to provide an intermediate diffraction path around a dynamic object or a static object based on the assumption that diffraction is not allowed between static objects and dynamic objects.

12. The apparatus according to claim 1, in, The plurality of diffraction objects includes at least one dynamic diffraction object. The renderer is configured as follows: Determine whether at least one dynamic diffraction object has been repositioned with respect to at least one of translation and rotation to obtain the repositioned dynamic object. Update the edges attached to the repositioned dynamic object; In the step of determining one or more effective intermediate diffraction paths, potential effective intermediate paths are examined for visibility between edge pairs, wherein, in the case where visibility is interrupted due to the repositioning of the repositioned dynamic object, potential effective intermediate diffraction paths are enhanced by additional paths generated due to the repositioned dynamic object to obtain an effective intermediate diffraction path among the one or more effective intermediate diffraction paths.

13. The apparatus according to claim 1, wherein, The renderer is configured to apply Uniform Diffraction Theory (UTD) to determine the associated filter information, or the renderer is configured to determine the associated filter information in a frequency-dependent manner.

14. The apparatus according to claim 1, wherein, The renderer is configured to: calculate the rotated audio source position based on either the effective intermediate diffraction path or the complete diffraction path, wherein the rotated audio source position differs from the audio source position due to diffraction effects produced by the effective intermediate diffraction path or depending on the complete diffraction path; and use the rotated audio source position when calculating the audio output signal of the audio scene, or The renderer is configured to calculate the audio output signal of the audio scene using, in addition to the filter representation, a sequence of edges associated with the complete diffraction path and a sequence of diffraction angles associated with the complete diffraction path.

15. The apparatus according to claim 14, wherein, The renderer is configured to: determine the distance from the listener's position to the rotated audio source position; and use the distance when calculating the audio output signal of the audio scene.

16. The apparatus according to claim 14, wherein, The renderer is configured to select one or more directional filters depending on the rotated position of the audio source and a predetermined output format for the audio output signal; Furthermore, when calculating the audio output signal, the one or more directional filters and the filter representations are applied to the audio signal.

17. The apparatus according to claim 14, wherein, The renderer is configured to determine an attenuation value based on the distance between the rotated audio source position and the listener position, and to apply the attenuation value to the audio signal in addition to one or more directional filters that represent or depend on the audio source position or the rotated audio source position.

18. The apparatus according to claim 14, wherein, The renderer is configured to determine the position of the rotated audio source in a series of rotation operations, including at least one rotation operation. Specifically, starting from the first diffraction edge of the complete diffraction path, the path portion from the first diffraction edge to the source position is rotated in the first rotation operation to obtain a straight line from the second diffraction edge to the first intermediate rotated audio source position, or a straight line from the listener position to the first intermediate rotated audio source position when the complete diffraction path only has the first diffraction edge. When the complete diffraction path only has the first diffraction edge, the first intermediate rotated audio source position is the rotated audio source position. In the second rotation operation, the result of the first rotation operation is rotated around the second diffraction edge to obtain a straight line from the third diffraction edge to the second intermediate rotated audio source position, or a straight line from the listener position to the second intermediate rotated audio source position when the complete diffraction path only has the first and second diffraction edges. When the complete diffraction path only has the first and second diffraction edges, the second intermediate rotated audio source position is the rotated audio source position. One or more rotation operations are performed until the complete diffraction path is processed and a straight line from the listener's position to the obtained rotated audio source position is obtained.

19. A method for rendering an audio scene including an audio source located at an audio source location and a plurality of diffraction objects, the method comprising: Provide multiple intermediate diffraction paths through the plurality of diffraction objects, wherein each intermediate diffraction path has the starting point and output edge of the plurality of diffraction objects and associated filter information of the intermediate diffraction path; and Rendering the audio source at the listener's location, wherein the rendering includes: Based on the output edge of the intermediate diffraction path and the listener's position, one or more valid intermediate diffraction paths are determined from the audio source position to the listener's position. For each of the one or more effective intermediate diffraction paths, a filter representation of the complete diffraction path from the audio source location to the listener location is determined using a combination of the associated filter information of the effective intermediate diffraction path and filter information describing the propagation of the audio signal from the output edge of the effective intermediate diffraction path to the listener location. The audio output signal of the audio scene is calculated using the audio signal associated with the audio source and the filter representation of each complete diffraction path.

20. A computer-readable storage medium having a computer program stored thereon, the computer program causing the computer or processor, when executed on a computer or processor, to perform the method according to claim 19.

Images