Apparatus and method for rendering multi-path sound diffraction with multi-layer grid map

Abstract

An apparatus (100) for decoding according to an embodiment is provided. The apparatus (100) comprises an audio signal decoder (110) for decoding an encoding of an audio object signal of an audio object. Further, the apparatus (100) comprises a metadata decoder (120) for decoding an encoding of metadata, wherein the metadata comprise, for each of a plurality of listener positions, information about two or more different sound wave propagation paths from a sound source position of the audio object to the listener position. Further, the apparatus (100) comprises a signal generator (130) for generating one or more audio output signals depending on the audio object signal and depending on the information about two or more different sound wave propagation paths from the sound source position to a current listener position of the plurality of listener positions.

Description

[0001] describe

[0002] This invention relates to the encoding, decoding, and rendering of multipath sound diffraction, and more particularly to the encoding, decoding, and rendering of multipath sound diffraction using multi-layer raster maps.

[0003] AR / VR systems generate visual and auditory virtual environments by visualizing and auralizing virtual scenes. For example, real-time and offline audio rendering of auditory scenes and environments is described in [1] (see also [1a]).

[0004] The auditory process simulates sound propagation in a virtual environment by considering acoustic effects such as sound wave blocking, reflection, and diffraction. These wave phenomena can be reproduced very accurately by solving the sound wave equations, but doing so over the entire audible spectrum is not feasible in real time.

[0005] AR / VR systems require low-latency audio rendering methods that use simplified models (such as geometric acoustics methods). For example, image models (IM) can be used to model sound reflections, while uniform diffraction theory (UTD) can be used to model sound diffraction on the edges of polygonal meshes [2], [3].

[0006] Another approach to simplifying the geometry of an acoustic environment is to use voxels (volume pixels), where the environment is discretized by a uniform grid of three-dimensional blocks. Shortest path search algorithms, such as Dijkstra's algorithm, A*, or jump point search, can be used to determine the direction of a listener arriving at a discrete location before the first wave [4], [5], [6].

[0007] By restricting the voxel lattice to a certain size or considering only a two-dimensional cross-section of geometry, shortest path search algorithms can be used to simulate diffraction effects in real time. However, considering only the shortest propagation path of diffracted sound is a strong constraint that can lead to clearly audible artifacts. For example, when a user of an AR / VR system moves around occluded and diffracted objects, at a certain point, the shortest propagation path of diffracted sound will jump from one side to the other.

[0008] The object of this invention is to provide an improved concept for encoding and decoding coordinates associated with audio-related and / or video-related data.

[0009] The object of the present invention is achieved by the apparatus according to claim 1, the apparatus according to claim 24, the method according to claim 45, the method according to claim 46, and the computer program according to claim 47.

[0010] An apparatus for decoding is provided according to an embodiment. The apparatus includes an audio signal decoder for decoding an encoding of an audio object signal of an audio object. Furthermore, the apparatus includes a metadata decoder for decoding an encoding of metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more distinct sound wave propagation paths from the sound source location of the audio object to the listener location. Additionally, the apparatus includes a signal generator for generating one or more audio output signals based on the audio object signal and based on the information about the two or more distinct sound wave propagation paths from the sound source location to the current listener location among the plurality of listener locations.

[0011] Furthermore, an encoding apparatus according to an embodiment is provided. The apparatus includes an audio signal encoder for encoding an audio object signal of an audio object to obtain an encoded audio object signal. Additionally, the apparatus includes a metadata encoder for encoding metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more distinct sound wave propagation paths from the sound source location of the audio object to the listener location.

[0012] Furthermore, a method for decoding is provided according to an embodiment. The method includes:

[0013] - Decode the encoded audio object signal of the audio object.

[0014] - Decode the encoded metadata, where, for each of multiple listener locations, the metadata includes information about two or more distinct sound wave propagation paths from the sound source location of the audio object to the listener location; and

[0015] - Generate one or more audio output signals based on the audio object signal and information about two or more different sound wave propagation paths for the current listener position among multiple listener positions.

[0016] Furthermore, a method for encoding according to an embodiment is provided. The method includes:

[0017] - Encode the audio object signal of the audio object to obtain the encoded audio object signal, and

[0018] - Encode the metadata, where, for each of the multiple listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location.

[0019] In addition, a computer program is provided for implementing one of the above methods when executed on a computer or signal processor.

[0020] When referring to coordinate positions below, this should be understood as a position defined relative to a coordinate system (e.g., a two-dimensional coordinate system or a three-dimensional coordinate system). For example, in a two-dimensional coordinate system, the coordinate position is defined by two coordinates of the coordinate system. For example, in a three-dimensional coordinate system, the coordinate position is defined by three coordinates of the coordinate system. For example, (2; 4) is a coordinate position in a two-dimensional coordinate system.

[0021] For example, a coordinate position is a position defined by two or more coordinates, such as a coordinate system.

[0022] The examples provide calculations and efficient encoding of multi-layer maps / graphics.

[0023] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings, wherein:

[0024] Figure 1 An apparatus for decoding according to an embodiment is shown.

[0025] Figure 2 An apparatus for encoding according to another embodiment is shown.

[0026] Figure 3 A system according to an embodiment is shown.

[0027] Figure 4 An example of a multi-layer raster map is shown, in which two raster map layers are used as the source to determine two diffraction paths around a rectangular diffraction object.

[0028] Figure 5 It shows in Figure 4 The propagation of sound waves is depicted in both lower-level and higher-level raster maps.

[0029] Figure 6 It shows the target of being made by Figure 4 and Figure 5 The examples depict raster data from lower-level and higher-level raster maps.

[0030] Figure 1 A device 100 for decoding according to an embodiment is shown.

[0031] The device 100 includes an audio signal decoder 110 for decoding the encoding of an audio object signal of an audio object.

[0032] In addition, the device 100 includes a metadata decoder 120 for decoding the encoding of metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location.

[0033] In addition, the device 100 includes a signal generator 130 for generating one or more audio output signals based on an audio object signal and based on information about two or more different sound wave propagation paths from the sound source location to the current listener location among a plurality of listener locations.

[0034] For example, for possible listener location information, the metadata includes at least two sound wave propagation paths from the sound source location to the listener location. Furthermore, the metadata includes not only such information for a single possible listener location, but also such information for two or more possible listener locations (“multiple listener locations”).

[0035] Figure 1 The embodiments implement information about two or more sound wave propagation paths for the actual / current listener location from among information about multiple possible listener locations when generating an audio output signal.

[0036] According to an embodiment, at least one of two or more different sound wave propagation paths from the sound source location to one of the multiple listener locations depends on the diffraction of the sound waves originating from the sound source location at the object.

[0037] For example, at least one sound wave can be diffracted.

[0038] In an embodiment, the first of two or more distinct sound wave propagation paths from the sound source location to one of the multiple listener locations represents the shortest path for the sound wave to propagate from the sound source location to said one of the multiple listener locations, and one or more of the other sound wave propagation paths are different from said shortest path.

[0039] The propagation path (for example, information about it can be provided in the metadata) includes the shortest sound wave propagation path and one or more other propagation paths.

[0040] According to an embodiment, information about a second sound wave propagation path in two or more different sound wave propagation paths from a sound source location to one of a plurality of listener locations reuses information about a first sound wave propagation path in two or more different sound wave segments.

[0041] Information about the second propagation path reuses information about the first propagation path.

[0042] In an embodiment, the first sound wave propagation path among two or more different sound wave propagation paths can be, for example, the shortest path for the sound wave to propagate from the sound source location to one of the multiple listener locations.

[0043] In particular, information about the shortest propagation path can be reused, for example, to provide information about other propagation paths; for example, higher-level raster maps reuse the lowest-level raster map.

[0044] According to an embodiment, the metadata may, for example, include information about a first sound wave propagation path among two or more different sound wave propagation paths, wherein the information may, for example, include first grid data of a first grid map in one or more grid maps, wherein the first grid data depends on the first sound wave propagation path. The signal generator 130 may, for example, be configured to process the first grid data to generate one or more audio output signals.

[0045] The first propagation path can be represented, for example, by information associated with the first grid map.

[0046] In an embodiment, the first grid map may be, for example, a two-dimensional grid map or a three-dimensional grid map.

[0047] According to an embodiment, for each of at least some of the multiple coordinate locations of a first grid map, the grid data indicates a subsequent coordinate location of the first grid map such that a line from the coordinate location to the subsequent coordinate location: when the subsequent coordinate location may be, for example, different from the sound source location, indicates a portion of a first sound wave propagation path in the opposite direction to, for example, a listener location located at the coordinate location; and when the subsequent coordinate location may be, for example, equal to the sound source location, indicates a complete first sound wave propagation path in the opposite direction to, for example, a listener location located at the coordinate location.

[0048] For example, a coordinate position is a position defined by two or more coordinates, such as a coordinate system.

[0049] The coordinate position is referenced from another coordinate position, and the line (not necessarily a straight line, but preferably a straight line) represents part of the propagation path.

[0050] Here, "in the opposite direction" means that when the sound wave propagates from the sound source location to the listener location, the arrow from the coordinate position to its subsequent coordinate position points in the opposite / reverse direction of the sound wave propagation.

[0051] In an embodiment, the metadata may, for example, include information about a second sound wave propagation path among two or more different sound wave propagation paths, wherein the information may, for example, include second grid data of a second grid map in one or more grid maps, wherein the second grid data depends on the second sound wave propagation path. The information about the second sound wave propagation path may also, for example, include information about the first grid map indicating the second sound wave propagation path. The signal generator 130 may, for example, be configured to process the second grid data to generate one or more audio output signals.

[0052] The second grid map can, for example, be linked to the first grid map, reuse information from the first grid map, and map the location coordinates from the second grid map to the locations in the first grid map.

[0053] According to an embodiment, the first grid map and the second grid map are two-dimensional grid maps. Alternatively, the first grid map and the second grid map are three-dimensional grid maps.

[0054] In an embodiment, for each of at least some of the multiple coordinate locations of the second grid map, the grid data indicates a subsequent coordinate location of the second grid map such that a line from the coordinate location to the subsequent coordinate location indicates a portion of a second sound wave propagation path in the opposite direction to, for example, a listener's location at the coordinate location.

[0055] In the second grid map, the coordinate position is referenced to other coordinate positions in the second grid map (as in the first grid map).

[0056] According to an embodiment, for each of one or more additional coordinate positions among a plurality of coordinate positions of the second grid map, the grid data indicates a subsequent coordinate position of the first grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of a second sound wave propagation path in the opposite direction to, for example, a listener position located at the coordinate position, and such that the first grid map indicates one or more additional portions of the second sound wave propagation path between the subsequent coordinate position and the sound source position.

[0057] The coordinates in the second grid map are referenced to the coordinates in the first grid map (thus reusing the first grid map).

[0058] For example, the second grid map can be, for instance, a two-dimensional grid map. For the coordinate position of the second grid, subsequent coordinate positions can be specified, for example, by its x-coordinate value, y-coordinate value, and map index, which indicates whether the next coordinate belongs to the second grid map or the first grid map.

[0059] For example, (x; y; mapChange) can indicate the next coordinate position, where x indicates the x coordinate value, y indicates the y coordinate value, and mapChange indicates that the next coordinate position can be, for example, the coordinate position of the second grid map (0) or the coordinate position of the first grid map (1).

[0060] For example, mapChange can be a binary value, for instance.

[0061] For example, with respect to the first grid map, in an embodiment, the coordinate position can be specified, for example, only by (x; y), since no map changes occur with respect to the first grid map.

[0062] Regarding the third grid map, the binary value mapChange can, for example, indicate whether subsequent coordinate positions are related to the third grid map (mapChange=0) or to the second grid map (mapChange=1).

[0063] These principles also apply to 3D raster maps, which also include z-coordinates. For example, in (x; y; z; mapChange), x indicates the x-coordinate value, y indicates the y-coordinate value, z indicates the z-coordinate value, and mapChange indicates whether the subsequent coordinate position can be, for example, in the current raster map (e.g., the second raster map) or in the previous raster map (e.g., the first raster map).

[0064] In an embodiment, the signal generator 130 may be configured, for example, to generate one or more audio output signals by gradually fading out the influence of the second sound wave propagation path on the generation of one or more audio output signals, based on the distance from the current listener's location to the collision region of the first sound wave propagating along the first sound wave propagation path and the second sound wave propagating along the second sound wave propagation path, relative to the influence of the first sound wave propagation path on the generation of one or more audio output signals.

[0065] For example, the sound waves can be gradually weakened at the collision zone along different propagation paths.

[0066] According to an embodiment, the signal generator 130 may be configured, for example, to generate one or more audio output signals by gradually reducing the influence of the second sound wave propagation path on the generation of one or more audio output signals as the current listener's position gets closer to the collision area.

[0067] For example, further attenuation of non-dominant sound waves can occur, for instance, near the collision region.

[0068] In an embodiment, the metadata may, for example, include information about a third sound wave propagation path among two or more different sound wave propagation paths, wherein the information includes raster data of a third raster map in one or more raster maps, wherein the third raster data depends on the third sound wave propagation path. For each of at least some of the multiple coordinate positions of the third raster map, the raster data indicates a subsequent coordinate position of the third raster map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to, for example, a listener position located at the coordinate position. For each of one or more additional coordinate positions of the multiple coordinate positions of the third raster map, the raster data indicates a subsequent coordinate position of a second raster map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to, for example, a listener position located at the coordinate position, and such that the second raster map indicates one or more additional portions of the third sound wave propagation path. The signal generator 130 may, for example, be configured to process the third raster data to generate one or more audio output signals.

[0069] Such an embodiment describes the provision and use of information about the third sound wave propagation path (this situation can be similar to, for example, the second sound wave propagation path), but with reference to coordinates in the second grid map.

[0070] According to an embodiment, the metadata decoder 120 may be configured, for example, to decode the encoding of raster data of one or more raster maps, wherein the raster data is encoded in a compressed manner.

[0071] For example, compression of raster data can be employed. This can be important, for example, because many coordinate locations will point to the same subsequent coordinate locations (e.g., at the corners of an object, as shown in the accompanying drawings provided in this invention report).

[0072] In an embodiment, the raster data may be entropy encoded, for example, in the encoding of the raster data.

[0073] According to an embodiment, the signal generator 130 may be configured, for example, to generate one or more audio output signals by attenuating an audio object signal at one or more diffraction angles along at least one of two or more different sound wave propagation paths.

[0074] In an embodiment, the signal generator 130 may be configured, for example, to generate one or more audio output signals by attenuating the audio object signal differently for different frequency components of the audio object signal based on one or more diffraction angles along at least one of two or more different sound wave propagation paths and based on the frequency.

[0075] Such embodiments cover frequency-dependent and diffraction-dependent attenuation.

[0076] According to an embodiment, the signal generator 130 may be configured, for example, to generate one or more audio output signals based on one or more diffraction angles, such that the higher frequency components of the audio object signal are attenuated more than the lower frequency components.

[0077] In other words, higher frequency components can attenuate more than lower frequency components.

[0078] In an embodiment, the metadata decoder 120 may be configured, for example, to determine the sum of all diffraction angles for at least one of two or more different sound wave propagation paths. The signal generator 130 may be configured, for example, to generate one or more audio output signals by attenuating the audio object signal according to the sum of all diffraction angles.

[0079] According to an embodiment, at least one of two or more different sound wave propagation paths from the sound source location to one of a plurality of listener locations depends on the reflection of the sound waves originating from the sound source location at the object.

[0080] In an embodiment, the sound source location may be, for example, one of two or more sound source locations, and the audio object signal may be, for example, one of two or more audio object signals, where the audio object may be, for example, one of two or more audio objects. The audio signal decoder 110 may be configured, for example, to decode the encoding of two or more audio object signals of two or more audio objects. The metadata decoder 120 may be configured, for example, to decode metadata for each of the two or more sound source locations, the metadata including information about two or more different sound wave propagation paths from the sound source location of one of the two or more audio objects to each of a plurality of listener locations. The signal generator 130 may be configured, for example, to generate one or more audio output signals for each of the two or more sound source locations, based on the two or more audio object signals and based on the information about the two or more different sound wave propagation paths from the sound source location to the current listener location.

[0081] Figure 2 An apparatus 200 for encoding according to an embodiment is shown.

[0082] The device 200 includes an audio signal encoder 210 for encoding the audio object signal of an audio object to obtain the encoded audio object signal.

[0083] In addition, the device 200 includes a metadata encoder 220 for encoding metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location.

[0084] According to an embodiment, at least one of two or more different sound wave propagation paths from the sound source location to one of the multiple listener locations depends on the diffraction of the sound waves originating from the sound source location at the object.

[0085] In an embodiment, the first of two or more distinct sound wave propagation paths from the sound source location to one of the multiple listener locations represents the shortest path for the sound wave to propagate from the sound source location to said one of the multiple listener locations, and one or more of the other sound wave propagation paths are different from said shortest path.

[0086] According to an embodiment, information about a second sound wave propagation path in two or more different sound wave propagation paths from a sound source location to one of a plurality of listener locations reuses information about a first sound wave propagation path in two or more different sound wave segments.

[0087] In an embodiment, the first sound wave propagation path among two or more different sound wave propagation paths may be, for example, the shortest path for the sound wave to propagate from the sound source location to the one of the multiple listener locations.

[0088] According to an embodiment, the metadata may include, for example, information about a first sound wave propagation path among two or more different sound wave propagation paths, wherein the information may include, for example, first raster data of a first raster map in one or more raster maps, wherein the first raster data depends on the first sound wave propagation path.

[0089] In an embodiment, the first grid map may be, for example, a two-dimensional grid map, or it may be, for example, a three-dimensional grid map.

[0090] According to an embodiment, for each of at least some of the multiple coordinate locations of a first grid map, the grid data indicates a subsequent coordinate location of the first grid map such that a line from the coordinate location to the subsequent coordinate location indicates a portion of a first sound wave propagation path in the opposite direction to, for example, a listener location that may be located at the coordinate location, when the subsequent coordinate location may be, for example, different from the sound source location, and indicates a complete first sound wave propagation path in the opposite direction to, for example, a listener location that may be located at the coordinate location, when the subsequent coordinate location may be, for example, equal to the sound source location.

[0091] In an embodiment, the metadata may, for example, include information about a second sound wave propagation path among two or more different sound wave propagation paths, wherein the information includes second grid data of a second grid map in one or more grid maps, wherein the second grid data depends on the second sound wave propagation path. The information about the second sound wave propagation path may, for example, also include information about a first grid map indicating the second sound wave propagation path.

[0092] According to an embodiment, the first raster map and the second raster map are two-dimensional raster maps. Alternatively, the first raster map and the second raster map are three-dimensional raster maps.

[0093] In an embodiment, for each of at least some of the multiple coordinate locations of the second grid map, the grid data indicates a subsequent coordinate location of the second grid map such that a line from the coordinate location to the subsequent coordinate location indicates a portion of a second sound wave propagation path in the opposite direction to, for example, a listener's location that may be located at the coordinate location.

[0094] According to an embodiment, for each of one or more additional coordinate locations among a plurality of coordinate locations of the second grid map, the grid data indicates a subsequent coordinate location of the first grid map such that a line from the coordinate location to the subsequent coordinate location indicates a portion of a second sound wave propagation path in the opposite direction to, for example, a listener's location that may be located at the coordinate location, and such that the first grid map indicates one or more additional portions of the second sound wave propagation path between the subsequent coordinate location and the sound source location.

[0095] In an embodiment, the metadata encoder 220 may be configured, for example, to generate the first raster data of the first raster map and the second raster data of the second raster map by employing a flooding algorithm, starting with the generation of the first raster data of the first raster map.

[0096] According to an embodiment, the metadata encodes information about the collision region between the first sound wave propagating along the first sound wave propagation path and the second sound wave propagating along the second sound wave propagation path.

[0097] In an embodiment, the metadata may, for example, include information about a third sound wave propagation path among two or more different sound wave propagation paths, wherein the information includes raster data of a third raster map in one or more raster maps, wherein the third raster data depends on the third sound wave propagation path. For each of at least some of the multiple coordinate positions of the third raster map, the raster data indicates a subsequent coordinate position of the third raster map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to, for example, a listener position located at the coordinate position. For each of one or more additional coordinate positions of the multiple coordinate positions of the third raster map, the raster data indicates a subsequent coordinate position of a second raster map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to, for example, a listener position located at the coordinate position, and such that the second raster map indicates one or more additional portions of the third sound wave propagation path.

[0098] According to an embodiment, the metadata encoder 220 may be configured, for example, to generate encodings of metadata, such that the encoding of raster data of one or more raster maps may be, for example, encoded in a compressed manner.

[0099] In an embodiment, the metadata encoder 220 may be configured, for example, to generate encodings of metadata such that raster data may be entropy encoded, for example, in the encoding of raster data.

[0100] According to an embodiment, the metadata encodes information about one or more diffraction angles along at least one of two or more different sound wave propagation paths.

[0101] In an embodiment, at least one of two or more different sound wave propagation paths from the sound source location to one of the multiple listener locations depends on the reflection of the sound waves originating from the sound source location at the object.

[0102] According to an embodiment, the sound source location may be, for example, one of two or more sound source locations, wherein the audio object signal is one of two or more audio object signals, and the audio object is one of two or more audio objects. The audio signal encoder 210 may be configured, for example, to encode two or more audio object signals from two or more audio objects. The metadata encoder 220 may be configured, for example, to encode metadata for each of the two or more sound source locations, the metadata including information about two or more distinct sound wave propagation paths from the sound source location of one of the two or more audio objects to each of a plurality of listener locations.

[0103] Figure 3 A system according to an embodiment is shown.

[0104] The system includes Figure 2 The device 200 shown is used for encoding.

[0105] In addition, the system includes Figure 1 The device 100 shown is for decoding.

[0106] The audio signal encoder 210 of the encoding device 200 is configured to encode the audio object signal of the audio object to obtain the encoding of the audio object signal.

[0107] The metadata encoder 220 of the encoding device 200 is configured to encode metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location.

[0108] The audio signal decoder 110 of the decoding device 100 is configured to decode the encoding of the audio object signal.

[0109] The metadata decoder 120 of the device 100 for decoding is configured to decode the encoding of metadata.

[0110] The signal generator 130 of the decoding device 100 is configured to generate one or more audio output signals based on the audio object signal and based on information about two or more different sound wave propagation paths from the sound source location to the current listener location of multiple listener locations.

[0111] Specific embodiments are described below. Such example embodiments are equally applicable to one or more locations in a two-dimensional raster map when referring to one or more voxels.

[0112] According to an embodiment, a method is provided for calculating multiple diffraction paths for a voxel-based geometry. The audio renderer uses a multi-layered raster map that stores the next waypoint for the shortest propagation path for each voxel. Each layer of the multi-layered raster map provides additional propagation paths from discrete sound source locations to discrete listener locations.

[0113] Multi-layered raster maps generate a directed graph, with voxel coordinates as nodes. Propagation paths can then be determined by tracing this graph from the listener voxel back to the origin (the voxel containing the sound source). Each layer of the multi-layered raster map includes waypoints belonging to either the same layer or the next lower layer. For example, if two diffraction paths are to be rendered for a given listener voxel, two raster map layers are used. The shortest propagation path is determined by backtracking from the listener voxel using the lower raster map layer. The second propagation path is determined by backtracking using the higher raster map layer. The number of supported propagation paths can be increased by adding additional raster map layers and thus increasing the number of entry nodes in the backtracking graph.

[0114] Figure 4 An example of a multi-layered raster map is shown, where two raster map layers are used to determine two diffraction paths around a rectangular diffraction object for the source. It should be noted that, for better readability, only relevant map entries are shown. S indicates the source location, while L indicates the listener location.

[0115] Raster map layers can be generated by performing a modified flooding algorithm on each layer, starting with the lowest layer. The iterative algorithm uses a list of voxels updated during previous iterations. For the lowest layer, this list initially contains only source voxels. In each iteration, the neighbors of the updated voxels are checked. If the waypoint of the updated voxel is not obstructed by a line of sight to its neighbor, and the resulting backtracking path is shorter than before or does not yet exist, then the waypoint of the updated voxel is stored as the waypoint of its neighbor, and the neighbor is added to the list of updated voxels for the next iteration. The algorithm terminates when the list of updated voxels remains empty.

[0116] For the next layer, voxels are identified where “waves” from two different directions collide. This occurs when at least one neighbor exists that is different from the previous one (neither the previous one nor the current one is the start of the current one). These colliding voxels are used as the initial list of voxels for the algorithm's update, which is then executed to produce the next map layer.

[0117] Knowledge of the positions of those wave-colliding voxels can also be used to gradually reduce higher-order diffraction paths. Such artifacts can occur when the listener moves to another voxel and the backtracking path changes. A smooth transition can be achieved by gradually reducing the diffraction path at a certain distance from the wave-colliding voxel. Optionally, the diffraction path can be gradually reduced if the difference in backtracking path length between two map layers exceeds a certain threshold.

[0118] Simulations of diffraction effects typically involve distance-dependent attenuation (sound pressure is inversely proportional to the distance from the point source) and frequency-dependent attenuation depending on the diffraction angle. When using multi-layer raster maps, the total length r of the backtrack path can be used to simulate 1 / r distance attenuation. In addition, the curvature angle of the backtrack path can be used to account for frequency-dependent attenuation. This can be done by calculating the frequency-dependent attenuation at each waypoint of the backtrack path using UTD[3], or by simplification, for example by using the cumulative curvature angle. To scale the prototype filter (in dB) This prototype filter It is for reference angle Confirmed:

[0119]

[0120] Another aspect of the embodiment is the efficient transmission of pre-calculated multi-layer raster maps: the data to be transmitted is voxel coordinates (including map layers) and corresponding voxel data, such as waypoints of the map.

[0121] According to the embodiments, since many voxels have neighbors with the same waypoints, the voxel data includes a large amount of redundancy that can be reduced through inter-voxel redundancy reduction and universal codebook reduction, see [7], [8].

[0122] Furthermore, according to the embodiments, if the data is transmitted sequentially, the voxel coordinates also include a large amount of redundancy, which can be reduced by voxel coordinate prediction, see [9].

[0123] The implementation provides multipath diffraction for voxel scenes (e.g., scenes where geometry is represented by voxels rather than meshes). For voxel-based AR / VR scenes, this significantly reduces artifacts in single-path diffraction rendering.

[0124] Applications are found in auditory rendering, such as real-time and offline audio rendering of auditory scenes and environments, see [1] (also see [1a]). This includes virtual reality (VR) and augmented reality (AR) systems, such as the MPEG-I 6-DOF immersive audio renderer.

[0125] In the following text, see references Figure 5 and Figure 6 Further specific embodiments are described, wherein... Figure 4 A specific embodiment of the embodiments.

[0126] Figure 5 Shown in Figure 4 The propagation of sound waves is depicted in both lower-level and higher-level raster maps.

[0127] Specifically, Figure 5It depicts the sound waves originating from the location of the sound source / audio object.

[0128] exist Figure 5 On the left side, we can see that the sound wave originates from the sound source location, is then diffracted at the object, and propagates further around the object. Figure 5 On the left side, on the left side of the object (in) Figure 5 (shown on the left side, in the top area of ​​the figure) or on the right side of the object (in ... top area of ​​the figure) Figure 5 (Shown in the bottom area of ​​the diagram on the left). After the sound wave has passed the object, it has already passed the object on the left ( Figure 5 The sound waves from the top left area and the object that has already passed on the right ( Figure 5 Sound waves from the left-side bottom region "collide" with each other at the collision zone; Figure 5 In the diagram, it is depicted by collision line 510. (In reality, physically, what occurs is sound wave overlap.) The sound waves then propagate further to the regions from which the other sound waves originated, but this is within... Figure 5 The left side is not shown, and the first grid map / lower layer should not represent further propagation of the sound wave.

[0129] However, when referring to Figure 5 When viewed from the right, it is indeed shown that the sound waves propagate further into other corresponding areas beyond the collision line 510. Therefore, Figure 5 The bottom area on the right shows the object that has passed by on its left (in Figure 5 How the sound waves (in the top area on the left) propagate further to another area beyond the collision line 510 (entering) Figure 5 The example is in the bottom area on the right. Similarly, Figure 5 The top area on the right shows the object that has passed to its right (in Figure 5 How the sound waves (in the bottom area on the left) propagate further to another area beyond the collision line 510 (entering) Figure 5 In the top region on the right side. Again, the sound wave collides at another collision area; in Figure 5 On the right side, it is depicted by collision line 520. All of these are represented by a second raster map / higher layer.

[0130] Similarly, the propagation of sound waves can be further described using a third grid map / second higher layer that describes the propagation of sound waves beyond the second collision line 520. Likewise, sound waves at even more layers can be described, for example, using a fourth grid map (third higher layer), a fifth grid map (fourth higher layer), a sixth grid map (fifth higher layer), etc.

[0131] Figure 6 It shows the target of being made by Figure 4 and Figure 5 The example depicts raster data for the first raster map (lower-level raster map) and the second raster map (higher-level raster map).

[0132] Specifically, Figure 6 The left side shows how sound wave propagation is fully represented for a lower layer / first grid map, for example, sound waves at the first collision line ( Figure 5 The propagation of sound waves before the collision at point 510 (in the image). Although Figure 6 The diagram shows raster data for a two-dimensional location, but in other embodiments, the method is extended to a three-dimensional location, for example, by adding a z-coordinate value.

[0133] Raster data implements a backtracking method. This means that for Figure 6 The grid map on the left side considers the coordinate position of each (useful, e.g., the location where sound waves can propagate), indicating the subsequent coordinate position, such that at least a portion of the propagation path of the sound wave from the sound source location to the considered position is described by a line between the considered coordinate position and the subsequent coordinate position, and such that the arrow from the considered coordinate position to the subsequent coordinate position points in the opposite / reverse direction of the direction of sound wave propagation.

[0134] like Figure 6 As can be seen on the left, for each (useful) coordinate position, a subsequent coordinate position is indicated to describe the propagation path in the opposite / reverse direction, so that finally, when following the indicated subsequent coordinate position, the sound source position is reached.

[0135] For example, in Figure 6 On the left side, all coordinate positions (x-coordinate values ​​between 1 and 3) in the first to third columns directly indicate coordinate positions (2; 4), that is, the coordinate positions of the audio object / sound source that will serve as the subsequent coordinate positions.

[0136] Figure 6 Other coordinate positions on the left (from which the sound source position is occluded by the object) reference another coordinate position to describe only a portion of the propagation path. For example, coordinate positions (6;2), (6;1), and other coordinate positions indicate coordinate position (3;2) as a subsequent coordinate position to describe a portion of the propagation path from the sound source to the corresponding position. However, then in coordinate position (3;2), coordinate position (3;2) again references the sound source position (2;4) as a subsequent coordinate position.

[0137] As an example, consider coordinate position (11; 3). The subsequent coordinate position is (8; 2). Coordinate position (8; 2) indicates coordinate position (3; 2) as the subsequent coordinate position. Then, coordinate position (3; 2) indicates coordinate position (2; 4), which is the sound source position as the subsequent coordinate position. Therefore, in this example, the propagation path of the sound wave starting from (2; 4) towards (11; 3) thus starts from (2; 4), then extends along (3; 2), (8; 2), and finally reaches coordinate position (11; 3).

[0138] For example, to indicate that a coordinate position (here (2; 4)) is indeed the location of a sound source, the coordinate position can, for example, use itself as a reference for subsequent coordinate positions. This allows the detection algorithm to determine that the position is indeed the location of a sound source. (For example, the coordinate position (2; 4) can, for example, indicate (2; 4) as a subsequent coordinate position to indicate that it itself is the location of a sound source.) In other embodiments, other concepts can be used, for example, to identify that a subsequent position is the location of a sound source, such as specifying how many steps it takes for the coordinate position to reach the sound source location, or, for example, using bits to indicate whether the coordinate position is the location of a sound source, etc.

[0139] Sound waves in the (first) collision region (here) Figure 5 The propagation outside the collision line 510 is caused by Figure 6 As shown on the right. This second raster map includes a third value for its coordinate position to indicate subsequent coordinate positions. (See diagram on the right.) Figure 6 On the left side of the graph, the first and second coordinate values ​​describe the subsequent coordinate positions through their x-coordinate and y-coordinate values.

[0140] To identify subsequent coordinate positions in the second raster map (higher layer), a third value is indicated. This third value indicates whether the subsequent coordinate position lies within the second raster map / higher layer raster map (in...). Figure 6 The right side of the map, indicated by a third value of 0), or whether the subsequent coordinates are located within the first raster map / lower-level raster map (in...). Figure 6 (This is indicated by a third value of 1 on the right side of the map). Therefore, such a third value can be implemented, for example, by a binary value that can be interpreted as "a change to the previous raster map"; see "mapChange" above. Other concepts for implementing the third value are also possible, such as an identifier that indicates the raster map in which the subsequent coordinate position is located.

[0141] For example, in Figure 6 In the higher-level raster map on the right, the coordinates (3; 8) can be considered:

[0142] As subsequent coordinate positions Figure 6The coordinates (3; 8) on the right indicate (9; 7; 0). That is, the subsequent coordinates are... Figure 6 The same coordinates (9; 7) in the same (higher layer / second) raster map on the right, because the third value is zero.

[0143] As subsequent coordinate positions Figure 6 The coordinate position (9; 7) on the right indicates (9; 2; 1). That is, the subsequent coordinate position is... Figure 6 The coordinates (9; 2) in the previous (lower layer / first) raster map on the left, because the third value is 1.

[0144] As subsequent coordinate positions Figure 6 The coordinates (9; 2) in the lower layer / first grid map on the left indicate (3; 2). That is, the subsequent coordinates are the coordinates (3; 2) in the lower layer / first grid map, such as... Figure 6 As shown on the left.

[0145] As subsequent coordinate positions Figure 6 The coordinates (3; 2) on the left indicate (2; 4), which is the location of the sound source.

[0146] Therefore, the found reverse / backpropagation path is defined by the coordinate positions: (3;8), (9;7), (9;2), (3;2), (2;4).

[0147] In the correct order, starting from the sound source location and extending toward the considered location, the second propagation path from the sound source location toward the coordinate position (3;8) is defined as: (2;4), (3;2), (9;7), (3;8).

[0148] Therefore, for two raster maps, two sound wave propagation paths are defined from the sound source location to a specific coordinate location. The first propagation path is defined only by the first raster map; the second propagation path is defined by both the second raster map and the first raster map. Figure 1 Limited to starting point.

[0149] For example, for coordinate position (3; 8), the first propagation path is defined as a direct propagation path from (2; 4) to (3; 8), as the first raster map indicates coordinate position (2; 8) as the subsequent coordinate position.

[0150] This concept also applies to other higher-level / raster maps, such as third-level, fourth-level, fifth-level, etc.

[0151] Each additional layer / raster map defines another propagation path for each coordinate location, which in turn defines subsequent coordinate locations for that location.

[0152] For example, for a third raster map, the third propagation path can be defined, for instance, by the third raster map together with the second raster map and the first raster map. In such an example, some coordinate positions of the third raster map indicate coordinate positions of the second raster map, and some coordinate positions of the second raster map indicate coordinate positions of the first raster map.

[0153] from Figure 6 As can be seen in the examples, many coordinate positions include the same subsequent coordinate positions. This allows for efficient encoding of subsequent coordinate positions. For example, entropy coding can be used.

[0154] Although some aspects have been described in the context of the apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of method steps also represent a description of a corresponding block or item or feature of the corresponding apparatus. Some or all of the method steps can be performed by (or using) hardware devices, such as microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important method steps can be performed by such devices.

[0155] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware or at least partially in software. The implementation may be performed using a digital storage medium (e.g., floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) on which electronically readable control signals are stored, which cooperate (or are capable of cooperating with) a programmable computer system to perform the corresponding methods. Therefore, the digital storage medium may be computer-readable.

[0156] Some embodiments of the invention include a data carrier having electronically readable control signals that are capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0157] Typically, embodiments of the present invention can be implemented as a computer program product having program code that, when run on a computer, is operable to perform one of these methods. The program code may, for example, be stored on a machine-readable medium.

[0158] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0159] In other words, therefore, one embodiment of the method of the present invention is a computer program having program code that, when the computer program is run on a computer, performs one of the methods described herein.

[0160] Therefore, another embodiment of the method of the present invention is a data carrier (or digital storage medium, or computer-readable medium) including a computer program recorded thereon for performing one of the methods described herein. Data carriers, digital storage media, or recording media are generally tangible and / or non-transitory.

[0161] Therefore, another embodiment of the method of the present invention represents a data stream or signal sequence for performing one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection, such as via the Internet.

[0162] Another embodiment includes a processing device, such as a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

[0163] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0164] Another embodiment of the invention includes an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. For example, the receiver may be a computer, mobile device, storage device, etc. The apparatus or system may, for example, include a file server for transmitting the computer program to the receiver.

[0165] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0166] The apparatus described herein can be implemented using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0167] The methods described herein can be performed using hardware devices, computers, or a combination of hardware devices and computers.

[0168] The above embodiments are merely illustrative of the principles of the invention. It should be understood that modifications and variations of the arrangements and details described herein will be readily apparent to those skilled in the art. Therefore, the intent is to be limited only by the scope of the pending claims and not by the specific details presented in the description and explanation of the embodiments herein.

[0169] References:

[0170] [1] ISO / IEC JT C1 / SC29 / WG6 N0211, “WD5 of ISO_IEC 23090-4, MPEG-IImmersive Audio”, October 2023, Hanover, Germany.

[0171] [1a] ISO / IEC JT C1 / SC29 / WG6 M61258 “Third version of Text of WorkingDraft of RM0”, 8th WG6 meeting, October 2022.

[0172] [2] Allen, JB and Berkley, DA (1979), Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America, 65(4), 943-950.

[0173] [3] RG Kouyoumjian and PH Pathak, “A uniform geometrical theory of diffraction for an edge in a perfectly conducting surface”, IEEE Proceedings, Vol. 62, No. 11, pp. 1448-1461, November 1974, doi: 10.1109 / PROC.1974.9651.

[0174] [4] Dijkstad, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik, 1(1), 269-271.

[0175] [5] PE Hart, NJ Nilsson and B. Raphael, “A Formal Basis for the Heuristic Determination of Minimum Cost Paths”, IEEE Transactions on Systems Science and Cybernetics, Vol. 4, No. 2, pp. 100-107, July 1968, doi:10.1109 / TSSC.1968.300136.

[0176] [6] Harabor, D. and Grastien, A. (August 2011). Online graph pruning for path finding on grid maps. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 25, No. 1, pp. 1114-1119).

[0177] [7] International patent application PCT / EP2022 / 069522, “Encoding of pre-computed data for rendering early reflections in AR / VR systems”.

[0178] [8] International patent application PCT / EP2022 / 069523, “Encoding of AR / VR metadata with generic codebooks”.

[0179] [9] European Patent Application, Application No. EP 22216666.2, “Apparatus and Method for Predicting Voxel Coordinates for AR / VR Systems”.

Claims

1. An apparatus (100) for decoding, wherein the apparatus (100) comprises: An audio signal decoder (110) is used to decode the encoded audio object signal of an audio object. Metadata decoder (120) is used to decode the encoding of metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location; as well as A signal generator (130) is configured to generate one or more audio output signals based on the audio object signal and information about two or more different sound wave propagation paths from the sound source location to the current listener location among a plurality of listener locations.

2. The apparatus (100) according to claim 1. At least one of the two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations depends on the diffraction of the sound wave originating from the sound source location at the object.

3. The apparatus (100) according to claim 1 or 2. The first of two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations represents the shortest path for the sound wave to propagate from the sound source location to the one of the plurality of listener locations, and one or more of the other sound wave propagation paths are different from the shortest path.

4. The apparatus (100) according to any one of the preceding claims. Information regarding the second sound wave propagation path among two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations reuses information regarding the first sound wave propagation path among the two or more different sound wave components.

5. According to claim 4, and further dependent on the apparatus (100) of claim 3. The first sound wave propagation path among the two or more different sound wave propagation paths is the shortest path for the sound wave to propagate from the sound source location to one of the plurality of listener locations.

6. The apparatus (100) according to any one of the preceding claims. The metadata includes information about a first sound wave propagation path among the two or more different sound wave propagation paths, wherein the information includes first grid data of a first grid map in one or more grid maps, wherein the first grid data depends on the first sound wave propagation path. The signal generator (130) is configured to process the first raster data to generate the one or more audio output signals.

7. The apparatus (100) according to claim 6. The first grid map is either a two-dimensional grid map or a three-dimensional grid map.

8. The apparatus (100) according to claim 6 or 7. For each of at least some of the multiple coordinate positions in the first raster map, the raster data indicates a subsequent coordinate position in the first raster map such that the line from the first coordinate position to the subsequent coordinate position is: When the subsequent coordinate position differs from the sound source position, it indicates a portion of the first sound wave propagation path in the opposite direction to the listener's position at the coordinate position. When the subsequent coordinate position is equal to the sound source position, it indicates a complete first sound wave propagation path in the opposite direction to the listener's position at the coordinate position.

9. The apparatus (100) according to any one of claims 6 to 8. The metadata includes information about a second sound wave propagation path among the two or more different sound wave propagation paths, wherein the information includes second raster data of a second raster map in one or more raster maps, wherein the second raster data depends on the second sound wave propagation path. The information regarding the second sound wave propagation path also includes information about the first grid map indicating the second sound wave propagation path. The signal generator (130) is configured to process the second raster data to generate the one or more audio output signals.

10. The apparatus (100) according to claim 9. Wherein the first raster map and the second raster map are two-dimensional raster maps; or The first and second raster maps are three-dimensional raster maps.

11. The apparatus (100) according to claim 9 or 10. For each of at least some of the multiple coordinate locations of the second grid map, the grid data indicates a subsequent coordinate location of the second grid map such that a line from the coordinate location to the subsequent coordinate location indicates a portion of a second sound wave propagation path in the opposite direction to the listener's location at the coordinate location.

12. The apparatus (100) according to any one of claims 9 to 11. For each of one or more additional coordinate positions among a plurality of coordinate positions in the second grid map, the grid data indicates a subsequent coordinate position in the first grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of a second sound wave propagation path in the opposite direction to the listener's position at the coordinate position, and such that the first grid map indicates one or more additional portions of the second sound wave propagation path between the subsequent coordinate position and the sound source position.

13. The apparatus (100) according to any one of claims 9 to 12. The signal generator (130) is configured to generate the one or more audio output signals by gradually reducing the influence of the second sound wave propagation path on the generation of the one or more audio output signals, based on the distance from the current listener's position to the collision region of the first sound wave propagating along the first sound wave propagation path and the second sound wave propagating along the second sound wave propagation path, compared to the influence of the first sound wave propagation path on the generation of the one or more audio output signals.

14. The apparatus (100) according to claim 13. The signal generator (130) is configured to generate the one or more audio output signals by gradually reducing the influence of the second sound wave propagation path on the generation of the one or more audio output signals as the current listener's position gets closer to the collision area.

15. The apparatus (100) according to any one of claims 6 to 14. The metadata includes information about a third sound wave propagation path among the two or more different sound wave propagation paths, wherein the information includes raster data of a third raster map in one or more raster maps, wherein the third raster data depends on the third sound wave propagation path. For each of at least some of the multiple coordinate positions in the third grid map, the grid data indicates a subsequent coordinate position in the third grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to the listener's position at the coordinate position. For each of one or more additional coordinate positions among the plurality of coordinate positions in the third grid map, the grid data indicates a subsequent coordinate position in the second grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to the listener's position at the coordinate position, and such that the second grid map indicates one or more additional portions of the third sound wave propagation path. The signal generator (130) is configured to process the third grid data to generate the one or more audio output signals.

16. The apparatus (100) according to any one of claims 6 to 15. The metadata decoder (120) is configured to decode the encoding of raster data of the one or more raster maps, wherein the encoding of the raster data is performed in a compressed manner.

17. The apparatus (100) according to claim 16. The raster data therein is entropy encoded in the encoding of the raster data.

18. The apparatus (100) according to any one of the preceding claims. The signal generator (130) is configured to generate the one or more audio output signals by attenuating the audio object signal according to one or more diffraction angles along at least one of the two or more different sound wave propagation paths.

19. The apparatus (100) according to claim 18. The signal generator (130) is configured to generate the one or more audio output signals by attenuating the audio object signal differently for different frequency components of the audio object signal according to the one or more diffraction angles along at least one of the two or more different sound wave propagation paths and according to the frequency.

20. The apparatus (100) according to claim 19. The signal generator (130) is configured to generate the one or more audio output signals based on the one or more diffraction angles, such that the higher frequency components of the audio object signal are attenuated more than the lower frequency components.

21. The apparatus (100) according to any one of claims 18 to 20. The metadata decoder (120) is configured to determine the sum of all diffraction angles for at least one of the two or more different acoustic propagation paths, and The signal generator (130) is configured to generate the one or more audio output signals by attenuating the audio object signal according to the sum of all diffraction angles.

22. The apparatus (100) according to any one of the preceding claims. At least one of the two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations depends on the reflection of the sound wave originating from the sound source location at the object.

23. The apparatus (100) according to any one of the preceding claims. The sound source location is one of two or more sound source locations, the audio object signal is one of two or more audio object signals, and the audio object is one of two or more audio objects. The audio signal decoder (110) is configured to decode the encoding of the signals of the two or more audio objects. The metadata decoder (120) is configured to decode the metadata for each of the two or more sound source locations, the metadata including information about two or more different sound wave propagation paths from the sound source location of one of the two or more audio objects to each of the multiple listener locations; The signal generator (130) is configured to generate one or more audio output signals for each of the two or more sound source locations, based on the two or more audio object signals and information about two or more different sound wave propagation paths from the sound source location to the current listener location.

24. An apparatus (200) for encoding, wherein the apparatus (200) comprises: An audio signal encoder (210) is used to encode the audio object signal of an audio object to obtain the encoded audio object signal. A metadata encoder (220) is used to encode metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location.

25. The apparatus (200) according to claim 24. At least one of the two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations depends on the diffraction of the sound wave originating from the sound source location at the object.

26. The apparatus (200) according to claim 24 or 25. The first of two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations represents the shortest path for the sound wave to propagate from the sound source location to the one of the plurality of listener locations, and one or more of the other sound wave propagation paths are different from the shortest path.

27. The apparatus (200) according to any one of claims 24 to 26. Information regarding the second sound wave propagation path among two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations reuses information regarding the first sound wave propagation path among the two or more different sound wave components.

28. The apparatus (200) according to claim 27, and also according to claim 26. The first sound wave propagation path among the two or more different sound wave propagation paths is the shortest path for the sound wave to propagate from the sound source location to one of the plurality of listener locations.

29. The apparatus (200) according to any one of claims 24 to 28. The metadata includes information about a first sound wave propagation path among the two or more different sound wave propagation paths, wherein the information includes first grid data of a first grid map in one or more grid maps, wherein the first grid data depends on the first sound wave propagation path.

30. The apparatus (200) according to claim 29. The first grid map is either a two-dimensional grid map or a three-dimensional grid map.

31. The apparatus (200) according to claim 29 or 30. For each of at least some of the multiple coordinate positions in the first raster map, the raster data indicates a subsequent coordinate position in the first raster map such that the line from the first coordinate position to the subsequent coordinate position is: When the subsequent coordinate position differs from the sound source position, it indicates a portion of the first sound wave propagation path in the opposite direction to the listener's position at the coordinate position. When the subsequent coordinate position is equal to the sound source position, it indicates a complete first sound wave propagation path in the opposite direction to the listener's position at the coordinate position.

32. The apparatus (200) according to any one of claims 29 to 31. The metadata includes information about a second sound wave propagation path among the two or more different sound wave propagation paths, wherein the information includes second raster data of a second raster map in one or more raster maps, wherein the second raster data depends on the second sound wave propagation path. The information regarding the second sound wave propagation path also includes information about the first grid map indicating the second sound wave propagation path.

33. The apparatus (200) according to claim 32. Wherein the first raster map and the second raster map are two-dimensional raster maps; or The first and second raster maps are three-dimensional raster maps.

34. The apparatus (200) according to claim 32 or 33. For each of at least some of the multiple coordinate locations of the second grid map, the grid data indicates a subsequent coordinate location of the second grid map such that a line from the coordinate location to the subsequent coordinate location indicates a portion of a second sound wave propagation path in the opposite direction to the listener's location at the coordinate location.

35. The apparatus (200) according to any one of claims 32 to 34. For each of one or more additional coordinate positions among a plurality of coordinate positions in the second grid map, the grid data indicates a subsequent coordinate position in the first grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of a second sound wave propagation path in the opposite direction to the listener's position at the coordinate position, and such that the first grid map indicates one or more additional portions of the second sound wave propagation path between the subsequent coordinate position and the sound source position.

36. The apparatus (200) according to any one of claims 32 to 35. The metadata encoder (220) is configured to generate the first raster data of the first raster map and the second raster data of the second raster map by employing a flooding algorithm, starting with the first raster data of the first raster map.

37. The apparatus (200) according to any one of claims 32 to 36. The metadata encodes information about the collision region between the first sound wave propagating along the first sound wave propagation path and the second sound wave propagating along the second sound wave propagation path.

38. The apparatus (200) according to any one of claims 29 to 37. The metadata includes information about a third sound wave propagation path among the two or more different sound wave propagation paths, wherein the information includes raster data of a third raster map in one or more raster maps, wherein the third raster data depends on the third sound wave propagation path. For each of at least some of the multiple coordinate positions in the third grid map, the grid data indicates a subsequent coordinate position in the third grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to the listener's position at the coordinate position. For each of one or more additional coordinate positions among the plurality of coordinate positions of the third grid map, the grid data indicates a subsequent coordinate position of the second grid map such that a line from the coordinate position to the subsequent coordinate position indicates a portion of the third sound wave propagation path in the opposite direction to the listener's position at the coordinate position, and such that the second grid map indicates one or more additional portions of the third sound wave propagation path.

39. The apparatus (200) according to any one of claims 29 to 38. The metadata encoder (220) is configured to generate the encoding of the metadata such that the encoding of the raster data of the one or more raster maps is in a compressed manner.

40. The apparatus (200) according to claim 39. The metadata encoder (220) is configured to generate the encoding of the metadata such that the raster data is entropy encoded in the encoding of the raster data.

41. The apparatus (200) according to any one of claims 24 to 40. The metadata therein encodes information about one or more diffraction angles along at least one of the two or more different sound wave propagation paths.

42. The apparatus (200) according to any one of claims 24 to 41. At least one of the two or more different sound wave propagation paths from the sound source location to one of the plurality of listener locations depends on the reflection of the sound wave originating from the sound source location at the object.

43. The apparatus (200) according to any one of claims 24 to 42. The sound source location is one of two or more sound source locations, the audio object signal is one of two or more audio object signals, and the audio object is one of two or more audio objects. The audio signal encoder (210) is configured to encode the signals of the two or more audio objects. The metadata encoder (220) is configured to encode the metadata for each of the two or more sound source locations, the metadata including information about two or more different sound wave propagation paths from the sound source location of one of the two or more audio objects to each of the multiple listener locations.

44. A system comprising: The encoding apparatus (200) according to any one of claims 24 to 43, and The device (100) for decoding according to any one of claims 1 to 23. The audio signal encoder (210) of the encoding device (200) is configured to encode the audio object signal of the audio object to obtain the encoding of the audio object signal. The metadata encoder (220) of the device (200) for encoding is configured to encode metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location. The audio signal decoder (110) of the device (100) for decoding is configured to decode the encoding of the audio object signal. The metadata decoder (120) of the means (100) for decoding is configured to decode the encoding of the metadata, and The signal generator (130) of the device (100) for decoding is configured to generate one or more audio output signals based on the audio object signal and information about two or more different sound wave propagation paths from the sound source location to the current listener location among the plurality of listener locations.

45. A method for decoding, wherein the method comprises: Decode the encoded audio signal of an audio object. Decoding the encoded metadata, wherein, for each of a plurality of listener locations, the metadata includes information about two or more distinct sound wave propagation paths from the sound source location of the audio object to the listener location; and One or more audio output signals are generated based on the audio object signal and information about two or more different sound wave propagation paths for the current listener position among the plurality of listener positions.

46. ​​A method for encoding, wherein the method comprises: Encode the audio object signal of the audio object to obtain the encoded audio object signal, and The metadata is encoded, wherein, for each of a plurality of listener locations, the metadata includes information about two or more different sound wave propagation paths from the sound source location of the audio object to the listener location.

47. A computer program, when executed on a computer or signal processor, for implementing the method according to claim 45 or 46.

Images