Methods and apparatus for processing voxel-based scene representations

Abstract

The present disclosure relates to a method for processing a voxel-based scene representation of an audio scene, where the voxel-based scene representation comprises a voxel matrix, and where entries of the voxel matrix are associated with respective voxels in the audio scene and valued from a set of values associated with respective material identifiers of the voxels. The method comprises, for a given value of the set of values: generating a binary matrix, an entry of the binary matrix indicating whether a corresponding entry in the voxel matrix has the given value, selecting an entry of the binary matrix indicating that the corresponding entry in the voxel matrix has the given value, and generating a set of values, the method further includes initializing a set of entries of the binary matrix with the selected entries, iteratively selecting a dimension from previously unselected dimensions of the binary matrix, and updating the set of entries based on the selected dimension, generating a representation of the resulting set of entries for encoding or rendering. The method further comprises, in each iteration: determining a respective maximum range along each previously unselected dimension with respect to a given entry in the binary matrix such that, for each entry in the set of entries, the maximum range is greater than the maximum range along each previously unselected dimension. All entries within a maximum range with respect to the respective entries indicate that the corresponding entries in the voxel matrix have the given value, selecting, among the previously unselected dimensions, a dimension that results in the largest maximum range, and updating the set of entries by adding, for each entry in the set of entries, all entries of the binary matrix within a maximum range of the selected dimension relative to the respective entry. The invention further relates to a corresponding device, a computer program and a computer readable storage medium.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 582,620, filed September 14, 2023, and European Application No. 23207674.5, filed November 3, 2023, the entire contents of each of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to techniques for processing voxel-based scene representations of audio scenes, such as for storage, transmission, and / or audio rendering. Specifically, this disclosure relates to computationally efficient real-time conversion between voxel-based scene representations of audio scenes. Background Technology

[0004] The Moving Picture Experts Group (MPEG) is a working group alliance jointly established by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) to set standards for media decoding, including audio decoding. MPEG is organized under ISO / IEC JTC1 / SC 29, and the audio group is currently referred to as Working Group (WG) 6. WG 6 is currently developing a new audio standard (also known as MPEG-I Immersive Audio, ISO / IEC 23090-4).

[0005] The new MPEG-I standard enables acoustic experiences from different viewpoints and / or perspectives or listening positions by supporting various motions in and around scenes, such as the use of various degrees of freedom (e.g., three-DOF or six-DOF) motion in virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or extended reality (XR) applications. 6DoF interaction extends the 3DoF spherical video / audio experience, which is limited to head rotation (pitch, yaw, and roll), to include translational motion (forward / backward, up / down, and left / right), thus allowing navigation within virtual environments (e.g., actually walking around a room) in addition to head rotation.

[0006] For audio rendering in VR, AR, MR, and XR applications, an object-based approach has been widely adopted, representing complex auditory scenes as multiple individual audio objects, each associated with parameters or metadata defining the object's position / orientation and trajectory within the scene. Alternatively, higher-order ambisonics (HOA) is also used for audio rendering in such environments. However, new uses for "voxels" in rendering audio scenes are now being explored, such as for novel immersive audio experiences. Voxels used for audio rendering are associated with media environments implemented in hardware and software, such as video games and / or VR, AR, MR, and XR environments.

[0007] A voxel is a spatial volume with audio rendering instructions or acoustic properties assigned to it. Voxel size can be an encoder configuration parameter and can be selected (manually or automatically) based on the level of detail in the scene geometry (e.g., in the range of 10cm-1m).

[0008] Voxels used for audio rendering can be obtained in the following ways:

[0009] ● Voxelization (or transformation) of mesh-based scene representation

[0010] ● Based on the scene representation used for scene generation (or even video rendering) (e.g., by upsampling smaller voxels).

[0011] Voxel-based scene representations are typically given in the form of a two-dimensional or three-dimensional voxel matrix. Each entry in the voxel matrix is ​​uniquely associated with a corresponding voxel. The values ​​of the voxel matrix entries represent the material identifier of each voxel. In the simplest case, voxel matrix entries can take values ​​from {0, 1}, where 0 represents a sound propagation medium (e.g., an air voxel) and 1 represents an obstruction medium in which sound cannot propagate. In other implementations, voxel matrix entries can take values ​​from a predefined set of (integer) values, where each value represents a corresponding material identifier (material type identifier), such as air, wood, concrete, etc.

[0012] On the other hand, the renderer interface defined in MPEG-I immersive audio cannot process voxel matrices, but instead relies on inputs of a predefined compressed representation of the audio scene based on voxels, as defined in, for example, patent application PCT / EP2023 / 055380 entitled "Methods, Apparatus, and Systems for Processing Audio Scenes for Audio Rendering" and disclosed as WO2023 / 169934. To date, the conversion techniques from voxel matrices to a compressed representation compatible with the MPEG-I renderer interface are computationally expensive and do not allow for real-time conversion of audio scene information on conventional encoder or decoder devices.

[0013] Therefore, there is a need for improved techniques for converting (e.g., compressing) voxel-based audio scene information. In particular, there is a need for techniques that can operate in real time, even on devices with limited processing power. Summary of the Invention

[0014] In view of this need, this disclosure provides a method for processing audio scenes based on voxel-based scene representation, an apparatus for processing audio scenes based on voxel-based scene representation, a computer program, and a computer-readable storage medium, which have the features of their respective independent claims.

[0015] One aspect of this disclosure relates to a method for processing a voxel-based scene representation of an audio scene. The voxel-based scene representation may include a voxel matrix. Entries in the voxel matrix may be (uniquely) associated with corresponding voxels in the audio scene and may be taken from a set of values ​​associated with a corresponding material identifier (material type identifier) ​​of the voxel. The method may include, for a given value in the set of values ​​(i.e., for a given material identifier), generating a binary matrix having entries indicating whether a corresponding entry in the voxel matrix has a given value. The method may also include, for a given value in the set of values, selecting entries in the binary matrix that indicate whether a corresponding entry in the voxel matrix has a given value. The method may further include, for a given value in the set of values, initializing a set of entries in the binary matrix with the selected entries. Equivalently, the set of voxels may be initialized with voxels associated with the selected entries. For example, the set of entries (or equivalently, voxels) may be initialized to consist of the selected entries (or equivalently, voxels associated with the selected entries). The method may further include, for a given value in the set of values, iteratively selecting a dimension (dimension direction) from previously unselected dimensions (dimension directions) of the binary matrix, and updating the set of entries (or equivalently, voxels) based on the selected dimension (dimension direction). The method may further include, for a given value in the set of values, generating a representation of the resulting set of entries (or equivalently, voxels) for encoding or rendering. The method may further include, in each iteration, determining the corresponding maximum range along each previously unselected dimension (dimension direction) relative to a given entry (or equivalently, voxel) of the binary matrix, such that for each entry (or equivalently, voxel) in the set of entries, all entries (or equivalently, voxels) within the maximum range relative to the corresponding entry indicate that the corresponding entry in the voxel matrix has a given value. The method may further include, in each iteration, selecting the dimension (dimension direction) among the previously unselected dimensions that produces the maximum range. The method may further include updating the set of entries (or equivalently, voxels) at each iteration by adding all entries of a binary matrix within the maximum range of the selected dimension relative to the corresponding entry to each entry in the set of entries.

[0016] Therefore, matrix-based representations (e.g., two-dimensional diffraction patterns or three-dimensional voxel matrices) can be converted into compressed formats, such as those compatible with the MPEG-I immersive audio renderer interface. Furthermore, this conversion can be performed in real-time for current audio encoding and rendering devices, enabling real-time updates of metadata describing the audio scene at the audio renderer.

[0017] In some embodiments, the representation of the resulting set of entries (or equivalently, voxels) can indicate the start entry, the end entry, and the material identifier of the entries in the set. For example, the voxel associated with the start entry (i.e., the start voxel) can be a corner voxel of a cube formed by the set of voxels associated with the entries in the set, and the voxel associated with the end entry (i.e., the end voxel) can be a diagonal voxel of the cube. Specifically, the start entry can be the entry in the set of entries that has the minimum coordinate value in each dimension (direction) of the binary matrix, and the end entry can be the entry in the set of entries that has the maximum coordinate value in each dimension (direction) of the binary matrix. Thus, a simple and compressed efficient representation of the cube voxel set can be achieved.

[0018] In some embodiments, the method may further include outputting the generated representation for encoding. For example, the generated representation may be output to a bitstream. The bitstream may be in a format suitable for processing by a rendering device, such as a rendering device compatible with the MPEG-I immersive audio standard.

[0019] In some embodiments, entries in a binary matrix may be selected as first entries along a predetermined scan order, indicating that the corresponding entry in the voxel matrix has a given value. The predetermined scan order may, for example, correspond to a grid scan order along the x, y, and z directions in sequence.

[0020] In some embodiments, the method may further include setting entries in the binary matrix included in the set of entries for the given value in the set of values ​​to values ​​indicating that the corresponding entry in the voxel matrix does not have the given value. In other words, entries already included in the determined set of entries can be removed from the binary matrix (e.g., in the sense that they are empty), thereby removing them from the voxel-based scene representation. Therefore, the resulting set of entries generated by the proposed method is non-overlapping, such that the reconstructed audio scene at, for example, the rendering device is independent of the order in which the representations of the entry sets are processed.

[0021] In some embodiments, the method may further include selecting another entry in the indicator voxel matrix of the binary matrix that has a given value.

[0022] In some embodiments, entries of a binary matrix may be selected for iterative processing until all entries of the binary matrix indicating that corresponding entries in the voxel matrix have given values ​​have been assigned to the voxel set. That is, successive selection of entries in the binary matrix used for iterative processing may be repeated until the binary matrix no longer includes entries indicating that corresponding entries in the voxel matrix have given values.

[0023] In some embodiments, the method may further include outputting a set of representations of the set of (multiple) entries for encoding or rendering. Here, it should be understood that this set of representations (of the (multiple) entry sets) may allow for complete (i.e., lossless) reconstruction of binary matrices (and, through extension, reconstruction of voxel-based audio scene representations). In some implementations, this set may be an ordered set. The reconstructed scene representation may not depend on a specific order because the set of entries is non-overlapping.

[0024] In some embodiments, the method may further include processing all values ​​in the group of values ​​according to a fixed processing order. For example, the values ​​in the group of values ​​may be processed in ascending numerical order, excluding 0.

[0025] In some embodiments, the method can be executed at an audio rendering device, and the output of the method can be used as input to the audio rendering process. In some implementations, the audio rendering process can be executed on the same rendering device. In other implementations, the audio rendering process can be executed at another rendering device after the output of the method is transmitted.

[0026] In some embodiments, the voxel matrix can be a two-dimensional matrix. In some embodiments, the voxel matrix can be associated with an audio diffraction pattern. For example, the encoded output can be provided to another audio renderer for audio rendering. The audio diffraction pattern may have already been generated based on a three-dimensional voxel matrix, for example, by projecting it onto a given plane or by intersecting with a given plane.

[0027] In some embodiments, the voxel matrix may be a three-dimensional matrix. In some embodiments, the voxel matrix may relate to a three-dimensional representation of an audio scene.

[0028] According to another aspect, an apparatus for processing audio scene information is provided. The apparatus may include a processor and a memory coupled to the processor and storing instructions for the processor. The processor may be configured to perform methods or method steps as outlined throughout this disclosure.

[0029] According to another aspect, an audio rendering apparatus is provided. The apparatus may include a processor and a memory coupled to the processor and storing instructions for the processor. The processor may be configured to perform methods or method steps as outlined in this disclosure.

[0030] According to another aspect, an audio encoding apparatus is provided. The apparatus may include a processor and a memory coupled to the processor and storing instructions for the processor. The processor may be configured to perform methods or method steps outlined throughout this disclosure.

[0031] According to another aspect, a computer program is described. The computer program may include executable instructions that, when executed by a computing device (e.g., a processor), are used to perform the methods or method steps outlined throughout this disclosure.

[0032] According to another aspect, a computer-readable storage medium is described. This storage medium can store a computer program adapted to execute on a computing device (e.g., a processor), and when executed on a computing device, to perform the methods or method steps outlined throughout this disclosure.

[0033] It should be noted that the methods and apparatuses, including those with preferred embodiments outlined in this disclosure, can be used alone or in combination with other methods and apparatuses disclosed in this document. Furthermore, all aspects of the methods and apparatuses outlined in this disclosure can be combined arbitrarily. In particular, the features of the claims can be combined with each other in any manner.

[0034] It should be understood that apparatus features and method steps can be interchanged in many ways. In particular, as those skilled in the art will understand, the details of the disclosed methods can be implemented by corresponding apparatuses, and vice versa. Furthermore, any of the foregoing statements regarding methods (and, for example, their steps) can be understood to apply equally to the corresponding apparatuses (and, for example, their blocks, stages, units), and vice versa. Attached Figure Description

[0035] The invention will now be explained by way of example with reference to the accompanying drawings, wherein...

[0036] Figure 1 An example of an interface between a voxel-based scene geometry engine and an audio renderer to which the technology according to embodiments of this disclosure can be applied is illustrated.

[0037] Figure 2 An example of an interface between a first audio renderer and a second audio renderer, to which the technology according to embodiments of this disclosure may be applied, is illustrated schematically.

[0038] Figure 3 An example of an interface between a voxel-based scene geometry engine and an audio renderer according to an embodiment of the present disclosure is illustrated schematically.

[0039] Figure 4An example of an interface between a first audio renderer and a second audio renderer according to an embodiment of the present disclosure is illustrated schematically;

[0040] Figure 5 This is a flowchart illustrating an example of a method for processing voxel-based audio scene representation according to embodiments of the present disclosure;

[0041] Figure 6 This illustrates an implementation of an embodiment according to the present disclosure. Figure 5 A flowchart illustrating the steps of a method;

[0042] Figures 7 to 10 An example illustrating an iterative update of the set of matrix entries associated with corresponding voxels of a voxel-based audio scene representation according to embodiments of the present disclosure is shown.

[0043] Figure 11 This is a block diagram schematically illustrating an example of an apparatus suitable for implementing a method according to an embodiment of the present disclosure. Detailed Implementation

[0044] In the following description, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. The same elements in the drawings may be denoted by the same reference numerals, and repeated descriptions thereof may be omitted.

[0045] Voxel-based audio scene representation

[0046] First, an overview of voxel-related concepts used for audio scene representation will be given.

[0047] What are voxels in audio rendering?

[0048] A voxel is understood as a spatial volume with audio rendering instructions or acoustic properties assigned to it.

[0049] What is the voxel size for audio rendering?

[0050] The voxel size can be an encoder configuration parameter. It can be selected (manually or automatically) based on the level of detail in the scene geometry (e.g., in the range of 10cm-1m).

[0051] How large of an audio scene can it handle?

[0052] Large audio scenes do not necessarily lead to a large number of voxels and high rendering complexity. For example, a large audio scene can be represented as...

[0053] ● A set of independent sub-scenes (and a method for "transferring" these representations without requiring a renderer "restart").

[0054] ● A set of scene updates (based on user location)

[0055] How to handle the discontinuity problem caused by voxel granularity?

[0056] Any strong discontinuities in sound levels (and jumps in the direction of diffraction signals) can be avoided by applying interpolation (for example, in time and space).

[0057] How to represent a voxel-based audio scene?

[0058] Any voxel-based representation of an audio scene can include indications of voxels that are not transmission voxels (e.g., occlusion voxels) (i.e., voxels in which sound cannot propagate or cannot propagate freely)—a representation of occlusion geometry. This indication can involve the coordinates of the corresponding voxel (e.g., center coordinates, angular coordinates, etc.). For example, the coordinates of these voxels can be represented by a grid index. Additionally, a voxel-based representation can include indications of material properties of voxels that are not transmission voxels, such as absorption coefficients, reflection coefficients, etc. Besides occlusion voxels, a voxel-based representation can also indicate transmission voxels (e.g., air voxels) (i.e., voxels in which sound can propagate)—a representation of the sound propagation medium. Therefore, for each voxel in a predefined portion of space (e.g., within the boundary surrounding the audio scene), some implementations of the voxel-based representation of the audio scene can include indications of the corresponding material properties.

[0059] Use Case Examples

[0060] In summary, this disclosure provides a scheme for converting voxel-based scene matrix representations (including voxel-based scene representations of voxel matrices) used, for example, in existing file formats and voxel scene engine memories, into MPEG-I immersive audio metadata syntax.

[0061] Specifically, this disclosure provides a method for lossless and real-time (meta)data conversion and compression of (integer) 3D (three-dimensional) and 2D (two-dimensional) matrices, such as voxel scenes or acoustic diffraction projection planes for encoding and / or audio rendering.

[0062] For example, the proposed techniques (e.g., metadata transformation and compression methods) can be used.

[0063] ● On the audio encoder side, for example, for online or offline processing of voxel-based audio scene representation,

[0064] ● On the audio renderer side, online or offline processing for voxel-based audio scene representation (specifically, representation of the acoustic diffraction projection plane) for transfer to another audio renderer, and

[0065] ● On the renderer side, real-time processing is used for voxel-based audio scene representation for audio rendering.

[0066] The following will describe illustrative examples of feasible use cases of the technology according to embodiments of this disclosure.

[0067] Example use case (A): Transformation of voxel scene geometry (3D voxel matrix)

[0068] In this example, the user can experience a virtual environment generated in real time by a scene engine (procedurally). During scene rendering, scenes of the virtual environment can be created around the user's location. Existing scene engines represent all scene parts using voxels arranged as a 3D matrix M, which is used for video rendering, for example, by generating a user-visible mesh. A similar 3D matrix representation M contains entries describing (or referencing) material properties such as occlusion and reflection coefficients. V It can be used for audio rendering.

[0069] Figure 1 An example of the interface between a voxel-based scene geometry engine 110 and an audio renderer 140, which is an MPEG-I immersive audio renderer, is schematically shown. The voxel-based scene geometry engine 110 generates a 3D matrix representation M. V 120. The audio renderer 140 receives scene matrix data using a metadata input interface defined by bitstream syntax. This interface supports a compressed format for voxel scene representation. The corresponding voxel scene V is defined as a vector set C. V 130 describes the voxel scene elements stored in a compressed format within the payload element voxSceneElements().

[0070]

[0071] Vector C V Indicates the voxel group that fills the cube volume. These voxels are of the same type or material. MatID represents the material ID of the voxel group, PackedPosS indicates the representation of the starting voxel of the voxel group (e.g., the corner of the cube volume), and PackedPosE indicates the representation of the ending voxel of the voxel group (e.g., the opposite corner of the cube volume).

[0072] Appendix A.1 describes the MPEG-I immersive audio specification, which provides information from C... V To M V The conversion process (e.g., decompression).

[0073] Example use case (B): Diffraction projection diagram (2D matrix)

[0074] In this example, the user experience is rendered by a virtual environment rendered by an instance (A) of the MPEG-I immersive audio renderer. It calculates (and caches) a virtual environment containing a 2D voxel matrix M. D The audio diffraction metadata of the diffraction projection pattern is represented by matrix M. D It can be used for acoustic diffraction pathfinding and subsequent generation of all (virtual) diffraction audio sources. Diffraction metadata can be shared with another instance (B) of the MPEG-I immersive audio renderer (e.g., in real-time or offline) to reduce computational complexity. In other words, the MPEG-I immersive audio renderer can act as an encoder for diffraction metadata used by itself or other audio renderers.

[0075] Figure 2 An example of an interface between two audio renderers used for diffraction metadata is schematically illustrated. The first MPEG-I immersive audio renderer (A) 210 generates a 2D voxel matrix M. D 220. The second MPEG-I immersive audio renderer (B) 240 uses a metadata input interface defined by bitstream syntax to receive diffraction matrix data. This interface supports a compressed format of voxel projection map representation. The corresponding diffraction projection map D is defined as a vector set C. D 230 describes the voxel scene elements stored in a compressed format within the payload element voxSceneDiffractionMap().

[0076]

[0077] From C D To M D The conversion process (e.g., decompression) is given in the MPEG-I Immersive Audio Specification described in Appendix A.2.

[0078] Technology for processing audio scene information

[0079] The MPEG-I immersive audio renderer interface supports 3D (and 2D) matrix M V (and M) D ) to bitstream metadata format C V (and C) D The conversion process for MPEG-I immersive audio is not currently specified. Several conversion methods may be feasible in this regard, but none of them can be performed in real-time on existing rendering devices.

[0080] This disclosure provides a fast (low-complexity) conversion and (real-time) compression algorithm.

[0081] Figure 3An example of the application of the technology according to embodiments of this disclosure to the interface between a voxel-based scene geometry engine and an audio renderer is illustrated.

[0082] In this example, Scene Engine 310 generates a (3D) matrix representation (voxel matrix) M. V This is input into the (encoder side) conversion (or compression) process 320. The resulting representation C V 330 describes voxel scene elements stored in a compressed format, which can then be input to the audio renderer 340 via a specified renderer interface.

[0083] Alternatively, conversion (e.g., compression) can be performed on the renderer side before the rendering process is input to the audio renderer 340 via the specified renderer interface.

[0084] exist Figure 4 The diagram schematically illustrates an example of the application of techniques according to embodiments of the present disclosure for the application of diffraction metadata to an interface between two audio renderers.

[0085] In this example, the first audio renderer 410 generates a (2D) matrix representation (diffraction pattern) M. D It is input to (the first renderer side) conversion (or compression) process 420. The resulting representation C D 430 describes voxel scene elements of a diffraction map stored in a compressed format, which can then be input to a second audio renderer 440 via a specified renderer interface.

[0086] Alternatively, conversion (e.g., compression) can be performed on the second renderer side before the rendering process is input to the second audio renderer 440 via the specified renderer interface.

[0087] Conversion Algorithm

[0088] The conversion (e.g., compression) techniques disclosed herein are based on a single-material voxel block “expansion” algorithm that focuses on low-complexity performance (rather than high compression ratio) to support real-time data communication.

[0089] The proposed method may include the following three repetitive basic (encoding) steps:

[0090] 1) Select a material type ID from all available material types (contained in the voxel matrix M). Essentially, select a value from a set of values ​​associated with the corresponding material identifier of the voxel.

[0091] 2) Create a Boolean matrix (binary matrix) containing information about the selected material type ID entry. This Boolean entry can have a value of "1" if the corresponding entry in the voxel matrix has the selected material type ID, otherwise it has a value of "0".

[0092] 3) Perform lossless encoding on the Boolean matrix (binary matrix) according to the bitstream metadata format C. The bitstream metadata format C can, for example, describe voxel scene elements stored in compressed format in the payload element voxSceneElements() (3D case), or describe voxel scene elements stored in compressed format in the payload element voxSceneDiffractionMap() (2D case).

[0093] The third encoding step can be performed using an iterative process, such as the "voxel block expansion" method without residual estimation.

[0094] exist Figure 5 The flowchart illustrates an example of a voxel-based scene representation method 500 for processing audio scenes, based on the above description. As mentioned above, it is assumed that the voxel-based scene representation comprises a voxel matrix M, wherein each entry in the voxel matrix M is uniquely associated with a corresponding voxel in the audio scene and is taken from a set of values ​​associated with the corresponding material identifier (material type identifier) ​​of the voxel. Thus, each entry in the voxel matrix M indicates the material type ID of the corresponding associated voxel in the audio scene.

[0095] Method 500 has steps S510 to S550, which are performed for a given material type ID (i.e., for a given value in the set of values ​​mentioned above).

[0096] In step S510, a binary matrix (Boolean matrix) is generated, the entries of which indicate whether the corresponding entries in the voxel matrix M have a given value. This can be done according to step 2 above.

[0097] In step S520, an entry of a binary matrix whose corresponding entry in the voxel matrix has a given value is selected. This entry of the binary matrix can be selected as the first entry along the binary matrix entries in a predetermined scan order, indicating that the corresponding entry in the voxel matrix has a given value. An example of selection is described below in step 3.1). For example, the predetermined scan order may correspond to a scan order along the x, y, and z directions.

[0098] In step S530, the set of entries of the binary matrix is ​​initialized with the selected entries. It should be noted that each entry in the binary matrix is ​​uniquely associated with a corresponding voxel, and it can be equivalently said that the set of voxels is initialized with the voxels associated with the selected entries, or initialized with the voxels associated with the selected entries.

[0099] In step S540, dimensions (dimensional directions) are iteratively selected from the previously unselected dimensions of the binary matrix, and the set of entries is iteratively updated based on the selected dimensions. For a three-dimensional voxel matrix, there are three dimensions, which can be represented by x, y, and z. For a two-dimensional voxel matrix (e.g., a diffraction pattern), there are two dimensions, which can be represented by x and y.

[0100] It should be noted that each entry in the binary matrix is ​​uniquely associated with the corresponding voxel. It can be said that a dimension (dimension direction) is iteratively selected from the previously unselected dimensions (dimension directions) of the binary matrix, and the voxel set is updated based on the selected dimension (dimension direction).

[0101] In step S550, a representation of the resulting set of entries is generated, for example, for encoding or rendering.

[0102] It should be noted that each entry in the binary matrix is ​​uniquely associated with the corresponding voxel, and it can be said that an equivalent representation of the resulting voxel set is generated.

[0103] For example, the generated representation can correspond to the aforementioned vector (or line) set C. V Or C D The vector (or line) in the vector. That is, the representation can correspond to the compressed format in the payload element voxSceneElements() (3D case) or the compressed format in the payload element voxSceneDiffractionMap() (2D case). In either case, the representation of the resulting set of entries (or voxels) can indicate the start entry (or voxel) of the resulting set of entries (e.g., PackedPosS), the end entry (or voxel) of the resulting set of entries (e.g., PackedPosE), and the material identifier (e.g., MatID) of the entries in the set of entries.

[0104] For example, the starting voxel could be a corner voxel of a cube formed by this set of voxels, and the ending voxel could be a diagonal voxel of the cube. The starting voxel could be a voxel in the set of voxels that has the minimum coordinate value in each dimension of the binary matrix, while the ending voxel could be a voxel in the set of voxels that has the maximum coordinate value in each dimension of the binary matrix.

[0105] The generated representation can be output for encoding or rendering. For example, the generated representation can be output to a bitstream.

[0106] As described below, processing a binary matrix typically yields several representations of a set of entries (or voxels), such that the output of the proposed method can include a set of representations of the set of entries(or voxels) for encoding or rendering. It should be understood that this set of representations of the set of entries (or voxels) allows for complete (i.e., lossless) reconstruction of the binary matrix. In some implementations (even if this is not necessary for lossless reconstruction), this set can be an ordered set.

[0107] Next, an example implementation of step 3) defined above will be described, namely, lossless encoding of a Boolean matrix (binary matrix) according to the bitstream metadata format C.

[0108] 3.1) Locate the first voxel of material type MatID (e.g., using the fastest array search method) and assign it to the initial voxel block C that shares the same material type MatID. (i=0) . Figure 7 An example is shown, where voxel 710 is the first voxel of material type MatID.

[0109] 3.2) (Virtually) Expand this voxel block C individually in each available dimensional direction. (i) This is assuming the expanded block contains only voxels of the same material type, MatID. This is equivalent to determining, for each available dimensional direction, the maximum extent (or maximum extension) along that dimensional direction relative to a given voxel, such that for voxel blocks (or sets) C... (i) Each voxel in the matrix, relative to all voxels within the maximum range of that voxel, has a material type ID as follows: For this material type ID, a Boolean matrix has been generated. This determination can be based solely on the Boolean matrix. Figure 7 In the example, the expansion in the first dimension (e.g., the x direction) generates a (virtual) expansion block 720, the expansion in the second dimension (e.g., the y direction) generates a (virtual) expansion block 730, and the expansion in the third dimension (e.g., the z direction) generates a (virtual) expansion block 740.

[0110] 3.3) Select the (virtual) expansion block obtained in 3.2) that results in the maximum number of covered voxels. Figure 7 In this example, this would be block 720. This is equivalent to selecting the dimension that produces the largest range among the available dimension directions.

[0111] 3.4) Place voxel block C (i) Expand to the corresponding selected expansion block to obtain voxel block C (i+1) . Figure 8 An example is shown where block 820 involves an extended voxel block C. (i+1)This is equivalent to expanding the voxel block C in the selected dimensional direction according to the maximum range of the selected dimensional direction. (i) .

[0112] 3.5) Repeat steps 3.2) through 3.4) until all dimensional directions (e.g., M) are reached. V There are 3, M D All two dimensions have been examined (i.e., for voxel block expansion). For example, Figure 8 Block 820 in the middle relates to a voxel block C obtained by extending in the first selected dimensional direction. (i=1) The (virtual) expansion in the other two dimensions resulted in blocks 830 and 840. Block 830 resulted in more voxels being covered than block 840, thus block 830 was chosen for voxel block C. (i=1) The extension to obtain voxel block C (i=2) . Figure 9 Another example is shown, where block 930 involves an extended voxel block C obtained by expanding in a second selected dimension. (i=2) Finally, in the current 3D example, for virtual extension ( Figure 9 Block 940 in the middle, only the third dimension remains, and the voxel block C (i=2) Expansion in the third dimension produces voxel block C. (i=3 ) This is due to Figure 10 Block 1040 is shown in the example.

[0113] Included in the final voxel block C (i=N) After the voxels in the matrix have been removed from the Boolean matrix (binary matrix), the method can then loop back to step 3.1), where N is the number of dimensional directions of the voxel matrix M.

[0114] Based on the above, an example implementation of step S540 of method 500 is provided by... Figure 6 Method 600 is given. Method 600 includes steps S610 to S630, which are performed in each iteration of step S540. Here, the number of iterations corresponds to the number of dimensions (dimensional directions) of the voxel matrix to be transformed (compressed). Although the foregoing use cases and examples involve three-dimensional and two-dimensional voxel matrices, the proposed method is equally applicable to higher-dimensional matrix representations.

[0115] In step S610, a corresponding maximum range is determined along each previously unselected dimension (dimensional direction) relative to a given entry in the binary matrix. This determination is performed for each previously unselected dimension (dimensional direction) such that for each entry in the set of entries, all entries within the maximum range relative to the corresponding entry have a given value in the corresponding voxel matrix. For a given dimension (dimensional direction), the maximum range relative to a given entry can involve extensions along the positive direction and the negative direction of the given dimension (dimensional direction).

[0116] It should be noted that each entry in the binary matrix is ​​uniquely associated with the corresponding voxel in the binary matrix (or voxel matrix), which can be said to determine the corresponding maximum range along each previously unselected dimension (dimensional direction) relative to a given voxel, such that for each voxel in the set of voxels, all voxels in the maximum range relative to the corresponding voxel have associated entries in the binary matrix, which indicate that the corresponding entry in the voxel matrix has a given value.

[0117] In step S620, the dimension (dimension direction) that produces the largest maximum range is selected from the previously unselected dimensions (dimension directions).

[0118] In step S630, the entry set is updated by adding all entries of the binary matrix within the maximum range of the selected dimension (dimensional direction) relative to the corresponding entry for each entry in the entry set.

[0119] It should be noted that each entry in the binary matrix is ​​uniquely associated with the corresponding voxel in the binary matrix (or voxel matrix), and the voxel set can be updated by adding all voxels within the maximum range of the selected dimension relative to the corresponding voxel for each voxel in the voxel set.

[0120] After all dimensions (dimensional directions) have been processed and used for expansion / updating the set of entries (or voxels), all those entries in the binary matrix contained in the set of entries for a given value from a set of values ​​are set to a value that indicates that the corresponding entry in the voxel matrix does not have that given value. That is, these entries are removed from the binary matrix (and, through expansion, from the voxel-based representation) to avoid double processing of entries and to achieve a non-overlapping set of entries.

[0121] Then, another entry in the binary matrix whose corresponding entry in the indicator voxel matrix has a given value (e.g., a given MatID) can be selected for iterative processing. Specifically, entries in the binary matrix can be selected for iterative processing (followed by removing the resulting set of entries from the binary matrix) until all entries in the binary matrix whose corresponding entries in the indicator voxel matrix have the given value have been assigned to the set of entries. In other words, the continuous selection of entries in the binary matrix for iterative processing can be repeated until the binary matrix no longer contains any entries whose corresponding entries in the indicator voxel matrix have the given value. The selection of binary matrix entries can be performed according to the aforementioned scan order. It should be noted that the previously encoded entries have been removed from the binary matrix, which is equivalent to repeatedly selecting the first entry in the indicator voxel matrix whose corresponding entry in the indicator voxel matrix has the given value along the (updated) binary matrix in a predetermined scan order.

[0122] A representation of the entire set of defined entries can be output for encoding or rendering.

[0123] The above processing involves a single value within the group of values ​​(e.g., a single material type ID or MatID). It should be understood that, as described above, all values ​​in the group of values ​​(e.g., all MatIDs) can be processed according to a fixed processing order (e.g., in numerical order, from smallest to largest, excluding 0).

[0124] Thus, the voxel matrix M will be decomposed into non-overlapping blocks of voxels (e.g., cubes), where all voxels in a given block have the same material type.

[0125] Below are examples of pseudocode for implementing steps 1) to 3) with sub-steps 3.1) to 3.5) (and correspondingly, method 500 with sub-steps according to method 600). Furthermore, Appendix B provides examples of the results of the conversion process, and Appendix C provides performance estimates.

[0126] Example pseudocode

[0127] % ******************************************************* /

[0128] % Compressed iteration

[0129] %**************************************************** /

[0130]

[0131]

[0132] %**************************************************** /

[0133] % For block magnification in one dimension

[0134] % ***************************************************** /

[0135]

[0136]

[0137]

[0138] Technological advantages

[0139] The conversion (e.g., compression) technology according to embodiments of this disclosure supports real-time metadata updates (e.g., L3 updates) on the encoder side or decoder / renderer side.

[0140] For use case (A) above, the proposed method allows the operation of an MPEG-I immersive audio renderer without encoding any voxel-related geometric metadata. All voxel information can be converted into a compressed bitstream syntax format (e.g., during scene initialization) and sent to the audio renderer, for example, via L3 updates. This makes the technique “encoder-free” and provides a “low bitrate” operating point for the MPEG-I immersive audio renderer.

[0141] For use case (B) above, the proposed method allows an MPEG-I immersive audio renderer to provide data for acoustic diffraction modeling tools to other renderer instances, thus acting as an encoder. This supports the “low-complexity” operation point of the MPEG-I immersive audio renderer.

[0142] In both use cases, the proposed method can also be used for conversion on the input side of various audio renderers.

[0143] Device

[0144] Although methods and processing chains have been described above, it should be understood that this disclosure also relates to apparatus (e.g., computer apparatus or, in general, apparatus with processing capabilities) for implementing these methods and processing chains (or, in general, techniques).

[0145] An example of this device 1100 is schematically shown in Figure 11As shown in the diagram. The apparatus 1100 includes a processor 1110 and a memory 1120 coupled to the processor 1110. The memory 1120 may store instructions executed by the processor 1110. The processor 1110 may be adapted to implement devices (e.g., encoders or decoders / renderers) throughout the present disclosure and / or perform methods (e.g., methods for processing voxel-based scene representations of audio scenes) throughout the present disclosure. The apparatus 1100 may receive input (e.g., voxel-based scene representations of audio scenes, etc.) and generate output (e.g., representations of entries / voxel sets, etc.), as described throughout the disclosure.

[0146] In one implementation, for example according to use case (A) above, the apparatus may relate to an audio encoder apparatus that performs a voxel-based representation (e.g., voxel matrix M) transformation (e.g., compression) at its output for providing the transformed output to an audio rendering apparatus. For example, this process may be performed offline.

[0147] In another implementation, such as according to use case (A) above, the apparatus may relate to an audio rendering apparatus that performs a transformation (e.g., compression) of a voxel-based representation (e.g., a voxel matrix M) at its input, and then consumes the transformed representation during audio rendering. This process can be performed in real time.

[0148] In either of the above implementations of use case (A), for example, the voxel matrix M can be a three-dimensional matrix related to the three-dimensional representation of the audio scene.

[0149] In one implementation, for example according to use case (B) above, the apparatus may relate to an audio rendering apparatus that performs a voxel-based representation (e.g., voxel matrix M) transformation (e.g., compression) at its output for providing the transformed output to another audio rendering apparatus. For example, this process may be performed offline.

[0150] In one implementation, for example according to use case (B) above, the apparatus may relate to an audio rendering apparatus that performs a transformation (e.g., compression) of a voxel-based representation (e.g., a voxel matrix M) at its input, and then consumes the transformed representation during audio rendering. For example, the process may be performed in real time.

[0151] In both implementations of use case (B) above, the voxel matrix can be a two-dimensional matrix associated with the audio diffraction pattern.

[0152] explain

[0153] The aspects of the systems described herein can be implemented in a suitable computer-based sound processing network environment (e.g., a server or cloud environment) for processing digital or digitized audio files. Parts of these systems may include one or more networks comprising any desired number of individual machines, including one or more routers (not shown) for buffering and routing data transmitted between computers. Such networks can be built on a variety of different network protocols and can be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0154] One or more of the components, blocks, processes, or other functional components may be implemented by a computer program that controls the execution of a processor-based computing device. It should also be noted that, depending on their behavior, register transfers, logical components, and / or other characteristics, the various functions disclosed herein can be described using any number of combinations of hardware, firmware, and / or data and / or instructions embodied in various machine-readable or computer-readable media. Computer-readable media in which such data and / or instructions may be embodied may include, but are not limited to, various forms of physical (non-transitory), non-volatile storage media, such as optical, magnetic, or semiconductor storage media.

[0155] Specifically, it should be understood that embodiments may include hardware, software, and electronic components or modules, which, for the purposes of discussion, may be shown and described as if most components were implemented solely in hardware. However, those skilled in the art, based on this detailed description, will recognize that in at least one embodiment, the electronic aspects may be implemented in software (e.g., stored on a non-transitory computer-readable medium) executable by one or more electronic processors (e.g., microprocessors and / or application-specific integrated circuits (“ASICs”)). Therefore, it should be noted that embodiments may be implemented using multiple hardware and software-based devices and multiple different structural components. For example, the computer-implemented neural network described herein may include one or more electronic processors, one or more computer-readable medium modules, one or more input / output interfaces, and various connectors (e.g., system buses) connecting the various components.

[0156] While one or more implementations have been described by way of example and according to specific embodiments, it should be understood that the one or more implementations are not limited to the disclosed embodiments. Rather, the invention is intended to cover various modifications and similar arrangements that will be obvious to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation to include all such modifications and similar arrangements.

[0157] Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The use of “including,” “comprising,” or “having,” and variations thereof, means including the items listed thereafter and their equivalents, as well as additional items. Unless otherwise stated or limited, the terms “installation,” “connection,” “support,” and “coupled,” and variations thereof, are used extensively and include both direct and indirect installation, connection, support, and coupling.

[0158] Listed example embodiments

[0159] Various aspects and implementations of the present invention can also be understood from the following exemplary embodiments (EEE), which are not claims.

[0160] EEE 1. A method for processing a voxel-based scene representation of an audio scene, wherein the voxel-based scene representation includes a voxel matrix, and wherein entries of the voxel matrix are associated with corresponding voxels in the audio scene and are taken from a set of values ​​associated with corresponding material identifiers of the voxels, the method comprising, for a given value in the set of values:

[0161] Generate a binary matrix, wherein each entry in the binary matrix indicates whether the corresponding entry in the voxel matrix has the given value.

[0162] Select an entry in the binary matrix that indicates the corresponding entry in the voxel matrix has the given value.

[0163] Initialize the set of entries of the binary matrix with the selected entries.

[0164] Iteratively select dimensions from the previously unselected dimensions of the binary matrix, and update the set of entries based on the selected dimensions.

[0165] Generate a representation of the resulting set of entries for encoding or rendering.

[0166] The method further includes, in each iteration:

[0167] Determine the corresponding maximum range along each previously unselected dimension relative to a given entry in the binary matrix, such that for each entry in the set of entries, all entries within the maximum range relative to the corresponding entry indicate that the corresponding entry in the voxel matrix has the given value.

[0168] Select the dimension that produces the largest maximum range from the previously unselected dimensions, and

[0169] The entry set is updated by adding all entries of the binary matrix within the maximum range of the selected dimension relative to the corresponding entry for each entry in the entry set.

[0170] EEE 2. The method according to EEE 1, wherein the representation of the resulting set of entries indicates the start entry of the resulting set of entries, the end entry of the resulting set of entries, and the material identifier of the entries in the resulting set of entries.

[0171] EEE 3. The method according to EEE 1 or EEE 2 further includes:

[0172] Output the generated representation used for encoding or rendering.

[0173] EEE 4. The method according to any one of EEE 1 to 3, wherein the entry of the binary matrix is ​​selected as a first entry along the entries of the binary matrix in a predetermined scan order, the first entry indicating that the corresponding entry in the voxel matrix has the given value.

[0174] EEE 5. The method according to any one of EEE 1 to 4, further comprising:

[0175] The entries in the binary matrix that will be included in the set of entries for the given value in the set of values ​​will be set to values ​​indicating that the corresponding entries in the voxel matrix do not have the given value.

[0176] EEE 6. The method according to EEE 5 further includes:

[0177] In the binary matrix, select another entry that indicates the corresponding entry in the voxel matrix has the given value.

[0178] EEE 7. The method according to EEE 6, wherein entries of the binary matrix are selected for performing iterative processing until all entries of the binary matrix indicating that the corresponding entries in the voxel matrix have the given value have been assigned to the entry set.

[0179] EEE 8. The method according to EEE 6 or EEE 7 further includes: outputting a representation set of the set of entries for encoding or rendering.

[0180] EEE 9. The method according to any one of EEE 1 to 8, further comprising:

[0181] All values ​​in the group are processed according to a fixed processing order.

[0182] EEE 10. The method according to any one of EEE 1 to 9, wherein the method is performed at an audio rendering device, and the output of the method is used as input to the audio rendering process.

[0183] EEE 11. The method according to any one of EEE 1 to 10, wherein the voxel matrix is ​​a two-dimensional matrix.

[0184] EEE 12. The method according to any one of EEE 1 to 11, wherein the voxel matrix relates to an audio diffraction pattern.

[0185] EEE 13. The method according to any one of EEE 1 to 10, wherein the voxel matrix is ​​a three-dimensional matrix.

[0186] EEE 14. The method according to any one of EEE 1 to 10 or 13, wherein the voxel matrix relates to a three-dimensional representation of the audio scene.

[0187] EEE 15. An audio rendering apparatus comprising a processor and a memory, the memory being coupled to the processor and storing instructions for the processor, wherein the processor is configured to perform a method according to any one of EEE 1 to 14.

[0188] EEE 16. An audio encoder device comprising a processor and a memory, the memory being coupled to the processor and storing instructions for the processor, wherein the processor is configured to perform a method according to any one of EEE 1 to 10, 13 or 14.

[0189] EEE 17. A program comprising instructions that, when executed by a processor, cause the processor to perform the method according to any one of EEE 1 to 14.

[0190] EEE 18. A computer-readable storage medium storing a program as described in EEE 17.

[0191] Appendix A.1

[0192] From C V To M V The conversion process (e.g., decompression) is given in the MPEG-I Immersive Audio Specification, as described below.

[0193]

[0194] Table 1 Syntax of voxSceneElements()

[0195] The syntax elements in Table 1 can be defined as follows.

[0196] voxSceneElements() provides a compact representation of a 3D voxel mesh for a 3D audio scene.

[0197] A SceneElement is defined by two points (x, y, z indices) on a voxel grid and their corresponding values. These two points span a cube, and all covered grid cells are assigned the value voxSceneElementMatID.

[0198] The bitstream element numberOfVoxSceneElements represents the number of SceneElements.

[0199] The bitstream element voxSceneElementMatID represents an integer value indicating the material type of the mesh cell of the SceneElement.

[0200] The bitstream element `voxSceneElementPosPackedS` represents a packed representation of the three indices of the starting grid cell of the SceneElement. It can be an array representing the set of all starting grid cells.

[0201] The bitstream element `voxSceneElementPosPackedE` represents a packed representation of the three indices of the ending grid cell of the SceneElement. It can be an array representing the set of all ending grid cells.

[0202] Both voxSceneElementPosPackedS and voxSceneElementPosPackedE are useful because they allow for a compact representation of the data, where a single voxSceneElementMatID is used for all grid cells between the two variables.

[0203] Uncompressed voxel scene data is represented by the 3D matrix variable VoxDataMatrix, defined by the payload elements voxSceneElements() using the following pseudocode:

[0204]

[0205] in

[0206]

[0207] Appendix A.2

[0208] From C D To M D The conversion process (e.g., decompression) is given in the MPEG-I Immersive Audio Specification, as described below.

[0209]

[0210] Table 2 Syntax of voxSceneDiffractionMap()

[0211] The syntax elements in Table 2 can be defined as follows.

[0212] voxSceneDiffractionMap() provides a compact representation of a 2D diffraction map (VoxDataDiffractionMap). This 2D representation is analogous to the 3D representation used for voxel-based 3D audio scenes.

[0213] A MapElement is defined by two points (x, y-indices) on the diffraction map and their corresponding values. These two points span a rectangle, and all covered grid cells are assigned the value voxDiffractionMapValue.

[0214] The numberOfVoxDiffractionMapElements table contains the number of MapElements in the bitstream.

[0215] The bitstream element `voxDiffractionMapValue` represents a binary value controlling the pathfinding algorithm. This is useful because the value indicates whether a path can pass through a grid cell. This value is defined for all entries on the diffraction map.

[0216] The bitstream element `voxDiffractionMapPosPackedS` represents a packed representation of the two indices of the starting grid cell of the `MapElement`. It can be an array representing the set of all starting grid cells.

[0217] The bitstream element `voxDiffractionMapPosPackedE` represents a packed representation of the two indices of the ending grid cell of the `MapElement`. It can be an array representing the set of all ending grid cells.

[0218] Both voxDiffractionMapPosPackedS and voxDiffractionMapPosPackedE are useful because they allow for a compact representation of the data, where a single voxDiffractionMapValue is used for all grid cells between the two variables.

[0219] The same method described in Appendix A.1 can be used to decompress the 2D matrix variable VoxDataDiffractionMap used in the diffraction modeling stage. If the 2D matrix variable VoxDataDiffractionMap is specified by the encoder (i.e., voxSceneDiffractionMapPresentFlag == 1), it is defined by the bitstream payload element voxSceneDiffractionMap(); otherwise (i.e., voxSceneDiffractionMapPresentFlag == 0), it is defined by the bitstream payload element voxSceneElements(), using the following pseudocode:

[0220]

[0221]

[0222] Appendix B

[0223] The following shows the three-dimensional voxel matrix M. V and two-dimensional diffraction pattern M D Examples of the results of the C↔M conversion process.

[0224]

[0225] Appendix C

[0226] The techniques according to embodiments of this disclosure can transform a voxel matrix M into metadata C comprising multiple lines (or vectors), each line corresponding to a representation of a set of entries (or voxels) defined in this disclosure. Simulation results show that the metadata C is significantly smaller in size than the initial voxel matrix M. Examples of simulation results are shown in Table 3 below, where the first column relates to a pre-configured example scenario, the second column relates to the time required for the transformation using a computing device with conventional computing capabilities, the third column indicates the number of rows in the metadata C, and the fourth column indicates the number of voxels in the corresponding example scenario.

[0227]

[0228] Table 3 Simulation Results

Claims

1. A method for processing a voxel-based scene representation of an audio scene, wherein the voxel-based scene representation includes a voxel matrix, and wherein entries of the voxel matrix are associated with corresponding voxels in the audio scene and are taken from a set of values ​​associated with corresponding material identifiers of the voxels, the method comprising, for a given value in the set of values: Generate a binary matrix, wherein each entry in the binary matrix indicates whether the corresponding entry in the voxel matrix has the given value. Select an entry in the binary matrix that indicates the corresponding entry in the voxel matrix has the given value. Initialize the set of entries of the binary matrix with the selected entries. Iteratively select dimensions from the previously unselected dimensions of the binary matrix, and update the set of entries based on the selected dimensions. Generate a representation of the resulting set of entries for encoding or rendering. in, The method also includes, in each iteration: Determine the corresponding maximum range along each previously unselected dimension relative to a given entry in the binary matrix, such that for each entry in the set of entries, all entries within the maximum range relative to the corresponding entry indicate that the corresponding entry in the voxel matrix has the given value. Select the dimension that produces the largest maximum range from the previously unselected dimensions, and The entry set is updated by adding all entries of the binary matrix within the maximum range of the selected dimension relative to the corresponding entry for each entry in the entry set.

2. The method according to claim 1, wherein, The representation of the resulting set of entries indicates the starting entry of the resulting set of entries, the ending entry of the resulting set of entries, and the material identifier of the entries in the set of entries.

3. The method according to claim 1 or 2, further comprising: Output the generated representation used for encoding or rendering.

4. The method according to any one of claims 1 to 3, wherein, The entry of the binary matrix is ​​selected as the first entry along the entries of the binary matrix in a predetermined scanning order, and the first entry indicates that the corresponding entry in the voxel matrix has the given value.

5. The method according to any one of claims 1 to 4, further comprising: The entries in the binary matrix that will be included in the set of entries for the given value in the set of values ​​will be set to values ​​indicating that the corresponding entries in the voxel matrix do not have the given value.

6. The method according to claim 5, further comprising: In the binary matrix, select another entry that indicates the corresponding entry in the voxel matrix has the given value.

7. The method according to claim 6, wherein, The entries of the binary matrix are selected for iterative processing until all entries of the binary matrix indicating that the corresponding entries in the voxel matrix have the given value have been assigned to the entry set.

8. The method according to claim 6 or 7, further comprising: Outputs a set of representations of the set of entries used for encoding or rendering.

9. The method according to any one of claims 1 to 8, further comprising: All values ​​in the group are processed according to a fixed processing order.

10. The method according to any one of claims 1 to 9, wherein, The method is executed at an audio rendering device, and the output of the method is used as input to the audio rendering process.

11. The method according to any one of claims 1 to 10, wherein, The voxel matrix is ​​a two-dimensional matrix.

12. The method according to any one of claims 1 to 11, wherein, The voxel matrix relates to an audio diffraction pattern.

13. The method according to any one of claims 1 to 10, wherein, The voxel matrix is ​​a three-dimensional matrix.

14. The method according to any one of claims 1 to 10 or 13, wherein, The voxel matrix relates to a three-dimensional representation of the audio scene.

15. An audio rendering apparatus comprising a processor and a memory, the memory being coupled to the processor and storing instructions for the processor, wherein the processor is configured to perform the method according to any one of claims 1 to 14.

16. An audio encoder device comprising a processor and a memory, the memory being coupled to the processor and storing instructions for the processor, wherein the processor is configured to perform the method according to any one of claims 1 to 10, 13 or 14.

17. A program comprising instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 14.

18. A computer-readable storage medium storing the program according to claim 17.

Images