Encoding and Decoding of Acoustic Environment

By using structural acoustic data encoding methods, the positional data of polygons is linked with the properties of acoustic materials, which solves the problem of low encoding efficiency in virtual acoustic environments and enables efficient audio signal rendering.

CN117529774BActive Publication Date: 2026-11-17FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202280038185.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-05-27
Filing Date
2022-05-25
Publication Date
2026-11-17
Estimated Expiration
2042-05-25

AI Technical Summary

Technical Problem

Existing triangular mesh data encoding methods cannot efficiently handle the specific properties of virtual acoustic environments, resulting in low encoding efficiency.

Method used

The structural acoustic data encoding method is adopted to link the position data of polygons with the properties of acoustic materials, and encodes and decodes them through bitstream. Audio source encoding blocks, structural acoustic data encoding blocks, and bitstream writers/readers are used to process audio and acoustic data.

Benefits of technology

It improves the coding efficiency of virtual acoustic environments, reduces data redundancy, and enhances the rendering fidelity of audio signals.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117529774B_ABST
    Figure CN117529774B_ABST
Patent Text Reader

Abstract

Apparatus and methods for encoding and decoding an acoustic environment are disclosed. According to an example, an apparatus is provided for decoding an acoustic environment including at least one audio source and at least one audio object represented by structural acoustic data that links position data of polygons to acoustic properties of an acoustic material, wherein, for each polygon, the position data includes the positions of its vertices. The apparatus includes: a bitstream reader for reading from a bitstream at least one audio stream to be rendered generated by at least one audio source in the acoustic environment and an encoded version of the structural acoustic data; an audio source decoding block for decoding at least one audio stream representing at least one audio source; and a structural acoustic data decoding block for decoding the structural acoustic data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Apparatus and methods for encoding and decoding acoustic environments are disclosed. Background Technology

[0002] Triangular mesh data is a crucial component of virtual acoustic environments. The mesh consists of a series of vertices and a series of triangular faces. Each vertex is a point in 3D space, located by its X, Y, and Z coordinates, and has an associated index in the vertex list. Each triangle identifies a simple surface and contains three vertex indices and an associated acoustic material. The vertex indices of the triangles are listed in a specific order, defining the external pointing normal of the simple surface.

[0003] Many exchange and compression formats exist for general-purpose triangular mesh data. However, these are generally designed for encoding visual triangular mesh data (typically visual triangular mesh data of objects and environments). In contrast, mesh triangular data of virtual acoustic environments and objects possess several specific properties. For example, mesh data typically contains only acoustically relevant surfaces of sufficient size. A large number of object surfaces lie on a few planes or have a layered structure. Surfaces that do not contain acoustic materials are invisible for acoustic purposes and can be discarded. Objects with regular shapes can also exhibit coordinate symmetry using a relative coordinate system centered on their apparent centroid. All these additional properties can be used to develop more efficient and less complex custom encoding schemes. Summary of the Invention

[0004] According to an example, an apparatus is provided for decoding an acoustic environment including at least one audio source and at least one audio object represented by structural acoustic data that links positional data of polygons to acoustic properties of an acoustic material, wherein, for each polygon, the positional data includes the positions of its vertices. The apparatus includes:

[0005] A bitstream reader for reading encoded versions of at least one audio stream and structured acoustic data to be rendered, generated by at least one audio source in an acoustic environment;

[0006] An audio source decoding block for decoding at least one audio stream representing at least one audio source; and

[0007] The structural acoustic data decoding block is used to decode structural acoustic data.

[0008] Also provided is an apparatus for encoding an acoustic environment including at least one audio source and at least one audio object, the at least one audio object being represented by at least one structural acoustic data, the at least one structural acoustic data linking polygonal positional data to acoustic properties of an acoustic material, wherein, for each polygon, the structural acoustic data includes the position of its vertices, the apparatus comprising:

[0009] An audio source encoding block is configured to encode at least one audio stream to be rendered, the at least one audio stream being associated with at least one audio source;

[0010] A structured acoustic data encoding block is configured to encode at least one structured acoustic data to obtain an encoded version of at least one structured acoustic data.

[0011] A bitstream writer is configured to write an encoded version of the at least one audio stream and at least one structured acoustic data into a bitstream.

[0012] A method for encoding an acoustic environment is also provided, the acoustic environment including at least one audio source and at least one audio object, the at least one audio object being represented by at least one structural acoustic data, the at least one structural acoustic data linking polygonal positional data to the structural acoustic properties of a material, wherein, for each polygon, the positional data includes the position of a principal polygon vertex and the positions of the remaining polygon vertices, the method comprising:

[0013] Encode at least one audio stream to be rendered that is associated with at least one audio source;

[0014] Encoding at least one structural acoustic data point to obtain an encoded version of at least one structural acoustic data point; and

[0015] Write at least one audio stream and at least one encoded version of structural acoustic data into a bitstream.

[0016] A bitstream for encoding audio information is also provided, wherein an acoustic environment is encoded, the acoustic environment including at least one audio source and at least one audio object, the at least one audio object being represented by at least one structured acoustic data list that maps polygonal position data onto an acoustic material, wherein for each polygon, the position data includes the position of a vertex, and the bitstream includes:

[0017] At least one audio stream to be rendered;

[0018] At least one encoded version of the structural acoustic data.

[0019] It also provides a non-transient memory location for storing instructions that, when executed by the processor, cause the processor to:

[0020] Decoding operations control an acoustic environment comprising at least one audio source and at least one audio object, the at least one audio object being represented by a structural acoustic data list that links polygon position data to the structural acoustic properties of a material, wherein, for each polygon, the position data includes the position of a primary structural acoustic vertex and the positions of the remaining structural acoustic vertices.

[0021] Controls the reading of encoded versions of at least one audio stream and structural acoustic data to be rendered, generated from at least one audio source in the acoustic environment;

[0022] Controlling the decoding of at least one audio stream; and

[0023] Decode the structural acoustic data.

[0024] It also provides a non-transient memory location for storing instructions that, when executed by the processor, cause the processor to:

[0025] A method for controlling the encoding operation of an acoustic environment, the acoustic environment including at least one audio source and at least one audio object, the at least one audio object being represented by at least one structural acoustic data, the at least one structural acoustic data linking polygon position data to the structural acoustic properties of a material, wherein, for each polygon, the position data includes the position of a main polygon vertex and the positions of the remaining polygon vertices;

[0026] Control the encoding of at least one audio stream to be rendered that is associated with at least one audio source;

[0027] Controlling the encoding of at least one structural acoustic data to obtain an encoded version of at least one structural acoustic data; and

[0028] The control writes at least one audio stream and at least one encoded version of structural acoustic data into a bitstream. Attached Figure Description

[0029] Figure 1 An example of a polygon (triangle) to be encoded / decoded is shown.

[0030] Figure 2 An example of a device for encoding an acoustic environment is shown.

[0031] Figure 3 An example of a device for decoding an acoustic environment is shown.

[0032] Figure 4An example of an operation to encode or decode a list of vertices is shown.

[0033] Figure 5 An example of a bounding box is shown.

[0034] Figure 6a and Figure 6b Examples of operations for encoding a list of triangles are shown at the encoder and decoder, respectively.

[0035] Figure 7 An example of a data structure that can be used in this example is shown.

[0036] Figure 8 It shows that it can be used as Figure 2 An example of a structured acoustic data encoding block, which is part of an encoder.

[0037] Figure 9 It shows that it can be used as Figure 3 An example of a structured acoustic data decoding block, which is part of the decoder.

[0038] Figures 10a to 10h The operation sequence at the encoder is shown. Detailed Implementation

[0039] encoder

[0040] Figure 2 An encoder 200 is shown, which can be understood as a means for encoding an acoustic environment 202. The acoustic environment can be understood as a way of representing an audio signal 211 in a specific acoustic environment to be encoded in a bitstream 204. The acoustic environment can be represented according to spatial coordinates. The acoustic environment can be represented according to a spatial coordinate system (e.g., Figure 1The acoustic environment can be represented by x, y, z). The acoustic environment may include at least one audio source virtually located within some parts of the environment. This environment can be understood as a virtual environment to be rendered at the highest possible fidelity. Encoder 200 may include a structural acoustic data encoding block 220, which links the positional data of a polygon to the properties associated with the acoustic material. The polygon may be a triangle. Each polygon (or more specifically a triangle) may be represented as a triplet of vertices. Therefore, the output of polygon data encoding block 222 can, in principle, be represented by a triplet of structural acoustic data and values ​​encoding the material. Thus, the polygon may be a surface of numerous material elements that influence the behavior of the audio signal virtually generated by the audio source at the location indicated by the audio source. Encoder 200 may include a bitstream writer 230 for writing to bitstream 204. Thus, structural acoustic data 222, representing the audio signal virtually generated by the audio source and its location in environment 212, as well as representing various materials in that environment, can be encoded in bitstream 204.

[0041] Generally, encoder 200 can be considered as a device for encoding an acoustic environment including at least one audio source and at least one audio object, the at least one audio object being represented by at least one structured acoustic data list that links polygonal positional data to the acoustic properties of an acoustic material. For each polygon, the positional data may include the position of a main polygon vertex (110ax, 110ay, 110az) and the positions of the remaining polygon vertices (110b, 110c, 120b). The device may include:

[0042] Audio source encoding block 210 is configured to encode at least one audio stream to be rendered;

[0043] Structural acoustic data encoding block 220 is configured to encode at least one version (222) of structural acoustic data (221); and

[0044] Bitstream writer 230 is configured to write at least one audio stream 212 and location data (including an encoded version of the structured acoustic data) into bitstream 204.

[0045] Furthermore, audio stream 212 is typically associated with audio source location data, such that the audio sources 211 represented by audio stream 212 can correspond to specific locations in the acoustic environment in which they are virtually generated. Generally, at least one audio source (which is encoded in bitstream 204 in association with the location in the acoustic environment where the audio source is virtually generated) is also encoded along with auxiliary information providing its virtual location in the acoustic environment. Thus, spatial data can also be encoded as auxiliary information for at least one audio stream 212, indicating the positional relationship between at least one audio source and the acoustic environment. Once decoded, the audio sources are rendered by taking into account the spatial relationship between the audio sources and at least one audio object.

[0046] decoder

[0047] Figure 3 A decoder 300 is shown, which renders the acoustic environment encoded in bitstream 304. Thus, audio signal 301 can be generated by decoder 300. Nevertheless, the claimed decoder 300 may have an acoustic environment 302 as output, which may be a best representation of the original acoustic environment 202 represented by renderer 350. Decoder 300 may include a bitstream reader 330 capable of reading bitstream 204. Thus, the bitstream reader can provide an encoded version 312 of at least one audio source encoded by audio source encoding block 210 (e.g., 212). Bitstream reader 330 can also provide an encoded version 322 of structural acoustic data 222 encoded by structural acoustic data encoding block 220. Audio source decoding block 310 can provide a decoded version 311 of the original audio source 211. Structural acoustic data decoding block 320 can provide a decoded version 321 of the original structural acoustic data 221. The decoded version 311 of the original audio source 211 and the decoded version 321 of the original structural acoustic data 221 can therefore be considered together as the decoded version 302 of the environment 202.

[0048] The renderer 350 receives the decoded environment 302 (including its components 311 and 321) to render the audio signal 301 to approximate the original audio signal 202 as closely as possible. Specifically, the renderer 350 can represent at least one audio source by considering its location in the acoustic environment (e.g., a virtual location) and the modulation (virtual or real) of the sound due to the presence of at least one audio object.

[0049] Generally, the audio source (which is encoded in bitstream 204 in association with the location where the audio source is virtually generated in the acoustic environment) is also encoded along with auxiliary information providing its virtual location in the acoustic environment. Thus, renderer 350 can represent sound virtually generated at a specific location (e.g., indicated by the location data of the audio source) under the influence of the presence (e.g., virtual presence) of at least one audio object.

[0050] Decoder 300 may be an apparatus for decoding an acoustic environment 302, which includes at least one audio source and at least one audio object represented by a structural acoustic data list that links polygon location data to the acoustic properties of an acoustic material. For each polygon, the location data may include the location of a primary structural acoustic vertex and the locations of the remaining structural acoustic vertices. The apparatus may include at least one of the following:

[0051] Bitstream reader 330 is configured to read encoded versions (322, 222) of at least one audio stream 212 and structured acoustic data 211 to be rendered generated by at least one audio source in acoustic environment 302 from bitstream 204.

[0052] Audio source decoding block 310 is used to decode at least one audio stream (312, 212);

[0053] The structural acoustic data decoding block 320 is used to decode the structural acoustic data 211.

[0054] Structural acoustic data

[0055] like Figure 1 As shown, structural acoustic data can be associated with polygons (or, more specifically, triangles in this example). The polygons can be polygons of a grid. Here, three triangles 110, 120, and 130 are shown. The first triangle has a principal vertex 110a and two other vertices 110b and 110c. The second triangle 120 has a principal vertex that coincides with the principal vertex 110a of the first triangle 110 (and is therefore indicated by the same reference numerals) and two other vertices 120b (not coinciding with another vertex of the first triangle 110) and 110c (coinciding with another vertex of the first triangle 110). The third triangle 130 has a principal vertex 130a and two other vertices 130b and 130c. In this case, the y-coordinate of the principal vertex 130a is exactly the same as the y-coordinate of vertex 110 of the first and second triangles.

[0056] Figure 7An example of how structural acoustic data can be understood is shown. It can be seen that triangles 110a, 110b, and 110c (triangle 130 is not shown here) are shown with vertices 110a, 110b, and 120b. It can be seen that the first vertex list (804, 3804, 400) contains the vertices in each record (e.g., 110a, 110b, 110c, 120b, etc.) combined with their coordinates (x-coordinate, y-coordinate, z-coordinate). Therefore, the link between each vertex index 403 in the vertex list and the coordinates of each vertex is illustrated (coordinate notation is indicated by adding "x", "y", or "z"). Thus, the vertex list (804, 3804, 400) associates vertex index 403 with triples (or n-tuples, depending on the dimension) of the spatial coordinates identifying the vertex position. In some cases, some vertices may be repeated because they are overlapping vertices of different triangles (e.g., vertices 110a and 11c are repeated because they overlap but are in different triangles).

[0057] Figure 7 A list of triangles (802, 3802) is also shown, linking each triangle to a vertex index 403 of that triangle (or more generally, a polygon). For example, associated with triangle 110, we see vertex index 0 (associated with the coordinates of vertex 110a), vertex index 1 (associated with vertex 110b), and vertex index 2 (associated with vertex 110c). In the list of triangles (802, 3802), triangle 120 is associated with vertex index 3 (associated with vertex 110a), vertex index 4 (associated with vertex 110c), and vertex index 5 (associated with vertex 120b). It can be seen that in the list of triangles (802, 3802), in addition to the triangle identifiers and vertex data mapped to the vertices in the vertex list (and subsequently mapped to the positions of the vertices), there are also stored acoustic features (806, 3806) associated with, for example, the acoustic properties of the material.

[0058] basically, Figure 7 An example of structural acoustic data 221 (to be encoded) is shown, which links triangles (110, 120, 130) and their positional data (e.g., coordinates of the vertices) to the acoustic properties of the material. It will be shown that this structural acoustic data can be compressed and written into bitstream 204.

[0059] Figure 4An example of a structured acoustic data list 400 (which could be an example of a vertex list 400 (802, 3802)) is shown, which lists material associated with the position data of the principal vertices and the remaining vertices of each of triangles 110, 120 in different records. Here, the structured acoustic data list 400 is shown as being divided into x-coordinates (for the x-dimensional), y-coordinates (for the y-dimensional), and z-coordinates (for the z-dimensional). For example, in the x-coordinate, for the first triangle 110, the structured acoustic data list 400 already stores:

[0060] The x-coordinate of the principal vertex 110a is 110ax; and

[0061] The x-coordinates of the other vertices 110b and 110c are 110bx and 110cx, respectively.

[0062] Similarly, in the corresponding record of the y-coordinate in structural acoustic data list 400 ( Figure 4 (Not shown in the image), the corresponding column for the main vertex includes the y-coordinate 110ay of the main vertex 110a, while the corresponding columns for the other vertices have already been inserted with the y-coordinates 110by and 110cy of the other vertices 110b and 110c, respectively. This also applies to the z-dimensional: in the corresponding record of the z-coordinate in the structural acoustic data list 400 (… Figure 4 (Not shown in the image), the corresponding column of the main vertex includes the z coordinate 110az of the main vertex 110a, while the corresponding columns 110b and 110c of the other vertices have been inserted with the y coordinates 110by and 110cy respectively.

[0063] In the second record (second horizontal row from the top) of the structural acoustic data list 400, the coordinates of the second triangle 120 are stored. It can be seen that the coordinates of the principal vertex 110ax (and also 110ay, 110az) are repeated (for example, the x-coordinate 110ax of the principal vertex 110a of the second triangle 120 is a duplicate of the same value stored for the principal vertex of the first triangle, although these values ​​are identical). The same applies to vertex 110c, whose coordinates 110cx, 110cy, 110cz are the same for both the first triangle 110 and the second triangle 120.

[0064] Audio source encoding / decoding

[0065] Audio source encoding block 210 and audio source decoding block 210 are important components of encoder 200 and decoder 300, respectively. The sound source to be encoded and decoded can be represented by at least one audio stream 212, 312. However, this is not the case. At least one sound source can be associated with location data (e.g., metadata) that locates the position of the at least one sound source in the acoustic environment (e.g., virtual position). Therefore, sound (audio signal) 301 can be rendered (e.g., by renderer 350) based on the location data of at least one audio object, the acoustic properties of the material (conceived as the material of the object), and the structural acoustic relationship between the location data of at least one audio source. This operation can be performed by renderer 350 at the decoder (which may be an external device).

[0066] For example, by considering the positional data of at least one object (and specifically vertices and triangles) and the structural properties of the material, at least one audio source can have positional data including coordinates that allow the at least one audio source to be located in an acoustic environment. Therefore, at least one audio source will be located at a specific location in the acoustic environment, and the listener will experience the sound as if it originates from that location and is influenced by the material properties.

[0067] Therefore, when discussing acoustic environments, it is not only about the spatial environment but also about the complete audio scene that needs to be encoded / decoded before rendering. An acoustic environment not only has its own spatial characteristics (e.g., compressed or uncompressed location data, such as lists of vertices and triangles), but also the properties of the materials that constitute the objects within that environment, and it also has sound that can be virtually generated at an audio source located at a specific location within the spatial environment and is virtually modulated by the structural acoustic data (location data and properties of acoustic materials) encountered within the spatial environment.

[0068] Structural acoustic data encoding block

[0069] Figure 8 An example of a structured acoustic data encoding block 220 of encoder 200 is shown. It can be seen that the input to the structured acoustic data encoding block 220 includes structured acoustic data 221. The structured acoustic data 221 may include, for example, a triangle list 802, a vertex list 804, and acoustic features 806. Acoustic features 806 may be part of the triangle list 802, but are shown here in a different manner for clarity.

[0070] The structural acoustic data encoding block 220 may include a vertex list encoder 800, which can encode a vertex list 804 to obtain an encoded vertex list 808. How the encoder vertex list 808 can be generated will be explained later.

[0071] The structured acoustic data encoding block 220 may include a triangle list encoder 850. The triangle list encoder may be input from a triangle list 802 including acoustic features 806, and from an encoded vertex list 808 if an encoded version is provided, or alternatively from a vertex list 804 in a non-encoded version. Therefore, in some cases, it is not necessary to provide both inputs 804 and 808 to the triangle list encoder 850. The triangle list encoder 850 may provide an encoded triangle list 852 in which the triangle list 802 is compressed. Although the discussion primarily uses the term "triangle". Figure 8 However, the same result can be obtained by using different polygons.

[0072] Structural acoustic data encoding block

[0073] Figure 9 An example of the structured acoustic data decoding block 320 of decoder 300 is shown. An encoded version of structured acoustic data 322 (which is encoded version 222 of structured acoustic data encoded by encoder 200) is obtained from bitstream 204. Bitstream reader 330 can provide an encoded version 3852 of the triangle list (which is a copy of encoder triangle list 852 encoded by triangle list encoder 850) and an encoded version 3808 of the vertex list (which is a copy of encoded vertex list 808 encoded by vertex list encoder 800). Triangle list decoder 3850 can be input from encoder triangle list 3852. Vertex list decoder 3800 can be input from encoded vertex list 3808 to provide decoded vertex list 3804. Triangle list decoder 3850 can output decoded triangle list 3802. Triangle list decoder 3850 can be input from encoded vertex list 3808 or vertex list 3804 output by vertex list decoder 3800. Therefore, the structural acoustic data 321 may include a triangle list 3802 (including acoustic features 3806) and a vertex list 3804. The triangle list 3802 may indicate the vertex indices obtained from the vertex lists 804 and 3804 for each triangle. Although the discussion primarily uses the term "triangle". Figure 9 However, the same result can be obtained by using different polygons.

[0074] Vertex index encoding and decoding

[0075] Theoretically, it would be simple to encode all coordinates of each vertex in bitstream 204. For example, for the main vertex, all its x, y, z coordinates (110ax, 110ay, 110az) could be encoded; the same would be done for the remaining vertices 110b and 110c of the first triangle 110, and all fields would be repeated for the second triangle 120 (i.e., all x, y, z coordinates representing the main vertex 110a and the remaining vertices 120bx and 120cx). However, it is understood that this would result in repetition of data fields. For example, the fact that the coordinates of the main vertex 110a (which are common to both triangles 110 and 120) are repeated increases the length of bitstream 204 and reduces efficiency.

[0076] Therefore, a preferred technique is employed, according to which, for at least one dimension (x, y, z) (and in some examples, for each dimension of the acoustic environment), the coordinates of a first triangle (e.g., 110) can be written only once, and at least one previously encoded coordinate is referenced when encoding at least one coordinate of a subsequent triangle (e.g., 120). Figure 1 In the example, this technique can therefore be advantageously applied to each of the coordinates x, y, z of the principal vertex 110a of the second triangle 120 and each of the coordinates x, y, z of the vertex 110c of the second triangle 120 (this is not possible for the coordinates of vertex 120b, and their values ​​should be inserted into bitstream 204). This technique can mean, for example, referencing previously encoded vertex coordinates via shortcodes. It will be shown that the already encoded coordinates can be stored in an ordered candidate list 450 (with instances 450x, 450y, 450z for different dimensions), and they can be addressed simply by encoding ordered values ​​(e.g., indices) that are the ordered values ​​in the candidate list 450 associated with the previously encoded coordinates.

[0077] The example above can also be applied to individual coordinates of each vertex. For example, if a set of vertices has the same x, z, or y coordinates, they can be encoded by referencing the previous vertex (it's worth noting that the encoder can decide to decode them in a closed, sequential manner, so that the stored coordinates remain in the candidate list before being updated). For example, the coordinates (whether x, y, or z) can be written to bitstream 204 only for the first vertex being encoded, while subsequent vertices can be encoded by simply referencing previously encoded coordinates. For example, the y coordinates 110ay and 130ay of vertex 110a (in triangles 110 and 120) are the same (see...). Figure 1Therefore, the coordinate value 110ay is preferably written into bitstream 204 (and previously into the encoded vertex list 808), and subsequently referenced by encoding the value as an ordered value in ordered candidate list 450. Since the encoder and decoder update the candidate list in the same way (in a copy manner), they share knowledge of the values ​​in ordered candidate list 450 (and its instances 450x, 450y, 450z). More generally, it is understood that in ordered candidate list 450, the coordinates of previously encoded vertices are stored in association with ordered value 455 (instantiated by 455x, 455y, 455z).

[0078] Figure 4 The diagram illustrates a first candidate list instance 450x for the x-coordinate, a second candidate list 450y for the y-coordinate, and a third candidate list 450z for the z-coordinate. It can be seen that in candidate list instance 450x, the first value (associated with ordered value 0) is stored as 110ax, because it is the x-coordinate 110ax of the first vertex processed by the first triangle 110. In the second ordered value 1, the value 110bx is stored, which refers to the x-coordinate 110bx of vertex 110b of triangle 110. At the third ordered value 2 (third index), the value 110cx obtained from the x-coordinate 110cx of vertex 110c of the first triangle 110 is stored. In the fourth ordered value 3 (fourth index), the x-coordinate 120bx of vertex 120b of the second triangle 120 is written. This will similarly occur for the candidate lists 450y for the y-coordinate and 450z for the z-coordinate, respectively (an ordered value exists for each candidate list (i.e., for each dimension)). It should be noted that the ordered list 450 is dynamically supplemented (stored) during the encoding of bitstream 204 (or the encoded version of structured acoustic data 422). In the example, this ordered candidate list 450 is supplemented whenever the encoded version 222 of structured acoustic data 221 is generated. For example, when the x-coordinate 110ax is written into the encoded version 222 of structured acoustic data 221 (and specifically the encoded vertex list 808), the values ​​110bx, 110cx, and 120bx still do not exist in the candidate list instance 450x for the x-coordinate. Therefore, the ordered candidate list 450 (and its instances 450x, 450y, and 450z) is dynamically updated while the encoded version 222 of structured acoustic data 221 is generated (and more specifically, the encoded vertex list 808 is generated).

[0079] Figure 4The encoding of vertex 130a is also shown. Since vertices 110a and 130a have the same y-coordinate (but different x and z coordinates), the y-coordinate of the vertex is not repeated in the candidate list instance 450y for the y-coordinate (and in fact, candidate list instance 450y stores fewer coordinates compared to candidate list instances 450x and 450z). Furthermore, although the same coordinate value 110ay is repeated in the y-coordinates of both vertices 110a and 130a! Additionally, when encoding version 222 of the structural acoustic data 221 (or more generally, when encoding version 808 of the vertex lists 400, 804), it is possible to substantially reference index 0 of candidate list instance 450y, which typically has a shorter bit length compared to longer codes.

[0080] Figure 4 An example is shown of an encoded version 222 of the structured acoustic data 221 to be written into bitstream 204 (and specifically encoded versions 808 and 3808 of the vertex list). Each encoding of each vertex includes a mask 160 for each vertex, indicating whether the coordinate values ​​are actually encoded or only encoded by reference to the ordered values ​​(450) stored in the candidate list (450x, 450y, 450z). In this case, the mask 160 is represented as three binary values ​​160x, 160y, and 160z, each indicating binary information selected among the following:

[0081] Encoding of vertex coordinates; and

[0082] Encoding of ordered values ​​(indices) from an ordered candidate list.

[0083] When encoding the principal vertex 110a of the first triangle 110, there are actually no other vertices previously stored in the candidate list 450: this means the candidate list 450 is empty, so it's impossible to reference the position 455 of any previously encoded coordinates. Therefore, all binary values ​​160x, 160y, and 160z in the mask 160 are 0 (imagine: 0 means the coordinates will be encoded in encoded version 222 of the structure acoustic data 221, while a binary value of 1 means only the ordered values ​​of the y-coordinate list 450 are encoded, but binary values ​​may have opposite meanings in different examples). Subsequently, vertex index 403 (or another identifier for the vertex) is encoded, and coordinate values ​​110ax, 110ay, and 110az are also encoded in the coordinate value data field 170c. The same operation is repeated to encode the remaining vertices 110b and 110c.

[0084] Figure 4The encoding of vertex 130a is also shown. Since vertices 110a and 130a have the same y-coordinate (but different x and z coordinates), it is not necessary to repeat the encoding of the y-coordinate value of vertex 130a. It can be seen that its value 110ay is already stored at the first position of instance 450y in candidate list 450. For this purpose, the ordered value 0 is inserted into the encoded version 222 of the structural acoustic data 221 (or more specifically, into the encoded versions 808 and 3808 of the encoded vertex list, and into the bitstream). It can be seen that for binary values ​​160x and 160z, the mask 160 is 0, but for binary value 160y, the mask 160 is 1. In fact, the ordered value data field 170v (carrying the ordered value 0, which is the ordered value of the reference coordinate 110ay in candidate list instance 450y) is subsequently encoded instead of its length coordinate value. As indicated by the binary values ​​160x and 160z, the coordinate values ​​130ax and 130az are not referenced by the ordered values ​​in the coordinate value data field 170c, but by all coordinate values.

[0085] In the example, vertex 110a is not encoded twice in encoding version 222 of the structural acoustic data 221 (or more specifically encoding versions 808, 3808 of the vertex list). Simply put, for both triangles 110 and 120, the triangle list will refer to the same vertex 110a.

[0086] Generally, for each coordinate of each vertex (the primary vertex or the remaining vertices), the structural acoustic data encoding block 220 (and specifically the vertex list encoder 800) encodes a value selected from the following:

[0087] - The value of the coordinates; and

[0088] - The ordered values ​​of the previously encoded coordinates (455x, 455y, 455z) (and thus previously encoded in encoding version 222 to be written into bitstream 204, or more specifically, previously encoded in encoding versions 808, 3808 of the encoded vertex list).

[0089] Based on whether the previously encoded coordinates are in the candidate list 450, a choice can be made between encoding the value coordinates and encoding the ordered values ​​455 (455x, 455y, 455z).

[0090] Of course, if the second vertex of a triangle shares the same coordinate axes as another triangle (but does not coincide), then the binary values ​​in fields 160x, 160y, and 160z of mask 160 can be different (because one or two coordinates may actually be encoded in encoded version 122 of the structural acoustic data 221), and at least one binary field should be 1 (and it should indicate which ordered value in encoded version 222 of the polygon data 221). This is the case for vertex 130a, which shares the same y-coordinate as vertex 110a. For this purpose, for each vertex, there may be some coordinates that can be written into encoded version 222 of the polygon data 221, while coordinates that have already been perfectly written (and stored in the ordered candidate list 450) can simply be defined as ordered values.

[0091] Candidate list 450 can be updated, for example, by excluding less frequent coordinate values ​​from candidate list 450 (e.g., by encoding more frequent coordinate values). Additionally or alternatively, candidate list 450 can be updated by allowing the last coordinate value encoded in bitstream 204 to take over from previously encoded coordinate values. These techniques can be combined: for example, a ranking can be established among the already encoded coordinates based on a score assigned to each already encoded coordinate based on a hybrid criterion that includes the frequency of coordinate encoding (by increasing the score of the most frequent coordinate) and the freshness of the coordinate (by increasing the score of the last coordinate), thereby assigning the first position in candidate list 450 (associated with a smaller bit length) to those encoded coordinates with higher scores, and excluding those encoded coordinates with lower scores from candidate list 450, up to excluding those encoded coordinates with the lowest scores.

[0092] Different polygons (e.g., triangles) that share the same coordinates with each other in a short, continuous manner can also be encoded to increase the probability that coordinates already exist in the candidate list 450. More generally, the encoding of the structural acoustic data 221 can be ordered such that polygons (e.g., triangles) with shared coordinates are encoded with closer step sizes compared to polygons (e.g., triangles) with vertices that do not share the same coordinates. In the example, the order of encoding can be chosen such that the more common coordinates, the closer the vertex encodings are.

[0093] With the help of the above techniques, the dimensions of the most frequently used indexes (ordered values) 450 are greatly reduced, which means that the encoded version 222 of the structured acoustic data 221 is also compressed and the length of the bitstream 204 is reduced.

[0094] It should be noted that, here, as described above, it was envisioned that the values ​​in the ordered candidate list 450 be dynamically encoded by subsequently updating the ordered candidate list. This could happen, for example, in the case of streaming.

[0095] Therefore, the structured acoustic data encoding block 220 can use an ordered candidate list 450 for at least one dimension (x, y, z) of the acoustic environment (or bounding box, see below), in which the coordinate values ​​of previously encoded polygon vertices are stored in order (index, ordered value 450). If the coordinate value of a current primary polygon vertex or other polygon vertex is the same as a coordinate value of a previously encoded primary polygon vertex or other polygon vertex stored at the determined ordered value in the candidate list, the structured acoustic data encoding block 220 can encode the ordered value of the candidate list 450. If the coordinate value of a current polygon is different from any coordinate value of a previously encoded primary polygon vertex or other vertex stored in the candidate list 450, then that coordinate value is encoded in the bitstream 204. Furthermore, the structured acoustic data decoding block 320 of the encoder 200 can also use an ordered candidate list for at least the same dimensions, in which the coordinate values ​​of previously decoded primary polygon vertices or other polygon vertices are stored in order. With the bitstream 204 already encoding specific ordered values ​​of the candidate list in the structured acoustic data decoding block 320, the structured acoustic data decoding block 320 can reconstruct the coordinate values ​​as values ​​associated with the ordered values ​​and store them in the candidate list 450.

[0096] Basically, the candidate list 450 at decoder 300 can be understood as a copy of the candidate list 450 at encoder 200.

[0097] The structure acoustic data encoding block 220 of encoder 200 can encode binary mask values ​​(160x, 160y, 160z) for at least one dimension (but preferably for each of the three dimensions), the binary mask values ​​indicating whether ordered values ​​or coordinate values ​​in the candidate list are encoded (in fields 170c or 170v). Furthermore, the structure acoustic data decoding block 320 of decoder 300 can estimate binary mask values ​​160 (160x, 160y, 160z) for each vertex, the binary mask values ​​indicating whether ordered values ​​or coordinate values ​​in the candidate list are encoded in the bitstream (204). Therefore, the structure acoustic data decoding block 320 can determine whether each coordinate is encoded as a coordinate value or an index (ordered value).

[0098] As mentioned above, if two vertices have only one or two identical coordinates (such as vertices 130a and 110a), then only the ordered values ​​(indexes) will be encoded / decoded for the identical coordinates, rather than the coordinate values ​​themselves. For the one or two different coordinates, the coordinate values ​​will be encoded / decoded independently.

[0099] As described above, the candidate list 450 can be divided into instances 450x, 450y, and 450z that can be processed independently. In the example, the structured acoustic data encoding block 220 is configured such that:

[0100] For the first dimension, if the coordinates of a current vertex are the same as the coordinates of a previously encoded vertex stored in the determined ordered values ​​of a candidate list instance related to the first dimension, the ordered values ​​of the candidate list instance are encoded, and

[0101] For the second dimension, the coordinate values ​​are encoded if the coordinate values ​​of the current vertex are different from any coordinate values ​​of a previously encoded vertex stored in the candidate list instance associated with the second dimension.

[0102] Similarly, at decoder 300:

[0103] For the first dimension, if the coordinates of a current vertex are the same as a previously decoded vertex in the determined ordinal values ​​stored in the candidate list instance associated with the first dimension (and this can be signaled in bitstream 204 (e.g., in a first binary value in the binary values ​​of mask 160)), the ordered values ​​of the candidate list instance are decoded, and

[0104] For the second dimension, the coordinates are decoded if the coordinates of the current vertex are different from any coordinates of a previously decoded vertex stored in the candidate list instance associated with the second dimension (and this can also be signaled in bitstream 204 (e.g., in a second binary value in the binary value of mask 160)).

[0105] Triangle list encoding / decoding

[0106] Figure 6aThis illustrates how the coded triangle list 852 is encoded. As illustrated, each triangle is encoded by linking the vertex indices of vertices from the vertex list (compressed forms 804, 3804, 400) to the acoustic characteristics of the material. For each triangle, the vertex indices of vertices can be encoded from either the vertex list in the compressed form (808) or from the indices in the second candidate list (also referred to here as the Moving Forward (MTF) list). The second candidate list (MTF list), which will be described in detail later, contains vertex indices that have been used previously. If a vertex index is already in the second candidate list, its position is encoded. Otherwise, the vertex index from the coded vertex list 808 is written. Symbols associated with different positions can have their bit lengths increased with distance from the first position. Whenever a vertex index is written instead of a position in the second candidate list, an additional value (which can be longer than any other value in the second candidate list) can be encoded. Figure 6a An example is provided. At step 602, the vertices to be written into bitstream 204 are obtained. At step 604, it is estimated whether the vertex index is already in the second candidate list (MTF list). If the vertex index is not in the second candidate list (MTF list), the symbol (e.g., 0b11111111) is written into the bitstream, and subsequently, the vertex index obtained from the encoded vertex list 808 or from the original vertex 804 is also written into the bitstream. At step 612, the second candidate list can be updated by writing the vertex index written in the bitstream (or more generally, in the encoded triangle list 852). Furthermore, the count of the occurrences of vertex indices can be modified by appropriately modifying the histogram associated with the vertex indices. Note that the order of steps 610 and 612 can be reversed or inverted. If at step 604 it is determined that the vertex index is already in the second candidate list (MTF list), its position is written into the bitstream (or more generally, the encoded list 852). Furthermore, in this case, the histogram and MTF list can be modified at step 608. Additionally, the order of steps 606 and 608 can be reversed. Subsequently, the new vertex indices can be encoded. It should be noted that both position and vertex index can be encoded using so-called arithmetic encoding, which requires both the encoder and decoder to know the histogram of probabilities for each vertex index.

[0107] Figure 6bAn example of triangle list decoding is shown. At step 3602, the new vertex index is decoded. At step 3604, it is determined whether the vertex index is already in the MTF list. This can be determined, for example, by examining the symbols in the bitstream. If the bitstream has symbols indicating that the vertex index was fully written (rather than its position from the second candidate list), steps 3610 and 3612 are called. At step 3610, the vertex index is read and written in the decoded version of the triangle list. At step 3612, the histogram and MTF list are updated. If the estimate at step 3604 provides that the vertex index is already in the second candidate list (MFT list), steps 3606 and 3608 are called. In step 3606, the position of the indexed vertex (the point at a specific ordered value in the second candidate list) is read from the second candidate list, and its value is read from the candidate list. At step 3608, the histogram in the MTF list is modified. In steps 3608 and 3612, modifying the histogram means that the probability of having that particular vertex index increases. When it is mentioned that the MTF list has been modified, it means that the vertex index has been inserted into the first position (the lowest position in the second candidate list). Steps 3606 and 3608 can be reversed. Steps 3610 and 3612 can be reversed.

[0108] Essentially, both the encoder and decoder include a second candidate list (MTF list), and the decoder's second candidate list is understood as a copy of the encoder's second candidate list. Basically, these operations are identical except for the fact that the encoded triangle list 852 is encoded in one case and decoded in another.

[0109] Figures 10a to 10g Examples of operation at the triangle list encoder 850 are shown (these can be easily adapted to operation at the triangle list decoder 3850). The encoder is referenced only for clarity, but the same examples can be reported for the decoder. Figure 10a The diagram shows initialization step 0, where the second candidate list (MTF list) 1450 has no values. Figure 10b In step 1, the first vertex indices 0 and 10 are encoded. These are both in the second candidate list 1450, and the histogram 1460 is updated. Specifically, the second candidate list 1450 has values ​​0 and 5 to be encoded at its first position. The histogram is associated with the occurrence count of 1 for each of the values ​​0 and 6. The occurrence count should be understood as being associated with probability. Figure 10c In the first step, vertex 5 is encoded. The second candidate list 1450 and histogram 1460 are then updated. In the second candidate list 1450, it can be seen that value 5 occupies the first position, while values ​​0 and 10 are moved to less important positions in the second candidate list. Figure 10d In the next step, vertex 7 is encoded. Vertex 7 is placed in the second candidate list 1450, and histogram 1460 is updated. Figure 10e In this process, vertex 5 will be written again. Therefore, in the encoded version 852 of the triangle list (and subsequently in bitstream 204), position 1470 (indicated by 0b10) of value 5 is encoded (it can be seen that value 5 is in the second position). At this point, vertex 5 also occupies the first position in the second candidate list (MTF list) 1450. It should be noted that, due to... Figure 6a The position was written with symbol 1470 as in step 606, therefore Figure 10e In essence, it shows Figure 6a Step 606 of the method shown. It should be noted that the position occupied is the second position before updating the second candidate list 1450 (step 608, as shown). Figure 10a (As shown). Figure 10f This simply illustrates how to encode other code. Figure 10g An example is shown where vertex 8 should be encoded, but 8 is not in candidate list 1450 (the candidate list is full). This is Figure 6a Example of step 610. In this case, code 0b11111111 (or another code indicating the same situation) is encoded to indicate that the code is not in the second candidate list 1450. It can be seen that the second candidate list 1450 is updated by placing the value 8 in the first position and by excluding (popping) the last value in the list.

[0110] Figure 10h An example of code associated with a position is shown. It can be seen that the first position is associated with 0b0 as the shortest code, while the final code 0b11111111 indicates that the position is not encoded, but the value of the vertex index will be encoded. The same applies to the decoder, but in this case, reading the encoded value should reveal whether a specific vertex index is searched from the second candidate list (MTF list) 1450 or whether the vertex index is encoded.

[0111] In any case, arithmetic coding can be used, where shorter codes are assigned to more repeating index values ​​to be encoded.

[0112] bounding box

[0113] By intelligently changing the spatial coordinate system from the original spatial coordinate system to a coordinate system with advantageous characteristics, the length of data to be encoded in encoding version 222 and / or bitstream 204 can be reduced.

[0114] For example, the coordinate system can be changed to bring the origin closer to the polygon vertices, thereby reducing the distance from the origin (and the length of the coordinates). For example, the volume to be encoded into the bounding box can be reduced, especially without excluding vertices. Specifically, the bounding box can be contained within an acoustic environment, and the structural acoustic data to be encoded in bitstream 204 and / or version 222 is encoded with reference to a spatial coordinate system defined by a specific vertex of the bounding box.

[0115] Figure 5 An example of a bounding box 500 is provided. The bounding box can be a parallelepiped volume (or more generally, a polyhedral volume), but in general it can be exemplified as a prism volume (especially with a rectangular base) and in some cases it can be a cube. Figure 5 Bounding box 500 is shown, and only polygon 510 contained within bounding box 500 is shown. Polygon 510 (triangle) has a principal vertex 510a and two remaining vertices 510b and 510c. Polygon 510 is contained within bounding box 500 (generally, bounding box 500 is selected to include all polygons). Figure 5 For simplicity, no other triangles are shown in the diagram. The bounding box 500 can be signaled, for example, by writing its positional and / or orientation features. For instance, the position of a specific vertex 502 (e.g., a vertex closer to the original origin of the coordinate system) can be encoded. In some cases, the other vertices of the bounding box 500 and / or the orientation information of the bounding box 500 can be encoded in the bitstream. Explicitly signaling the shape and orientation of the bounding box 500 allows the decoder to reconstruct the positions of vertices 510a, 510b, and 510c relative to the bounding box and, consequently, relative to the origin of the original axis. The bounding box 500 can also form a new spatial coordinate system through translation, rotation, or more generally, rotation-translation. Figure 5 In the simplest case, the coordinates along dimensions y and z remain the same between the old and new coordinate systems, but x is shifted by an amount of "bounding_box_min" because the origin is moved to vertex 502 corresponding to bounding box 500. This is advantageous when there are no vertices in the gap between vertex 502 and the origin O of the original spatial coordinate system. Therefore, when the coordinates of vertices 510a, 510b, and 510c of polygon 510 are encoded (in version 222 and / or bitstream 204), the bit length of the coordinates in the x-direction is reduced. Thus, bounding box 500 can be defined such that it contains all the vertices to be encoded, but the space between the original origin of the coordinate system and the new origin of the coordinate system (which in this case corresponds to vertex 502 of bounding box 500) is reduced. Essentially, the change of coordinate system is achieved by reducing the length of the coordinates to be encoded in bitstream 204 and / or version 222 of the structured acoustic data.

[0116] Additionally or alternatively, other kinds of optimizations associated with the definition of the bounding box can be performed. For example, possible repeating patterns can be estimated. If at least one repeating pattern exists in the acoustic environment, the bounding box can be restricted to at least one repeating pattern. The repeating pattern can be, for example, a symmetrical pattern. For example, the symmetry can be radial symmetry or planar symmetry (other symmetries are also possible). In the case of symmetry (or more generally, repeating patterns), it is not necessary to encode all vertices of all polygons, because only the coordinates of the repeating pattern (e.g., half of the symmetric volume in the case of planar symmetry) can be encoded to reduce the number of vertices to be encoded in bitstream 204. Basically, the bounding box 500 can be defined as containing the repeating pattern once, without re-encoding other repeating patterns (e.g., in the case of planar symmetry, the bounding box 500 needs to contain more than just half of the symmetric volume from the symmetry plane toward one of the two directions). In bitstream 204, the repeating pattern data is signaled (e.g., in the case of planar symmetry, the symmetry data is encoded such that encoder 200 can reconstruct the shape of the represented acoustic object; for example, in the case of planar symmetry, information defining the plane of symmetry can be simply provided, allowing decoder to reconstruct the final shape by re-inserting the unencoded half of the symmetric shape). The same operation can be performed when the repeating pattern is a periodic shape: the bounding box can be limited to the periodically repeating shape, and the repeating pattern data can include information allowing decoder to reconstruct the final shape of the acoustic object (e.g., including spatial periodicity, such as in three dimensions, etc.). The same operation can also be applied in the case of variable symmetry, where only the angular shape is defined according to the variable symmetry, and the repeating pattern data is signaled with respect to the symmetry point and / or symmetry radius, allowing decoder 300 to reconstruct the final radially symmetric shape based on the symmetry data.

[0117] As will become apparent in the following paragraphs, a bounding box can also be defined such that when the spatial coordinate system is changed to the spatial coordinate system defined by the bounding box, the coordinates whose values ​​have a greatest common divisor greater than 1 are maximized.

[0118] In the example, the audio source encoding block 210 of encoder 200 can thus define the bounding boxes contained in the acoustic environment and encode the structural acoustic data within the bounding boxes, thereby avoiding writing the structural acoustic data into regions outside the bounding boxes. The bounding boxes can exclude portions of the acoustic environment that do not contain any principal vertices and any other polygon vertices. Information about the bounding boxes, including their position data, is signaled in bitstream 204 to allow the bounding boxes to be located in the acoustic environment. The structural acoustic data can thus undergo a change in coordinate system to a new coordinate system defined by the bounding boxes, and the coordinates of the polygon vertices can thus be encoded with reference to this new coordinate system. Furthermore, the audio source decoding block 310 of decoder 300 can read information about the bounding boxes (and specifically, position data) in the auxiliary information of bitstream 204. Therefore, the audio source decoding block 310 can locate the bounding boxes within the environment. In addition, the audio source decoding block 310 can decode the structural acoustic data within the bounding boxes, and based on the location performed using the bounding box position data, the audio source decoding block 310 can reconstruct the position data of the bounding boxes in the environment. For example, by performing a coordinate change opposite to the coordinate change performed at encoder 200, audio source decoding block 310 performs a coordinate change from the coordinate system defined by the bounding box to the original coordinate system of the environment.

[0119] As described above, the audio source encoding block 210 of encoder 200 can also estimate whether at least one repeating pattern exists in the acoustic environment and constrain the bounding box to at least one repeating pattern. Therefore, repeating pattern data can be signaled in bitstream 204. In this case, the audio source decoding block 310 of decoder 300 can reconstruct at least one acoustic object by repeatedly applying (e.g., by symmetry, by periodicity, etc.) the repeating pattern within the bounding box.

[0120] For example, at least one repeating pattern can be a symmetrical pattern (e.g., a planar symmetrical pattern), and the repeating pattern data can therefore be symmetrical data that can be signaled in bitstream 204 (e.g., positional data indicating the position and / or orientation of the symmetrical plane). Furthermore, the audio source decoding block 310 of decoder 300 can reconstruct at least one object by generating structural acoustic data symmetrically at locations symmetrical with respect to the positions of the main vertex and the remaining polygon vertices in the bounding box (e.g., relative to the symmetrical plane).

[0121] Greatest common divisor

[0122] In the example, with or without a bounding box, the encoder may search for vertices that share a common greatest common divisor in at least one coordinate. Therefore, the encoder 200 is also configured to encode segmented versions of the coordinate values ​​by searching for at least one common divisor of the coordinates of the main polygon vertices or the remaining polygon vertices, for a specific dimension of the acoustic environment to be encoded and for multiple main polygon vertices or the remaining polygon vertices.

[0123] exist Figure 5 In the middle, we can consider the x-coordinate of vertex 510a. a x-coordinate of vertex 510b b (For example, coordinate x) a and x b (It may have already been discretized into integers). If the coordinate x a and x b If the greatest common divisor g > 1, then x a and x b Factorize x so that a =g*d a And x b =g*d b , and among them, d a and d b Both are integers, d a <x a And d b <x b And g <x a And g <x b Therefore, instead of (in the encoded version 222 of the structural acoustic data and / or in bitstream 204) large numbers (with long bit lengths) x a and x b Encoding can be performed on small numbers (with short bit length) g, d a and d b Encoding is performed. Therefore, a suitable reduction in bit length can be achieved, especially when searching for the greatest common divisor of multiple vertex coordinates.

[0124] In the example, the audio source encoding block 210 of encoder 200 can thus search for a common factor different from (greater than) 1 among the coordinates of multiple main polygon vertices or remaining polygon vertices for at least one dimension of the environment or bounding box, thereby encoding the common factor and the result of dividing the coordinates by the common factor in bitstream 204. Therefore, in this document, bitstream 204 has already encoded at least two coordinate values ​​of at least two distinct vertices in factorial form according to the common factor. This is signaled in bitstream 204 (also encoding the common factor). Furthermore, the audio source decoding block 310 of decoder 300 can reconstruct the at least two coordinate values ​​encoded in factorial form by multiplying each of the at least two coordinate values ​​by the common factor.

[0125] exist Figure 4 The diagram shows a bitstream 204 with a coordinate value data field 170c encoding coordinate values ​​and an ordered value data field 170v encoding ordered values ​​of a candidate list 450. The value data field 170c typically has a longer bit length than the ordered value data field 170v.

[0126] Quantification

[0127] It should be noted that the structure acoustic data encoding block can pre-quantize the structure acoustic data 221, thereby eliminating duplicate vertices and degraded polygons. When this applies, the examples discussed above (see [reference]) Figure 1 and Figure 4 Basically, it can be managed through quantification.

[0128] discuss

[0129] The new triangular mesh encoding method consists of several stages, each contributing to improved efficiency. Strictly speaking, not all stages need to be executed together.

[0130] The first stage uses an encoder-selectable quantization step to uniformly quantize vertex coordinates and excludes all duplicate vertices and all duplicate and degraded triangles. The second stage computes the bounding box of the entire vertex list. The third stage acts as a preprocessor, detecting implicitly reduced coordinate precision, meaning all vertex coordinates are multiples of some integer in each dimension. The fourth stage utilizes the geometry of a large number of vertices lying on a common plane parallel to the coordinate axes. The fifth stage refines the fourth stage by considering and creating statistical models of recent information for duplicate coordinates in each dimension. Each of these stages computes several model parameters to obtain the best representation found by the encoder, which uses a range encoder to efficiently encode these parameters along with the data itself as auxiliary information.

[0131] The first stage uses a selectable quantization step by the encoder to uniformly quantize the vertex coordinates. The quantization step size is typically chosen to be small enough that the quantization process does not introduce any acoustic artifacts, usually in the range of 1mm to 2cm. After quantization, all duplicate vertices and all duplicate and degraded triangles are eliminated. Depending on the algorithm used to generate the triangle mesh, identical duplicate vertices may appear many times.

[0132] The second stage calculates the exact bounding boxes for the entire vertex list to exclude any unused ranges from the encoding. The bounding boxes are encoded very efficiently, optimized for several frequently encountered patterns. One common pattern is that the bounding box coordinate ranges are symmetrical around zero, such as -150 and 150, where only the absolute values ​​are encoded once. This is suitable for acoustic environments or objects that are symmetrical relative to these coordinates. Another pattern is that the bounding box coordinate ranges have zero width, such as 150 and 150, where only one value is encoded once. This is suitable for acoustic objects that are completely flat.

[0133] The third stage acts as a preprocessor, detecting implicitly reduced coordinate precision, meaning that all vertex coordinates are multiples of some integer in each dimension. For example, if we assume the quantization precision is set to 1mm and all X-coordinates are actually represented as multiples of 10mm, then all quantized X-coordinates, and therefore all quantization sizes at the X-coordinates, are multiples of 10. Furthermore, if the coordinates are relative to the bounding box, complete translation invariance is achieved (e.g., all sizes can be multiples of 10, but coordinates can all be shifted by 1). When these common multiples differ from 1, they can be removed from the values ​​to narrow the data range. The common multiples of each coordinate are encoded very efficiently, optimized for some frequently encountered patterns, such as for values ​​of 1 (meaning no common multiple was found) and for values ​​exactly equal to the corresponding width of the bounding box (meaning that the coordinate has exactly two distinct values). Therefore, the cube aligned with the coordinate axes uses the width of the bounding box for each coordinate as a common multiple. Thus, after preprocessing, only values ​​of 0 and 1 are retained for each coordinate.

[0134] The fourth stage utilizes geometry where a large number of vertices lie on a common plane parallel to the coordinate axes. For example, if multiple vertices lie on a plane parallel to the X and Y coordinate axes, this means that all Z coordinate values ​​of these vertices are identical. The ways to utilize repeated coordinate values ​​on each axis are either to encode each coordinate value as an index into a list of previously encoded unique values, or to explicitly encode each coordinate value as a new value, which adds it to the list of previously encoded unique values. Indicating which of the two ways the value is actually encoded requires separate “mask” bits for each coordinate. An adaptive binary probability estimator is used to optimally encode these 3 “mask” bits for each vertex. If the number of distinct values ​​for each coordinate is significantly less than the number of vertices, the size of the encoding is significantly smaller, even with the overhead of the “mask” bits. The encoder can optimally determine whether this representation is more efficient than a direct representation based solely on the number of unique values ​​for the coordinates and the number of vertices.

[0135] The fifth stage refines the fourth stage by considering and creating statistical models of recent information for repeating coordinates separately in each dimension. The fourth stage uses a uniform distribution to encode the index of repeating values ​​into a list of previously encoded unique values. However, a large proportion of repeating values ​​map to recently used indices. A parameter, nr, representing the maximum number of recently used index values ​​to remember, is introduced to create a statistical model that is more efficient than encoding the last unique index value used by nr compared to all other index values. A forward-moving (MTF) list of nr+1 entries is used to track the values ​​of the most recent nr index values, with the last entry representing all other indices. If an index value is found in the MTF list, its position in the list is encoded, and that index value is moved to the beginning of the MTF list. Otherwise, the position nr in the list is encoded to indicate that the index was not recently used, followed by a uniform encoding of the index value itself. An adaptive probability estimator is used to encode the positions in the MTF list to best match the relative recent distribution. Increasing nr improves encoding efficiency; however, a smaller value of 8 for nr has already achieved near-optimal results, allowing for low-complexity implementation.

[0136] Alternative Examples

[0137] It should be noted here that all the alternatives or aspects discussed above, as well as all aspects defined by the independent claims in the appended claims, can be used individually; that is, there are no other alternatives or objectives different from the contemplated alternatives, objectives, or independent claims. However, in other embodiments, two or more alternatives or aspects or independent claims can be combined with each other, and in other embodiments, all aspects or alternatives and all independent claims can be combined with each other. The encoded signals of the present invention can be stored on a digital storage medium or a non-transitory storage medium, or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0138] Although some aspects have been described in the context of the apparatus, it will be clear that these aspects also represent a description of the corresponding method, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also indicate a description of the features of the corresponding block or item or the corresponding apparatus.

[0139] Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. Implementation may be performed using a digital storage medium (e.g., floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or flash memory) on which electronically readable control signals are stored, in cooperation with (or capable of cooperating with) a programmable computer system, such that the corresponding methods are executed.

[0140] Some embodiments of the invention include a data carrier having electronically readable control signals, capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0141] Typically, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. This program code may, for example, be stored on a machine-readable medium.

[0142] Other embodiments include a computer program stored on a machine-readable carrier or non-transitory storage medium for performing one of the methods described herein.

[0143] In other words, embodiments of the method of the present invention are therefore computer programs having program code for performing one of the methods described herein when the computer program is run on a computer.

[0144] Therefore, other embodiments of the method of the present invention are data carriers or digital storage media or computer-readable media on which a computer program is recorded, the computer program being used to perform one of the methods described herein.

[0145] Therefore, another embodiment of the method of the present invention represents a data stream or signal sequence of a computer program used to perform one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0146] Another embodiment includes a processing means, such as a computer or a programmable logic device, which is configured or adapted to perform one of the methods described herein.

[0147] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0148] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0149] The above embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and variations of the arrangements and details described herein will be readily apparent to those skilled in the art. Therefore, it is intended to be limited only by the scope of the appended claims and not by the specific details given by way of the description and explanation of the embodiments herein.

Claims

1. An apparatus (300) for decoding an acoustic environment (302), the acoustic environment (302) comprising at least one audio source and at least one audio object, the at least one audio object being represented by structural acoustic data linking position data (110ax, 120bx) of a polygon (110, 120) with acoustic properties of an acoustic material (150), wherein, For each polygon, the location data includes the positions of the vertices, and the device includes: Bitstream reader (330) for reading from bitstream (204) at least one audio stream (312) to be rendered and an encoded version (322) of structural acoustic data generated by at least one audio source in the acoustic environment (302). An audio source decoding block (310) is used to decode the at least one audio stream (312) representing the at least one audio source; A structural acoustic data decoding block (320) is used to decode the structural acoustic data (211). The device is configured to read information of a bounding box (500) contained in the acoustic environment, which is signaled in the bit stream (204), to locate the bounding box (500) in the environment, wherein the information of the bounding box (500) includes location data, and the device is further configured to decode structural acoustic data within the bounding box.

2. The apparatus of claim 1, further comprising: A renderer (350) is used to render an audio signal (301) obtained from the at least one audio stream (312, 212) based on the structural and positional relationship between the at least one audio source and the decoded structural acoustic data (211).

3. The apparatus of claim 1, wherein, The structure acoustic data decoding block (320) includes: a vertex list decoder (3800) for decoding a vertex list (3802) indicating the position of vertices, each vertex having a vertex index.

4. The apparatus of claim 3, wherein, The vertex list decoder (3800) is configured to: for each coordinate of each vertex, retrieve the vertex index or coordinate value of the previously encoded vertex associated with the ordered value in the candidate list.

5. The apparatus of claim 4, configured to read signaling in the bit stream (204), the signaling indicating whether the coordinate value or the vertex index is encoded in the bit stream.

6. The apparatus of claim 1, wherein, The structural acoustic data decoding block (320) uses an ordered candidate list for at least one dimension (x, y, z), in which the coordinate values ​​of previously decoded vertices are stored sequentially. The structured acoustic data decoding block (320) is configured to reconstruct the coordinate values ​​as the values ​​associated with the ordered values ​​stored in the candidate list (450) when the ordered values ​​of the candidate list are encoded in the bit stream (204).

7. The apparatus of claim 6, wherein, The structured acoustic data decoding block is configured to estimate binary mask values ​​(160x, 160y, 160z) for each vertex, the binary mask values ​​(160x, 160y, 160z) indicating whether the ordered value from the candidate list or the coordinate value is encoded in the bitstream (204).

8. The apparatus of claim 6, configured to: based on signaling from the bit stream, select between activating and deactivating the ordered candidate list for at least one dimension, thereby deactivating the ordered candidate list.

9. The apparatus of claim 6, configured to: determine the multiplicity of the coordinates of a vertex, so as to assign higher-ranking ordered values ​​and / or ordered values ​​with fewer bits to coordinates with higher multiplicity.

10. The apparatus of claim 6, configured to dynamically update the candidate list based on coordinate values ​​and / or ordered values ​​decoded from the bitstream.

11. The apparatus according to claim 6, wherein, The candidate list (450) includes one candidate list instance for each dimension (450x, 450y, 450z).

12. The apparatus according to claim 11, wherein, The structured acoustic data decoding block is configured as follows: For the first dimension, if the coordinates of a current vertex are the same as the coordinates of a previously decoded vertex stored at a determined ordered value in a candidate list instance associated with the first dimension, then the ordered value of the candidate list instance is decoded, and... For the second dimension, the coordinate value is decoded if the coordinate value of the current vertex is different from any coordinate value of a previously decoded vertex stored in the candidate list instance associated with the second dimension.

13. The apparatus of claim 1, configured to decode structural acoustic data using arithmetic coding.

14. The apparatus of claim 1, wherein at least one structural acoustic data is decoded using a second candidate list, and the at least one structural acoustic data is decoded from a position in the second candidate list according to the second candidate list.

15. The apparatus of claim 14, configured such that: if the at least one structural acoustic data is not in the second candidate list, the at least one structural acoustic data is read entirely from the bitstream.

16. The apparatus according to claim 14, wherein, The specific code has a longer bit length than the code used to indicate the position in the second candidate list.

17. The apparatus according to claim 14, wherein, The final decoded structural acoustic data is placed in the first position of the second candidate list, and the other decoded structural acoustic data in the second candidate list are shifted.

18. The apparatus according to claim 14, wherein, The code indicating the first position in the second candidate list has a shorter bit length compared to the code indicating the last position in the second candidate list.

19. The apparatus of claim 14, wherein the second candidate list is used to decode the polygon data list.

20. The apparatus of claim 19, wherein the second candidate list is used to decode the polygon data list, wherein the vertex index of the vertex is indicated in the vertex list.

21. The apparatus according to claim 1, wherein, The decoder is configured to reconstruct the position of each vertex based on information from the bounding box, including position data.

22. The apparatus of claim 1, configured such that: when the bitstream (204) signals that the acoustic environment has at least one repeating pattern, the at least one acoustic object is reconstructed by repeatedly applying the repeating pattern within the bounding box.

23. The apparatus of claim 22, configured such that: when the bit stream (204) signals that the at least one repeating pattern is a symmetrical pattern enclosed by the bounding box, at least one object is reconstructed by generating structural acoustic data symmetrically at positions symmetrical to the positions of the vertices in the bounding box.

24. The apparatus according to claim 23, wherein, The symmetry is planar symmetry, and the symmetry data signaled in the bitstream includes information associated with the symmetry plane, wherein the apparatus is configured to reconstruct the at least one object by generating structural acoustic data symmetrically at locations symmetrical with respect to the symmetry plane and the positions of the vertices in the bounding box.

25. The apparatus of claim 1, configured to: change the coordinates of the vertex from a bounding box space coordinate system defined by at least one defined vertex of the bounding box to the original coordinate system.

26. The apparatus of claim 1 is further configured to: when the bit stream (204) signals that at least two coordinate values ​​of at least two vertices are encoded in factored form according to a common divisor, multiply each of the at least two coordinate values ​​encoded in factored form by the common divisor to reconstruct the at least two coordinate values.

27. The apparatus according to claim 26, wherein, The common divisor is the greatest common divisor.

28. The apparatus according to claim 1, wherein, The polygon is a triangle.

29. An apparatus for encoding an acoustic environment, the acoustic environment comprising at least one audio source and at least one audio object, the at least one audio object being represented by at least one structural acoustic data, the at least one structural acoustic data linking positional data (110ax, 120bx) of a polygon (110, 120) to acoustic properties of an acoustic material (150), wherein, For each polygon, the structural acoustic data includes the positions of vertices (110ax, 110ay, 110az), and the device includes: An audio source encoding block (210) is configured to encode at least one audio stream to be rendered, the at least one audio stream being associated with the at least one audio source; A structural acoustic data encoding block (220) is configured to encode at least one structural acoustic data (221) to obtain an encoded version (222) of the at least one structural acoustic data (221). A bitstream writer (230) is configured to write an encoded version (222) of the at least one audio stream (212) and the at least one structured acoustic data (221) into a bitstream (204). The structure acoustic data encoding block (220) includes a vertex list encoder (800) for encoding a vertex list (802) indicating vertex positions, wherein each vertex has a vertex index.

30. The apparatus of claim 29 is further configured to: search for a common factor other than 1 among the coordinates of a plurality of vertices for at least one dimension, thereby encoding the common factor and the result of dividing the coordinates by the common factor in the bitstream (204).

31. The apparatus according to claim 30, wherein, The common divisor is the greatest common divisor.

32. The apparatus of claim 30, when subordinate to claim 29, is configured to: define a boundary block such that the number of coordinates having a greatest common divisor greater than 1 relative to the spatial coordinate system defined by the boundary box (500) is maximized.

33. The apparatus according to claim 29, wherein, The vertex list encoder (800) is configured to, for each coordinate of each vertex, choose whether to write the coordinate value in the encoded version (222) of the structure acoustic data (202) or to write the vertex index of the previously encoded vertex associated with the ordered value in the candidate list.

34. The apparatus according to claim 29, wherein, The vertex list encoder (800) uses an ordered candidate list for at least one dimension (x, y, z), wherein the coordinate values ​​of previously decoded vertices are stored sequentially in the ordered candidate list. The vertex list encoder (800) is configured to encode the ordered value of the candidate list when the coordinate value of a current vertex is the same as the coordinate value of a previously encoded vertex stored at a determined ordered value in the candidate list. If the coordinates of a current vertex are different from any coordinates of a previously encoded vertex stored in the candidate list, the coordinates are encoded.

35. The apparatus according to claim 34, wherein, The vertex list encoder (800) is configured to encode binary mask values ​​(160x, 160y, 160z) for at least one dimension, the binary mask values ​​indicating whether an ordered value in the candidate list or the coordinate value is encoded.

36. The apparatus according to claim 34, wherein, The vertex list encoder (800) is configured to: determine the multiplicity of the coordinates of the vertices, such that if the sum of the multiplicity is less than a predetermined threshold, the ordered candidate list is deactivated for at least one dimension, such that for at least the dimension in which the ordered candidate list is deactivated, all coordinates are encoded only by encoding the coordinate values.

37. The apparatus according to claim 34, wherein, The vertex list encoder (800) is configured to determine the multiplicity of the vertex coordinates so as to assign higher-ranking ordered values ​​and / or ordered values ​​with fewer bits to coordinates with higher multiplicity.

38. The apparatus according to claim 34, wherein, The candidate list is dynamically updated based on the coordinate values ​​and / or ordered values ​​encoded in the bitstream.

39. The apparatus according to claim 34, wherein, The ordered values ​​have different bit lengths, such that more frequent coordinate values ​​are assigned ordered values ​​with shorter bit lengths.

40. The apparatus according to claim 34, wherein, The candidate list (450) includes one candidate list instance (450x, 450y, 450z) for each dimension.

41. The apparatus according to claim 40, wherein, The structured acoustic data encoding block is configured as follows: For the first dimension, if the coordinates of a current vertex are the same as the coordinates of a previously encoded vertex stored at a determined ordered value in a candidate list instance associated with the first dimension, the ordered value of the candidate list instance is encoded, and For the second dimension, the coordinate value is encoded if the coordinate value of the current vertex is different from any coordinate value of a previously encoded vertex stored in the candidate list instance associated with the second dimension.

42. The apparatus of claim 29, configured to encode structural acoustic data using arithmetic coding.

43. The apparatus of claim 1 or 29, wherein at least one structural acoustic data is encoded using a second candidate list, wherein the at least one structural acoustic data is encoded as a position in the second candidate list according to the second candidate list.

44. The apparatus of claim 43, configured such that: if the at least one structural acoustic data is not in the second candidate list, the at least one structural acoustic data is encoded as a whole and signaled with a specific code.

45. The apparatus according to claim 43, wherein, The specific code has a longer bit length than the code used to indicate a position in the second candidate list.

46. ​​The apparatus according to claim 43, wherein, The final encoded structural acoustic data is placed in the first position of the second candidate list, and the other encoded structural acoustic data in the second candidate list are shifted.

47. The apparatus according to claim 43, wherein, The code indicating the first position in the second candidate list has a shorter bit length compared to the code indicating the last position in the second candidate list.

48. The apparatus of claim 43, wherein the second candidate list is used to encode the polygon data list.

49. The apparatus of claim 48, wherein the polygon data list is encoded using the second candidate list by indicating the vertex index of the vertex in the vertex list.

50. The apparatus of claim 29, configured to: define a bounding box (500) contained in the acoustic environment, and encode structural acoustic data within the bounding box (500), the position data of the bounding box (500) being signaled in the bitstream.

51. The apparatus of claim 50, configured to define the bounding box (500) by excluding at least portions of the acoustic environment that do not contain any vertices.

52. The apparatus of claim 50, configured to: estimate whether the acoustic environment has at least one repeating pattern in order to limit the bounding box (500) to the at least one repeating pattern; the apparatus is further configured to signal repeating pattern data in the bitstream.

53. The apparatus according to claim 52, wherein, The at least one repeating pattern is a symmetrical pattern, and the device is further configured to signal symmetrical data in the bit stream (204).

54. The apparatus according to claim 53, wherein, The symmetry is planar symmetry, and the symmetric data to be signaled in the bitstream includes information associated with the symmetry plane.

55. The apparatus of claim 50, configured to: change the coordinates of the vertex from the original spatial coordinate system to a bounding box (500) coordinate system defined by at least one determined vertex of the bounding box (500).

56. The apparatus according to claim 29, wherein, The polygon is a triangle.

Citation Information

Patent Citations

  • Audio decoder, audio encoder, method for providing a decoded audio signal, method for providing an encoded audio signal, audio stream, audio stream provider and computer program using a stream identifier

    CN110476207A

  • Dynamic acoustic rendering

    US6973192B1