Bit stream indicating voice in environment

Abstract

To provide a device for generating a bit stream.SOLUTION: In an encoder, a metadata generator 203 generates metadata of voice data of a plurality of voice elements indicating sound sources in an environment. Metadata includes acoustic environment data of an environment, and acoustic environment data describes a characteristic to affect sound propagation of the sound source in the environment. At least a part of acoustic environment data can be applied to a plurality of listening poses in the environment, and the characteristic includes both a static characteristic and a dynamic characteristic. A bit stream generator 205 generates a bit stream including metadata. The bit stream often further includes voice data representing the voice elements of the sound source in the environment. A decoder includes a receiver for receiving the bit stream and a renderer for rendering voice in a voice environment based on acoustic environment data and voice data of the voice element.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to bitstreams representing audio environments, and to devices for generating and processing such bitstreams, in particular but not exclusively to bitstreams representing virtual audio environments, for example for virtual reality applications. [Background technology]

[0002] In recent years, the variety and range of experiences based on audiovisual content has grown significantly, and new services and methods for using and consuming such content are continually being developed and introduced. In particular, many spatially interactive services, applications, and experiences have been developed to provide users with more engaging and immersive experiences.

[0003] Examples of such applications include virtual reality (VR), augmented reality (AR), and mixed reality (MR) applications, which are rapidly becoming mainstream, with many solutions targeting the consumer market. Also, various standards are being developed by various standards bodies. Such standardization activities are actively developing standards for various aspects of VR / AR / MR systems, e.g., streaming, broadcasting, and rendering.

[0004] VR applications tend to provide a user experience that corresponds to the user being in a different world / environment / scene, while AR (including mixed reality MR) applications tend to provide a user experience that corresponds to the user being in a current environment with additional information or virtual objects or information added. Thus, VR applications tend to provide fully immersive artificial worlds / scenes, while AR applications tend to provide partially artificial worlds / scenes overlaid on a real scene in which the user is physically present. However, these terms are often used interchangeably and overlap significantly. In the following, the term virtual reality / VR will be used to refer to both virtual reality and augmented / mixed reality.

[0005] Communicating audiovisual data, and especially audio data, representing an environment, and especially an audio environment, in a way that allows for a flexible representation to allow adaptation on the part of the user, for example to provide a VR experience, is a very challenging task. The communicated data preferably represents the environment in a way that can be used locally to render a dynamic experience that reflects changes in the (virtual) listening position and changes in the environment itself.

[0006] Much research has been done to arrive at advantageous approaches for efficiently communicating data representing such environments. Various proposals for suitable data streams and formats have been made, most of which involve individualized models where individual sound sources are presented separately and associated with metadata describing various characteristics such as the source's location. Also, general data describing the audio environment may be provided, such as data describing reverberation, decay, etc.

[0007] However, defining a bitstream format that provides efficient (e.g., reduced data rate) communication of such information is very challenging, and a favorable approach requires careful consideration and balancing of many issues, characteristics, and tradeoffs. The Moving Picture Experts Group (MPEG) has initiated a standardization approach to develop a standard called MPEG-I for bitstreams suitable for VR and similar experiences.

[0008] Therefore, improved approaches and data formats / bitstreams for supporting audio in immersive applications and services such as VR and AR would be advantageous. In particular, approaches / bitstreams / formats that enable improved operation, increased flexibility, reduced complexity, easier implementation, improved audio experience, reduced complexity, reduced computational burden, improved audio quality, lower data rates, improved tradeoffs, and / or improved performance and / or operation would be advantageous. Summary of the Invention

[0009] SUMMARY OF THE DISCLOSURE Accordingly, the Invention seeks to preferably mitigate, reduce or eliminate one or more of the above mentioned disadvantages singly or in any combination.

[0010] According to an aspect and optional features of the invention, there is provided an apparatus for generating a bitstream, the apparatus comprising: a metadata generator (203) for generating metadata for audio data of a plurality of audio elements representing sound sources in an environment, the metadata comprising acoustic environment data of the environment, the acoustic environment data describing characteristics influencing sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics comprising both static and dynamic characteristics; and a bitstream generator for generating a bitstream comprising the metadata.

[0011] This approach can improve the performance and operation of many applications, including immersive, flexible, and dynamic audiovisual applications (e.g., many VR and AR applications). This approach can improve the trade-off between different desires in many cases, such as the desire to provide accurate, complete, and / or dynamic data of an environment versus the desire to provide a low data rate bitstream. This approach can provide a high degree of flexibility in many cases, and can facilitate, improve, or even enable adaptation and customization on the rendering side. This approach can facilitate and / or improve the rendering of the sound of the environment, and in particular the dynamic rendering of a changing environment and / or a changing listening position.

[0012] This approach in particular may provide a carefully adapted audiovisual bitstream that is particularly suited to dynamic applications, where the bitstream provides a carefully selected and at least partially optimized data representation of an audio environment, including both the sound sources and the acoustic properties of the environment, and possibly even individual objects within the environment.

[0013] In some embodiments, the apparatus may further include an audio data generator configured to generate audio data for a plurality of audio elements representing sound sources in the environment, and the bitstream generator may be configured to include the audio data in the bitstream.

[0014] A static property may be a property that is time-invariant (at least over the time interval covered by the property value). A dynamic property may be a property that is time-varying (at least over the time interval covered by the property value).

[0015] In many embodiments, the at least one characteristic is position and orientation dependent. In many embodiments, the acoustic environment data of the at least one characteristic exhibits a position and / or orientation dependency. The position and / or orientation dependency may be a dependency on the orientation and / or position of the sound source and / or the orientation and / or position of the listener pose.

[0016] In some embodiments, the acoustic environment data in the bitstream is divided into a series of multiple data groups, at least some of the groups providing respective characteristic values ​​for properties that affect sound propagation.

[0017] According to an optional feature of the invention, the acoustic environment data includes a data group describing a data format of at least a part of a representation of a characteristic value of at least one of the characteristics influencing sound propagation, and a plurality of data groups each including data describing the at least one characteristic value using the representation.

[0018] In some embodiments, the acoustic environment data includes a data group describing a data format of one of the characteristics affecting sound propagation, and a plurality of data groups, each data group including data describing a characteristic value of that characteristic according to the data format.

[0019] A data group may be one or more data values, specifically a set of bits.A data group may be a set of data, specifically one or more data fields of a bitstream.

[0020] In some embodiments, the acoustic environment data includes a data group describing a frequency grid and a plurality of data groups each including data describing a frequency dependent one of the characteristics using the frequency grid.

[0021] The frequency grid may be a subdivision of the frequency range into frequency subranges, for example by defining center frequencies of the frequency subranges.

[0022] In some embodiments, the bitstream includes an indicator that indicates whether the bitstream includes a data group that describes a frequency grid.

[0023] In some embodiments, the data group includes an indication of a format of the data describing the frequency grid.

[0024] In some embodiments, the acoustic environment data includes a data group describing a frequency grid and a plurality of data groups each including data describing a frequency dependent one of the characteristics using the frequency grid, the bit stream including an indicator of whether the bit stream includes the data group describing the frequency grid, and the data group including an indication of the format of the data describing the frequency grid.

[0025] In some embodiments, the data group includes at least one of data indicative of a predetermined default grid, data indicative of starting frequencies and frequency ranges of at least some subranges of the frequency grid, and data indicative of individual frequencies.

[0026] At least one subrange or individual frequency represented by the data of the data group is aligned with a fraction of an octave band.

[0027] According to an optional feature of the invention, the acoustic environment data includes a data group describing a frequency grid and a plurality of data groups each including data describing a frequency dependent one of the characteristics using the frequency grid, the bit stream includes an indicator indicating whether the bit stream includes the data group describing the frequency grid, the data group including an indication of a format of the data describing the frequency grid, and the data group including at least one of data indicative of a predetermined default grid, data indicative of a start frequency and a frequency range of at least some sub-ranges of the frequency grid, and data indicative of individual frequencies.

[0028] In some embodiments, the acoustic environment data includes a data group describing an orientation representation format for representing the orientation characteristic and at least one data group including data describing the orientation characteristic of the characteristics using the orientation representation format.

[0029] The bitstream may include an indicator that indicates whether the bitstream includes a data group that describes an orientation representation format.

[0030] In some embodiments, the group of data includes at least one of: data indicative of a predetermined default orientation representation; data indicative of a predetermined set of angles; and data indicative of an angle on a quantization grid.

[0031] According to an optional feature of the invention, the acoustic environment data includes a data group describing an orientation representation format for representing an orientation characteristic and at least one data group including data describing an orientation characteristic of the characteristics using the orientation representation format, the data group including at least one of data indicative of a predetermined default orientation representation, data indicative of a set of predetermined angles, and data indicative of an angle on a quantization grid.

[0032] According to an optional feature of the invention, the acoustic environment data includes a first data field for a first bit representing a value of a first one of the properties influencing sound propagation, and a second data field indicating whether the acoustic environment data includes extension data for a second bit representing a value of the first property.

[0033] According to an optional feature of the invention, the second bit extends the range of values ​​that the first characteristic can take.

[0034] The second bit may be a more significant bit than the first bit of the data word that represents the value of the first characteristic.

[0035] According to an optional feature of the invention, the second bit increases the resolution of the possible values ​​of the first characteristic.

[0036] The second bit may be a less significant bit than the first bit of the data word representing the value of the first characteristic. The second bit may extend the precision of the values ​​that the first characteristic can take.

[0037] In some embodiments, the first characteristic is a characteristic selected from the group including a temporal characteristic, a spatial characteristic, an amount, a gain characteristic, a volume characteristic, a frequency characteristic, an index characteristic, and a discrimination characteristic.

[0038] According to an optional feature of the invention, the metadata generator (203) generates the acoustic environment data to include a global indicator indicating that the environment is a spatially constrained environment, the metadata generator restricting (at least one) data value of the acoustic environment data to conform to a predetermined restricted format for the data value for the global indicator indicating that the environment is spatially constrained.

[0039] The spatially restricted environment may be an environment having a spatial extent below a threshold. The predetermined restricted format may be a data format for at least some values ​​of the characteristics affecting sound propagation that uses fewer bits than a data format that would be used for the bitstream if the environment was not a spatially restricted environment.

[0040] The global indicator may be an optional indicator. In some embodiments, the global indicator may indicate whether the environment is a spatially constrained environment or a less spatially constrained environment (or an environment with no spatial constraints).

[0041] In some embodiments, the acoustic environment data includes an animation indication of at least a first audio element, the animation indication indicating whether at least one characteristic of the first audio element changes during the time interval.

[0042] According to an optional feature of the invention, the acoustic environment data includes an animated indication of at least a first audio element, the animated indication indicating whether at least one characteristic of the first audio element changes during a time interval, and for an animated indication indicating that the first audio element has at least one changing characteristic, the acoustic environment data includes data describing a variation of the at least one changing characteristic.

[0043] In some embodiments, if the animation indication indicates that at least one characteristic of the first sound element changes during the time interval, the acoustic environment data further includes a further animation indication for each of the at least two characteristics, the further animation indication indicating whether the corresponding characteristic is animated or not, and if the animation indication indicates that there are no characteristics of the first sound element that change during the time interval, the acoustic environment data does not include a further animation indication for the at least two characteristics.

[0044] In some embodiments, if the animation indication indicates that at least one characteristic of the first audio element changes during the time interval, further animation indications for at least two characteristics are present, the further animation indications indicating whether the corresponding characteristic is animated or not, and if the animation indication indicates that there is not at least one characteristic of the first audio element that changes during the time interval, the animation indication is excluded.

[0045] In some embodiments, the acoustic environment data includes at least two values ​​of at least one characteristic that varies during a time interval, and interpolation data describing a temporal interpolation for interpolating between the at least two values.

[0046] According to an optional feature of the invention, the audio elements include a plurality of sound effect elements, and the sound environment data includes data linking a user-controlled change to the environment with a first sound effect element of the plurality of sound effect elements.

[0047] In some embodiments, the acoustic environment data is arranged as consecutive data sets, each data set including data spanning a time interval.

[0048] The time interval varies for each data set.

[0049] According to an optional feature of the invention, the acoustic environment data is arranged as a plurality of consecutive data sets, each data set comprising data spanning a time interval, and a first of the consecutive data sets comprising a first characteristic value of at least one of the sound affecting characteristics and a time indication of the first characteristic value indicative of a time within the time interval represented by the first data set.

[0050] In some embodiments, a first one of the consecutive data sets comprises at least two characteristic values ​​of at least one of the sound affecting characteristics and a time indication of the at least two characteristic values, the time indication indicating a time within a time interval represented by the first data set.

[0051] According to an optional feature of the invention, the acoustic environment data is arranged as a number of consecutive data sets, each data set comprising data spanning a time interval, and the bitstream generator determines whether a characteristic value of a first one of the characteristics affecting sound propagation is provided for a default time within the time interval represented by the first data set, and if so includes the first characteristic value in the first data set without a time indication, and if not includes the first characteristic value in the first data set with a time indication for the first characteristic value.

[0052] In some embodiments, the acoustic environment data includes several complete rendering data sets that contain all the data necessary to render the sound of the environment, and several partial rendering data sets that require additional data from other data sets to render the sound of the environment.

[0053] In some embodiments, the acoustic environment data for at least some elements of the environment includes identification data and parent identification data for elements of a scene graph of the environment.

[0054] At least some of these elements may be, for example, objects, sound sources (sound elements), and / or acoustic properties of the environment.

[0055] According to an optional feature of the invention, the acoustic environment data of the first sound element includes an indication of a first applicability region and a second applicability region for a first characteristic value of a first characteristic of the characteristics affecting sound propagation, the first applicability region indicating a region of a position of the first sound element to which the first characteristic value applies and the second applicability region indicating a region of a listening position to which the first characteristic value applies.

[0056] According to an aspect and optional features of the invention, an apparatus for generating rendered audio is provided, the apparatus including: a first receiver (303) that receives audio data of a plurality of audio elements representing sound sources in an environment; a second receiver (305) that receives a bitstream including metadata of the audio data of the plurality of audio elements representing the sound sources in the environment, the metadata including acoustic environment data of the environment, the acoustic environment data describing characteristics affecting sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics including both static and dynamic characteristics; and a renderer (307) that generates output audio data of the environment in response to the audio data and the acoustic environment data.

[0057] According to an aspect and optional features of the invention, a bitstream is provided comprising metadata of audio data of a plurality of audio elements representing sound sources in an environment, the metadata comprising acoustic environment data of the environment, the acoustic environment data describing characteristics influencing sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics comprising both static and dynamic characteristics.

[0058] According to an aspect and optional features of the invention, there is provided a method for generating a bitstream, the method comprising the steps of generating metadata for audio data of a plurality of audio elements representing sound sources in an environment, the metadata including acoustic environment data of the environment, the acoustic environment data describing characteristics influencing sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics including both static and dynamic characteristics, and generating a bitstream including the metadata.

[0059] According to an aspect and optional features of the invention, there is provided a method for generating rendered audio, the method comprising the steps of receiving audio data for a plurality of audio elements representing sound sources in an environment; receiving a bitstream including metadata for the audio data for the plurality of audio elements representing the sound sources in the environment, the metadata including acoustic environment data of the environment, the acoustic environment data describing characteristics affecting sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics including both static and dynamic characteristics; and generating output audio data for the environment in response to the audio data and the acoustic environment data.

[0060] These and other aspects, features and advantages of the invention will be elucidated and elucidated with reference to the embodiment(s) described hereinafter. [Brief description of the drawings]

[0061] Embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which: [Figure 1] FIG. 1 shows an example of an audiovisual distribution system. [Diagram 2] FIG. 2 illustrates example elements of an encoding device according to some embodiments of the present invention. [Diagram 3] FIG. 3 illustrates example elements of a decoding device according to some embodiments of the present invention. [Figure 4]FIG. 4 illustrates an example of a data structure of a bitstream according to some embodiments of the present invention. [Diagram 5] FIG. 5 illustrates examples of varying property values ​​represented in a bitstream according to some embodiments of the present invention. [Figure 6] FIG. 6 illustrates a data structure of a bitstream according to some embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0062] The following description focuses on audiovisual applications, such as virtual reality (VR) applications, although it will be appreciated that the principles and concepts described can be used in many other applications and embodiments.

[0063] The following description focuses on generating a bitstream that provides an audio representation of an environment. In many examples, the audio representation is complemented by a representation of the visual environment, and the VR application is configured to generate both audio and video for presentation to the user. The audio representation may supplement the visual representation of the virtual scene / environment, or the bitstream may provide a representation of, for example, a real-world or hybrid environment. The bitstream may include data representing individual elements in the environment / scene, such as, for example, sound sources and objects. Additionally, more general information may be provided, such as general acoustic or visual data (e.g., data describing reverberation, background color, etc.).

[0064] However, it will be appreciated that the principles and concepts described can be used in many other applications and embodiments.

[0065] A virtual reality experience may enable a device, such as a computer, to generate a virtual experience for a user by rendering three-dimensional audio and video for a virtual scene and presenting it to the user. The user may typically move around, so the observer / listening pose may change dynamically. In many embodiments, the virtual scene / environment may be a dynamic scene, for example where objects move or change their shape and multiple sound sources emit sound at different times.

[0066] In the art, the terms configuration and pose are used as general terms to represent a position and / or a direction / orientation. For example, a combination of a position and a direction / orientation of an object, a camera, a head, or a view may be called a pose or configuration. Thus, an indication of a configuration or pose may comprise six values / components / degrees of freedom. Each value / component typically describes an individual characteristic of the position / location or direction / orientation of the corresponding object. Of course, in many situations, a configuration or pose may be considered or represented with fewer components, for example, when one or more components are considered to be constant or irrelevant (e.g., when all objects have the same height and are considered to be horizontal in orientation, it may be possible to fully represent the pose of an object with four components). Hereinafter, the term pose is used to refer to a position and / or orientation that can be represented with one to six values ​​(corresponding to the maximum possible degrees of freedom). The term pose may be replaced by the term configuration. The term pose may be replaced by the term position and / or orientation. The term pose can be replaced with the terms position and orientation (when the pose provides both position and orientation information), position (when the pose provides (possibly only) position information), or orientation (when the pose provides (possibly only) orientation information).

[0067] In many approaches, the VR application may be provided locally to the viewer, for example, by a standalone device without any remote VR server or even access to a remote VR server. In other applications, however, the VR application may be based on data received from a remote or central server. For example, audio or visual data may be provided to the VR device from a remote central server and processed locally to generate the desired VR experience.

[0068] 1 shows an example of a VR system in which a remote VR client device 101 communicates with a VR server 103 over a network 105, such as the Internet. The server 103 may be configured to support a potentially large number of client devices 101 simultaneously.

[0069] The VR server 103 can support the virtual reality experience, for example, by transmitting data to the client device 101 that defines elements of the virtual environment, and virtual objects. The data can specifically describe the visual features and geometric properties of a number of virtual objects that can be used by the client device 101 to generate graphics that can be presented to the user. In some embodiments, the data can also include various information that can be presented to the user. Additionally, the server 103 can provide audio data to the client device 103 that can be used to locally generate virtual sounds / voices that can further enhance the user experience, particularly the sense of immersion.

[0070] The following description focuses on the generation of audio bitstreams that provide a representation of an acoustic scene and environment (typically including both a representation of the sound sources and a representation of the acoustic properties of the environment).

[0071] Figure 2 shows an example of an apparatus for generating a bitstream, which in particular may be (included in) the server 103 of Figure 1. The apparatus may in particular be an encoder / transmitter. Figure 3 shows an example of an apparatus for receiving and processing a bitstream such as the bitstream generated by the apparatus of Figure 2. The apparatus of Figure 3 may be a decoder / receiver, in particular may be (part of) the client device 101 of Figure 1. In the following, the apparatus of Figure 2 is also called an encoder and the apparatus of Figure 3 is also called a decoder.

[0072] In this example, the encoder generates a bitstream describing the audio environment. In this specific example, the bitstream includes both audio data of sounds produced by audio sources in the environment and metadata describing the acoustic environment (typically including both metadata for the individual audio sources and metadata for the acoustic environment). However, in some embodiments, the audio data and metadata may be provided in separate bitstreams, and in particular a separate bitstream may be generated that includes only metadata for the audio data, with the actual audio data being provided separately in another bitstream. Indeed, in some embodiments the audio data may be provided from one source / provider and the additional metadata provided from another source / provider.

[0073] The encoder includes an audio data generator 201 configured to generate audio data for multiple audio elements representing multiple sound sources in the environment. The audio generator may generate the audio data from received audio data describing the audio from individual sound sources or audio elements representing, for example, sounds picked up by a microphone, or may generate the audio data itself, for example based on an audio model for the scene. In some embodiments, the audio data generator 201 may extract the audio data for a particular audio source from, for example, a local store that contains an appropriate representation of the audio from various audio sources. In other embodiments, instead or additionally, the audio data may be generated from, for example, a microphone input capturing live audio.

[0074] The audio data generator 201 may generate the audio data to conform to any suitable data format, and any suitable encoding, representation, etc. may be used. Thus, the audio data may be generated in any suitable manner, and may represent audio elements in any suitable manner.

[0075] The audio elements may be audio objects, audio clips, audio channels, first order or higher order Ambisonics (FOA, HOA), or other audio elements. In particular, each audio element may be represented by a set of audio data that characterizes the sounds that may be generated in the environment. The audio data is generally generated to include multiple sets of audio data for multiple different audio elements, and in particular multiple sets of audio data for multiple different sound sources represented by individual audio elements.

[0076] The encoder further includes a metadata generator 203 configured to generate metadata. The metadata includes acoustic environment data of the environment, which describes characteristics of sound sources in the environment that affect sound propagation. The characteristics of the acoustic environment data may include acoustic characteristics of the environment (e.g., reverberation, reflection characteristics, etc.), characteristics of objects in the environment that may affect sound propagation (e.g., object position, orientation, size, material, attenuation, reflection, etc.), or characteristics of sound sources / audio elements (e.g., position, orientation, size, volume, etc.).

[0077] The bitstream may include data groups of several data symbols / bits that together provide an indication of characteristic values ​​of various characteristics that affect the sound perceived in the environment, and may also include data groups that provide various other data, such as auxiliary parameters or format definitions for other data of the acoustic environment data.

[0078] A data group may be a simple bit sequence indicating a data value / format etc. or may be a more complex combination of data providing the appropriate information. A data group may, for example, be considered to correspond in many scenarios to one or more data fields of a bitstream. Data fields may contain sub-data fields, i.e. hierarchical organisation of data fields may be applied which are themselves combinations of data fields.

[0079] The acoustic environment data may include metadata describing, for example, the pose (position and / or orientation) of one or more sound sources represented by audio elements, acoustic properties of the environment or individual objects within the environment (e.g., (possibly frequency-dependent) attenuation, reverberation characteristics, object shape that may affect the propagation of sound waves, material properties such as sound absorption, acoustic reflection, acoustic scattering, acoustic coupling, or acoustic transmission parameters, etc.), signal references, reference distances, rendering control flags, sound effect characteristics, user interaction metadata, etc.

[0080] The acoustic environment data may include data that may be applicable to multiple listening poses, and in particular may include data that the renderer can use to adapt the rendering of the audio elements in a manner that depends (differently for each listener pose) on different listener poses (in particular different positions and / or orientations). Furthermore, the acoustic environment data may include data for both static and dynamic characteristics. In particular, the acoustic environment data may be time-independent and may include data describing time-invariant characteristics (values) at least in the time interval for which the data is provided (and thus such characteristics may be static). The acoustic environment data may further include data describing time-dependent and time-varying characteristics (values) at least in the time interval for which the data is provided (and thus such characteristics may be dynamic). For at least one characteristic, the acoustic environment data may include data describing the time change of (the value of) the characteristic. The characteristic may be a characteristic of one of the sound sources, the audio elements, and / or the sound propagation / sound propagation in the environment.

[0081] The encoder further includes a bitstream generator 205 configured to generate a bitstream including the audio data and metadata (or in some embodiments only the metadata). The bitstream may be generated to satisfy one or more aspects and features of a particular data format, which will be described in more detail below.

[0082] The bitstream generator 205 is coupled to an output processor 207 configured to output the generated bitstream. The output processor 207 may have the functionality necessary to communicate the bitstream as desired for a particular application. For example, the output processor may have a network interface, wireless functionality, WiFi circuitry, internet connectivity, etc.

[0083] The decoder shown in Figure 3 includes a receiver or input processor 301 configured to receive the bitstream generated by the encoder. The input processor 301 may have the functionality necessary to receive the bitstream as desired for a particular application. For example, the input processor 301 may have a network interface, wireless functionality, WiFi circuitry, Internet coupling functionality, etc. In many embodiments, the input processor 301 may have functionality complementary to that of the output processor 207 of the encoder.

[0084] The input processor 301 is coupled to an audio data processor 303 configured to extract audio data from the bitstream. The audio data processor 303 is therefore configured to extract, and often process, audio data representative of a number of audio elements of the environment. Typically, each audio element may correspond to one or more sound sources in the environment.

[0085] In the described example, the bitstream includes audio data describing the audio elements. In other embodiments, the audio data is not included in the bitstream and may be received from another source, such as an internal source of the decoder / client device in some cases. For example, an internal memory may store the audio data and the server 103 may provide additional metadata that may provide an enhanced experience, for example by providing information that enhances the audio to present dynamic animations of sound sources moving in the environment. In some embodiments, the input processor 301 may be configured to receive audio data from an external source different from the server, for example the audio data may be received as part of a different bitstream provided by a different server than the server 103 that provides the metadata.

[0086] The decoder further includes a metadata processor 305 configured to extract metadata from the bitstream. The metadata processor 305 is thus configured to extract, and often process, audio element / environment metadata. The metadata processor 305 may be configured to extract the data and generate appropriate property values ​​for one, multiple, or all of the properties described by the metadata.

[0087] The decoder in this example has a processor that processes the audio data based on the metadata and the property value. Specifically, the decoder may have a renderer 307 configured to render one or more audio elements based on a value of at least one of the properties represented by the metadata, where the value of the property is determined from the metadata. Typically, the renderer 307 may be configured to render an audio scene corresponding to an environment by rendering one or more audio elements based on the metadata, for example, pose (and changes thereto) determined by the metadata, frequency-dependent attenuation based on metadata describing the environment and objects in the environment, reverberation representing the reverberation characteristics of the environment described by the metadata, etc. may be used.

[0088] It will be appreciated that many algorithms, approaches, and techniques for rendering audio data based on environmental and contextual data are known to those skilled in the art (including, e.g., HRTF rendering, reverberation modeling, etc.) and for the sake of brevity and clarity, this will not be described in further detail.

[0089] The encoder is configured to generate a bitstream, and the decoder is configured to receive, decode, and process the bitstream according to a data format that includes one or more of the features and aspects described below. Although the approaches are described with reference to a particular bitstream / format that may include most or all of the specific features and characteristics, it will be understood that in many embodiments, a bitstream is generated that may include, for example, only some of the features or approaches. Indeed, the different features and elements of the described bitstream are not necessarily used together, but are individual features that may be used separately or combined in any way with other described features.

[0090] In a specific example, a metadata bitstream is generated that includes acoustic environment data organized as a series of multiple data sets (data sets may also be referred to as data frames). Each data set includes data for a time interval. The acoustic environment data may be organized such that data values ​​for a time interval are grouped together, specifically as a series of data sets in which all encoded audio data streams that apply only to a single time interval are included, and the data set does not include acoustic environment data that applies only to a time interval different from the single time interval. To obtain all data specific to a given time interval, a decoder need only decode the data set. In some embodiments, the acoustic environment data for a given time interval may also apply to other time intervals (e.g., some static acoustic environment data may apply to all time intervals). Such multi-interval data may be included in a data set for one of the multiple time intervals, in a data set for two or more of the multiple time intervals, in all data sets, or outside the data sets (e.g., a common data set provided separately from the data sets for the individual time intervals).

[0091] A bitstream may therefore contain multiple data sets for different time intervals, each data set being hereinafter also referred to as a data point of the bitstream.

[0092] The generated metadata bitstream is likely to include metadata that is often relatively static and has relatively slow animation / changing characteristics (e.g., position, orientation, signal gain). Thus, in this example, the bitstream is not organized as frames having durations in the range of a few milliseconds, but rather into data sets or data points that represent time intervals on a much longer time scale. The time intervals of the data sets / data points may be, for example, 0.5 seconds, 1 second, 2 seconds, or 5 seconds or more.

[0093] A bitstream may contain independent data points that contain enough data to begin decoding the bitstream when starting to read randomly or when multiple bitstreams are combined. The data in such independent data points does not depend on the data in preceding data points. An example bitstream using this approach is shown in Figure 4.

[0094] The duration of the time interval represented by each data set / point may be flexible and may vary from group to group. Typically, the duration may be on the order of a few seconds. The data points may be defined relative to a timestamp, which may specifically be a start point, an end point, or a midpoint for the data point. The metadata of the data points may represent data regarding the characteristics of the acoustic environment data of the time interval (e.g., specifically, data indicating a specific characteristic value). For example, the indicated value may be the value at the end of the time interval (e.g., where the next data point is carried over, or where the scene ends). Furthermore, data may be included that allows for reconstructing the characteristic value at other points in the time interval, if the value varies over time within the time interval. For example, values ​​within the time interval may be determined by interpolation.

[0095] In this example, the time interval may be indicated by a data value in the bitstream. Similarly, a data value within the time interval may be referenced and indicated as applying to the particular time interval indicated by the data field / value. Specifically, a value referred to as targetOffset[n] may be used, where n is an index.

[0096] With this approach, when a time change occurs, for example when an animation starts, stops, or changes speed within its range, this can be supported using a variety of approaches such as: - Waypoints - Update the data points

[0097] If a metadata field is animated and the animation needs to be updated multiple times within a data point's time interval, multiple waypoints can be included by including multiple targetOffsets within the data point's time interval. For example, - TargetOffset[0]=48000 - TargetOffset[1]=20000 - TargetOffset[2]=38643

[0098] A data point may contain target values ​​for one or more characteristics linked to a particular point in time within a time interval. That point in time is indicated by a time indication, specifically in the form of a targetOffset field. By providing multiple target values, each paired with a targetOffset reference, multiple waypoints can be generated whose values ​​are interpolated. Figure 5 shows how this can be used to provide varying parameter values.

[0099] Alternatively or additionally, dynamic changes may be supported by multiple update data points, which may be non-independent data points. An update data point may not provide enough data to allow complete rendering of its time interval, but may rely on data provided elsewhere in the bitstream. An update data point may include only a subset of the acoustic environment data that describes the characteristics during the time interval represented by the update data point. In some embodiments, an update data point may always be linked to at least one independent data point.

[0100] An updated data point may in some cases only include data related to time-varying characteristics (changing during a time interval). An updated data point may also have associated data indicating the time interval to which the updated data point applies, e.g., start time, end time, midpoint, etc. An updated data point may have a maximum time interval duration that does not extend beyond the next independent data point.

[0101] An advantage of updated data points may be that they are useful for live streaming of scene elements (eg, movements and actions of other users) where the corresponding fields are transmitted at a higher rate than independent data points.

[0102] Thus, the bitstream may include multiple consecutive data sets / data points containing one or more characteristic values ​​for characteristics of the environment that affect sound propagation. Additionally, a time indication (e.g., targetOffset) may be included to indicate when a characteristic value is appropriate. In some embodiments, the data set may include two or more characteristic values ​​for one or more characteristics. The data set may include a time indication for a variable characteristic indicating when the multiple characteristic values ​​should be applied.

[0103] In some embodiments, a time indication may be included only if it is different from a default time within the time interval. For example, the property values ​​may be shown by default for the end of the time interval, and no specific time indication may be provided for the property values ​​provided at the end of the time interval. However, property values ​​provided for other times within the time interval may be associated with a time indication.

[0104] Thus, in some embodiments, the bitstream generator 205 is configured to determine whether the characteristic value of a given characteristic is the one provided for a default time within the represented time interval, if so, the first characteristic value is included in the data point without a time indication, otherwise the characteristic value includes a time indication.

[0105] The bitstream may be generated to include both data sets that are independent and do not require additional data for rendering, and data sets that are not independent and require acoustic environment data from other data sets to enable a complete rendering of the acoustic environment. Thus, the acoustic environment data may include several complete rendering data sets that include all the data necessary to render the audio of the environment, and several partial rendering data sets that additionally rely on and require data from other data sets to render the audio of the environment.

[0106] With the notion that data points are independent and may cover gaps of up to several seconds, it may be useful to include both the beginning and end of the data point time interval (e.g., indicating where a data point begins and where an interpolation interval ends). This is optional, since in many cases it is acceptable to specify only the end point, and when a decoder jumps into the stream, potential deviations may only occur over the duration of the interval of the first decoded data point.

[0107] In many cases, the end of the source pose may be considered the most important information, and this may include position and orientation data. In addition, an optional starting position / orientation may be provided. This may be differentially coded with respect to the end, to save bitrate through more efficient coding. For non-interpolatable data (e.g., flags, identifiers, indices), there may be an offset where the change occurs instantaneously, and the value at the start of the data point may be indicated (hence not differential). For flags, the starting value may not be indicated, as it is considered to be the opposite of the value following the provided indication.

[0108] In a JSON structure, an example of such an approach is as follows: - ObjectSource[] ID Signal Index PreGain PreGainInterpMethod (optional) PreGainInterpLengthIdx (optional) PreGainDelta (optional) % presence tells us there is animation in that data point ·position PositionInterpMethod (optional) PositionInterpLengthIdx (Optional) PositionDelta (optional) %Presence tells us there is animation in that data point ·direction OrientationInterpMethod (optional) OrientationInterpLengthIdx (optional) OrientationDelta (optional) %Presence tells us there is animation in that data point ·rendering RenderUpdateOffsetIdx (optional) %Presence tells the data point that there is an animation in it ·Directional ID DirectivityIDUpdateOffsetIdx (Optional) DirectivityIDStart (optional) %Presence tells us that there is an animation within that data point

[0109] Here the end point is [position, orientation] and the start point is [position - PositionDelta, orientation - OrientationDelta]. This means that the end position can be provided as either an absolute or relative value, allowing the start position to be reconstructed if necessary.

[0110] For example, various interpolation methods can be indicated by a field InterpMethod, which defines one of the following: - 'linear'<default> - 'instant' (makes an instant change in the target timestamp with no interpolation. Useful for example for transforming SignalIndex, ExtentID, or any Boolean field) - 'spherical' - 'logarithmic'

[0111] An end target may be the start of the next independent data point, any intermediate update data point, or an intermediate waypoint. Multiple potential targets may be enumerated within the general part of the bitstream, and other parts of the bitstream may effectively reference them to indicate valid targets for the data they cover. A target may be the target value / intended value at a given time.

[0112] Below, we describe an example of the data format and syntax for a single data point / data set. This description follows the approach used in MPEG Audio standards such as MPEG-H 3D Audio (ISO / IEC23008-3) to describe bitstream formats. The syntax description is structured as pseudocode, with function calls indicating that the syntax described below that function is inserted where the function call is made. Fields that occupy bits in the bitstream are shown in bold, with the second and third columns indicating the number of bits and the bit format using mnemonics. Some fields may have a variable number nr of bits depending on the value represented. These fields are associated with lookup tables that describe the codewords and the corresponding values. The codewords are designed so that when you start reading bit by bit, a shorter codeword never overlaps with the early bits of a longer codeword. This is known to those skilled in the art as lossless data encoding or entropy encoding, and examples include Huffman encoding, run-length encoding, or arithmetic encoding. Data read in the first half of the syntax may be used in the second half of the syntax to control the decoding of the other parts. For example, by signaling the number of bits, the number of elements, the method of encoding, the presence of certain data, etc.

[0113] In the description, the following specific acronyms are used (taken from ISO / IEC23008-3, MPEG-H 3D Audio specification): - bslbf - bit string, left bit first. "Left" is the order in which bit strings are written in ISO / IEC14496. A bit string is written as a string of 1s and 0s enclosed in single quotes, e.g. '1000 0001'. Whitespace within a bit string is included for readability and has no significance. Used for booleans - tcimsbf - two's complement integer, most significant (sign) bit first. - uimsbf - Unsigned Integer Most Significant Bit First. Used for uint - vlclbf - variable length code, left bit first. "Left" refers to the order in which the variable length code is written.

[0114] Datasets / points may be provided according to the following format / syntax: DataPoint() [Table 1]

[0115] Below, various fields / portions / elements of such data points are described in more detail, but it will be understood that the described fields / portions / elements are not limited to specific data points or syntax and may be used individually or in any combination within any bitstream containing acoustic environment data.

[0116] Further description of the feature / field / subelement of the data point.

[0117] dpType Data point type: Indicates the type of data point. [Table 2]

[0118] reservedBits Reserved bits. These can be used to introduce extension mechanisms.

[0119] Thus, a data point may begin with two bits that indicate the particular type of data point, specifically whether the data point is an independent data point or an update point.

[0120] GeneralData() Field GeneralData() may provide data that provides information about some parameters in the general configuration or rendering data. [Table 3]

[0121] bsVersion Bitstream version. [Table 4]

[0122] isSmallScene If set to true, indicates that there are no coordinates outside the range [-100..100] meters.

[0123] In some embodiments, the metadata generator 203 is configured to generate acoustic environment data that includes a global indicator that indicates that the environment is a spatially constrained environment. For example, the global indicator may be in the form of isSmallScene, which may be set to true or false. When set to true, isSmallScene is restricted to be expressed in coordinates no greater than [-100..100] meters.

[0124] When a global indicator is set to indicate a spatially restricted environment, many data values ​​are restricted to conform to a predefined restricted format for data values. Specifically, spatial coordinates (specifically position values) may be restricted not to exceed a threshold, such as being restricted to an interval of [-100..100] meters. In some embodiments, when a global indicator is set to indicate a spatially restricted environment, other parameters may be restricted to limited ranges. For example, for a small environment, the maximum duration or time constant of reverberation (e.g., T 60 value) may be restricted.

[0125] The spatially restricted environment may be an environment having a spatial extent below a threshold. The predetermined restricted format may be a data format for at least some values ​​of the characteristics affecting sound propagation that uses fewer bits than a data format that would be used for the bitstream if the environment was not a spatially restricted environment.

[0126] In some embodiments, a global flag or indicator may be included that indicates limits on one or more parameter values.

[0127] usacSamplingFrequencyIndex An index of the sample rate used for audio signals, as defined in 23003-3. [Table 5] TIFF2025031710000007.tif12477

[0128] usacSamplingFrequency If usacSamplingFrequencyIndex=0, the decoder's output sampling frequency is coded as an unsigned integer value.

[0129] useDefaultSpeedOfSound Flag indicating whether to use the default speed of sound for material-free media (343 m / s at 20°C) or specify a custom value.

[0130] speedOfSound Custom sound speed value for materialless media. Materialless media is considered to be the space that is not occupied by geometry that has a different material assigned to it.

[0131] TargetData() The field TargetData() may contain the target offset for the animation of the attribute, referenced by index from other parts of the data point. [Table 6]

[0132] FreqGridData() The field may contain a frequency grid definition that is referenced by index from other parts of the data point, which typically tells some bitstream parser how many frequency dependent elements are to be coded next.

[0133] The field FreqGridData() may provide data providing information about a frequency grid definition referenced by index from other parts of the data points. In particular, a frequency grid / subdivision into multiple frequency ranges may be described and other parameters / other data of the bitstream may be provided according to the defined frequency grid. For example, different frequency-dependent filtering may be provided by indicating attenuation for multiple different ranges of defined frequencies of the frequency grid. Thus, a filter may be described simply as a set of attenuation values ​​without the need to explicitly describe the frequencies that correspond to these attenuation values.

[0134] In some embodiments, the acoustic environment data may be configured to include one or more data groups / fields describing a frequency grid and a plurality of data groups / fields each including data describing frequency-dependent ones of the characteristics using the frequency grid. The data describing frequency-dependent ones of the characteristics using the frequency grid may do this, for example, by providing data values ​​for frequency values ​​and / or one or more of the frequency values ​​defined by the frequency grid.

[0135] The frequency grid may be a subdivision of the frequency range into frequency subranges, for example by defining center frequencies of the frequency subranges.

[0136] In some embodiments, the bitstream may include an indicator that indicates whether the bitstream includes a data group that describes a frequency grid. For example, a single bit, bFgdPresent, may indicate whether a frequency grid definition is included (e.g., in the next bit).

[0137] In some embodiments, the bit stream may include an indication of a format of data describing the frequency grid. For example, the bit stream may include an indication describing whether the frequency grid is described by reference to a predefined grid, for example by indicating an index of the grid among a set of predefined / predefined frequency grids. As another example, the bit stream may include data indicating a start frequency and a frequency range for at least some subranges of the frequency grid, typically for all subranges of the frequency grid. In some embodiments, the frequency grid may be indicated by a set of transition frequencies indicating boundary frequencies of frequency ranges / frequency intervals. For example, a first frequency interval / range may be indicated by data indicating a start frequency of the first frequency interval / range and a start frequency of a next frequency interval / range.

[0138] In many embodiments, the frequency range may be constant in a logarithmic representation of the frequency scale.

[0139] Frequency banding and division into ranges / intervals may be done based on octaves. A difference of one octave represents a doubling of the frequency (e.g., 125, 250, 500, 1000 Hz). The bitstream indicates, for example, whether there is banding in the octave bands or another subdivision (e.g., 1 / 3 octave bands (125, 160, 200, 250, 315, 400, 500, 630, 800, 1000) that places two more values ​​between the octave bands). In some embodiments, at least one subrange or individual frequency may be indicated by data aligned to a fraction of the octave band.

[0140] In some embodiments, the bitstream may include data indicative of a distinct frequency, e.g., a set of multiple distinct frequencies, such that within the bitstream, other frequency-dependent properties may be represented simply by sets of property values ​​for those distinct frequencies, without the need to explicitly state those distinct frequencies for each property.

[0141] In many embodiments, the frequency grid may then be described / defined by data in the bitstream, e.g., different modes of the frequency grid may be used and the data may indicate the mode used to describe the frequency grid.

[0142] For example, the field fgdMethod may indicate which of the following modes is used: - Default Grid For example, aligned to fractions of an octave band - Start frequency + frequency hop size + amount of bands For example, aligned to fractions of an octave band - Individual frequencies For example, aligned to fractions of an octave band

[0143] An example of the frequency grid format / syntax is as follows: [Table 7] TIFF2025031710000010.tif47170

[0144] bFgdPresent Flag indicating whether a frequency grid is defined.

[0145] fgdMethod The way the frequency grid is coded. [Table 8]

[0146] fgdCenterFreq The center frequency of each band in each frequency grid is given in Hz.

[0147] frequencyHopCode A code indicating the hop factor for frequency banding. [Table 9]

[0148] fgdDefaultBanding defines one or more default banding schemes. [Table 10]

[0149] Thus, in some embodiments, the bitstream is generated to include a data group describing a frequency grid that is used in other data groups to describe frequency-dependent characteristics of the acoustic environment data. Thus, a frequency grid is provided in the datastream and used to describe frequency changes. Furthermore, the bitstream includes a specific indicator indicating whether the bitstream includes a description of a frequency grid or not. Thus, a flexible approach is provided that allows the bitstream generation to be adapted to use an optionally customizable frequency grid that can be optimized for the specific characteristics and frequency dependencies to be communicated. As a simple example, the frequency resolution or frequency range may be flexibly adapted, and in fact the entire frequency grid description can be optionally omitted by appropriately setting the indicator of whether the description is included or not.

[0150] It may also be arranged such that the data group not only contains the frequency grid description but also specifically indicates the format used for the frequency grid description, so that an indication may be provided that allows the receiver to interpret the frequency grid description data, leaving the bitstream generator free to choose a format for the frequency grid description that is particularly suited to specific characteristics and their specific frequency dependences.

[0151] The data group describing the frequency grid may include data indicative of a predefined default grid, a starting frequency and a frequency range of at least some subranges of the frequency grid, and / or individual frequencies.

[0152] This approach can therefore enable the inclusion of frequency change characteristics in the bitstream by using a specific narrow approach for encoding and representing such acoustic environment data within the bitstream, which can provide a very efficient, adaptable and flexible approach for including such frequency-dependent acoustic environment characteristics in the bitstream.

[0153] AcousticElementData() This part of the syntax covers the data describing the acoustic elements. These are typically generic elements that act as hierarchical elements of the scene graph. Typically, the characteristics of such elements are inherited by their child nodes, i.e. the pre-gain values ​​of an acoustic element also apply to audio sources organized under the corresponding acoustic element. The position and orientation of these nodes transform the position and orientation of their child nodes, unless they have a pose dependency set to ignore the orientation or pose of their parent.

[0154] In some embodiments, the acoustic environment data includes an animation indication for at least one audio element. The animation indication indicates whether a characteristic of the audio element changes during a time interval, and thus indicates whether the characteristic is a dynamic or static characteristic. In particular, a flag or indication may be included for a data point / group that indicates whether the audio element is static or time-varying within the time interval of that data point / group.

[0155] If the animation indication indicates that the corresponding sound is time-varying, the acoustic environment data may further include data describing the change in the varying characteristic. The acoustic environment data for such an audio element may include data indicating, for at least one characteristic of the audio element, whether the characteristic is a time-varying characteristic or not. Thus, an indication as to the characteristics may be included indicating which characteristics are time-varying and which characteristics are not time-varying (within the time interval of the data set / point).

[0156] In many embodiments, the acoustic environment data may include data describing how to determine characteristic values ​​at different times within a time interval; in particular, the acoustic environment data may include data describing an interpolation approach to be applied to determine characteristic values ​​at different times.

[0157] The acoustic environment data may include two or more values ​​for a particular time-varying characteristic, each value may be for a particular time point, and the characteristic values ​​for other times may be determined by interpolation. The encoded audio data stream, and in particular the individual data points, may include data describing the characteristics of the temporal interpolation to be performed for the time-varying characteristic.

[0158] In some cases, one or more of the values ​​used for interpolation within a time interval of a data point may be provided or determined from values ​​provided outside the time interval / data point. For example, one or more values ​​may be derived from data of another data point, which may have been transmitted earlier in the bitstream, for example. Such a data value may be associated with a particular point in time (e.g., indicated by a timestamp), which may be a time before or after the time interval of the data point to which the interpolation is applied (and for which the interpolation may be defined).

[0159] In some embodiments, a different interpolation may be described for each data point, and therefore for each time interval, and different interpolations may be described for different properties (including different properties for the same data point / time interval).

[0160] The interpolation method may be indicated, for example, by a code represented by the flag *InterpMethod. The code may be, for example, as shown in Table 1 below. [Table 11]

[0161] In some embodiments, the acoustic environment data for at least some elements of the environment may include identification data and parent identification data of elements of the scene graph of the environment, which may be, for example, objects, sound sources (audio elements), and / or acoustic characteristics of the environment.

[0162] For example, multiple different objects and / or sound sources may be located within a scene graph, and the acoustic environment data may include data indicative of this scene graph. This may be achieved in particular by providing identification data and parent identification information for individual elements, as this allows reconstruction of the scene graph, and thus may be configured to represent a hierarchy.

[0163] In some embodiments, animation of elements may be indicated (and possibly described) in the acoustic environment data.

[0164] For example, the acoustic environment data may include ElementAnimated, SourceAnimated, and AttributeAnimated flags / indications for each element / source to indicate whether it is animated (within a time segment), in which case a flag / indication may be included for each attribute / property to indicate whether the attribute / property is animated, in which case data describing the animation / time change may be further included.

[0165] As an example of the above functionality, the following format may be used: [Table 12] TIFF2025031710000016.tif255168TIFF2025031710000017.tif147170

[0166] aedPresent A flag indicating whether the audio element is defined.

[0167] aedNrElements Reports the number of audio elements defined.

[0168] elementAnimated A flag indicating whether the corresponding element is animated at this data point.

[0169] attributeAnimated A flag indicating whether the corresponding attribute is animated at that data point. Also used to indicate whether data exists for more waypoints.

[0170] aedID The ID of the audio element.

[0171] aedParentID The ID of the acoustic element's parent in the scene graph.

[0172] aedPoseDependency Indicates what the pose of the audio element is relative to.

[0173] aedPosition Indicates the position coordinates (x,y,z) of the acoustic element in meters. If the same element appears multiple times, this indicates that there are multiple waypoints within the range of the data point.

[0174] aedPositionInterpMethod Indicates the interpolation method to use for position animation (see table 1).

[0175] aedPositionInterpTargetIdx An index into the TargetOffset, indicating the offset from the data point's timestamp for which the preceding position target value is valid. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points.

[0176] aedPositionDelta The position delta value that allows to reconstruct the position value at the data point's timestamp. PositionAtDPStart = aedPosition - aedPositionDelta If there are multiple target values, the first target value is used as the basis.

[0177] aedOrientation Indicates the orientation (yaw, pitch, roll) of the acoustic element in radians. If the same element appears multiple times, this indicates that there are multiple waypoints within the range of the data point.

[0178] aedOrientationInterpMethod Indicates the interpolation method to use for orientation animations (see table 1).

[0179] aedOrientationInterpTargetIdx An index into the TargetOffset, indicating the offset from the data point's timestamp for which the preceding orientation target value is valid. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points.

[0180] aedOrientationDelta The orientation delta value that allows to reconstruct the orientation value at the data point's timestamp. OrientationAtDPStart = aedOrientation - aedOrientationDelta If there are multiple target values, the first target value is used as the basis.

[0181] aedPregain The pre-gain value (in dB) for all sources hierarchically located under the corresponding acoustic element. If the same element appears multiple times, this indicates that there are multiple waypoints within the range of the data point.

[0182] aedPregainInterpMethod Indicates the interpolation method to use for pre-gain animation (see Table 1).

[0183] aedPregainInterpTargetIdx An index into the TargetOffset, indicating the offset from the data point's timestamp for which the preceding pre-gain target value is valid. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points.

[0184] aedPregainDelta The pre-gain delta value that allows reconstructing the pre-gain value at the data point's timestamp. PregainAtDPStart = aedPregain - aedPregainDelta If there are multiple target values, the first target value is used as the basis.

[0185] aedRender A rendering flag that, if false, indicates not to render any sources hierarchically located under the corresponding audio element.

[0186] aedRenderUpdateTargetIdx An index into the TargetOffset, indicating the offset from the timestamp of the data point at which the rendering flags will flip state to the preceding aedRender value. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points, with each target index indicating a binary inversion of the state of a flag.

[0187] AudioSourceData()AudioSourceData() This part of the syntax brings together various types of sources. [Table 13]

[0188] ObjectSourceData() This part of the syntax describes the characteristics of the object source. [Table 14] TIFF2025031710000020.tif255170TIFF2025031710000021.tif255169

[0189] osdPresent A flag indicating the presence or absence of an object source.

[0190] osdNrElements Indicates the number of object source elements that follow.

[0191] sourceAnimated A flag indicating whether the corresponding source is animated at this data point.

[0192] aedID The ID of the audio element.

[0193] aedParentID The ID of the acoustic element's parent in the scene graph.

[0194] osdSignalIndex Indicates the index into the signal input buffer of the signal corresponding to the source.

[0195] osdIsContinuousSource Flag indicating whether this is associated with a continuous signal or with a decoder-side triggered sound effect.

[0196] osdReferenceDistance Indicates the reference distance of the object source.

[0197] osdPoseDependency Indicates what the source pose is relative to.

[0198] attributeAnimated A flag indicating whether the corresponding attribute is animated at that data point. Also used to indicate whether data exists for more waypoints.

[0199] osdPosition The object source's position coordinates (x,y,z) in meters. If the same source appears multiple times, this indicates that there are multiple waypoints within range of the data point.

[0200] osdPositionInterpMethod Indicates the interpolation method to use for position animation (see table 1).

[0201] osdPositionInterpTargetIdx An index into the TargetOffset, indicating the offset from the data point's timestamp for which the preceding position target value is valid. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points.

[0202] osdPositionDelta The position delta value that allows reconstructing the object source position value at the data point's timestamp. PositionAtDPStart = osdPosition - osdPositionDelta If there are multiple target values, the first target value is used as the basis.

[0203] osdOrientation The orientation (yaw, pitch, roll) of the object source in radians. If the same source appears multiple times, this indicates that there are multiple waypoints within range of the data point.

[0204] osdOrientationInterpMethod Indicates the interpolation method to use for orientation animations (see table 1).

[0205] osdOrientationInterpTargetIdx An index into the TargetOffset, indicating the offset from the data point's timestamp for which the preceding orientation target value is valid. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points.

[0206] osdOrientationDelta The orientation delta value that allows reconstructing the object source orientation value at the data point's timestamp. OrientationAtDPStart = osdOrientation - osdOrientationDelta If there are multiple target values, the first target value is used as the basis.

[0207] osdPregain Pre-gain value (dB) of the object source. If the same element appears multiple times, this indicates that there are multiple waypoints within the range of the data point.

[0208] osdPregainInterpMethod Indicates the interpolation method to use for pre-gain animation (see Table 1).

[0209] osdPregainInterpTargetIdx An index into the TargetOffset, indicating the offset from the data point's timestamp for which the preceding pre-gain target value is valid. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points.

[0210] osdPregainDelta The pre-gain delta value that allows reconstructing the pre-gain value of the object source at the data point's timestamp. PregainAtDPStart = osdPregain - osdPregainDelta If there are multiple target values, the first target value is used as the basis.

[0211] osdRender The rendering flag, which if false indicates that the object source should not be rendered.

[0212] osdRenderUpdateTargetIdx An index into TargetOffset, indicating the offset from the data point's timestamp at which the rendering flags will flip state to the preceding osdRender value. Multiple target indexes may be provided such that there is one target index for each waypoint within the range of the data points, with each target index indicating a binary inversion of the state of a flag.

[0213] osdIsOmniDirectional A flag indicating whether the source is omnidirectional.

[0214] osdDirectivityPatternID The ID of the directional pattern that should be used.

[0215] SoundEffectData() This part of the syntax describes the characteristics of sound effects. These do not represent continuously rendered audio signals, but short segments of an audio signal that are triggered by user interaction, etc.

[0216] In some embodiments, the acoustic environment data may provide metadata associated with sound effects, clips, and other audio elements, and the metadata may include, among other things, data for sound effect audio elements associated with particular user actions or interactions. For example, a sound effect element may be provided for a creaking door being opened. Such a sound effect may be associated with a user action such that when a user provides input corresponding to a door being opened in the environment, a renderer may extract and render the creaking door sound effect.

[0217] In some embodiments, the audio elements may include multiple sound effect elements, and the acoustic environment data may include data linking a user-controlled environmental change with one (or more) of such sound effect elements. The user-controlled environmental change may be or may be determined based on (or result from) user input or interaction, for example.

[0218] The acoustic environment data may indicate changes for a particular user interaction, for example by referencing target IDs (which identify elements), target attributes, and / or corresponding target values. A list of these changes may be performed in response to a trigger ID.

[0219] As a specific example, the start of rendering a sound effect may be indicated using SoundEffectID as an attribute of the sound source, and the sound effect ID of the sound effect to be played may be included as a target value.

[0220] The sound effect data may be provided, for example, according to the following syntax: [Table 15]

[0221] sedPresent A flag indicating whether sound effect data is included.

[0222] sedNrElements The number of sound effect elements defined below is shown.

[0223] sedID The ID of the sound effect.

[0224] acdParentID The id of the sound effect's parent in the scene graph.

[0225] sedDuration The duration of the sound effect.

[0226] sedPregain Sound effect pre-gain value.

[0227] In some embodiments, the acoustic environment data may include data associated with a particular region. For example, a characteristic value may be provided along with an indication of the particular region to which the characteristic value applies. For example, for a given sound source, a characteristic (e.g., frequency-dependent attenuation) may depend on whether the listener is in the same room as the sound source or the user is in another room. Thus, a characteristic may have an indication of a linked region within which the listening position must be in order for the value to be valid. If the listening position is outside the zone of application, the characteristic cannot be legitimately used. Instead, another characteristic value may be included that has a different scope of application, for example including the current listening position.

[0228] In many embodiments, the metadata may include multiple linked regions for one or more properties, specifically a first and a second region of application. A first region may be provided for the location of the audio element / audio source, and a second region may be provided for the listening position. Thus, a property value may be associated with two regions, one related to the listening position and the other related to the audio source position. The decoder / renderer then evaluates whether the listening position and the audio source position are within the appropriate region. If not, another property value may be used, for example a default value, a value associated with another validity region including the listening position and the audio source position, or an alternative property value provided as an alternative to the original value and indicated for use if the validity region is not met.

[0229] As a specific example, the bitstream may contain fields / data denoted applicableRegion and internalSourceRegion that respectively represent the validity regions of the listening position and sound source / audio element position, respectively.

[0230] As an example, the bitstream may contain data for a first characteristic value that depends on the listening position and whether the listening position is within the region indicated by the value in the data field applicableRegion, where this value is then directly applicable to sources that are within the region indicated by the value in the data field internalSourceRegion, and where another characteristic value is applied if they are outside the region indicated by the data field internalSourceRegion.

[0231] An example of bitstream data using such an approach is included below.

[0232] AcousticEnvironmentData() This part of the syntax describes the (overall) characteristics of the (general) acoustic environment / ambience, specifically the reverberation characteristics. [Table 16]

[0233] acdPresent A flag indicating whether acoustic environment data is available.

[0234] acdNrElements Indicates the number of acoustic environments defined.

[0235] acdID The ID of the acoustic environment definition.

[0236] acdParentID The ID of the acoustic environment definition's parent in the scene graph.

[0237] applicableRegionID The ID of the geometric element that describes the region the parameters apply to. Position and orientation are offset by the parent of the acoustic environment, unless the pose dependency is global.

[0238] internalSourceRegionID The ID of the geometric element that describes the region where all the energy of all contained sources contributes to the reverberation. Position and orientation are offset by the parent of the acoustic environment, unless the pose dependency is global.

[0239] freqGridIdx An index into the list of frequency grids defined in FreqGridData().

[0240] dsrOffset The offset (in seconds) from where the DSR is calculated in the RIR. Offset=0 is equivalent to being emitted at the source, so the offset should be higher.

[0241] T60 The T60 time calculated from the 0 to -30 dB point on the linear portion of the EDC after the initial decay.

[0242] DSR Diffuse to Source energy Ratio. The diffuse reverberation energy is calculated for one user sample point after the RIR lag dsrOffset. The source energy is the sum of the emitted source energy that causes the diffuse energy.

[0243] dsrCode A code indicating the DSR value. [Table 17] TIFF2025031710000025.tif46169

[0244] GeometricElementData() This part of the syntax describes a geometric element. [Table 18]

[0245] gedPresent A flag indicating whether acoustic environment data is available.

[0246] gedNrElements Indicates the number of geometric elements defined.

[0247] gedID The id of the geometric element.

[0248] gedType Next, define the type of geometric element to be transmitted. [Table 19]

[0249] cornerPos1 It contains one corner of an axis-aligned box. The position is expressed in global coordinates.

[0250] cornerPos2 Contains the second corner of the axis-aligned box diagonally from cornerPos1. The position is expressed in global coordinates.

[0251] boxParentID The ID of the parent node of the corresponding box geometric element.

[0252] boxPoseDependency Indicates what the pose of the audio element is relative to.

[0253] boxPosition The (x,y,z) coordinates of a box geometry, measured in meters.

[0254] boxOrientation The orientation (yaw, pitch, roll) of the box geometry, measured in radians.

[0255] boxXDim The dimension of the box geometry along the x-axis before rotation.

[0256] boxYDim The dimension of the box geometry along the y-axis before rotation.

[0257] boxZDim The dimension of the box geometry along the z-axis before rotation.

[0258] UserInteractionData() This part of the syntax describes the user interaction data: how a user interaction can be triggered and what changes should be made in response to the trigger, by describing for which element, which attribute should be changed to which value, or whether this value is provided from an external entity (for example in the case of interactions that are entirely controlled by the user, such as picking up a source and moving it in the scene).

[0259] A more semantic definition of user interaction can be defined at a higher bitstream level, covering all aspects of 6DoF media rendering: vision, sound, haptics, etc. A user interaction can affect all or some of these aspects. This system layer (in the case of MPEG-I, covered by the MPEG Systems working group WG03) defines how a user can trigger a specific interaction. This may mean defining activating a trigger button on a controller within a certain spatial region of the scene, or it may be a more abstract meaning where other layers link abstract meaning to hardware-dependent control. For example, if a user opens door 5, it activates user interaction G.

[0260] Such user interaction triggers at the system level may send dedicated user interaction triggers to the respective media renderers. For example, system level user interaction G may be linked to audio user interaction 12, and the audio renderer may then perform the changes associated with the user interaction with triggerID=12.

[0261] In many embodiments, such user interaction triggers may be accompanied by further parameters for more immersive user interaction: for example, the position coordinates of a sound source being picked up and moved by the user may be provided.

[0262] Such user interactions that are triggered from outside the audio renderer may also be referred to as external triggers. Other user interactions may rely on being triggered by the audio renderer itself. The distinction between such triggers may be indicated by a triggerType property in the bitstream that describes the change in user interaction. [Table 20] TIFF2025031710000029.tif255162TIFF2025031710000030.tif255164TIFF2025031710000031.tif39170

[0263] uidPresent A flag indicating whether user interaction is required.

[0264] triggerType Defines how user interactions trigger scene updates. [Table 21]

[0265] triggerIdx The index used in a user interaction message from an external source to trigger an update of this particular scene.

[0266] triggerID The ID of the state element that describes the state that describes this particular scene update.

[0267] condTransition An update is triggered when the state changes to a specified transition value.

[0268] updateDelay How long the update should be delayed after the trigger.

[0269] nrChanges Indicates the number of changes in this scene update.

[0270] usePreviousID A flag indicating whether the change is for the same ID as the previous change.

[0271] immediateChange A flag that indicates whether the change is immediate or interpolated to the target value over a period of time.

[0272] duration Interpolation period of change.

[0273] targetAttribute The name of the attribute to change.

[0274] attribCode A code indicating which attributes can be changed. [Table 22]

[0275] externalParameter A flag that indicates whether the parameter value is provided by an external process.

[0276] parameterIdx The index of the parameter in the message from the external process that is mapped to this attribute.

[0277] paramIdxCode A code indicating the parameter index. [Table 23]

[0278] moreParameters Flag indicating whether additional parameters are required.

[0279] targetValue The target value for the change. Depending on the type of the target attribute, the value may be encoded differently according to the switch statement in the syntax.

[0280] Supporting Elements The bitstream may include various supporting elements that may support the provision of data representing the property values.

[0281] Also, in many embodiments, the number of bits used to indicate the value of the property may be variable within the bitstream. In particular, the property value may be indicated by a field containing a predefined number of bits. However, a flag indicator may be included to indicate that one or more extension fields providing additional bits for the property value are included. The extension fields may specifically provide additional bits to extend the range of the property value. In particular, the extension fields may further include most significant ( / upper) bits that are combined with the data bits of the default field to generate a property value with a (dynamic) range (specifically, may provide additional bits that allow higher values ​​to be represented).

[0282] In other scenarios, the extension field may specifically provide additional bits to extend the precision of the characteristic value, in particular, the extension field may further include additional least significant ( / lower) bits that are combined with the data bits of the default field to generate a more precise characteristic value.

[0283] In some embodiments, the acoustic environment data may include a first data field providing a first bit representing a value of the given characteristic and a second data field that may provide an indication for the first data field / given characteristic. The indication may include an indication of whether a further extension data field is included providing further bits representing the value of the first characteristic. The indication may be, for example, an indication that the extension field includes bits for extending the range of data values ​​provided and / or includes bits for increasing the precision / resolution of the data values ​​provided. In particular, the indication may indicate that the value is represented by a larger data word resulting from a combination of bits from both the first default field and the extension field.

[0284] In some embodiments, the further bits extend the range of values ​​that the characteristic can take. The further bits may be more significant bits than the first bit of the data word that represents the value of the characteristic. In some embodiments, the further bits increase the resolution of the values ​​that the characteristic can take. The further bits may be less significant bits than the first bit of the data word that represents the value of the first characteristic. The further bits may extend the precision of the values ​​that the first characteristic can take.

[0285] In many embodiments, the default bit and the further bit may each represent a value that is not necessarily combined by bit concatenation, but is instead combined in other ways.

[0286] Thus, in some embodiments, the acoustic environment data may include an indication / flag that further bits are used to determine the value of the characteristic. This approach may be used for a variety of characteristic values, such as, for example, spatial characteristics, volume, gain characteristics, volumetric characteristics, frequency characteristics, index characteristics, and discrimination characteristics.

[0287] For example, the following fields may be used to indicate that a wider range is offered: - For time scale: addSeconds - For spatial scale: addHectometers - For a measure of quantity: isLargerNumber - (integer) For ID numbers: largerValue

[0288] The following fields may be used to indicate that more accurate (higher resolution) values ​​are provided. - For time scale: addMilliseconds - For spatial scale: addCentimeters - For frequency scale: moreAccuracy - For angle scale: addFineAngle - For gain scale: addFineGain.

[0289] The various support elements for the bitstream are listed below: As mentioned above, the support elements may use variable range / resolution values.

[0290] GetID() Returns an integer ID. [Table 24]

[0291] idVal An ID value or a partial ID value.

[0292] largerValue A flag indicating whether the ID value is greater than.

[0293] GetCountOrIndex() Returns a number in the range [0..1023]. [Table 25]

[0294] countOrIndexLoCode A code indicating the low-order bits of the count or index value. [Table 26]

[0295] isLargerNumber A flag indicating whether more bits are transmitted to indicate a larger number.

[0296] countOrIndexHiCode A code indicating the most significant bits of a count or index value. [Table 27]

[0297] GetDuration() Returns the temporal duration in samples. [Table 28]

[0298] LUT() Executes a query to a lookup table whose value corresponds to the field specified as an argument.

[0299] deciSecondsCode A code indicating the decisecond duration offset. [Table 29]

[0300] addMilliseconds Next, a flag indicating whether a millisecond duration offset is sent.

[0301] milliSecondsCode A code indicating the millisecond duration offset. [Table 30]

[0302] addSamples Next, a flag indicating whether a sample-based duration offset is sent or not.

[0303] Sample Code A code indicating the sample number duration offset. [Table 31]

[0304] addSeconds Next, a flag indicating whether a seconds duration offset is sent or not.

[0305] secondsCode A code indicating the seconds duration offset. [Table 32]

[0306] GetFrequency() Returns the frequency in Hz in the range [16...49717]. [Table 33]

[0307] LUT() Executes a query to a lookup table whose value corresponds to the field specified as an argument.

[0308] frequencyCode A code indicating the center frequency (Hz) of a 1 / 3 octave band. [Table 34]

[0309] More Accuracy A flag indicating whether data for more precise frequencies is transmitted.

[0310] frequencyRefine A field indicating the value by which to adjust the frequency value.

[0311] GetPosition() Returns the position [x,y,z] in meters. [Table 35]

[0312] GetPositionDelta() Returns the position Δ[dx,dy,dz] in meters. [Table 36]

[0313] GetDistance() Returns the distance in meters. [Table 37]

[0314] LUT() Executes a query to a lookup table whose value corresponds to the field specified as an argument.

[0315] metersCode A code indicating a meter coordinate offset. [Table 38]

[0316] addHectometers Next, a flag indicating whether a hectometer coordinate offset is sent.

[0317] hectometersCode A code indicating the hectometer coordinate offset. [Table 39]

[0318] addKilometers Next, a flag indicating whether or not a kilometer coordinate offset is sent.

[0319] kilometersCode A code indicating a kilometer coordinate offset. For distances greater than 10km, multiple occurrences may be provided. [Table 40]

[0320] addCentimeters Next, a flag indicating whether centimeter coordinate offsets are sent or not.

[0321] centimetersCode A code indicating a centimeter coordinate offset. [Table 41]

[0322] addMillimeters Next, a flag indicating whether or not a millimeter coordinate offset is sent.

[0323] millimetersCode A code indicating a millimeter coordinate offset. [Table 42]

[0324] Representation of Orientation Properties The acoustic environment data will often, and in fact most often, include values ​​for one or more orientation properties.

[0325] In some embodiments, this may be advantageously achieved by acoustic environment data that includes a representation of orientation values.

[0326] In some embodiments, the acoustic environment data includes a data group / data field that describes an orientation representation format for expressing the orientation characteristic. In many embodiments, the acoustic environment data can include data (within the same or multiple different data sets / points) that describes or defines multiple different orientation representation formats. Each orientation representation format may provide a (data / bit) format for expressing an orientation value. In that case, multiple data sets / points may include data that describes the orientation characteristic by using one of the defined orientation representation formats.

[0327] In some embodiments, the acoustic environment data may include an indicator of whether the bitstream includes a data group / field describing an orientation representation format, e.g., a flag may be indicated to indicate that a data field describing an orientation representation format is included.

[0328] A flag / indicator may also be provided for each individual orientation value to indicate whether the orientation value is provided in accordance with an orientation representation format defined in the acoustic environment data. A flag / indicator may also be included for each individual value, for example to indicate which of multiple orientation representation formats is used for that particular value.

[0329] Orientation representation formats may include, for example, one or more of the following:

[0330] An indication of a predefined default orientation representation. The number of default orientation representations may be predetermined (e.g., by definition of a standard). Data referencing such a default orientation may be included. For example, the field orientationCode may indicate the default orientation.

[0331] A set of predefined angles. The orientation representation format may define a set of predefined angles, and an orientation value may be indicated, for example, by simply referring to one of such predefined angles. For example, each predefined angle may be represented by an index, and a given orientation value may be represented by indicating the appropriate index. For example, each angle of the orientation value may be represented by a default angle from a narrow range of angles.

[0332] A set of angles (e.g., on a quantization grid). The representation of the angles may be by explicit angle values. The angle values ​​may be represented by a given word length and therefore may have a given quantization level. Each orientation angle is thus indicated by one of a larger range of angles.

[0333] Below are provided some examples of approaches to providing orientation values ​​following the above examples.

[0334] GetOrientation() Returns the orientation [yaw, pitch, roll] in radians. [Table 43]

[0335] LUT() Executes a query to a lookup table whose value corresponds to the field specified as an argument.

[0336] orientationCode An orientation code indicating either the default orientation, or one of two escaped values ​​where the orientation is defined by further data. [Table 44]

[0337] defaultYawCode Code for default yaw angle. [Table 45]

[0338] defaultPitchCode Code for default pitch angle. [Table 46]

[0339] defaultRollCode Code for default roll angle. [Table 47]

[0340] coarseAngleCode Code for coarse angle indication in 1 / 36pi increments. [Table 48] TIFF2025031710000060.tif50170

[0341] addFineAngle Flag indicating whether finer granularity angle data is sent.

[0342] fineAngleCode Code for fine angle indication in 1 / 1800pi increments. [Table 49]

[0343] GetGain() Returns the gain value (dB). [Table 50]

[0344] coarseGainCode Coarse gain value (dB) code. [Table 51]

[0345] addFineGain A flag indicating whether more data will be sent to provide a finer resolution gain value.

[0346] fineGainCode Code for finer gain resolution (1 dB resolution). [Table 52]

[0347] GetGainDelta() Returns the gain delta value (dB). [Table 53]

[0348] GetRenderingConditions() Contains information about how to render the source. [Table 54]

[0349] isNormalConditions A flag indicating whether the rendering condition is normal or not. This means that no acoustic features are explicitly turned off and the rendering of the source, or specific acoustic features of the source, is decided by the decoder.

[0350] doReverb Flag indicating whether to render reverberation for the corresponding source.

[0351] doEarlyReflections Flag indicating whether to render early reflections of the corresponding source.

[0352] Doppler Flag indicating whether to render Doppler hardening for the corresponding source.

[0353] doDistanceAtt Flag indicating whether to render the distance attenuation of the corresponding source.

[0354] doDirectPath A flag indicating whether to render the direct path for the corresponding source.

[0355] regionDependentActivation A flag that indicates whether further data specifies whether the source should be active or not, depending on the user's position.

[0356] activateGoingIn A flag that, when true, indicates that the source is activated when the user moves into the specified region.

[0357] regionDependentDeactivation A flag indicating whether further data specifies whether the source should be deactivated or not depending on the user's location.

[0358] deactivateGoingIn A flag that, when true, indicates that the source will be deactivated when the user moves into the specified region.

[0359] Whereas the terms voice and sound source are used above, it will be understood that this is equivalent to the terms tone / sound and sound source. References to the term "voice" may be replaced with references to the term tone / sound.

[0360] For clarity, the above description describes embodiments of the invention in terms of different functional circuits, units, and processors. However, it will be appreciated that functionality may be suitably distributed among different functional circuits, units, or processors without detracting from the invention. For example, functionality described as being performed by multiple separate processors or controllers may be performed by the same processor or controller. Thus, references to specific functional units or circuits should not be construed as indicative of a strict logical or physical structure or organization, but rather as references to suitable means for providing the described functionality.

[0361] The invention may be implemented in any suitable form including hardware, software, firmware or any combination of these. The invention may be implemented at least in part as computer software running on one or more data processors and / or digital signal processors. The elements and components of the embodiments of the invention may be physically, functionally and logically implemented in any suitable way. Indeed functionality may be implemented in a single unit, in multiple units or as part of other functional units. Thus, the invention may be implemented as a single unit or may be physically and functionally distributed between several different units, circuits and processors.

[0362] Although the present invention has been described in relation to several embodiments, the present invention is not limited to the specific forms described in the specification. The scope of the present invention is limited only by the appended claims. Moreover, even if a feature appears to be described in relation to a specific embodiment, a person skilled in the art will recognize that various features of the above-mentioned embodiments can be combined in accordance with the present invention. In the claims, the terms "comprise", "include" and the like do not exclude the presence of other elements or steps.

[0363] Furthermore, although individually recited, a plurality of means, elements, circuits, or method steps may be implemented by, for example, a single circuit, unit, or processor. Moreover, although individual features are included in different claims, they may be combined as desired, and the inclusion of such features in different claims does not imply that the combination of features is not feasible and / or advantageous. Furthermore, the inclusion of a feature in one claim category does not necessarily limit the feature to this category, but the feature may be equally applicable to other claim categories, as appropriate. Furthermore, the order of features in the claims does not imply a particular order in which the features should be acted upon, and in particular the order of individual steps in a method claim does not imply that the steps must be performed in that order. The steps may be performed in any suitable order. Furthermore, the singular does not exclude the plural. Thus, the use of "a", "an", "first", "second", etc. does not exclude a plural. Reference signs in the claims are merely examples for clarity and do not limit the scope of the claims in any way.

Claims

1. 1. An apparatus for generating a bitstream, comprising: a metadata generator for generating metadata for audio data of a plurality of audio elements representing sound sources in an environment, said metadata including acoustic environment data of said environment, said acoustic environment data describing characteristics influencing sound propagation of said sound sources in said environment, at least a portion of said acoustic environment data being applicable to a plurality of listening poses in said environment, said characteristics including both static and dynamic characteristics; a bitstream generator for generating the bitstream including the metadata; The acoustic environment data includes a data group describing an orientation expression format for expressing an orientation characteristic, and at least one data group including data describing an orientation characteristic among the characteristics using the orientation expression format, the data group including: Data indicative of a predetermined default orientation representation; Data indicative of a set of predetermined angles; and data indicative of an angle on a quantization grid.

2. The acoustic environment data includes a data group describing a frequency grid and a plurality of data groups each including data describing a frequency dependent one of the characteristics using the frequency grid, the bit stream including an indicator of whether the bit stream includes the data group describing the frequency grid, the data group including an indication of a format of the data describing the frequency grid, the data group including: Data indicative of a predetermined default grid; data indicative of start frequencies and frequency ranges of at least some subranges of the frequency grid; and data indicative of the individual frequencies.

3. The data group is Data indicative of a set of predetermined angles; 3. The apparatus of claim 1 or 2, further comprising at least one of: data indicative of an angle on a quantization grid;

4. The apparatus of claim 1 , wherein the metadata generator is configured to select the orientation representation format as one of the predetermined default orientation representation, the predetermined set of angles, and an angle on a quantization grid.

5. An apparatus, wherein each predetermined angle of the set of predetermined angles is represented by an index, and wherein the orientation characteristic is represented by an indication of the index of the predetermined angle.

6. 6. Apparatus according to claim 1, wherein the group of data includes data indicative of a set of predetermined angles.

7. An apparatus according to claim 1 , wherein the data groups comprise data indicative of angles on a quantisation grid.

8. 8. The apparatus of claim 7, wherein the acoustic environment data includes a further data group describing a further orientation representation format for representing an orientation characteristic, and at least one further data group including data describing an orientation characteristic of the characteristics using the further orientation representation format, the further data group including data indicative of a set of predetermined angles.

9. 9. The apparatus of claim 1, wherein the acoustic environment data includes a first data field for a first bit representing a value of a first characteristic among the characteristics affecting sound propagation, and a second data field indicating whether the acoustic environment data includes extension data for a second bit representing the value of the first characteristic.

10. 10. The apparatus of claim 1, wherein the metadata generator generates the acoustic environment data to include a global indicator indicating that the environment is a spatially constrained environment, and the metadata generator restricts data values ​​of the acoustic environment data to conform to a predetermined restricted format for data values ​​for the global indicator indicating that the environment is spatially constrained.

11. 11. The apparatus of claim 1, wherein the acoustic environment data is arranged in a plurality of consecutive data sets, each data set comprising data spanning a time interval, and a first data set of the consecutive data sets comprising a first characteristic value of at least one of the characteristics affecting sound and a time indication of the first characteristic value indicative of a time within the time interval represented by the first data set.

12. 12. The apparatus of claim 1, wherein the acoustic environment data is arranged in a number of consecutive data sets, each data set comprising data spanning a time interval, and wherein the bitstream generator determines whether a characteristic value of a first one of the characteristics affecting sound propagation is provided for a default time within the time interval represented by the first data set, and if so includes the first characteristic value in the first data set without a time indication, and if not includes the first characteristic value in the first data set together with a time indication for the first characteristic value.

13. 13. The apparatus of claim 1, wherein the acoustic environment data for a first sound element includes an indication of a first area of ​​applicability and a second area of ​​applicability for a first characteristic value of a first characteristic of the characteristics affecting sound propagation, the first area of ​​applicability indicating an area of ​​a position of the first sound element to which the first characteristic value applies and the second area of ​​applicability indicating an area of ​​a listening position to which the first characteristic value applies.

14. 1. An apparatus for generating a rendered audio, the apparatus comprising: a first receiver for receiving audio data for a plurality of audio elements representative of sound sources in an environment; a second receiver for receiving a bitstream comprising metadata of the audio data of the plurality of audio elements representing sound sources in the environment, the metadata comprising acoustic environment data of the environment, the acoustic environment data describing characteristics influencing sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics comprising both static and dynamic characteristics; a renderer for generating output audio data of the environment in response to the audio data and the acoustic environment data; The acoustic environment data includes a data group describing an orientation expression format for expressing an orientation characteristic, and at least one data group including data describing an orientation characteristic among the characteristics using the orientation expression format, the data group including: Data indicative of a predetermined default orientation representation; Data indicative of a set of predetermined angles; and data indicative of an angle on a quantization grid.

15. A bitstream comprising metadata of audio data of a plurality of audio elements representing sound sources in an environment, said metadata comprising acoustic environment data of said environment, said acoustic environment data describing characteristics influencing sound propagation of said sound sources in said environment, at least a portion of said acoustic environment data being applicable to a plurality of listening poses in said environment, said characteristics comprising both static and dynamic characteristics; The acoustic environment data includes a data group describing an orientation expression format for expressing an orientation characteristic, and at least one data group including data describing an orientation characteristic among the characteristics using the orientation expression format, the data group including: Data indicative of a predetermined default orientation representation; Data indicative of a set of predetermined angles; and data indicating an angle on a quantization grid.

16. 1. A method for generating a bitstream, the method comprising: generating metadata for audio data of a plurality of audio elements representing sound sources in an environment, said metadata including acoustic environment data of said environment, said acoustic environment data describing characteristics influencing sound propagation of said sound sources in said environment, at least a portion of said acoustic environment data being applicable to a plurality of listening poses in said environment, said characteristics including both static and dynamic characteristics; generating the bitstream including the metadata; The acoustic environment data includes a data group describing an orientation expression format for expressing an orientation characteristic, and at least one data group including data describing an orientation characteristic among the characteristics using the orientation expression format, the data group including: Data indicative of a predetermined default orientation representation; Data indicative of a set of predetermined angles; and data indicative of an angle on a quantization grid.

17. 1. A method for generating a rendered audio, the method comprising: receiving audio data for a plurality of audio elements representing sound sources in an environment; receiving a bitstream comprising metadata of the audio data of the plurality of audio elements representing sound sources in the environment, the metadata comprising acoustic environment data of the environment, the acoustic environment data describing characteristics influencing sound propagation of the sound sources in the environment, at least a portion of the acoustic environment data being applicable to a plurality of listening poses in the environment, the characteristics comprising both static and dynamic characteristics; generating output audio data of the environment in response to the audio data and the acoustic environment data; The acoustic environment data includes a data group describing an orientation expression format for expressing an orientation characteristic, and at least one data group including data describing an orientation characteristic among the characteristics using the orientation expression format, the data group including: Data indicative of a predetermined default orientation representation; Data indicative of a set of predetermined angles; and data indicative of an angle on a quantization grid.

Images