Stabilizing rendering with varying details

Abstract

The decoder's method for decoding a bitstream includes receiving metadata in the bitstream, wherein the metadata includes a first parameter indicating the level of detail of the metadata for the received frame, and determining whether the metadata active state variable state is set to the initial state or to the level of detail of the metadata for the received frame. In response to determining that the metadata active state variable state is neither set to the initial state nor to the level of detail of the metadata for the received frame, the method increments a metadata change counter and determines whether the metadata change counter is equal to the maximum change count. In response that the metadata change counter is not equal to the maximum change count, the metadata from memory is used for rendering the received frame.

Description

Technical Field

[0001] The present disclosure generally relates to communications, and more particularly, to encoding and decoding methods, and related devices and nodes that support encoding and decoding.

Background Art

[0002] Virtual reality or augmented reality has become common in online games and is also gaining momentum in social media. The playback of a virtual scene, which is called a rendering process, involves creating a representation of the scene that includes sound, vision, and in some cases, haptic feedback such as force feedback. The quality of rendering depends not only on the rendering technology but also on the representation of the scene. In the case of audio rendering, the scene can be described using acoustic signals emitted from different sources, along with descriptions of the sources or objects, such as orientation, position, width, etc. Such a description is often called a spatial audio object representation. The parameters associated with the acoustic signals are sometimes called the metadata of the object.

[0003] In a communication scenario, the sound and metadata need to be encoded to be transmitted over a network such as a cellular network. The connection characteristics may impose certain constraints on the transmission quality, meaning that the available bandwidth needs to match the transmission characteristics.

Summary of the Invention

[0004] Currently, there is (one or more) certain problem. In the field of communications, the network transmission characteristics may change as a result of various conditions. When the channel conditions vary, the resolution of the sound and metadata may vary over time. Especially considering the metadata, the scene may exhibit abrupt changes due to abrupt changes in the metadata. Specific aspects of this disclosure and their embodiments can provide solutions to these or other problems. In various embodiments, hysteresis schemes are provided for handling fluctuating metadata resolution in decoders and renderers. Various embodiments aim to avoid abrupt changes in perceived quality that result when conditions fluctuate by introducing several conditions based on changes in the level of detail of the metadata state and adjusting the relevant parameters in response to the changes that occur.

[0005] Some embodiments provide a method performed by a decoder to decode a bitstream. The method includes receiving metadata in a bitstream, wherein the metadata includes metadata and a first parameter, the first parameter indicating the level of detail of the metadata for the received frame; and determining whether the metadata active state variable state is set to an initial state or to the level of detail of the metadata for the received frame. In response to determining that the metadata active state variable state is neither set to an initial state nor to the level of detail of the metadata for the received frame, the method includes incrementing a metadata change counter; determining whether the metadata change counter is equal to a maximum change count; using the metadata from memory as retrieved metadata in response that the metadata change counter is not equal to a maximum change count; setting the metadata change counter to 0 in response that the metadata change counter is equal to a maximum change count; setting the metadata active state variable state to the level of detail of the metadata for the received frame; and decoding the metadata for the received frame to be used as retrieved metadata. The method further includes rendering the received frame using the retrieved metadata.

[0006] The method may further include setting the metadata change counter to 0 in response to determining that the metadata active state variable state is set to its initial state or to the metadata detail level of the received frame, setting the metadata active state variable state to the metadata level of the received frame, decoding the metadata of the received frame for use as retrieved metadata, and rendering the received frame using the retrieved metadata.

[0007] Decrypting metadata may include decrypting the basic metadata as the first part of the retrieved metadata, decrypting the extended metadata for use as the second part of the retrieved metadata in response to the first parameter flag being set, and reconfiguring the extended metadata for use as the second part of the retrieved metadata in response to the first parameter flag not being set.

[0008] Using metadata from memory may further include decrypting the base metadata as the first part of the retrieved metadata and using the extended metadata from memory as the second part of the retrieved metadata.

[0009] This method may further include storing metadata in memory in order to decode and render subsequent frames when it is determined that the metadata will be used as metadata obtained to render the received frame.

[0010] This method may further include initializing each metadata parameter to its default value at the start of bitstream decoding, and storing the initialized metadata parameters in memory.

[0011] Storing initialized metadata parameters may include storing only extended metadata parameters. Initializing each metadata parameter means that each metadata parameter is This may include setting according to TIFF2026517633000002.tif24170, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source.

[0012] This method may further include setting and storing the metadata detail level in the metadata active state variable as an initial state at the start of the decryption process.

[0013] The maximum change count may include the maximum change count over a certain time period. In some embodiments, the maximum change count may include 5 change counts over a time period.

[0014] This method may further include remembering the last metadata used by the audio renderer when rendering frames of the bitstream, and performing a smooth transition from the last metadata to the currently used metadata. Performing a smooth transition may include applying a smoothing transition of rendering parameters, which is an intermediate step in rendering.

[0015] Rendering parameters may include gain parameters calculated from the orientation and distance of the audio object relative to the listener.

[0016] The level of detail may indicate whether or not extended metadata exists within the frame.

[0017] The first parameter may include the EXT_MD_BS parameter, the metadata state active variable may include the EXT_MD_ACTIVE variable, and the metadata change counter may include the EXT_MD_COUNTER counter.

[0018] Some embodiments provide a computer program comprising program code to be executed by a decoder processing circuit, thereby causing the decoder to perform operations in one of the above-described methods by executing the program code. Some embodiments provide a computer program product comprising a non-temporary storage medium containing program code to be executed by a decoder processing circuit, thereby causing the decoder to perform operations in any of the above-described methods by executing the program code.

[0019] Similar decoders, computer program products, and computer programs are provided in further embodiments.

[0020] Certain embodiments may offer one or more of the following technical advantages. The various embodiments described herein can achieve the resolution of undesirable effects of switching in a rendered scene due to changes in network transition conditions. The characteristics of the previous scene are maintained and updated based on the duration of the change in conditions to avoid constant switching when the duration of the shift during transmission is short.

[0021] To provide a further understanding of this disclosure, the accompanying drawings, which are included and incorporated into this application and form part of this application, illustrate certain non-limiting embodiments of the inventive concept. [Brief explanation of the drawing]

[0022] [Figure 1] This is a block diagram of an example of an operating environment for various embodiments. [Figure 2] This is a block diagram of a decoder according to several embodiments. [Figure 3] Figure 2 is a block diagram of a metadata decoder for a decoder that, in several embodiments, is capable of handling various levels of detail in metadata as a result of potentially varying bitrates. [Figure 4]A flowchart showing the operation of a decoder according to some embodiments. [Figure 5] A flowchart showing the operation of a decoder according to some embodiments. [Figure 6] A flowchart showing the operation of a decoder according to some embodiments. [Figure 7] A diagram showing an exemplary case of a change in the detailed level of metadata during transmission. [Figure 8] A flowchart showing the operation of a decoder according to some embodiments. [Figure 9] A flowchart showing the operation of a decoder according to some embodiments. [Figure 10] A block diagram of a decoder according to some embodiments. [Figure 11] A block diagram of a host computer communicating with an encoder and / or decoder according to some embodiments. [Figure 12] A block diagram of a virtualized environment according to some embodiments. **DETAILED DESCRIPTION OF THE INVENTION**

[0023] Next, some of the embodiments discussed in this specification will be described more fully with reference to the accompanying drawings. The embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art, and examples of embodiments of the inventive concept are shown therein. However, the inventive concept can be embodied in many different forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be complete and thorough, and will fully convey the scope of the inventive concept to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be implicitly assumed to be present / used in another embodiment.

[0024] As mentioned above, network transmission characteristics can change as a result of various conditions. When channel conditions fluctuate, the resolution of sound and metadata can fluctuate over time. Particularly considering metadata, a scene can exhibit abrupt changes due to sudden changes in metadata. The various embodiments described below provide hysteresis schemes for decoders / renderers to handle fluctuating metadata resolution. The solutions aim to avoid abrupt changes in perceived quality that result from fluctuating conditions by introducing several conditions based on changes in the level of detail of the metadata state and adjusting the relevant parameters in response to the changes that occur. In some of these various embodiments, the hysteresis logic delays rendering changes until sufficient observation of the new level of detail has been observed, allowing the use of parameters stored in memory while receiving metadata of the level of detail that is not currently being used.

[0025] The use of hysteresis in various embodiments can resolve undesirable effects of switching in the rendered scene due to changes in network transmission conditions. The properties of the previous scene are maintained and updated based on the duration of the condition change to avoid constant switching when the duration of the shift during transmission is short.

[0026] Before describing various embodiments, we will describe the operating environment in which various embodiments may be implemented. Figure 1 shows an example of an operating environment in which various embodiments of this disclosure may be implemented. Referring to Figure 1, in exemplary operating environment 100, encoder 102 receives data to be encoded, such as audio files and possibly metadata, from an entity such as host 106 and / or storage 108, via network 104. In some embodiments, host 106 may communicate directly with encoder 102. Encoder 102 encodes the audio files and scene descriptions via metadata and stores the encoded information in storage 108, or transmits the encoded audio files to decoder 112 via network 110. Decoder 112 decodes the audio files and the scene descriptions in the metadata and transmits the decoded audio files to audio player 114 for playback. Audio player 114 may be or be included in user equipment, terminals, mobile phones, etc. In other embodiments, host 106 may transmit the encoded audio files to decoder 112 via network 110.

[0027] One embodiment of the decoder 112 is shown in Figure 2. The decoder 112 is configured to render a scene of an encoded audio object bitstream. The decoder 112 receives the encoded audio object bitstream to be decoded and supplies the relevant portion to the audio decoder 210 and the metadata decoder 220. The audio decoder 210 decodes the audio from the encoded audio object bitstream, and the metadata decoder decodes the metadata parameters from the encoded audio object bitstream. The audio renderer 230 renders the audio scene using the decoded audio received from the audio decoder 210 and the decoded metadata parameters from the metadata decoder 220, and outputs the rendered audio.

[0028] Figure 3 shows another embodiment of decoder 112 configured to process various levels of detail in metadata as a result of potentially different bitrates. Referring to Figure 3, the encoded audio object bitstream is received by decoder 112 and fed to audio decoder 220 and stabilized metadata decoder 320. The audio object bitstream is processed in time segments, generally called frames. Each input audio frame yields a corresponding bitstream frame, which yields a rendered frame. The length of the frame may determine the update rate of the metadata parameters. The metadata may have two or more levels of detail, for example, location at a given distance from the listener (basic metadata, also called common metadata or level 1 metadata), and combinations of basic metadata with location at various distances and source orientations (extended metadata, also called level 2 metadata). The decoded metadata and decoded audio file are then transferred to audio renderer 230, which renders the audio file using the decoded audio file and decoded metadata. The rendered audio file is provided as the resulting output audio.

[0029] The audio renderer 230 within the decoder 112 outputs decoded and rendered audio when the decoder receives the encoded bitstream. Transmission can occur between various channel conditions, triggering the encoder to use various bitrates for the encoded bitstream. Information about the characteristics of the scene is carried via metadata that shapes the end user's experience. The decoder 112 can receive two levels of detail about the metadata. When the first level of detail is used, the basic metadata can describe the position in polar coordinates using angles, i.e., azimuth angle θ and elevation angle φ. The azimuth angle represents the position in the horizontal plane around the listener, and the elevation angle corresponds to the position in the vertical plane. The position of the source is defined using these angles, assuming a radius r of r=1.0 that places the object at a distance of 1 unit from the origin. In this case, the orientation of the source, described by angle yaw and pitch, is assumed to have values ​​of yaw=0 and pitch=0. Yaw defines the horizontal orientation of the source, while pitch is used for vertical orientation. At level 1 detail of metadata, only the basic metadata decoder is active. When a second or extended level of metadata detail is used, the radius r and orientation angles yaw and pitch are also decoded by the extended metadata decoder. Note that the first level of detail, azimuth angle θ, and elevation angle φ are subsets of the extended detail level. The two metadata levels can be summarized as follows: Metadata level 1)(azimuth angle θ, elevation angle φ) 2) (azimuth angle θ, elevation angle φ, radius r, yaw, pitch)

[0030] At the start of the decryption process, metadata parameters for both detail levels are initialized with default values ​​and stored in memory 324. A suitable set of default values ​​may be: TIFF2026517633000003.tif24170

[0031] The metadata detail level is stored in the metadata detail state 328. This may be initialized to the metadata detail level of the first received frame, or it may be set to indicate the initial state. At startup, a counter 326 that tracks the number of received frames of the opposite detail level is initialized to zero. Level 1 metadata details, also called basic metadata, are always encoded and decoded regardless of transmission conditions. If the encoding bitrate allows and input data is available, this set can be expanded to form level 2 details, providing enhanced rendering of the audio scene. If transmission conditions change, the metadata change counter 326 tracks the duration of such changes before updating the metadata detail state 328. The metadata change counter controls the status of the metadata detail state 328 and enables updates in the metadata memory 324 accordingly. In this embodiment, the basic part of the metadata (level 1 details) is decoded for use in the rendering module 230. Decoding the basic part is the same for both detail levels in the metadata. In general, the strategies described herein for stabilizing metadata parameters can be applied to any subset of the metadata, or entirely to the set. In particular, partial decoding and application of metadata is impossible when metadata levels do not share any overlap. In other words, basic metadata is decoded and used in every frame without regard to whether or not extended metadata exists, as its resolution does not change. Note that the principles described herein also apply when a set of metadata parameters is the same at at least two levels of detail, where only the resolution of the parameters differs. Switching between metadata parameter resolutions can result in abrupt changes in the rendered scene, which can be mitigated by the hysteresis logic described herein.

[0032] Figure 4 is a flowchart illustrating the operation performed by decoder 112 using, for example, a stabilized metadata decoder 320. In step 401, decoder 112 receives a frame of metadata bitstream, which contains the parameter EXT_MD_BS. The EXT_MD_BS parameter indicates the level of metadata detail in the received frame. In some embodiments, the frame contains the EXT_MD_BS flag EXT_MD_BS∈{TRUE,FALSE} from the bitstream. The EXT_MD_BS flag indicates whether the received metadata detail in the bitstream is level 2 (or higher) or not. In some embodiments, a true EXT_MD_BS flag indicates that the received metadata detail is level 2 (or higher), and a false EXT_MD_BS indicates that the received metadata is not level 2 (or higher) (i.e., level 1). In other embodiments, a true EXT_MD_BS flag indicates that the received metadata detail is not level 2 (or higher), and a false EXT_MD_BS indicates that the received metadata is level 2 (or higher). In the following explanation, a true EXT_MD_BS flag indicates that the received metadata details are level 2 (or higher), while a false EXT_MD_BS flag indicates that the received metadata is not level 2 (or higher).

[0033] In other words, this carries information about the level of detail in bitstream metadata. For example, Level 2 detail metadata contains higher detail in the scene, resulting in different rendering behavior. This enhanced format of metadata (Level 2 detail) enables a richer end-user experience.

[0034] The metadata active state variable EXT_MD_ACTIVE ∈ {INIT, TRUE, FALSE} (shown in Figure 3 as metadata detail state 328) indicates whether the current metadata detail state is active. The value INIT is used for the first frame when the decoder is started, in which case the frame has not yet been decoded. TRUE indicates that metadata level 2 detail (or higher) is active, and FALSE indicates that metadata level 1 detail is active. These values ​​can be represented using integer values, for example, {-1, 0, 1}, respectively. In other embodiments, TRUE indicates that metadata level 1 detail is active, and FALSE indicates that metadata level 2 detail (or higher) is active. In the following description, TRUE indicates that metadata level 2 detail (or higher) is active, and FALSE indicates that metadata level 1 detail is active.

[0035] In step 403, the decoder 112 reviews the value of EXT_MD_ACTIVE to determine whether the received frame is the first frame (i.e., EXT_MD_ACTIVE=INIT) or whether EXT_MD_ACTIVE=EXT_MD_BS. In other words, the decoder 112 determines whether the received frame is the first frame being decoded (i.e., EXT_MD_ACTIVE=INIT) or whether the received metadata detail state EXT_MD_BS is the same as the metadata level currently in use (i.e., EXT_MD_ACTIVE=EXT_MD_BS).

[0036] If any one or both of these conditions are true, the decoder 112 sets the metadata change counter EXT_MD_COUNTER to 0 in step 405 and sets EXT_MD_ACTIVE (stored in metadata detail state 328) to the received metadata detail state EXT_MD_BS in step 407: TIFF2026517633000004.tif9170 Here, TIFF2026517633000005.tif3170 indicates the allocation.

[0037] EXT_MD_COUNTER corresponds to a counter for the number of frames since a change in the conditions occurred, and is shown in Figure 3 as metadata change counter 326.

[0038] In step 409, the decoder 112 decodes the received metadata to be used as acquired metadata. If the level 1 metadata is a subset of the level 2 metadata, the level 1 metadata may be called basic metadata, and the additional features of the level 2 metadata may be called extended metadata.

[0039] Refer briefly to Figure 5, which illustrates an embodiment for decoding metadata. Referring to Figure 5, in step 501, the decoder 112 decodes the basic metadata to be used as the first part of the acquired metadata. In step 503, the decoder 112 determines whether the received metadata format (EXT_MD_BS) is TRUE, meaning that the extended metadata is included in the bitstream (e.g., Level 2 metadata). If the received metadata format (EXT_MD_BS) is TRUE, the decoder 112 decodes the extended metadata in step 505 to be used as the second part of the acquired metadata. In some embodiments, this example corresponds to encoding and decoding Level 2 detailed metadata (extended metadata) without changing the transmission conditions.

[0040] If the received metadata format (EXT_MD_BS) is not set (for example, FALSE), i.e., if the received metadata is at the basic metadata detail level, while the value of the active metadata (metadata detail state) EXT_MD_ACTIVE is also FALSE (level 1), the decoder 112 resets the extended metadata memory to their initial values ​​in step 507. This action ensures that if both the detail level and metadata detail state of the current frame are 1, the extended metadata memory is reset to its default values ​​and can be used as a second part of the retrieved metadata.

[0041] Returning to Figure 4, if EXT_MD_ACTIVE is not equal to EXT_MD_BS or INIT in step 403, it corresponds to a frame that follows initialization where the metadata detail state EXT_MD_ACTIVE is different from the received metadata detail level EXT_MD_BS. This step allows the system to recognize a condition, in this case a change in the detail level in the metadata of the received frame. This can be expressed as when the following condition is true: (EXT_MD_ACTIVE≠EXT_MD_BS) and (EXT_MD_ACTIVE≠INIT) In step 411, decoder 112 increments EXT_MD_COUNTER by 1.

[0042] Following the increment, in step 413, decoder 112 compares EXT_MD_COUNTER to a threshold to determine whether the counter has reached the maximum number of frame changes (over a specified time period). EXT_MD_COUNTER=MAX_CHANGE_FRAMES?

[0043] This condition ensures that settings from the last frame are saved to avoid glitches for shorter-term changes within the condition. However, if the change lasts for a long period (i.e., equal to MAX_CHANGE_FRAMES), the change in the condition takes effect.

[0044] Therefore, in step 413, if EXT_MD_COUNTER reaches a threshold, that is, if a certain number of changes in the level of detail have occurred over a predetermined time window, the decoder 112 sets EXT_MD_COUNTER to 0 in step 405 and updates the metadata detail state at the current level of detail (for example, metadata level 1 or metadata level 2 (or higher)) in step 407. TIFF2026517633000006.tif9170

[0045] In step 409, the decoder 112 decodes the received metadata to be used as acquired metadata. If the level 1 metadata is a subset of the level 2 metadata, the level 1 metadata may be called basic metadata, and additional features of the level 2 metadata may be called extended metadata, and these can be decoded as described above in the steps shown in steps 501-507 of Figure 5.

[0046] Instead, in step 413, decoder 112 EXT_MD_COUNTER≠MAX_CHANGE_FRAMES If it is determined that this is the case, the decoder 112 uses the metadata from the metadata memory in step 415.

[0047] If Level 1 metadata is a subset of Level 2 metadata, the additional features of Level 2 metadata are sometimes called extended metadata. In this case, the use of metadata from memory 415 can be further detailed as shown in Figure 6. In step 601, the basic metadata is decoded as the first part of the retrieved metadata. In step 603, the decoder 112 determines the level of detail of the received metadata by checking whether EXT_MD_BS is set (for example, set to TRUE). If EXT_MD_BS is set, extended metadata is present in the bitstream. This can be decoded without being used in step 605 as a way for the decoder to maintain bitstream synchronization. In step 607, the extended metadata from memory is used as the second part of the retrieved metadata to form the retrieved metadata to be used in the current frame. If EXT_MD_BS is not set in step 603, the extended metadata is not present in the bitstream, and the decoder 112 proceeds directly to step 607, where the extended metadata from memory is used together with the decoded basic metadata to form the retrieved metadata to be used in the current frame.

[0048] Returning to step 415 in Figure 4, this terminates the process if there has been a change in conditions such as the level of detail of the metadata. However, the change only occurs within a short time window. Therefore, updates to the metadata are withheld for a period of time until the conditions stabilize. As a result, the metadata from metadata memory is used to render the current frame to avoid abrupt switching.

[0049] Following steps 409 and 415, the metadata obtained from either step 409 or 415 is used to render the received frame in step 416. The metadata used for rendering the current frame is maintained in memory 324 for processing subsequent frames. Continuing to update this memory can be implicitly done by addressing it when updating the metadata for the current frame. This is shown as an optional step 417.

[0050] Figure 7 illustrates an exemplary example of the change in the level of detail of metadata during transmission. In the top plot of Figure 7, the current level of detail (EXT_MD_BS) in the metadata from the bitstream is shown on the y-axis via the transmission timeline on the x-axis. Level "1" represents an extended metadata format (corresponding to Level 2 metadata in the description above) that includes radius, yaw, and pitch along with azimuth and elevation, while Level "0" represents a basic metadata format (corresponding to Level 1 metadata in the description above) that includes only azimuth and elevation for the position of the sound object.

[0051] The second plot from the top in Figure 7 shows the values ​​of the different detail counter EXT_MD_COUNTER, which counts the number of frames when the conditions of the top plot change. When the counter reaches the threshold MAX_CHANGE_FRAMES, which is shown as "5" in this figure, the counter is reset to "0".

[0052] Active metadata (also known as detailed memory) is shown in the third plot from the top in Figure 7 and represents EXT_MD_ACTIVE. In the initial frame, this is set to the value in the first plot, EXT_MD_BS. Subsequently, whenever there is a change in the conditions trackable by the first plot, the counter in the second plot begins counting the number of frames during the change period. When the counter EXT_MD_COUNTER reaches the threshold MAX_CHANGE_FRAMES, the detailed memory EXT_MD_ACTIVE in the third plot is updated and set to the current level of detail from the first plot, EXT_MD_BS.

[0053] The bottom plot in Figure 7 corresponds to the update of the extended metadata. The extended metadata, consisting of yaw, pitch, and radius, is updated only when the counter is "0," which means either that there is no change in the condition or that the change is valid for longer than the specified time window. In these cases, the extended metadata memory is updated with the received metadata for level 2 details and with the default value for level 1 details.

[0054] As mentioned above, at the start of the decoding process, the decoder 112 initializes the metadata parameters for both levels of detail to their default values ​​and stores the initialized metadata parameters in memory 324. This is shown in Figure 8. Referring to Figure 8, at the start of decoding the bitstream, in step 801, the decoder 112 initializes each metadata parameter to its default value.

[0055] In some embodiments, as described above, the decoder 112 processes each metadata parameter, Each metadata parameter is initialized by setting it according to TIFF2026517633000007.tif24170, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source.

[0056] In step 803, the decoder 112 stores the initialized metadata parameters in memory. In step 805, the decoder 112 sets the metadata level in EXT_MD_ACTIVE to its initial state and stores it.

[0057] Even with the stabilization described herein, there may be abrupt changes in the metadata input to the audio renderer 230. In one embodiment, the audio renderer is configured to use a smoothing technique to equalize abrupt changes in the scene description, rather than allowing abrupt jumps between frames. Smoothing can be enabled by including a memory of the last metadata in the audio renderer 230 and performing a smooth transition from the last to the current metadata. This smoothing may also be part of the stabilization metadata decoder 320. Furthermore, this may be applied to rendering parameters that are intermediate steps in rendering, such as gain parameters calculated from the orientation and distance of audio objects relative to the listener. This is shown in Figure 9, where in step 901, the decoder 112 stores the last metadata used by the audio renderer when rendering frames of the bitstream, and in step 903, performs a smooth transition from the last metadata to the current metadata being used.

[0058] In another exemplary embodiment, the encoded metadata has at least two levels of detail, and parameters are present at all levels of detail but quantized at different resolutions. Changes in parameter resolution can cause undesirable artifacts and discontinuities in audio rendering. The parameters used and the audio rendering can be stabilized using the principles described above. The aforementioned embodiments having levels of detail with different numbers of metadata parameters can also be viewed as different levels, where lower levels use 0 bits to encode metadata parameters that do not exist. For 0-bit parameters, default values ​​are used. Default values ​​can be seen as a codebook for bitless decoding sent for these parameters.

[0059] Figure 10 shows an audio decoder 112 (for example, a decoder) according to several embodiments, where the audio decoder 112 is implemented as a standalone device. As used herein, an audio object renderer refers to a device that is configured, set up, and / or operable to decode encoded objects and to communicate with network nodes, encoders, and / or decoders. Examples of audio object renderers include, but are not limited to, smartphones, mobile phones, cell phones, voice over IP (VoIP) phones, wireless local loop phones, desktop computers, personal digital assistants (PDAs), wireless cameras, gaming consoles or devices, storage devices, playback devices, wearable terminal devices, wireless endpoints, mobile stations, tablets, laptop computers, laptop embedded equipment (LEE), laptop mounted equipment (LME), smart devices, wireless customer premises equipment (CPE), and vehicle-mounted or vehicle-embedded / integrated wireless devices.

[0060] The audio decoder 112 may support D2D (device-to-device) communication by implementing 3GPP standards for, for example, sidelink communication, dedicated short-range communication (DSRC), V2V (Vehicle-to-Vehicle), V2I (Vehicle-to-Infrastructure), or V2X (Vehicle-to-Everything). In other examples, the decoder may not necessarily have a user in the sense of a human user who owns and / or operates the associated device.

[0061] The audio decoder 112 includes a processing circuit 1002 operably coupled via the bus 1004 to an input / output interface 1006, a power supply 1008, a memory 1010, a communication interface 1012, and / or any other components, or any combination thereof. Some decoders may utilize all or a subset of the components shown in Figure 10. The level of integration between components may vary from decoder to decoder. Furthermore, some decoders may include multiple instances of components, such as multiple processors, memories, transceivers, transmitters, and receivers.

[0062] The processing circuit 1002 is configured to process instructions and data and may be configured to implement any sequential state machine capable of executing instructions stored in memory 1010 as machine-readable computer programs. The processing circuit 1002 may be implemented as one or more stored computer programs, general-purpose processors, or any combination of the above, such as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), programmable logic with appropriate firmware, or microprocessors or digital signal processors (DSPs) with appropriate software. For example, the processing circuit 1002 may include multiple central processing units (CPUs).

[0063] In this example, the input / output interface 1006 may be configured to provide one or more interfaces to an input device, an output device, or one or more input and / or output devices. Examples of output devices include speakers, sound cards, video cards, displays, monitors, printers, actuators, emitters, smart cards, other output devices, or any combination thereof. Input devices may allow a user to capture information to the audio decoder 112. Examples of input devices include touch-sensitive or presence-sensitive displays, cameras (e.g., digital cameras, digital video cameras, webcams, etc.), microphones, sensors, mice, trackballs, directional pads, trackpads, scroll wheels, smart cards, etc. A presence-sensitive display may include capacitive or resistive touch sensors for detecting user input. Sensors may include, for example, accelerometers, gyroscopes, tilt sensors, force sensors, magnetometers, light sensors, proximity sensors, biosensors, or any combination thereof. Output devices may use the same type of interface port as input devices. For example, a Universal Serial Bus (USB) port may be used to provide input and output devices.

[0064] In some embodiments, the power supply 1008 is constructed as a battery or battery pack. Other types of power sources may be used, such as an external power source (e.g., an electrical outlet), a photovoltaic device, or a battery. The power supply 1008 may further include a power circuit for delivering power to various parts of the audio decoder 112 from the power supply 1008 itself and / or from an external power source via an interface such as an input circuit or a power cable. Delivering power may be, for example, for charging the power supply 1008. The power circuit may perform any formatting, converting, or other modifications to the power from the power supply 1008 to make that power suitable for each component of the audio decoder 112 being powered.

[0065] Memory 1010 is or can be configured to include random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and other types of memory. In one example, memory 1010 includes one or more application programs 1014, such as an operating system, a web browser application, a widget, a gadget engine, or another application, and corresponding data 1016. Memory 1010 may store any of a variety of operating systems or combinations of operating systems for use by the audio decoder 112.

[0066] Memory 1010 may be configured to include several physical drive units such as a redundant array of independent disks (RAID), flash memory, a USB flash drive, an external hard disk drive, a thumb drive, a pen drive, a key drive, a high-density digital versatile disk (HD-DVD) optical disk drive, an internal hard disk drive, a Blu-ray optical disk drive, a holographic digital data storage (HDDS) optical disk drive, an external mini dual in-line memory module (DIMM), a synchronous dynamic random access memory (SDRAM), an external microDIMM SDRAM, a smart card memory such as a tamper-resistant module in the form of a universal integrated circuit card (UICC) containing one or more subscriber identification modules (SIMs) such as USIM and / or ISIM, other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC commonly known as a "SIM card". Memory 1010 may enable the audio decoder 112 to access instructions, application programs, etc., stored in temporary or non-temporary memory media, to offload data, or to upload data. Products that utilize a communication system, such as manufactured goods, may be tangibly embodied as memory 1010 or within memory 1010, and memory 1010 may be a device-readable storage medium or may include a device-readable storage medium.

[0067] The processing circuit 1002 may be configured to communicate with an access network or other networks using a communication interface 1012. The communication interface 1012 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1022. The communication interface 1012 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or network node in the access network). Each transceiver may include a transmitter 1018 and / or receiver 1020 suitable for providing network communication (e.g., optical, electrical, frequency allocation, etc.). Furthermore, the transmitter 1018 and receiver 1020 may be coupled to one or more antennas (e.g., antenna 1022) and may share circuit components, software or firmware, or alternatively, be implemented separately.

[0068] In the illustrated embodiment, the communication functions of the communication interface 1012 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communication such as Bluetooth, near-field communication, location-based communication such as the use of the Global Positioning System (GPS) for determining location, other similar communication functions, or any combination thereof. The communication may be implemented in accordance with one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMAX, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0069] Regardless of the sensor type, the audio object renderer can provide the decoded data output to a network node via a wireless connection through its communication interface 1012.

[0070] An audio decoder, in the form of an Internet of Things (IoT) device, can be a device for use in one or more application areas, including, but not limited to, urban wearable technology, augmented industrial applications, and healthcare. Non-exclusive examples of such IoT devices include connected refrigerators or freezers, TVs, connected lighting devices, electricity meters, robotic vacuum cleaners, voice-controlled smart speakers, home security cameras, thermostats, electric door locks, connected doorbells, autonomous vehicles, surveillance systems, weather monitoring devices, vehicle parking monitoring devices, electric vehicle charging stations, smartwatches, fitness trackers, head-mounted displays for augmented reality (AR) or virtual reality (VR), and wearables for haptic augmentation or perceptual augmentation, or devices incorporated into them. A decoder in the form of an IoT device comprises circuitry and / or software depending on the intended use of the IoT device, in addition to the other components described with respect to the audio decoder 112 shown in Figure 10.

[0071] Figure 11 is a block diagram of host 1100 in various embodiments described herein. Host 1100 as used herein is or may comprise various combinations of hardware and / or software, including standalone servers, blade servers, cloud implementation servers, distributed servers, virtual machines, containers, or processing resources within a server farm. Host 1100 may provide one or more services to one or more encoders and / or decoders and one or more UEs.

[0072] The host 1100 includes an input / output interface 1106, a network interface 1108, a power supply 1110, and a processing circuit 1102 operably coupled via a bus 1104 to memory 1112. Other components may be included in other embodiments. The features of these components may be substantially the same as those described with respect to the devices in previous figures, such as Figure 10, and therefore their descriptions are generally applicable to the corresponding components of the host 1100.

[0073] Memory 1112 may include one or more computer programs, each containing one or more host application programs 1114 and data 1116, the data 1116 of which may include user data, for example, data generated by the UE for host 1100, or data generated by host 1100 for the UE. Embodiments of host 1000 may utilize only a subset or all of the components shown. Host application programs 1114 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Multipurpose Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementation forms of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application program 1114 can also provide user authentication and licensing checks and periodically report health, route, and content availability to central nodes such as devices within or on the edge of the core network. Thus, host 1100 can select and / or direct different hosts for over-the-top services for encoders or decoders. The host application program 1114 can support various protocols such as HTTP Live Streaming (HLS), Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), and Dynamic Adaptive Streaming over HTTP (MPEG-DASH).

[0074] Figure 12 is a block diagram showing a virtualization environment 1200 in which functions implemented by several embodiments of the audio decoder 112 or components of the audio decoder 112 may be virtualized. In this context, virtualization means creating a virtual version of a device or apparatus, which may include virtualizing hardware platforms, storage devices, and networking resources. The virtualization used herein may apply to any device or component thereof described herein and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functionality described herein may be implemented as virtual components, run by one or more virtual machines (VMs) implemented in one or more virtual environments 1200 hosted by one or more hardware nodes, such as a decoder, encoder, network node, UE, core network node, or hardware computing device acting as a host. Furthermore, in embodiments in which the virtual node does not require wireless connectivity (e.g., a core network node or host), the node may be fully virtualized.

[0075] Application 1202 (which may alternatively be referred to as a software instance, virtual appliance, network function, virtual node, virtual network function, etc.) runs in the virtualization environment 1200 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0076] Hardware 1204 includes processing circuits, memory for storing software and / or instructions executable by the hardware processing circuits, and / or other hardware devices described herein, such as network interfaces and input / output interfaces. The software is executed by the processing circuits to instantiate one or more virtualization layers 1206 (also called hypervisors or virtual machine monitors (VMMs)), providing VM1208A and 1208B (one or more of which may commonly be referred to as VM1208), and / or may implement any of the functions, features, and / or benefits described with respect to some embodiments described herein. The virtualization layer 1206 can present a virtual operating platform that appears to the VM1208 as networking hardware.

[0077] VM1208 may feature virtual processing, virtual memory, virtual networking, or interfaces, and virtual storage, and may be powered by the corresponding virtualization layer 1206. Different embodiments of instances of the virtual appliance 1202 may be implemented in one or more of the VM1208, and the implementation may be carried out in different ways. Hardware virtualization is referred to in some contexts as network function virtualization (NFV). NFV can be used to aggregate many types of network equipment on industry-standard high-volume server hardware, physical switches, and physical storage that can be located in data centers and customer premises equipment (CPE).

[0078] In the context of NFV, VM1208 can be a software implementation of a physical machine that runs the program as if it were running on a physical, non-virtualized machine. Each VM1208, and that portion of the hardware 1204 running its VM, forms a separate virtual network element, whether it is dedicated hardware for that VM or / or hardware shared by that VM and other VMs. Furthermore, in the context of NFV, the virtual network function is responsible for handling specific network functions running on one or more VM1108 on the hardware 1204, corresponding to application 1202.

[0079] Hardware 1204 may be implemented in a standalone network node with general or specific components. Hardware 1204 may implement several functions through virtualization. Alternatively, hardware 1204 may be part of a larger cluster of hardware (e.g., within a data center or CPE) where many hardware nodes cooperate and are managed via management and organization 1210, which oversees, among other things, the lifecycle management of application 1202. In some embodiments, hardware 1204 is coupled to one or more radio units, each including one or more transmitters and one or more receivers that can be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more suitable network interfaces and may be used in combination with virtual components to provide a virtual node with radio capabilities, such as a radio access node or base station. In some embodiments, some signaling may be provided using a control system 1212, which may be used alternatively for communication between hardware nodes and radio units.

[0080] The computing devices described herein (e.g., decoders, audio object renderers, encoders, hosts) may include the shown combinations of hardware components, but other embodiments may comprise computing devices with different combinations of components. It should be understood that these computing devices may comprise any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determining, calculating, acquiring, or similar operations described herein may be performed by processing circuits, which may process information by, for example, converting acquired information to other information, comparing acquired or converted information with information stored in network nodes, and / or performing one or more operations based on the acquired or converted information and as a result of the processing making decisions. Furthermore, although components are shown as a single box located within a larger box, or as a single box nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that constitute a single shown component, and functions may be separated between the distinct components. For example, a communication interface may be configured to include any of the components described herein, and / or the functions of the components may be separated between the processing circuit and the communication interface. In another example, the non-computation-intensive functions of any of such components may be implemented in software or firmware, while the computation-intensive functions may be implemented in hardware.

[0081] In certain embodiments, some or all of the functions described herein may be provided by a processing circuit that executes instructions stored in memory, which in certain embodiments may be a computer program product in the form of a non-temporary computer-readable storage medium. In alternative embodiments, some or all of the functions may be provided by a processing circuit without executing instructions stored in a separate or individual device-readable storage medium, such as in a hardwired manner. In any of those particular embodiments, whether or not it executes instructions stored in a non-temporary computer-readable storage medium, the processing circuit may be configured to perform the functions described. The benefits provided by such functions are enjoyed by the processing circuit alone, or by the computing device as a whole, but not limited to other components of the computing device, and / or generally by the end user and the wireless network. Embodiment Embodiment 1. A method performed by a decoder (112, 1102) to decode a bitstream, Receiving a frame of metadata bitstream (401), wherein the frame contains the parameter EXT_MD_BS, and the EXT_MD_BS parameter indicates the level of detail of the metadata of the received frame (401), Determine whether the metadata active state variable EXT_MD_ACTIVE state is set to the initial state or to the level of detail of the metadata of the received frame (403), In response to the determination that the EXT_MD_ACTIVE state is not set to the initial state and is not set to the level of detail of metadata for the received frame, Increment the metadata change counter EXT_MD_COUNTER (411), Determine whether EXT_MD_COUNTER is equal to the maximum change count (413), In response to EXT_MD_COUNTER not being equal to the maximum change count, use metadata from memory to render the received frame (415), In response that EXT_MD_COUNTER is equal to the maximum change count, Set EXT_MD_COUNTER to 0 (405), Setting the EXT_MD_ACTIVE status to the metadata detail level of the received frame (407), Decrypting and using the metadata of the received frame for use when rendering the received frame (409) Methods that include... Embodiment 2. In response to determining that the EXT_MD_ACTIVE state is set to the initial state or to the metadata detail level of the received frame, Set EXT_MD_COUNTER to 0 (405), Setting the EXT_MD_ACTIVE state to the metadata level of the received frame (407), Decrypting and using the received metadata of the received frame when rendering the received frame (409) The method according to Embodiment 1, further comprising: Embodiment 3. Decrypting the received metadata, In response to the EXT_MD_BS flag being true (503), decode and use the metadata of the received frame when rendering the received frame (505), In response to the EXT_MD_BS flag being false (507), the Level 1 metadata is decoded for use when rendering the received frame, and the metadata is reset for use when rendering the received frame. Render the received frame using metadata determined based on the EXT_MD_BS flag (416) A method according to any one of Embodiments 1 to 2, including the method described above. Embodiment 4. When metadata is determined to be used to render the received frame, Storing metadata in memory in order to decode and render subsequent frames (417) The method according to any one of embodiments 1 to 3, further comprising: Embodiment 5. At the start of bitstream decoding, Initialize each metadata parameter in Level 1 metadata and Level 2 metadata to its default value (801), Storing initialized metadata parameters in memory (803) The method according to any one of embodiments 1 to 4, further comprising: Embodiment 6. Initializing each metadata parameter means that each metadata parameter is The method according to Embodiment 5, comprising setting according to TIFF2026517633000008.tif24170, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source. Embodiment 7. Set the level of detail of the metadata in EXT_MD_ACTIVE to its initial state and store it (705). A method according to any one of embodiments 1 to 6, further comprising: Embodiment 8. The method according to any one of Embodiments 1 to 7, wherein the maximum change count includes the maximum change count over a certain time period. Embodiment 9. The method according to Embodiment 8, wherein the maximum change count includes 5 change counts over a period of time. Embodiment 10. Storing the last metadata used by the audio renderer when rendering a bitstream frame (901), To perform a smooth transition from the last metadata to the currently used metadata (903) and The method according to any one of embodiments 1 to 8, further comprising: Embodiment 11. The method according to Embodiment 10, comprising applying a smoothing transition of rendering parameters, wherein performing a smooth transition is an intermediate step in rendering. Embodiment 12. The method according to Embodiment 11, wherein the rendering parameters include gain parameters calculated from the orientation and distance of the audio object to the listener. Embodiment 13. Decoders (112, 1102) adapted to perform an operation, wherein the operation is Receiving a frame of metadata bitstream (401), wherein the frame contains the parameter EXT_MD_BS, and the EXT_MD_BS parameter indicates the level of detail of the metadata of the received frame (401), Determine whether the metadata active state variable EXT_MD_ACTIVE state is set to the initial state or to the level of detail of the metadata of the received frame (403), In response to the determination that the EXT_MD_ACTIVE state is not set to the initial state and is not set to the level of detail of metadata for the received frame, Increment the metadata change counter EXT_MD_COUNTER (411), Determine whether EXT_MD_COUNTER is equal to the maximum change count (413), In response to EXT_MD_COUNTER not being equal to the maximum change count, use metadata from memory to use for rendering the received frame (415), In response that EXT_MD_COUNTER is equal to the maximum change count, Set EXT_MD_COUNTER to 0 (405), Setting the EXT_MD_ACTIVE status to the metadata detail level of the received frame (407), Decrypting and using the metadata of the received frame for use when rendering the received frame (409) Decoders (112, 1102), including the above. Embodiment 14. The decoder (112, 1102) is In response to determining that the EXT_MD_ACTIVE state is set to the initial state or to the metadata detail level of the received frame, Set EXT_MD_COUNTER to 0 (405), Setting the EXT_MD_ACTIVE state to the metadata level of the received frame (407), Decrypting and using the received metadata of the received frame for use when rendering the received frame (409) The decoder (112, 1102) described in Embodiment 13 is further adapted to perform operations including the above. Embodiment 15. Decrypting and using the received metadata, In response to the EXT_MD_BS flag being true (503), the metadata of the received frame is decoded and used for use when rendering the received frame (505), In response to the EXT_MD_BS flag being false (507), the Level 1 metadata is decoded for use when rendering the received frame, and the metadata is reset for use when rendering the received frame. Render the received frame using metadata determined based on the EXT_MD_BS flag (416) Decoders (112, 1102) according to any one of embodiments 13 to 14, including the above. Embodiment 16. Decoders (112, 1102) are When metadata is determined to be used to render the received frame, Storing metadata in memory in order to decode and render subsequent frames (417) Decoders (112, 1102) according to any one of embodiments 13 to 15, which are further adapted to perform operations including the above. Embodiment 17. Decoders (112, 1102) are At the start of bitstream decoding, Initialize each metadata parameter in Level 1 metadata and Level 2 metadata to its default value (801), Storing initialized metadata parameters in memory (803) Decoders (112, 1102) according to any one of embodiments 13 to 16, which are further adapted to perform operations including the above. Embodiment 18. Initializing each metadata parameter, each metadata parameter, The method according to Embodiment 17, comprising setting according to TIFF2026517633000009.tif24170, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source. Embodiment 19. Decoders (112, 1102) are Set the metadata detail level within EXT_MD_ACTIVE to its initial state (805) and save it. Decoders (112, 1102) according to any one of embodiments 13 to 18, which are further adapted to perform operations including the above. Embodiment 20. A decoder according to any one of Embodiments 13 to 19 (112, 1102), wherein the maximum change count includes the maximum change count over a certain time period. Embodiment 21. The decoder (112, 1102) according to Embodiment 20, wherein the maximum change count includes 5 change counts over a time period. Embodiment 22. Decoders (112, 1102) are Remembering the last metadata used by the audio renderer when rendering a bitstream frame (901), To perform a smooth transition from the last metadata to the currently used metadata (903) and Decoders (112, 1102) according to any one of embodiments 13 to 21, which are further adapted to perform operations including the above. Embodiment 23. The decoder (112, 1102) according to Embodiment 22, which includes applying a smoothing transition of rendering parameters, wherein performing a smooth transition is an intermediate step in rendering. Embodiment 24. The decoder (112, 1102) according to Embodiment 23, wherein the rendering parameters include gain parameters calculated from the orientation and distance of the audio object to the listener. Embodiment 25. Processing circuit (802) and, Memory (801) coupled to the processing circuit and A decoder (112, 1102) comprising, When the memory is executed by the processing circuit, the decoder (112, 1102) receives the following information: Receiving a frame of metadata bitstream (401), wherein the frame contains the parameter EXT_MD_BS, and the EXT_MD_BS parameter indicates the level of detail of the metadata of the received frame (401), Determine whether the metadata active state variable EXT_MD_ACTIVE state is set to the initial state or to the level of detail of the metadata of the received frame (403), In response to the determination that the EXT_MD_ACTIVE state is not set to the initial state and is not set to the level of detail of metadata for the received frame, Increment the metadata change counter EXT_MD_COUNTER (411), Determine whether EXT_MD_COUNTER is equal to the maximum change count (413), In response to EXT_MD_COUNTER not being equal to the maximum change count, use metadata from memory to use for rendering the received frame (415), In response that EXT_MD_COUNTER is equal to the maximum change count, Set EXT_MD_COUNTER to 0 (405), Setting the EXT_MD_ACTIVE status to the metadata detail level of the received frame (407), Decrypting and using the metadata of the received frame for use when rendering the received frame (409) Decoders (112, 1102) that include instructions to perform an operation including the above. Embodiment 26. When the memory is executed by the processing circuit, the decoder (112, 1102) receives the following information: In response to determining that the EXT_MD_ACTIVE state is set to the initial state or to the metadata detail level of the received frame, Set EXT_MD_COUNTER to 0 (405), Setting the EXT_MD_ACTIVE state to the metadata level of the received frame (407), Decrypting and using the received metadata of the received frame for use when rendering the received frame (409) Decoder(112, 1102) according to Embodiment 25, including further instructions to perform further operations including the above. Embodiment 27. Decrypting and using the received metadata, In response to the EXT_MD_BS flag being true (503), the metadata of the received frame is decoded and used for use when rendering the received frame (505), In response to the EXT_MD_BS flag being false (507), the Level 1 metadata is decoded for use when rendering the received frame, and the metadata is reset for use when rendering the received frame. Render the received frame using metadata determined based on the EXT_MD_BS flag (507) Decoders (112, 1102) according to any one of embodiments 25 to 26, including the above. Embodiment 28. When the memory is executed by the processing circuit, the decoder (112, 1102) receives the following information: When metadata is determined to be used to render the received frame, Storing metadata in memory in order to decode and render subsequent frames (417) A decoder (112, 1102) according to any one of embodiments 25 to 27, including further instructions to perform further operations including the above. Embodiment 29. When the memory is executed by the processing circuit, the decoder (112, 1102) receives the following information: At the start of bitstream decoding, Initialize each metadata parameter in Level 1 metadata and Level 2 metadata to its default value (801), Storing initialized metadata parameters in memory (803) A decoder (112, 1102) according to any one of embodiments 25 to 28, including further instructions to perform further operations including the above. Embodiment 30. Initializing each metadata parameter, The method according to Embodiment 29, comprising setting according to TIFF2026517633000010.tif24170, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source. Embodiment 31. When the memory is executed by the processing circuit, the decoder (112, 1102) receives the following information: Set the metadata detail level within EXT_MD_ACTIVE to its initial state (805) and save it. A decoder (112, 1102) according to any one of embodiments 25 to 30, including further instructions to perform further operations including the above. Embodiment 32. A decoder according to any one of Embodiments 25 to 31 (112, 1102), wherein the maximum change count includes the maximum change count over a certain time period. Embodiment 33. The decoder (112, 1102) according to Embodiment 32, wherein the maximum change count includes 5 change counts over a time period. Embodiment 34. When the memory is executed by the processing circuit, the decoder (112, 1102) receives the following information: Remembering the last metadata used by the audio renderer when rendering a bitstream frame (901), To perform a smooth transition from the last metadata to the currently used metadata (903) and A decoder (112, 1102) according to any one of embodiments 25 to 33, including further instructions to perform further operations including the above. Embodiment 35. The decoder (112, 1102) according to Embodiment 34, which includes applying a smoothing transition of rendering parameters, wherein performing a smooth transition is an intermediate step in rendering. Embodiment 36. The decoder (112, 1102) according to Embodiment 35, wherein the rendering parameters include gain parameters calculated from the orientation and distance of the audio object to the listener. Embodiment 37. A computer program comprising program code to be executed by a processing circuit (802) of a decoder (112, 1102), wherein the computer program causes the decoder (112, 1102) to perform the operation described in any one of Embodiments 1 to 12 by executing the program code. Embodiment 38. A computer program product comprising a non-temporary storage medium containing program code to be executed by a processing circuit (802) of a decoder (112, 1102), wherein the program code is executed to cause the decoder (112, 1102) to perform the operation described in any one of Embodiments 1 to 12.

Claims

1. A method performed by a decoder (112) to decode a bitstream, Receiving metadata in a bitstream (401), wherein the metadata includes a first parameter, the first parameter indicating the level of detail of the metadata of the received frame (401), (403) Determining whether the metadata active state variable state is set to the initial state or to the level of detail of the metadata of the received frame, In response to the determination that the metadata active state variable state is not set to the initial state and is not set to the level of detail of the metadata of the received frame, Incrementing the metadata change counter (411), (413) Determining whether the metadata change counter is equal to the maximum change count, In response to the metadata change counter not being equal to the maximum change count, the metadata retrieved from memory is used as the metadata (415), In response that the metadata change counter is equal to the maximum change count, Setting the metadata change counter to 0 (405), Setting the metadata active state variable state to the level of detail of the metadata of the received frame (407), Decoding the metadata of the received frame so that it can be used as the acquired metadata (409), Rendering the received frame using the acquired metadata (416) Methods that include...

2. In response to determining that the metadata active state variable state is set to the initial state or to the metadata detail level of the received frame, Setting the metadata change counter to 0 (405), Setting the metadata active state variable state to the metadata level of the received frame (407), Decoding the metadata of the received frame so that it can be used as the acquired metadata (409), Rendering the received frame using the acquired metadata (416) The method according to claim 1, further comprising:

3. Decrypting the metadata (409) Decoding the basic metadata so that it can be used as the first part of the acquired metadata (501), In response to the first parameter flag being set (503), the extended metadata is decoded (505) to be used as a second part of the acquired metadata, In response to the fact that the first parameter flag is not set (507), the extended metadata is reconfigured to be used as a second part of the acquired metadata. The method according to any one of claims 1 to 2, including

4. Using metadata from memory (415) Decoding the basic metadata so that it can be used as the first part of the acquired metadata (601), The second part of the acquired metadata is to use extended metadata from memory (607) The method according to any one of claims 1 to 3, including

5. When it is determined that the metadata will be used as the acquired metadata to render the received frame, To decode and render subsequent frames, store the metadata in memory (417) The method according to any one of claims 1 to 4, further comprising:

6. At the start of decoding the bitstream, Initialize each metadata parameter to its default value (801), The initialized metadata parameters are stored in memory (803) The method according to any one of claims 1 to 5, further comprising:

7. The method according to claim 6, wherein storing the initialized metadata parameters includes storing only the extended metadata parameters.

8. Initializing each metadata parameter means that each metadata parameter The method according to claim 6, comprising setting according to the formula, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source.

9. At the start of the decryption process, the level of detail of the metadata in the metadata active state variable is set to its initial state (805) and stored. The method according to any one of claims 1 to 8, further comprising:

10. The method according to any one of claims 1 to 9, wherein the maximum change count includes a maximum change count over a certain time period.

11. The method according to claim 10, wherein the maximum change count includes 5 change counts over the time period.

12. (901) storing the last metadata used by the audio renderer when rendering the frame of the bitstream, (903) To perform a smooth transition from the last metadata mentioned above to the currently used metadata and The method according to any one of claims 1 to 11, further comprising:

13. The method according to claim 12, wherein the smoothing of the transition is performed by applying a smoothing transition of rendering parameters, which is an intermediate step in rendering.

14. The method according to claim 13, wherein the rendering parameters include gain parameters calculated from the orientation and distance of the audio object to the listener.

15. The method according to any one of claims 1 to 14, wherein the level of detail indicates whether or not extended metadata exists within the frame.

16. The method according to any one of claims 1 to 15, wherein using the metadata from memory includes using the extended metadata from memory.

17. The method according to any one of claims 1 to 16, wherein the first parameter includes the EXT_MD_BS parameter, the metadata state active variable includes the EXT_MD_ACTIVE variable, and the metadata change counter includes the EXT_MD_COUNTER counter.

18. A decoder (112) adapted to perform an operation, wherein the operation is Receiving metadata in a bitstream (401), wherein the metadata includes a first parameter, the first parameter indicating the level of detail of the metadata of the received frame (401), (403) Determining whether the metadata active state variable state is set to the initial state or to the level of detail of the metadata of the received frame, In response to the determination that the metadata active state variable state is not set to the initial state and is not set to the level of detail of the metadata of the received frame, Incrementing the metadata change counter (411), (413) Determining whether the metadata change counter is equal to the maximum change count, In response to the metadata change counter not being equal to the maximum change count, metadata from memory is used as the retrieved metadata (415), In response that the metadata change counter is equal to the maximum change count, Setting the metadata change counter to 0 (405), Setting the metadata active state variable state to the level of detail of the metadata of the received frame (407), Decoding the metadata of the received frame for use as acquired metadata (409), Rendering the received frame using the acquired metadata (416) A decoder (112) including this.

19. The decoder (112,) In response to determining that the metadata active state variable state is set to the initial state or to the metadata detail level of the received frame, Setting the metadata change counter to 0 (405), Setting the metadata active state variable state to the metadata level of the received frame (407), Decoding the metadata of the received frame for use as acquired metadata (409), Rendering the received frame using the acquired metadata (416) The decoder (112) according to claim 18, further adapted to perform operations including the operation described above.

20. Decrypting the aforementioned extended metadata Decoding the basic metadata for use as the first part of the acquired metadata (501), In response to the first parameter flag being set (503), the extended metadata is decoded (505) for use as a second part of the acquired metadata, In response to the fact that the first parameter flag is not set (507), the extended metadata is reconfigured for use as a second part of the acquired metadata. A decoder (112) according to any one of claims 18 to 19, including the decoder (112) according to any one of claims 18 to 19.

21. Using metadata from memory (415) Decoding the basic metadata for use as the first part of the acquired metadata (601), The second part of the acquired metadata is to use extended metadata from memory (607) A decoder (112) according to any one of claims 18 to 20, including the decoder (112) according to any one of claims 18 to 20.

22. The decoder (112) is When it is determined that the metadata will be used to render the received frame, To decode and render subsequent frames, store the metadata in memory (417) A decoder (112) according to any one of claims 18 to 21, further adapted to perform operations including those described above.

23. The decoder (112) is At the start of decoding the bitstream, Initializing each metadata parameter to its default value (701), (703) storing the initialized metadata parameters in memory A decoder (112) according to any one of claims 18 to 22, further adapted to perform operations including those described above.

24. Initializing each metadata parameter means that each metadata parameter The method according to claim 23, comprising setting according to the formula, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source.

25. The decoder (112) is Set the level of detail of the metadata in the metadata active state variable to its initial state (705) and store it. A decoder (112) according to any one of claims 18 to 24, further adapted to perform operations including those described above.

26. The decoder (112) according to any one of claims 18 to 25, wherein the maximum change count includes the maximum change count over a certain time period.

27. The decoder (112) according to claim 26, wherein the maximum change count includes 5 change counts over the time period.

28. The decoder (112) is (801) storing the last metadata used by the audio renderer when rendering the frame of the bitstream, (803) Perform a smooth transition from the last metadata to the currently used metadata. A decoder (112) according to any one of claims 18 to 27, further adapted to perform operations including those described above.

29. The decoder (112) according to claim 28, wherein performing the smooth transition includes applying a smoothing transition of rendering parameters, which is an intermediate step in rendering.

30. The decoder (112) according to claim 29, wherein the rendering parameters include gain parameters calculated from the orientation and distance of the audio object to the listener.

31. The decoder according to any one of claims 18 to 30, wherein the level of detail indicates whether or not extended metadata exists within the frame.

32. The decoder according to any one of claims 18 to 31, wherein using the metadata from memory includes using the extended metadata from memory.

33. The decoder according to any one of claims 18 to 32, wherein the first parameter includes the EXT_MD_BS parameter, the metadata state active variable includes the EXT_MD_ACTIVE variable, and the metadata change counter includes the EXT_MD_COUNTER counter.

34. Processing circuit (802), The memory (801) coupled to the processing circuit and A decoder (112) comprising, When the memory is executed by the processing circuit, the decoder (112) receives the following information: Receiving metadata in a bitstream (401), wherein the metadata includes a first parameter, the first parameter indicating the level of detail of the metadata of the received frame (401), (403) Determining whether the metadata active state variable state is set to the initial state or to the level of detail of the metadata of the received frame, In response to the determination that the metadata active state variable state is not set to the initial state and is not set to the level of detail of the metadata of the received frame, Incrementing the metadata change counter (411), (413) Determining whether the metadata change counter is equal to the maximum change count, In response to the metadata change counter not being equal to the maximum change count, metadata from memory is used as the retrieved metadata (415), In response that the metadata change counter is equal to the maximum change count, Setting the metadata change counter to 0 (405), Setting the metadata active state variable state to the level of detail of the metadata of the received frame (407), Decoding the metadata of the received frame for use as the acquired metadata (409), Rendering the received frame using the acquired metadata (416) A decoder (112) that includes an instruction to perform an operation including the above.

35. When the memory is executed by the processing circuit, the decoder (112) receives the following information: In response to determining that the metadata active state variable state is set to the initial state or to the metadata detail level of the received frame, Setting the metadata change counter to 0 (405), Setting the metadata active state variable state to the metadata level of the received frame (407), Decoding the metadata of the received frame for use as the acquired metadata (409), Rendering the received frame using the acquired metadata (416) The decoder (112) according to claim 34, comprising further instructions for performing further operations including the above.

36. Decrypting the aforementioned extended metadata Decoding the basic metadata for use as the first part of the acquired metadata (501), In response to the first parameter flag being set (503), the extended metadata is decoded (505) for use as a second part of the acquired metadata, In response to the fact that the first parameter flag is not set (507), the extended metadata is reconfigured for use as a second part of the acquired metadata. A decoder (112) according to any one of claims 34 to 35, including the decoder (112) according to any one of claims 34 to 35.

37. Using metadata from memory (415) Decoding the basic metadata for use as the first part of the acquired metadata (601), The second part of the acquired metadata is to use extended metadata from memory (607) A decoder (112) according to any one of claims 34 to 36, including the decoder (112) according to any one of claims 34 to 36.

38. When the memory is executed by the processing circuit, the decoder (112) receives the following information: When it is determined that the metadata will be used as the acquired metadata to render the received frame, To decode and render subsequent frames, store the metadata in memory (417) A decoder (112) according to any one of claims 34 to 37, comprising further instructions for performing further operations including the above.

39. When the memory is executed by the processing circuit, the decoder (112) receives the following information: At the start of decoding the bitstream, Initializing each metadata parameter to its default value (701), The initialized metadata parameters are stored in memory (703) A decoder (112) according to any one of claims 34 to 38, comprising further instructions for performing further operations including the above.

40. Initializing each metadata parameter means that each metadata parameter The method according to claim 39, comprising setting according to the formula, where θ is the azimuth angle, φ is the elevation angle, r is the radius from the origin, yaw is the horizontal orientation of the source, and pitch is the vertical orientation of the source.

41. When the memory is executed by the processing circuit, the decoder (112) receives the following information: (705) Set the level of detail of the metadata in the metadata active state variable to its initial state and store it. A decoder (112) according to any one of claims 34 to 40, comprising further instructions for performing further operations including the above.

42. The decoder (112) according to any one of claims 34 to 41, wherein the maximum change count includes a maximum change count over a certain time period.

43. The decoder (112) according to claim 42, wherein the maximum change count includes 5 change counts over the time period.

44. When the memory is executed by the processing circuit, the decoder (112) receives the following information: (801) storing the last metadata used by the audio renderer when rendering the frame of the bitstream, (803) Perform a smooth transition from the last metadata to the currently used metadata. A decoder (112) according to any one of claims 34 to 43, comprising further instructions for performing further operations including the above.

45. The decoder (112) according to claim 44, wherein performing the smooth transition includes applying a smoothing transition of rendering parameters, which is an intermediate step in rendering.

46. The decoder (112) according to claim 45, wherein the rendering parameters include gain parameters calculated from the orientation and distance of the audio object to the listener.

47. The decoder according to any one of claims 34 to 46, wherein the level of detail indicates whether or not extended metadata exists within the frame.

48. The decoder according to any one of claims 34 to 47, wherein using the metadata from memory includes using the extended metadata from memory.

49. The decoder according to any one of claims 34 to 48, wherein the first parameter includes the EXT_MD_BS parameter, the metadata state active variable includes the EXT_MD_ACTIVE variable, and the metadata change counter includes the EXT_MD_COUNTER counter.

50. A computer program comprising program code to be executed by a processing circuit (1102) of a decoder (112), wherein the computer program causes the decoder (112) to perform the operation described in any one of claims 1 to 17 by executing the program code.

51. A computer program product comprising a non-temporary storage medium containing program code to be executed by a processing circuit (1102) of a decoder (112), wherein the decoder (112) is caused to perform the operation described in any one of claims 1 to 17 by executing the program code.

Images