Implementation of improvement coding schemes in transmission applications

Description

1 / 42 “IMPLEMENTATION OF ENHANCED CODING SCHEMES IN TRANSMISSION APPLICATIONS” FIELD OF THE INVENTION

[0001] The present invention relates to methods, systems, computer programs and computer-readable media for implementing enhancement coding schemes in multimedia transmission applications, in particular for video transmission applications. BACKGROUND

[0002] Streaming applications allow video content to be delivered remotely to users quickly and efficiently. In general, video content is pre-encoded using a traditional encoding scheme at different resolutions to ensure that the video content is compatible with a wide range of user devices while providing the best possible experience.

[0003] As modern players, capable of displaying higher resolution video content, become more common and are commonly connected via wireless connections, various techniques must be employed to improve bandwidth utilization so that video content distribution is not only reliable but also provides a seamless experience for the end user.

[0004] Much of the video content on the Internet is still encoded using H.264 (also known as MPEG-4 Part 10, Advanced Video Coding - MPEG-4 AVC). This content is typically provided to decoding devices as a single video stream that has a one-to-one relationship with available hardware and / or software video decoders; for example, a single stream is received, analyzed, and decoded by a single video decoder to produce a reconstructed video signal. Many video decoder implementations are then developed according to this framework.

[0005] To be compatible with different conditions of Petition 870250112790, dated 08 / 12 / 2025, page 9 / 58 2 / 42 In transmission, client devices are generally configured with a simple switching mechanism that is triggered based on metadata that identifies a stream format. An example is adaptive bitrate streaming, or ABR, and a typical and standardized mechanism that implements it is Dynamic Adaptive Streaming over HTTP (DASH), also known as MPEG-DASH. Other common HTTP-based streaming protocols include HTTP Live Streaming (HLS) and Real-Time Messaging Protocol (RTMP), which function similarly to HTTP (DASH), although RTMP is not based on HTTP but on TCP / UDP.

[0006] In ABR transmission, a video is encoded in multiple resolutions or bitrate combinations, and each of these is typically called a “rendering” (or a representation, see below in the context of manifest files). Client devices determine the available bandwidth and retrieve the appropriate rendering according to the available bandwidth and in a format that they can support. The collection of renderings is typically called a ladder or profile.

[0007] To implement this, in DASH, each client device retrieves a manifest file, known as an MPEG-DASH MPD (Media Presentation Description) file, which comprises metadata to allow the client device to construct a URL (uniform resource locator) to retrieve the appropriate stream based on available conditions (such as device capability, resolution, available bandwidth, buffer state, etc.). Each manifest file contains a set of representations, each describing one of the versions of the content. The manifest file describes how the video was divided into parts by a packager and how it is organized so that the client device can request fragments of the video for subsequent playback.In general, representations include specific video and audio tracks to describe a version of the media stream, bitrates, resolution, and other associated information. Petition 870250112790, dated 08 / 12 / 2025, page 10 / 58 3 / 42 media content. The manifesto can also be called a playlist in other formats.

[0008] Existing multi-layered encoding schemes include the Scalable Video Coding (SVC) extension for H.264, scalable extensions for H.265 (MPEG-H Part 2, High Efficiency Video Coding - SHVC), and more recent standards such as MPEG-5 Part 2, Low Complexity Enhancement Video Coding (LCEVC), and SMPTE ST-2117 (VC-6). SVC and SHVC operate by creating different encoding layers and feeding each one with a different spatial resolution. Each layer encodes the input according to a normal AVC or HEVC encoder, with the possibility of leveraging information generated by the lower encoding layers. LCEVC, on the other hand, generates one or more layers of enhancement residues compared to a decoded version of the base coding, where the base coding can typically have a lower spatial resolution.

[0009] Unlike traditional encoding schemes, LCEVC encodes a lower-quality or base version of a source image using an existing codec, and an enhancement layer is encoded and can be used to upscale and bring the decoded base version to a higher quality. The enhancement layer adds a further degree of freedom in how a video stream can be implemented and provides new capabilities such as expanding the compression capacity of the base codec, reducing encoding and decoding complexity, and providing a platform for further future enhancements.

[0010] In order for ABR (and DASH) transmission to accommodate different user devices and adapt to changes in network conditions, content must be encoded and stored at different bitrates and resolutions so that each device can retrieve an appropriate transmission. Similarly, while some devices may be able to transmit content Petition 870250112790, dated 08 / 12 / 2025, page 11 / 58 4 / 42 encoded using a specific encoding scheme, legacy devices cannot. For content to be transmitted to the largest possible number of users, it must be encoded and stored using schemes available to those user devices. This generally means there must be at least one representation of the content for each possible combination of device configuration, bitrate, and resolution. For example, advanced configurations with high available bandwidth can transmit media at high bitrate, framerate, or resolution, while basic configurations, or those with low bandwidth, can transmit media at low quality.

[0011] Therefore, to provide compatibility with the transmission In ABR (Augmented Rendering of Multimedia Content) encoded using enhancement coding schemes, there is a significant increase in the number of versions encoded and stored for distribution, and a significant increase in the number of representations available in each manifest file. The increased flexibility of enhancement coding schemes comes with an inherent trade-off: the proliferation of representations for each variant, balanced combination and permutation, with the inherent flexibility and enhanced quality for the available conditions resulting from the ability to transmit these representations.

[0012] In short, one difficulty in implementing enhancement coding schemes is that managing encoders, packers, and decoders is much more complicated than managing them in relation to a traditional “single-layer” coding scheme. This is because when an enhancement coding scheme (which results in an enhancement coding layer) is introduced, there are many more combinations and permutations available for encoders, packers, and decoders. SUMMARY OF THE INVENTION

[0013] According to one aspect of the invention, a method of transmitting encoded multimedia content using an enhanced encoding scheme can be provided. The method may comprise: a Petition 870250112790, dated 08 / 12 / 2025, page 12 / 58 5 / 42 retrieval of a manifest file, the manifest file comprising metadata for constructing a locator for segments of the encoded multimedia content, the metadata comprising: a first representation pointing to a first version of the encoded multimedia content using a base encoding scheme; and location metadata for a first enhancement layer of the encoded multimedia content associated with the first version; and parsing the manifest file to construct a locator for retrieving the first representation and enhancement layer segments; downloading segments from the first representation and enhancement layer segments to reconstruct the encoded multimedia content.

[0014] According to one aspect of the invention, a method can be provided for packaging multimedia content encoded using an enhancement coding scheme for transmission, the method comprising: constructing a manifest file, the manifest file comprising metadata for constructing a locator for segments of the encoded multimedia content, the metadata comprising: a first representation pointing to a first version of the multimedia content encoded using a base coding scheme; and location metadata for a first enhancement layer of the encoded multimedia content associated with the first version.

[0015] In this way, the number of variants of encoded multimedia content that need to be maintained and compatible can be reduced. By presenting the enhancement layer and the base layer of the encoded content separately, it is possible for a client device to retrieve only the base layer, depending on the available bandwidth and device conditions, but, when appropriate, retrieve the base layer and enhancement data for combination and reconstruction, even if they are stored and presented separately. Legacy devices can retrieve legacy encoded data without the associated enhancement data, thus utilizing all available bandwidth, while this legacy encoded data can be used by Petition 870250112790, dated 08 / 12 / 2025, page 13 / 58 6 / 42 devices enabled for enhancement as encoded base data for higher quality content reconstruction using the enhancement data presented separately.

[0016] The encoded multimedia content can be a video stream. The enhancement encoding scheme can be LCEVC, and the base encoding scheme can be any “single-layer” encoding scheme, such as AVC, HEVC, AV1, or VVC. The manifest file can comprise a Media Presentation Description (MPD). The locator can be a link or a Uniform Resource Locator (URL). The first version of the multimedia content encoded using a base encoding scheme can be the base layer of the multimedia content encoded using the enhancement encoding scheme, i.e., the base layer associated with the first specific enhancement layer. Segments can be downloaded according to the guidelines defined in MPEG-DASH, HTTP Live Streaming (HLS), or Microsoft Smooth Streaming.In other words, MPEG-DASH and other transmission protocols are adequate guidelines for indicating the order and retrieval of segments. The download itself is done by the player and therefore differs for different players based on configurable parameters that control segment downloads based on, for example, bandwidth or quality and many other parameters.

[0017] Although the term manifest is used, as it is the official term for transmission protocols, it should be understood that it can be considered interchangeable with the term playlist, depending on the format used. In some circumstances, the manifest can be passed as a playlist.

[0018] The concepts according to the present invention facilitate adaptive bitrate transmission of a base layer of multimedia content encoded separately from its enhancement layer, wherein a client device performs the function of deciding which bitrate segments should be downloaded, based on the current state of the network. Petition 870250112790, dated 08 / 12 / 2025, p. 14 / 58 7 / 42

[0019] The second representation may be at a higher quality level than the first representation. By quality level, we mean resolution, bitrate, color depth, and file size, among others. In the sense of representation, you mean that the higher quality level likely requires more advanced client device compatibility or better conditions (e.g., network or device, such as buffer capacity).

[0020] The method can be performed by a client device, player or decoder.

[0021] Preferably, the metadata comprises a second representation that points to the first layer of enhancement of the encoded multimedia content. Localization metadata can be included in the second representation. In this way, the manifest file presents multiple representations for selection by the client device, each representation corresponding to a different layer of the enhancement encoding scheme. By downloading the two layers according to network conditions, they can be combined later to render the multimedia content on the client device. In the implementations, each representation corresponds to a single track. With the use of the second representation, the base layer and the enhancement layer can be provided separately with an external link that associates the two together.

[0022] More preferably, the metadata further includes a dependency attribute that associates the second representation with the first representation. The dependency attribute can be a dependency ID attribute from an MPEG-DASH manifest file. The dependency attribute provides an association of the second representation with the first representation, so that they can be provided separately with an external link, i.e., the dependency. The dependency attribute provides the use of an existing mechanism for a previously unforeseen purpose, thus providing an immediate path to implementation without significant modification of the Petition 870250112790, dated 08 / 12 / 2025, page 15 / 58 8 / 42 existing standards and devices. The dependency attribute means that any implementation can be deployed without affecting currently deployed devices and without requiring software updates for those devices to function.

[0023] Along with a dependency attribute, the representation may also comprise a resolution of the encoded multimedia content. The second representation may comprise a resolution of the combined base and enhancement layers when constructed. In addition, the representation may also comprise a bandwidth attribute, minimum, maximum, or average, to retrieve the representation. The bandwidth attribute of the second representation may correspond to the bandwidth required to retrieve only the enhancement layer or to retrieve both the enhancement layer representation and the associated base layer representation.

[0024] The method may also include determining a dependency between the second representation and the first representation. In other words, configuring the second representation based on the dependency. The method may also include analyzing the dependency attribute to identify the association between the second representation and the first representation. In the context of MPEG DASH terminology, the dependency may be a dependencyId.

[0025] Preferably, the first and second representations are contained within their respective Adaptation Sets. By using the respective Adaptation Sets, an ABR algorithm for client devices that are compatible with one or both base and enhancement coding schemes can traverse an ABR ladder of that scheme. An ABR ladder can contain enhancement layer representations while retrieving base layer representations through the use of dependency. Dependency can associate representations between different Adaptation Sets or within the same Adaptation Set. Alternatively, the first and second representations can be contained within the same Adaptation Set. Petition 870250112790, dated 08 / 12 / 2025, page 16 / 58 9 / 42 Adaptation for selection by an ABR algorithm that traverses an ABR ladder within the same Adaptation Set. Optionally, the first representation can be repeated in different Adaptation Sets, with the different adaptation sets being associated with different MIME types or compatible codecs.

[0026] In optional implementations, the metadata may also include, for each representation, a bandwidth to retrieve the respective representation, and the method may also include: determining a total bandwidth to transmit the multimedia content by adding the bandwidth associated with the second representation and the bandwidth of its respective first associated representation. In this way, the client device can select an appropriate representation according to the available network conditions using different representations. As mentioned above, the representation may also include a bandwidth associated with the total bandwidth to retrieve the second representation and its first associated representation.

[0027] Preferably, the first representation comprises an attribute indicating the base encoding scheme; and the second representation comprises an attribute indicating the enhancement encoding scheme. The attribute may comprise a “codecs” attribute. Examples include “avc1.64001f” and “lvc1”. The attribute may also comprise a MIME type attribute. Examples include video / h264. Defining attributes in this way allows client devices to select ABR profiles according to the codecs compatible with the devices.

[0028] Metadata may also include: a third representation pointing to a second version of the multimedia content encoded using the base encoding scheme, wherein the first and second versions are versions of the multimedia content at a different quality level; and a fourth representation pointing to a second enhancement layer associated with the second version of the encoded multimedia content. The Petition 870250112790, dated 08 / 12 / 2025, page 17 / 58 10 / 42 a different quality level may be a higher quality level, as set forth elsewhere in this document.

[0029] The first representation can be 340p with an associated second representation of 720p, and the third representation can be 540p with an associated fourth representation of 1080p. In this way, four representations can be provided from just two encoded and stored videos. This example demonstrates how multiple representations can be presented using few encoded and stored videos and tracks, using base layers as low-quality representations in the ABR ladder and different base layers from the enhancement layers at higher quality levels in the ABR ladder.

[0030] Next, the preferred implementations of the above method for transmitting encoded multimedia content using an enhancement encoding scheme.

[0031] The method may also include: the creation of a set of virtual representations from the first and second representations and the dependency attribute, wherein the virtual representations comprise: a first virtual representation comprising the first representation; and a second virtual representation comprising the first and second representations. The method may also include the selection of a virtual representation for retrieval according to the ABR rules. The virtual representation allows an ABR algorithm to traverse the virtual representations according to the available conditions, wherein the virtual representations comprise representations of the base and enhancement layers presented separately in the manifest file.

[0032] The method may further comprise: detecting the available bandwidth and / or device conditions; and downloading segments from the first representation or segments from the first representation and segments from the enhancement layer to reconstruct the encoded multimedia content according to the available conditions. The method may also comprise Petition 870250112790, dated 08 / 12 / 2025, page 18 / 58 11 / 42 Selection of a download representation based on available bandwidth and / or device conditions. Bandwidth conditions can be considered network conditions. Device conditions include examples such as buffer status, power status, decoder capacity, or hardware capacity. In this way, representations can be selected and ABR profiles traversed according to a set of predefined rules.

[0033] The method may also comprise: passing the downloaded segments from the enhancement layer to an enhancement decoder to generate decoded enhancement data; and reconstructing the multimedia content from the decoded enhancement data and a decoded version of the video stream encoded using the base coding scheme decoded from the downloaded segments of the first representation. Optionally, the downloaded segments from the base layer may be passed to a base decoder directly or via the enhancement decoder. The base decoder may then operate under the control of the enhancement decoder or independently of it.

[0034] The following are preferred implementations of the above method of packaging encoded multimedia content using an enhancement coding scheme for broadcast.

[0035] Preferably, the method may also include: Receive an encoded version of the multimedia content encoded using the enhancement encoding scheme comprising a base layer and an enhancement layer; generate a first set of segments representing the base layer; and generate a second set of segments representing the enhancement layer, where the first set of segments is independent of the second set of segments. Independent can be considered as separate from. In this way, the packager can store the segments separately for later retrieval. The segments can be stored in separate files or in a respective file with defined byte ranges. Alternatively, the segments from both layers can be Petition 870250112790, dated 08 / 12 / 2025, page 19 / 58 12 / 42 stored in a file with defined byte ranges, with the wrapper generating the segments so that the manifest file includes locations for separate representations to show the retrieval of these segments separately.

[0036] The first set of segments and the second set of segments can be generated as respective single-strip representations.

[0037] The construction of the manifest file may include: inserting an association between the second set of segments and the first set of segments; obtaining a location for the first and second sets of segments; and inserting metadata to construct a locator for the location of the first and second sets of segments.

[0038] The modes in this document can generally be independent of the player and protocol; for example, different containers can be used, such as the ISO-based media file format (e.g., MP4 file format) or the MPEG-2 transport stream, as well as different clients, such as those based on HTML5 or other transport stream mechanisms. Theoretically, if the DASH dependencyId is implemented, the modes presented here will work on any LCEVC-compatible platform.

[0039] According to aspects of the present invention, a method is provided comprising analyzing a file (e.g., a manifest file) and selecting a set of renderings based on a given available bandwidth.

[0040] A method is described that comprises analyzing an attribute to determine a dependency between a low-level representation of a hierarchical coding scheme (e.g., as a base layer representation, e.g., AVC, HEVC, VVC, and so on) and a higher level of said hierarchical coding scheme (e.g., as an enhancement layer representation, such as LCEVC). The determination may be called a configuration. A manifest file may Petition 870250112790, dated 08 / 12 / 2025, page 20 / 58 13 / 42 understand this attribute. The analysis may be the analysis of a manifest file.

[0041] The method may also include the creation of a list of representations. The list may include one or more representations comprising a base layer (e.g., and no enhancement layer). The list may include one or more representations comprising a base layer and an enhancement layer (e.g., associated). The list may include a first representation comprising a first version of a base layer. The list may include a second representation comprising a second version of a base layer. The list may include a third representation comprising the first version of a base layer associated with a first version of an enhancement layer.The list may include a fourth representation comprising the second version of a base layer associated with a second version of an enhancement layer. The second versions may have higher quality (e.g., higher resolution, larger file size, etc.) than the first versions of the base and / or enhancement layers. Each version of the enhancement layer may include a dependency identifier configured to identify a base representation upon which the enhancement layer version depends.

[0042] The method may comprise determining a total bandwidth associated with each representation in the list of representations. The method may comprise determining the total bandwidth associated with each representation in the list of representations by summing a bandwidth associated with each of the constituent elements of the representation. For example, the method may comprise determining a total bandwidth associated with the third representation by combining (e.g., summing) a bandwidth associated with the first version of a base layer and a bandwidth associated with the first version of an enhancement layer. For example, the method may comprise determining a total bandwidth associated with the fourth representation by combining (e.g., summing) a bandwidth Petition 870250112790, dated 08 / 12 / 2025, page 21 / 58 14 / 42 of the bandwidth associated with the second-first version of a base layer and a bandwidth associated with the second-first version of an enhancement layer.

[0043] The method may comprise the determination of available bandwidth. The method may comprise the determination of an encoding attribute of a device associated with the method (e.g., that performs it). The encoding attribute may be an indication that the device is compatible with an encoding scheme associated with the enhancement layer (and / or the base layer). The “available bandwidth” mentioned above may be the available bandwidth between the device (i.e., the player, the set-top box, etc.) and the transmission device (e.g., the content server).

[0044] The method may comprise selecting a representation from the list of representations based on the determined available bandwidth. The method may comprise selecting a representation from the list of representations based on the determination of the encoding attribute of a device associated with the method (e.g., that performs it). The method may comprise selecting a representation associated (e.g., having, comprising, and so on) with a bandwidth that corresponds to the determined bandwidth.

[0045] The method may involve downloading segments (for example, a version of a base layer and, if applicable, a version of the enhancement layer associated with the selected representation) associated with the selected representation. The method may involve downloading the segments in a synchronous manner.

[0046] The method may also include sending the segment(s) to the appropriate decoder. The sending may be performed via an interface. For example, a segment associated with a base layer version may be sent to a base layer decoder. For example, a segment associated with an enhancement layer version may be sent to a Petition 870250112790, dated 08 / 12 / 2025, page 22 / 58 15 / 42 enhancement decoder.

[0047] The method may comprise selecting a representation that does not include (and / or is not associated with) a segment of the enhancement layer (for example, the selected representation may be associated only with a segment of the base layer and / or the selected representation is associated only with a segment of the base layer); this selection may occur due to a determined low bandwidth and / or the fact that the device codec attribute does not match the encoding scheme associated with the enhancement layer. The method may comprise, in response to this selection (i.e., the selection of a representation that includes only and / or is associated with a segment of the base layer), downloading only the segment of the base layer. The method may comprise sending these segments of the base layer to a base decoder.

[0048] The method may comprise processing decoded segments to enable playback of the content. The method may comprise combining the decoded base segment with an associated decoded enhancement segment to generate an output / display video.

[0049] The method can be performed by a decoder, a player, a client device, and so on.

[0050] According to one aspect of the invention, a client device can be provided for transmitting multimedia content encoded using an enhancement encoding scheme configured to accomplish the method of any of the above aspects.

[0051] According to one aspect of the invention, a packer can be provided for packaging multimedia content encoded using an enhancement-for-transmission coding scheme configured to accomplish the method of any of the above aspects.

[0052] According to one aspect of the invention, a system comprising a customer device and a packager of any of the above aspects can be provided. The system may further comprise: a Petition 870250112790, dated 08 / 12 / 2025, page 23 / 58 A 16 / 42 encoder is configured to encode multimedia content using an enhancement encoding scheme; and a content delivery network is configured to serve the segments and the manifest file.

[0053] According to other aspects of the invention, a computer-readable means can be provided comprising instructions which, when executed by a computer, cause the computer to perform the method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Examples of systems and methods according to the invention will now be described with reference to the accompanying drawings, in which:

[0055] Figure 1 shows a known high-level scheme of an LCEVC encoding, decoding and transport process;

[0056] Figure 2 shows a known high-level schematic of an ABR process for implementing enhancement coding;

[0057] Figure 3 shows an example of a known ABR profile for enhancement coding;

[0058] Figure 4 shows an example of a known approach for encoding and storing representations for ABR;

[0059] Figure 5 shows an example of an approach to encoding and storing representations for ABR in accordance with the principles of this disclosure;

[0060] Figure 6 shows an example of an approach to encode, store, present, and retrieve representations for ABR in accordance with the principles of this disclosure;

[0061] Figure 7 shows a high-level schematic of an ABR process for implementing enhancement coding in accordance with the principles of this disclosure;

[0062] Figure 8 shows a high-level approach to encapsulation according to the present disclosure; and,

[0063] Figure 9 is an example of a process to be Petition 870250112790, dated 08 / 12 / 2025, page 24 / 58 17 / 42 performed on a client device in accordance with the principles of this disclosure. DETAILED DESCRIPTION

[0064] For context, an enhancement coding scheme, LCEVC, suitable for use with the concepts of this disclosure will be described first. LCEVC will be described in the context of Figure 1. Throughout the description, the terms “hierarchical coding” and “enhancement coding” may be used interchangeably, and although the examples are described in the context of LCEVC, it is understood that the concepts described may be applicable to any similar hierarchical or enhancement coding scheme.

[0065] LCEVC adopts a multi-layered approach where any base codec (e.g., h.264, HEVC, AV1, and others) is enhanced through an additional low-bitrate stream. The LCEVC data stream is defined by two component streams: a base stream typically decodable by a hardware decoder, and an enhancement stream consisting of one or more enhancement layers suitable for implementing software processing with sustainable power consumption. The enhancement provides greater compression efficiency to existing codecs and reduces encoding and decoding complexity for live and on-demand streaming applications.

[0066] Figure 1 below illustrates how LCEVC operates on the encoding and decoding channels. Base encoding (whether H.264, HEVC, or others) is performed on a downscaled input at a lower resolution, typically one-quarter of the desired output resolution. LCEVC enhancement data is calculated at both resolutions, providing two levels of correction and enhancement. The LCEVC encoder generates the enhancement stream from two inputs: the base encoding and the original uncompressed full-resolution video, effectively correcting the quality difference between the two. LCEVC data can be packaged along with the stream. Petition 870250112790, dated 08 / 12 / 2025, page 25 / 58 18 / 42 basic elementary (for example, as Supplemental Enhancement Information (SEI) from the Network Abstraction Layer (NAL), such as frame metadata in a WebM container or as a Packet Identifier (PID) of additional data in an MPEG-2 TS stream.

[0067] The LCEVC decoder operates at an individual video frame level. As input, it receives the low-resolution image decoded from the base video decoder, which is typically provided by a hardware decoder on the device, and enhances the decoded LCEVC in software to produce a full-resolution image ready to be rendered on the monitor display. Examples of LCEVC decoding implementations are described in WO2022 / 023739 and WO2023 / 118851, which are incorporated by reference.

[0068] As illustrated in Figure 1, in encoder 100, a full-resolution input video, i.e., a source video 102, is processed to generate several encodings. A first encoding (base 110 encoding) is produced by feeding a base 106 encoder (e.g., AVC, HEVC, VP9, ​​or any other codec) with a subsampled version of the input video, which is produced by subsampling 104 from the input video 102. In the example shown, the subsampling is done to a quarter of the resolution, but this is optional, as will be explained elsewhere. The base 110 encoding can be called the base layer.

[0069] A second encoding (level 112 encoding) (1, an example of enhancement coding) is produced to create first-level 116 corrections by applying an encoding operation to the residues obtained by the difference between a reconstructed basecodec video and the subsampled version of the input video. The reconstructed basecodec video is obtained by decoding the output of the base106 encoder with a base decoder. In typical implementations, level 112 encoding is optional. This level 112 encoding can be called the first layer of enhancement. Petition 870250112790, dated 08 / 12 / 2025, page 26 / 58 19 / 42

[0070] A third encoding (level 114 encoding) 2, another example of enhancement coding) is produced to create first-level corrections 120 by processing the residuals obtained by the difference between an oversampled version (i.e., normative oversampling 118) of a corrected version of the reconstructed base-coded video and the input video 102. This level 2 coding 114 can be called second-layer enhancement.

[0071] Typically, the enhancement layer(s) and the base layer are combined (i.e., as illustrated by mux 122) and the full-resolution video is layered. This is then typically transmitted using standard packaging and transmission protocols, an example of which will be described below in the context of Figure 2.

[0072] Although the corrections that make up the enhancement layer often involve oversampling to a higher resolution, generally any improvement in quality can be provided by the enhancement layer. For example, resolution, visual quality (VQ), bit depth (e.g., 8 to 10b), or color space (e.g., HDR).

[0073] In decoder 140, the encoded video is separated (i.e., as illustrated by demux 142) into an auxiliary data stream 144 and a video stream 146. The decoder receives the layers (a base encoding, an optional level 1 encoding, and a level 2 encoding) along with headers containing other decoding information. The base encoding, i.e., in video stream 146, is decoded by a base decoder 148 corresponding to the base decoder used in the encoder. In an enhancement decoder 150, which receives the headers and enhancement layers in the auxiliary data stream 144, its output is combined with the decoded residues obtained by decoding the level 1 encoding (if any). The combined video is upsampled and combined with the decoded residues obtained by applying a decoding operation to the level 2 encoding to generate the full-resolution video 152. Petition 870250112790, dated 08 / 12 / 2025, page 27 / 58 20 / 42

[0074] Figure 2 illustrates a schematic example of enhancement coding implementation using standard packaging and transmission protocols. Source video 102 is encoded by encoder 100 to produce the layered full-resolution video. An example of how encoder 100 can be implemented using an encoder integration layer is described in document WO2022 / 023747, which is incorporated here by reference. Any suitable container format can be used for the video, but here an mp4 container format containing both the base layer and the enhancement layer has been illustrated.

[0075] The encoder will be instructed to produce a set of videos in different qualities, referred to here as bitrate variants, depending on the desired settings. For example, the set of files may include videos in different resolutions (e.g., 1080p, 720p, etc.) and different qualities, frame rates, or bitrates at the same resolution (e.g., 720p, 60 fps and 720p, 30 fps, etc.).

[0076] LCEVC was designed so that older devices can still decode and play back the base video, even if the stream with the enhancement and base layers is recovered. In other words, LCEVC is inherently backward compatible, since devices that have not yet been updated to decode LCEVC play back the underlying base codec fine. However, for these legacy devices, in this situation, they are transmitting data that they are not using and therefore underutilizing the available bandwidth.

[0077] To resolve this, just as with full-resolution video files encoded in layers, in order to provide compatibility with older devices, the encoder will also be instructed to generate encoded variants using only the base codec. This allows these files to be retrieved by older client devices without the accompanying enhancement data, thus utilizing more available bandwidth on viewable video data. Petition 870250112790, dated 08 / 12 / 2025, page 28 / 58 21 / 42

[0078] The set of variants, that is, the set of encodings or renderings, is packaged by a packaging service referred to here as a packager 202. An example of a packager is the Shaka packager. The packager 202 divides each rendering into fragments of a specific duration (e.g., 2 or 4 seconds) and records how the video is divided and the order of the fragments in a media presentation description (MPD) file, also called a manifest file 205. The packaged videos and manifest files are then placed in storage 204. Once stored, the video fragments can be called segments, and the segments and the manifest file are made available for retrieval on a network 206.

[0079] The manifest file is a document, usually in XML (Extensible Markup Language), that contains metadata required by a client device to construct appropriate URLs (uniform resource locators) for accessing the segments. The manifest is structured hierarchically. The media presentation contains a sequence of one or more periods; a period contains one or more Adaptation Sets; an Adaptation Set contains one or more representations; a representation contains one or more segments; and the segments contain the locations of the actual media data and associated metadata.

[0080] The media presentation (i.e., the overall hierarchy) contains information about all the different media types in the content. The most common media types are video, audio, and closed caption data. At the top level, the MPD contains information including the MPD profile, minimum buffer time, presentation duration, maximum segment duration, and title. Video is structured into multiple periods, for example, for ad insertion or chapters. An Adaptation Set catalogs the available media representations. For example, as shown in the diagram in Figure 2, there might be an adaptation set for the base video and an adaptation set for the LCEVC video. There might also be an Adaptation Set for a caption track. Petition 870250112790, dated 08 / 12 / 2025, page 29 / 58 22 / 42 and an audio track, for example. In each Adaptation Set, the player will select only one Representation for a given Segment, but the Representation may change from one Segment to another. Video Adaptation Sets typically contain multiple Representations, one for each resolution / bitrate, allowing the media player to select the best available quality without buffering. If the video is available in more than one codec, each codec will be in a different Adaptation Set. In an Adaptation Set, a Representation describes one of the versions of the content. Each Representation typically includes the average bandwidth of the Representation to allow the client device to choose which representation to retrieve. Each representation comprises a set of segments containing the information necessary to construct the URLs for downloading the content.MPD can provide a list of segment URLs or a template that the player uses to dynamically create the URLs.

[0081] The structure of an MPEG DASH implementation will be well understood by a versed user and, therefore, this description is provided for context purposes only. More information can be found in the ISO / IEC 23009 standards documentation.

[0082] Figure 2 illustrates two 208 client devices, 210. To help demonstrate the principles of the present invention, it is assumed that client device #1 208 may be a device with LCEVC capability and client device #2 210 may be a legacy device without LCEVC video decoding capability. When referring to the functions of the client device, the functionality may be incorporated into a player on the device, a decoder module, or any other module on the device, or the functionality may be divided into several modules, such as an ABR handler, a data handler, a media handler, a player, and a decoder, etc. Client devices 208, 210 may first retrieve (i.e., download) the manifest file 205 over the network 206, for example, as served by a CDN content delivery network. The manifest file is parsed by the device of Petition 870250112790, dated 08 / 12 / 2025, page 30 / 58 23 / 42 client to understand the video delivery mechanism. Client device 208, 210 detects its available bandwidth and selects an available representation according to the available bandwidth and the representation compatible with the device. Then, the client device requests fragments of one of the video bitrate variants, depending on the buffer size and available bandwidth. In other words, the client device selects segments for download according to the available bandwidth and the device status (e.g., compatible video and buffer conditions). In the example in Figure 2, client device #208 is LCEVC compliant and has 2 Mbps available.In this way, it analyzes the manifest file to identify that the most suitable segments to download are those that conform to the biei representation and, from the metadata in the manifest file, constructs a suitable URL to retrieve the segments to fill its buffer. The retrieved segments are passed to an LCEVC decoder for decoding. Client device 208 continuously detects its bandwidth conditions and buffer levels and transmits the content adaptively. According to ABR principles, if bandwidth conditions or buffer levels change, for example, the device may retrieve segments from a different representation, for example, within the same Adaptation Set. To conclude the example, it is observed above that client device #2 210 is not LCEVC compliant. It is assumed that it detects the same available bandwidth as client device #1 208.Therefore, it would choose the AVC Adaptation Set, since it does not offer compatibility with LCEVC, and would choose to retrieve segments from the b2 Representation, since this indicates an average bandwidth of 2 Mbps, and then construct an appropriate URL from the metadata.

[0083] Representations in an Adaptation Set can be called an ABR ladder or ABR profile. Figure 3 illustrates an example of an ABR profile, from a report written by industry analyst Jan Ozer in April 2020. The H.264 ABR ladder profiles, encoded using x264 (medium preset), are in accordance with Apple's recommended (RTM) HLS creation specification. Petition 870250112790, dated 08 / 12 / 2025, page 31 / 58 24 / 42 The LCEVC H.264 ABR ladder profiles were optimized to achieve increased quality and reduced average total transmitted bitrate. As shown, an ABR ladder using LCEVC balances quality and efficiency. In other words, there can be a lower bitrate for the top profile with quality similar to or better than x264, while improving the quality and resolution of the lower profiles. To achieve maximum efficiency, it may be possible to reduce the data rate as much as possible while matching x264 quality across all profiles to decrease costs (e.g., transcoding, CDN) and maximize network capacity (e.g., number of simultaneous users).

[0084] Figure 4 illustrates an alternative view of a traditional approach, with or without LCEVC. As shown, each resolution is encoded to require a defined bitrate so that this encoded video stream can be properly recovered. As described above, a difficulty in implementing enhancement coding schemes is that the management of encoders, packers, and decoders is much more complex than managing them in relation to a traditional “single-layer” coding scheme. This is because when an enhancement coding scheme, resulting in an enhancement coding layer, is introduced, there are many more combinations and permutations available for encoders, packers, and decoders.

[0085] Although the use of representations works well for most conventional configurations, it is evident that it can be detrimental to the interoperability of unusual or older configurations. Since each representation defines a specific bit rate and resolution, the manifest should present representations for as many common configurations as possible. However, since each representation is stored separately and independently of the others, the amount of storage required increases exponentially with the number of representations.

[0086] It would not be feasible to store representations for Petition 870250112790, dated 08 / 12 / 2025, page 32 / 58 25 / 42 all possible permutations are included, and to achieve balance, unusual or older configurations tend to be left out and are therefore forced to match a specific representation. Similarly, players cannot alter or modify the encoded representations and must simply select what is already available.

[0087] Examples in this disclosure describe methods and systems that can benefit from the scalable capabilities of enhancement coding schemes (such as MPEG-5 LCEVC) in broadcast applications. According to the principles of this disclosure, the base and enhancement layers are stored and provided separately from each other. The representations are also presented separately from each other in the ABR ladder, i.e., in the manifest file. In an optional implementation, a dependency is introduced between the representations of the enhancement sets. This dependency creates an association between the segments and frames of the base layer and the enhancement layer. In other words, the enhancement layer is associated with the base layer that is configured to enhance through the use of the dependency, specifically, in the examples, the dependencyId of MPEG DASH.

[0088] This concept and those described in this document can be called Scalable Mode. When used in scalable mode, the encoder is configured so that the base layer, which corresponds to an existing distribution method such as AVC, HEVC, AV1, can be forced to match an existing standard ABR scaling profile (e.g., 540p AVC, 1080p HEVC). The rate control mode can be chosen to match the pre-existing system configuration, e.g., CBR, limited CRF, limited VBR, etc., and to ensure that the target bitrates for the base layer and the total base layer plus the enhancement layer are achieved.

[0089] The LCEVC enhancement layer on top can be used to efficiently compress a higher resolution (e.g., 1080p AVC, 2160p HEVC). The enhancement fragment will then be Petition 870250112790, dated 08 / 12 / 2025, page 33 / 58 26 / 42 combined with the base to produce the highest resolution video on the decoder.

[0090] For example, assuming the target bitrate ranges are 2700 Kbps and 1900 Kbps, LCEVC can be encoded in 1080p (with 540p BL) or 720p (with 360p BL) - or other intermediate resolutions - in CBR targeting 2700 Kbps, with a target base ratio of approximately 75%, to accommodate two profiles with a single encoding.

[0091] This mode can be used for distribution by HTTP with LCEVC-enabled players able to choose whether or not to add the LCEVC enhancement. Legacy players will only select and decode the base file. Older players will not need to download the enhancement file.

[0092] In this document, the following terminology will be used: BL = Base layer (e.g., AVC, HEVC, VVC); and EL = Enhancement layer (LCEVC).

[0093] Figure 5 illustrates a comparison with Figure 4, in which an advantageous implementation of the scalable mode can be illustrated. In the illustrated implementation, the division of base and enhancement can be leveraged to present a condensed set of variants in an approach called multiscale with LCEVC support. A first set of representations is optionally presented as unaffected base profiles, for example, unaffected AVC profiles. In this example, videos encoded only with basic codecs are neither stored nor presented. Shown here as the 360p and 540p profiles. Instead, the encoder produces enhanced videos with LCEVC, and the base layer of these enhanced LCEVC videos is provided separately as profiles for the lower bandwidths. In other words, the base layer of the LCEVC encoding process is provided as a separate representation in the ABR ladder.

[0094] In the specific example shown, an additional benefit is provided by encoding the LCEVC video using different base settings. This facilitates an overall increase in available representations. Petition 870250112790, dated 08 / 12 / 2025, page 34 / 58 27 / 42 without significantly increasing the number of encodings and stored videos. Higher resolutions can be encoded using different base settings, so more base representations can be presented without increasing the encodings and stored videos. In this example, four representations can be derived from two LCEVC encodings, with each LCEVC encoding using different base settings. In the specific example in Figure 5, the enhancement layer adds a small bitrate to the 540p and 360p profiles to present 720p and 1080p representations. They reuse the bitrate of the base layer, specifically, reusing the approximately 1.9 Mbps (1.1 Mbps) base layer of 1080p as the 540p profile and the base layer of the 720p LCEVC encoding as the 360p representation. This allows 4 much smaller representations to be derived from 2 base layers and 2 enhancement layers.The specific example shown in the comparison of Figures 4 and 5 leads to an approximately 54% reduction in the total bandwidth required to present all profiles. Note that 720p would normally be derived from the 360p base due to the normative scaling modes in the LCEVC standard, i.e., x2 scaling horizontally and vertically. In other words, two videos encoded with enhancement can be presented as four representations for the ABR logic to alternate.

[0095] At this point, it is worth noting that ABR typically has rules regarding which representation to consider. For example, based on codec compatibility or maximum resolution. The client device typically navigates the ladder in a predetermined pattern based on the manifest, and aspects such as startup speed are considered to optimize the user experience. The configuration proposed in this document facilitates the improvement of these rules, allowing for easy navigation through the ladder to enhance the user experience.

[0096] The following pseudocode is an example of this implementation: trackSelectorParameters = new Petition 870250112790, dated 08 / 12 / 2025, page 35 / 58 28 / 42 DefaultTrackSelector.ParametersBuilder() setMaxVideoSize(maxVideoWidth, maxVideoHeight) .setMaxVideoBitrate(maxVideoBitrate) .setForceHighestSupportedBitrate(false) .build();

[0097] Figure 6 illustrates how the implementation of the above example of Scalable Mode can be implemented at different points in the pipeline.

[0098] Encoder 100 is configured to produce two LCEVC-enhanced encoded videos from a source video 102. In a specific example, this comprises 2 fmp4 files, each with 2x tracks, 1x 'acvc1' (i.e., a video encoded using AVC) and 1x 'lvc1' (i.e., an enhancement layer that provides an LCEVC enhancement to a respective base layer which is a respective AVC-encoded video). In the example, this is a 720p LCEVC-enhanced video and a 1080p LCEVC-enhanced video. These videos are fed to a modified packer 602.

[0099] The 602 packager, as described above, selects the manifest file and presents each track of the two enhanced-encoded videos as separate, independent profiles. The decoder (i.e., the client device or player) will select the appropriate representation according to the profile defined by the packager. In the example, an LCEVC-enabled 608 client device will download a base file and an enhancement file and combine them to recreate the video (or combine them to create the representation, in optional implementations). A legacy 210 client device (decoder or player) will download only the base segments, avoiding having to download the enhancement layer it cannot process (or avoiding the packager presenting two representations, comprising the base, to legacy and LCEVC-enabled devices). By legacy, it is meant not LCEVC-enabled.

[0100] Figure 7 illustrates this concept using the pipeline of Petition 870250112790, dated 08 / 12 / 2025, page 36 / 58 29 / 42 Figure 2. As described herein, the delivery of LCEVC encoded streams in scalable mode can be done using MP4 ISO BMFF transport and MPEG-DASH distribution. Other mechanisms are considered within the scope of this disclosure, but the examples are presented using MPEG-DASH for brevity. In the examples, modifications can be made to the encoder, packager, and decoder to adapt them to the distribution method.

[0101] As before, a source video 102 is encoded using encoder 100 to produce a plurality of variants of the source video encoded using the enhancement codec. Here, the use of LCEVC encoding the video into an mp4 file has been illustrated. Depending on the implementation of the LCEVC encoder, modifications may or may not be necessary. The important thing is that the enhancement layer and the base layer can be separated. Therefore, in some embodiments, the encoder can be configured to generate an encoded version of the source video, encoded using an enhancement codec that comprises separate base and enhancement tracks.

[0102] The LCEVC enhancement layer can be loop-encoded with the base layer, downscaling the image before base encoding and upscaling the reconstructed image to allow calculation of residual EL information, which is then encoded and transmitted. The LCEVC encoder will produce LCEVC NALUs according to the standard (ISO / IEC230942). The LCEVC NAL unit types can have the following meaning: nal_unit_type Name of the nal_unit_type NAL unit content and RBSP syntax structure NAL unit type class 0.27 UNSPEC0...UNSPEC27 Not specified Not VCL 28 LCEVC_NON_IDR Non-IDR segment VCL / Non-VCL 29 LCEVC_IDR IDR segment VCL / Non-VCL 30 LCEVC_RSV Reserved VCL / Non-VCL 31 UNSPEC31 Not specified Not VCL Petition 870250112790, dated 08 / 12 / 2025, page 37 / 58 30 / 42

[0103] It is expected that, for each IDR image in the database, the corresponding LCEVC image will also be IDR.

[0104] Alternative approaches are contemplated to facilitate the synchronization of segments (and frames) of enhancement layer data and base layer data sent separately. For example, ideally, a snapshot decoder update image of the base layer and enhancement layer should be synchronized to facilitate the combining of enhancement data residues. There are techniques in well-known multi-layer codecs such as SVC and SHVC.

[0105] The frames of the base layer video and the enhancement layer video can also be kept synchronized using the techniques described in WO2023 / 118851, which is incorporated here by reference. This technique is relevant for HTML element implementations where the player may not have access to information in the base layer. It is described here how an elementary stream in packets (PES) can be constructed from a plurality of transport stream (TS) packets. A presentation timestamp (PTS) is provided in packets for the base stream. The enhancement stream uses a presentation timestamp in the enhancement stream to synchronize the frames of the two streams during HTML reconstructions.In other words, the frames in the enhancement stream are indexed according to the PTS, and this is compared with timing metadata, such as a "media time" variable or a current playback time provided in the base layer, to pair the enhancement layer frame with the base layer frame. In other words, in HTML5, the player can be provided with RGB data and timing metadata (but not timestamp information) that can be used to discover the timestamp information and combine it, as appropriate, with the enhancement data.

[0106] Alternative approaches involve the use of SEI messages to transport timing information that can be used to synchronize enhancement and baseline. Petition 870250112790, dated 08 / 12 / 2025, page 38 / 58 31 / 42

[0107] To preserve timing information and ensure synchronization, an example of LCEVC encapsulation in MP4 file format is provided. The encapsulation of an LCEVC bitstream in MP4 file format can be summarized in Figure 8.

[0108] The two main concepts used to link the LCEVC enhancement to its corresponding XVC base (where X stands for “any encoding format”, for example, AVC, HEVC, EVC, VVC) are:

[0109] (1) link the enhancement of LCEVC to the base of XVC is identified by means of an "sbas" box that identifies the corresponding database;

[0110] (2) use the same time base for the “stts” boxes in the enhancement, as they are used in the base, to perform picture-to-picture synchronization.

[0111] An 800 process is shown in Figure 8, in which an LCEVC NALU 802 is linked to the base NALU 804 using a box < <sbas>The LCEVC NALU 802 is transformed into an LCEVC sample with a box < <stts>equal to <stts>Sampling time that the Base 808 sample created from the NALU Base. These samples can then be combined into the LCEVC 810 image.

[0112] Examples of encoder implementations can be configured to return two separate outputs, one for the BL NALUs and the other for the EL NALUs, with the necessary synchronization between BL and EL. Packaging in the transport mechanism by the multiplexer encoder can then generate: a single mp4 file with BL and EL on two separate tracks or two files, one with BL and the other with EL, both on a single track.

[0113] In one implementation example, the first can be produced, i.e., a single mp4 with BL and EL on separate tracks. The separation of the tracks into different independent fragments can be left to the packager.

[0114] Returning to Figure 7, a modified packer 702 is configured to separate the encoded variants into base and enhancement layers for retrieval as a set of segments and build a Petition 870250112790, dated 08 / 12 / 2025, page 39 / 58 32 / 42 manifest file comprising metadata suitable for constructing URLs to retrieve these segments. As usual, this metadata may comprise a modified URL or a URL template to be combined with the information from each segment to construct a URL to retrieve that segment. The separate segments are then stored in storage 204 for later retrieval, as usual. Each profile created by the bundler may also comprise a suitable indication of the average bandwidth and other available metadata that can be used by an ABR handler or ABR rules to identify and retrieve the appropriate segments.

[0115] In one implementation, the packager will generate segments that are represented by separate MP4 files (or a single file with byte ranges), those containing only BL and those containing EL as a single-track representation. BL and EL are linked into an elementary stream by the encoder using the method described above. Optionally, the packager provides a way for the player to associate the BL MP4 fragments and the EL MP4 fragments. This can be done at the manifest level. In the implementations described here, this association is done using a dependency attribute in the manifest file.

[0116] To allow smooth playback, the packager can signal fragments in the BL and EL at the same start and end times / frames. In certain implementations, this means that the packager will have to recognize the NALUs of the LCEVC IDR (nal_unit_type = 29) and cut the segment in correspondence with the equivalent AVC / HEVC / VVC IDR.

[0117] The matching of IDR frames is preferably guaranteed by the encoder, as explained above.

[0118] As explained elsewhere in this document, a dependency attribute is a possible implementation used to create an association between EL segments and the BL segments they are enhancing. A dependency attribute can be defined by the packager 702 in the manifest file 705. In the example shown in Figure 7, this is shown as Petition 870250112790, dated 08 / 12 / 2025, pages 40 / 58 33 / 42 "dID" and links the representation in the Adaptation Set for the enhancement layer to a representation in the Adaptation Set for the base layer. The bandwidth of the enhancement adaptation set can represent the bandwidth of the combined BL and EL representation or, optionally, the bandwidth of the enhancement layer only for the client device to combine it with the base bandwidth requirement to identify the total bandwidth. The latter may be preferable, but this may depend on the standards and how they are designed.

[0119] The contemporary use of the Adaptation Set dependency ID attribute has been limited to a single Adaptation Set for purposes such as HDR. To date, no use or mechanism has been proposed or documented for combining two video adaptation sets. Current players know how to combine segments retrieved from multiple Adaptation Sets, for example, for video and audio or subtitles, but players have not been adapted for combining two Adaptation Sets comprising video representations.

[0120] In certain modalities, the Set of Adaptation that includes the enhancement layer can be labeled as Data instead of Video to facilitate retrieval and matching by players and client devices.

[0121] This disclosure introduces the concept of separating base and enhancement layers for storage and presentation and provides mechanisms to allow ABR logic to move between representations that represent a video stream encoded using the combined base and enhancement layers, as well as the base-encoded video alone. The use of a dependency ID attribute to link enhancement layer segments with their respective base layer segments is only one possible preferred implementation.

[0122] In this exemplary illustrated scenario, a single Adaptation Set is expected, but with separate Representations. It is clear that Petition 870250112790, dated 08 / 12 / 2025, page 41 / 58 34 / 42 other adaptation sets can be understood in MPD, but, for the purposes of explanation here, there are several representations in each adaptation set for each MIME type. Each representation points to an MP4 segment containing a single track, for BL only the legacy bitstream, e.g., AVC; for EL only the LCEVC encoded enhancement. The link from EL to BL is external, from EL MP4 to BL MP4, and is provided using the @DependencyID attribute in MPEG-DASH.

[0123] This mechanism is described in "DASH-IF Interoperability - Guidelines for Implementations”, Appendix G: Sample of a dual-layer MPD. The example reported in the guidelines is shown below: <period> <adaptationset mimetype=""video / mp4”" subsegmentalignment=""true”" subsegmentstartswithsap=""1”" framerate=""24”"> Representation id="base-layer” codecs="hev1.2.4.H153.b0” bandwidth="29200362” width="3840” height="2160”> <baseurl>BL_DashIf_BL-HDR10_UHD.mp4< / baseurl> <segmentbase indexrange=""687-862”">initialization range="0-686” / > < / segmentbase> Representation id="enhancement-layer” dependencyId="baselayer” codecs="dvhe.07.03” bandwidth="9320458” width="1920” height="1080”> <baseurl> EL_DashIf_BL-HDR10_UHD.mp4< / baseurl> <segmentbase indexrange=""723-898”"> <initialization range=""0-722” / "> < / initialization> < / segmentbase> < / adaptationset> < / period> Petition 870250112790, dated 08 / 12 / 2025, page 42 / 58 35 / 42

[0124] In the case of an LCEVC enhancement, the only attribute that changes is the 4cc for codecs, which for LCEVC is "lvc1". An example of a manifest for LCEVC is shown below: <?xml version="1.0” encoding="utf-8”?> <MPD xmlns:xsi=http: / / www.w3.org / 2001 / XMLSchema-instance xmlns="urn:mpeg:dash:schema:mpd:2011” xmlns:xlink=http: / / www.w3.org / 1999 / xlink xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http: / / standards.iso.org / ittf / PubliclyAvailableStandards / MPEG- DASH_schema_files / DASH-MPD.xsd” profiles="urn:mpeg:dash:profile:isoff-live:2011” type="static” mediaPresentationDuration="PT10.0S” maxSegmentDuration="PT5.0S” minBufferTime="PT20.0S”> <programinformation> < / programinformation> <servicedescription id=""0”"> < / servicedescription>

[0125]

[0126] <!-- Nesse exemplo, 4 representações são usadas para informar o player sobre a existência de 4 perfis, 2 dos quais são derivados da combinação de base e aprimoramento>

[0127] <!-- o mecanismo de vinculação é a dependênciaID-><Period id="0” start="PT0.0S”><AdaptationSet id="0” contentType="video” startWithSAP="1” segmentAlignment="true” bitstreamSwitching="true” frameRate="25 / 1 ” Petition 870250112790, dated 08 / 12 / 2025, pp. 43 / 58 36 / 42 maxWidth="1920” maxHeight="1080” par="16:9”>

[0128] Representation id="0” mimeType="video / mp4” codecs="avc1.64001f” bandwidth="1100000” width="640” height="360” sar="1:1”> <SegmentTemplate timescale="12800” initialization="initstream$RepresentationID$.m4s” media="chunk-stream$RepresentationID$$Number%05d$.m4s” startNumber="1”><SegmentTimeline> <S t="1024” d="128000” / ><S d="1024” / >< / SegmentTimeline>< / SegmentTemplate>< / Representation>

[0129] <Representation id="1” mimeType="video / mp4” codecs-"avc1. < / s>64001f” bandwidth="1400000” width="960” height="540” sar="1:1”><SegmentTemplate timescale="12800” initialization="initstream$RepresentationID$.m4s” media="chunk-stream$RepresentationID$$Number%05d$.m4s” startNumber="1”><SegmentTimeline><S t="1024” d="128000” / ><S d="1024” / >< / SegmentTimeline>< / SegmentTemplate>< / Representation>

[0130] <Representation id="2” mimeType="video / mp4” codecs="lvc1” Petição 870250112790, de 08 / 12 / 2025, pág. 44 / 58 37 / 42 bandwidth="800000” width="1280” height="720” sar="1:1” dependencyId="0”><SegmentTemplate timescale="12800” initialization="initstream$RepresentationID$.m4s” media="chunk-stream$RepresentationID$$Number%05d$.m4s” startNumber="1”><SegmentTimeline> <S t="1024” d="129024” / >< / SegmentTimeline>< / SegmentTemplate>< / Representation>

[0131] Representation id="3” mimeType="video / mp4” codecs-1vc1” bandwidth="900000” width="1920” height="1080” sar="1:1” dependencyId="1”> <SegmentTemplate timescale-"12800” initialization-"initstream$RepresentationID$.m4s” media-"chunk-stream$RepresentationID$$Number%05d$.m4s” startNumber="1”><SegmentTimeline><S t-"1024” d-"129024” / >< / SegmentTimeline>< / SegmentTemplate>< / Representation>< / AdaptationSet>< / Period>< / MPD>

[0132] The concept of Dependent Representation (which is the case of an LCEVC EL, depending on its corresponding BL) is introduced in 23009-1, clause 5.3.5.1.

[0133] The following is a quote from 23009-1, clause 5:

[0134] "Dependent Representations are described by a Representation element containing a @dependencyId attribute. Dependent Petition 870250112790, dated 08 / 12 / 2025, p.< / s>45 / 58 38 / 42 Representations are regular Representations except that they depend on a set of complementary Representations for decoding and / or presentation. The @dependencyId contains the values ​​of the @id attribute of all the complementary Representations, i.e. Representations that are necessary to present and / or decode the media content components contained in this dependent Representation.”

[0135] @codingDependency is defined in 23009-1, clause 5.3.7.2 (Semantics) and clause 5.3.7.3 (Syntax), as a common attribute for AdaptationSet and Representation.

[0136] Other relevant information can be found in 230090 - Dynamic adaptive streaming over HTTP (DASH), Part 3: Implementation guidelines, Section 10. Mapping the ISOBMFF track reference type to the DASH MPD. The following is a quote from 23009-3, clause 10:

[0137] "Dependencies (see the definition of "Dependent "Representations" in Section 5.3.5.1.(from DASH Part 1):

[0138] For a given track reference type (one row of the (Table), when the "Mapping to DASH" column of the Table indicates "Dependency", it means that the Representation@dependencyId must be defined for the Representation corresponding to the range, declaring the range reference of the data type to the Representation(s) corresponding to the range(s) having its range_ID listed in the 'tref' box. The list of representation ids in the @dependencyId attribute will match the track_ID order from the 'tref' box.

[0139] In another possible implementation detail, regarding the MIME types for the Base Layer and Enhancement Layer, the Base Layer will have the Sample Description Box according to the base codec (e.g., "avc1", "hvc1", "evs1", "vvc1"...), while the latter will consistently have its Sample Description Box (e.g., "lvc1").

[0140] As mentioned above, the link between the EL and its corresponding BL can be implemented using the "sbas" box. Note that, in other modes established in this document, the current time of the media element is used to compare the points on the enhancement side and, Petition 870250112790, dated 08 / 12 / 2025, pages 46 / 58 39 / 42 therefore, this example is not necessary.

[0141] Returning to Figure 7, in the known manner, to retrieve a video stream, a client device first retrieves the manifest file. In the known manner, a legacy 210 device, i.e., a device not enabled for LCEVC, will retrieve and parse the manifest file and retrieve a representation according to the Adaptation Set it is compatible with, constructing a URL for the video encoded in the base layer and downloading segments of the video to fill its available buffer according to the current conditions.References to client devices here can also refer to a player or set-top box.

[0142] A client device modified according to the principles of the present invention will retrieve the manifest file and analyze the manifest file to identify a representation of the enhancement layer and, from the dependency attribute, will be able to identify an associated representation of the base layer. In a first example, the client device will determine the appropriate representation of the enhancement adaptation set and retrieve the associated base layer segments as appropriate. In a second example, the client device may create a set of virtual representations from the manifest file, the virtual representations indicating the combination of the base layer and the enhancement layer indicated by the dependency attribute.

[0143] References to client devices here can also refer to a player or set-top box. An example of a player to be modified is dash.js.

[0144] The client device will retrieve the base layer and enhancement layer segments and pass them to an appropriate enhancement decoder for decoding and combining for rendering by the client device.

[0145] In an example process, a client device, referred to here as decoder 708, will have to analyze the manifest file Petition 870250112790, dated 08 / 12 / 2025, pages 47 / 58 40 / 42 and choose from a set of interpretations based on available bandwidth. The player (i.e., the player on the client device or decoder, such as dash.js) is expected to create a combined virtual set of representations where an enhanced profile will have a bandwidth equal to the sum of EL and BL. The example process might involve the following:

[0146] 1.The DASH manifest parser must analyze the @DependencyID attribute to define the dependency between the BL and EL representations.

[0147] 2. A DASH ABR mechanism can maintain its list of Representations by considering BL as one Representation, and EL along with its dependency BL as another Representation. In the manifest example in Figures 5 and 6 above, the DASH ABR mechanism would maintain 4 representations:

[0148] a. Representation 1 with BL 360p only, with bandwidth = 1100000

[0149] b. Representation 2 with BL 540p only, with bandwidth = 1400000

[0150] c. Representation 3 with 720p EL with its dependency 360p BL, with bandwidth=1900000, which is the sum of the 720p EL bandwidth (Representation id 2) and the 360p BL bandwidth (Representation id 0).

[0151] d.Representation 4 with 1080p EL with 720p BL breast dependency, with bandwidth=2300000, which is the sum of the bandwidth of 1080p EL (Representation id 3) and the bandwidth of 540p BL (Representation id 1).

[0152] 3. When the DASH ABR mechanism detects sufficient bandwidth and codec capacity (i.e., LCEVC compatibility) to be able to choose the bitrate corresponding to the bandwidth of the representation containing the EL:

[0153] a. The DASH downloader can download the BL segment and the related EL segment. The downloader can ensure that the Petition 870250112790, dated 08 / 12 / 2025, pages 48 / 58 41 / 42 BL and EL segments should be downloaded asynchronously, as much as possible, so that all the necessary EL and BL data is available to decode a complete image. In other words, the segments should be downloaded together, as opposed to synchronous download, which would mean one after the other.Synchronous mode is also included.

[0154] b. The playback pipeline can feed the data BL is sent to the base decoder via the available interface (e.g., mediacodec on Android), and EL data is sent to the LCEVC decoder to decode the complete image. Further details can be found in document WO2022 / 023739, incorporated by reference, which describes a decoder integration layer (DIL).

[0155] 4. If the ABR mechanism detects a bandwidth or codec capacity that matches only the BL stream, only the BL data can be downloaded and fed into the base decoder, as is currently done in contemporary processes.

[0156] A simplified example of this process 900 from the client device perspective is illustrated in Figure 9. In step 902, the modified manifest file is obtained. The manifest file is then parsed in step 904.Based on the available conditions, a Representation for recovery is identified in step 906, and a dependency is identified from the dependency attribute in step 908. In step 910, the base layer segment and the related enhancement layer segment are downloaded based on the dependency. In step 912, they are passed to a decoder for decoding and reconstruction of the enhancement video.

[0157] As indicated above, the concepts set forth herein can be implemented in a client device, player-on-device, or set-top box. Similarly, the concepts can be incorporated by modifications in an encoder, packager, and / or content distribution network. In each of the entities, methods and processes described herein can be incorporated as code (e.g., software code) and / or data. A Petition 870250112790, dated 08 / 12 / 2025, page.49 / 58 42 / 42 functionality can be implemented in hardware or software, as is well known in the art of data compression and video transmission. For example, hardware acceleration using a specifically programmed Graphics Processing Unit (GPU) or a specifically designed Field Programmable Gate Array (FPGA) can provide certain efficiencies. Finally, these codes and data can be stored on one or more computer-readable media, which can include any device or medium that can store code and / or data for use in a computer system. When a computer system reads and executes the code and / or data stored on a computer-readable medium, the computer system executes the methods and processes embedded as data structures and code stored on the computer-readable storage medium.In certain embodiments, one or more steps of the methods and processes described herein may be executed by a processor (for example, a processor in a computer system or data storage system).

[0158] Generally, any functionality described in this text or illustrated in the figures can be implemented using software, firmware (e.g., fixed logic circuits), programmable or non-programmable hardware, or a combination of these implementations. The terms "component" or "function," as used herein, generally represent software, firmware, hardware, or a combination thereof. For example, in the case of a software implementation, the terms "component" or "function" may refer to the program code that performs specific tasks when executed on one or more processing devices.The illustrated separation of components and functions into distinct units may reflect any real or conceptual physical grouping and allocation of such software and / or hardware and tasks. Petition 870250112790, dated 08 / 12 / 2025, pages 50 / 58-->.< / stts> < / stts> < / sbas>

Claims

1 / 6 CLAIMS 1. A method for transmitting encoded multimedia content using an enhancement coding scheme, wherein the method is characterized in that it comprises: retrieving a manifest file, the manifest file comprising metadata for constructing a locator for segments of the encoded multimedia content, the metadata comprising: a first representation pointing to a first version of the encoded multimedia content using a base coding scheme; and, location metadata for a first enhancement layer of the encoded multimedia content associated with the first version; and, parsing the manifest file to construct a locator for retrieving the first representation and enhancement layer segments; downloading segments from the first representation and enhancement layer segments to reconstruct the encoded multimedia content.

2. Method, according to claim 1, characterized in that the metadata comprises a second representation that points to the first layer of enhancement of the encoded multimedia content.

3. Method, according to claim 2, characterized in that the metadata further comprises a dependency attribute that associates the second representation with the first representation.

4. Method, according to claim 2 or 3, characterized in that the first and second representations are included in their respective Adaptation Sets.

5. A method, according to any one of claims 2 to 4, characterized in that the metadata further comprises, for each representation, a bandwidth to retrieve the respective representation, and the method further comprises: Petition 870250093027, dated 10 / 10 / 2025, pp. 82 / 91 2 / 6 determining a total bandwidth to transmit the multimedia content by summing the bandwidth associated with the second representation and the bandwidth of its respective first associated representation.

6. Method, according to claims 2 to 5, characterized in that: the first representation comprises an attribute indicating the base coding scheme; and the second representation comprises an attribute indicating the enhancement coding scheme.

7. A method, according to any one of claims 2 to 6, characterized in that the metadata further comprises: a third representation pointing to a second version of the multimedia content encoded using the base encoding scheme, wherein the first and second versions are versions of the multimedia content at a different quality level; and a fourth representation pointing to a second enhancement layer associated with the second version of the encoded multimedia content.

8. A method according to any one of claims 3 to 7, characterized in that the method further comprises: creating a set of virtual representations from the first and second representations and the dependency attribute, wherein the virtual representations comprise: a first virtual representation comprising the first representation; and, a second virtual representation comprising the first and second representations.

9. A method, according to any of the preceding claims, characterized in that it further comprises: detecting the available bandwidth and / or conditions of the device; and downloading segments from the first representation or segments from both the first representation and enhancement layer segments to reconstruct the encoded multimedia content according to the available conditions.

10. A method, according to any of the preceding claims, characterized in that it comprises: passing the downloaded segments from the enhancement layer to an enhancement decoder to generate decoded enhancement data; and reconstructing the multimedia content from the decoded enhancement data and a decoded version of the video stream encoded using the base encoding scheme decoded from the downloaded segments of the first representation.

11. A method for packaging encoded multimedia content using an enhancement coding scheme for transmission, wherein the method is characterized by the fact that it comprises: constructing a manifest file, the manifest file comprising metadata for constructing a locator for segments of the encoded multimedia content, the metadata comprising: a first representation that points to a first version of the multimedia content encoded using a base coding scheme; and, location metadata for a first enhancement layer of the encoded multimedia content associated with the first version.

12. Method, according to claim 11, characterized in that the metadata comprises a second representation that points to the first layer of enhancement of the encoded multimedia content.

13. Method, according to claim 12, characterized in that the metadata further comprises a dependency attribute that associates the second representation with the first representation. Petition 870250093027, dated 10 / 10 / 2025, pp. 84 / 91 4 / 6 14. Method, according to claim 12 or 13, characterized in that the first and second representations are included in their respective Adaptation Sets.

15. A method, according to any one of claims 12 to 14, characterized in that the metadata further comprises, for each representation, a bandwidth to retrieve the respective representation, and the method further comprises: determining a total bandwidth to transmit the multimedia content by summing the bandwidth associated with the second representation and the bandwidth of its respective first associated representation.

16. Method, according to claims 12 to 15, characterized in that: the first representation has a type indicating the basic coding scheme; and, the second representation has a type indicating the enhanced coding scheme.

17. A method according to any one of claims 12 to 16, characterized in that the metadata further comprises: a third representation pointing to a second version of the multimedia content encoded using the base encoding scheme, wherein the first and second versions are versions of the multimedia content at a different quality level; and a fourth representation pointing to a second enhancement layer associated with the second version of the encoded multimedia content.

18. A method, according to any one of claims 11 to 17, characterized in that it further comprises: receiving an encoded version of multimedia content encoded using the enhancement coding scheme comprising a base layer and an enhancement layer; Petition 870250093027, dated 10 / 10 / 2025, pp. 85 / 91 5 / 6 generating a first set of segments representing the base layer; and, generating a second set of segments representing the enhancement layer, wherein the first set of segments is independent of the second set of segments.

19. A method according to any one of claims 11 to 17, characterized in that the first set of segments and the second set of segments are generated as respective single-strip representations.

20. A method, according to any one of claims 11 to 19, characterized in that the construction of the manifest file comprises: inserting an association between the second set of segments and the first set of segments; obtaining a location of the first and second sets of segments; and inserting metadata for constructing a locator for the location of the first and second sets of segments.

21. Client device for transmitting encoded multimedia content, characterized in that it uses an enhancement encoding scheme configured to perform the method as defined in any one of claims 1 to 10.

22. A packager for packaging encoded multimedia content, characterized in that it uses a transmission enhancement coding scheme configured to perform the method as defined in any one of claims 1 to 20.

23. System characterized in that it comprises a client device, as defined in claim 21, and a packager, as defined in claim 22.

24. System according to claim 23, characterized in that it further comprises: Petition 870250093027, dated 10 / 10 / 2025, pp. 86 / 91 6 / 6 an encoder configured to encode multimedia content using an enhancement encoding scheme; and a content delivery network configured to serve the segments and the manifest file.

25. Computer-readable means, characterized in that it comprises instructions of claims which, when executed by a computer, cause a computer to perform the method as defined in any one of claims 1 to 10 or 11 to 20. Petition 870250093027, dated 10 / 10 / 2025, pp. 87 / 91

Images