Decoding method and device, encoding method and device, bitstream storage method, and bitstream transmission method

Abstract

To provide a method for efficiently signaling coding tool parameters used to compress video data in video coding.SOLUTION: A method for decoding a video sequence from a bitstream includes receiving a bitstream comprising a first header parameter set (HPS) containing a first type of coding tool parameters, a second HPS containing a second type of coding tool parameters, a slice header, and a slice associated with the slice header. The method further includes determining that the slice header contains a first reference to the first HPS and a second reference to the second HPS. The method further includes decoding the slice using the first type of coding tool parameters and the second type of coding tool parameters based on the determination that the slice header contains the first reference and the second reference. The method further includes forwarding the slice for display as part of a decoded video sequence.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present disclosure generally relates to video coding, and more particularly, to efficient signaling of coding tool parameters used to compress video data in video coding.

Background Art

[0002] The amount of video data required to represent even a relatively short video can be substantial, which can cause difficulties when data is to be streamed or otherwise communicated across a communication network with limited bandwidth capacity. Thus, video data is generally compressed before being communicated across today's telecommunications networks. The size of the video can also be an issue when the video is stored on a storage device, since memory resources may be limited. Video compression devices often use software and / or hardware at the source to encode the video data before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. Due to limited network resources and the ever-increasing demand for even higher video quality, improved compression and decompression techniques that improve the compression ratio without sacrificing picture quality at all or almost at all are desirable.

Summary of the Invention

[0003] In an embodiment, the disclosure is a method implemented in a decoder, comprising receiving, by a receiver of the decoder, a bitstream having a first Header Parameter Set (HPS) including first type coding tool parameters, a second HPS including second type coding tool parameters, a slice header, and a slice associated with the slice header; determining, by a processor of the decoder, that the slice header includes a first reference to the first HPS and a second reference to the second HPS; decoding, by the processor, the slice using the first type coding tool parameters and the second type coding tool parameters based on the determination that the slice header includes the first reference and the second reference; and transferring, by the processor, the slice for display as part of a decoded video sequence. The HPS, also known as an Adaptive Parameter Set (APS), may be used to describe video data with a lower granularity than a Picture Parameter Set (PPS) and a higher granularity than a slice header. The disclosed aspect enables a single slice header to reference multiple types of HPS. By providing a mechanism that enables a single slice header to reference multiple types of HPS, various coding tool parameters can be signaled at the HPS level. This allows coding tool parameters to vary between slices within the same picture / frame without loading extra data into the slice header. Thus, the encoder has more flexibility when performing Rate Distortion Optimization (RDO) in that coding tool parameters that vary between slices within the same picture do not have to be loaded into any slice header. Furthermore, the average coding efficiency increases in that the encoder may have access to more coding options when finding an optimal coding solution.This, in turn, reduces the use of memory resources and network resources in both the encoder and the decoder when storing and transmitting video data.

[0004] Optionally, in any of the above aspects, other implementations of the aspect provide that the first HPS and the second HPS include an Adaptive Loop Filter (ALF) HPS, a Luma Mapping with Chroma Scaling (LMCS) HPS, a Scaling List Parameter HPS, or a combination thereof.

[0005] Optionally, in any of the above aspects, other implementations of the aspect provide that the bitstream further has a plurality of HPSs including the first HPS and the second HPS, and each of the plurality of HPSs is restricted from referring to coding tool parameters from other HPSs within the plurality of HPSs. In some systems, the current HPS may inherit coding parameters by reference to a previous HPS. In theory, this allows the current HPS to contain only the differences between the current HPS and the previous HPS. However, in practice, this approach often creates long inheritance chains, which in turn require the decoder to hold a number of old HPSs in the buffer as long as subsequent HPSs may refer back to old HPSs. This causes both buffer memory problems and increases the likelihood of coding errors if an HPS is lost during transmission. Some aspects of the present disclosure address this issue by requiring that each HPS include all relevant coding parameters without referring to other HPSs.

[0006] Optionally, in any of the above aspects, another implementation of the aspect provides that the first HPS is included in an access unit associated with a time identifier (ID), and the first HPS includes a time ID associated with the access unit including the first HPS. In one example, each HPS may be required to hold the same time ID as the access unit that includes that HPS.

[0007] Optionally, in any of the above aspects, another implementation of the aspect provides that a slice is a part of a picture, the picture is associated with a time identifier (ID), and the first HPS includes a time ID associated with the picture. In some examples, each HPS may be required to hold the same time ID as the picture associated with the first slice that references that HPS.

[0008] Optionally, in any of the above aspects, another implementation of the aspect provides that a bitstream includes a plurality of pictures each including one or more slices, the bitstream further has a plurality of HPSs including a first HPS and a second HPS, each of the plurality of HPSs and each of the one or more slices is associated with one of a plurality of time IDs, and each slice having a first time ID is restricted from referencing any HPS having a second time ID greater than the first time ID. In some examples, the bitstream of pictures and slices can be associated with one of a plurality of time IDs (e.g., one of three). Each time ID is associated with a corresponding frame rate. Data items having a higher frame rate may be ignored when a lower frame rate is being rendered. In this example, a slice is made unable to reference an HPS having a higher time ID and thus a higher frame rate. This approach ensures that a slice does not reference an HPS with a higher frame rate that would be ignored when the slice's lower frame rate is being rendered. This can ensure that the HPS is actually available to the slice and is not ignored due to a frame rate mismatch.

[0009] In an embodiment, the disclosure is a method implemented by an encoder, comprising partitioning, by a processor of the encoder, a plurality of pictures into a plurality of slices; encoding, by the processor, the plurality of slices into a bitstream, wherein the plurality of slices are encoded by at least a first type of coding tool based on first type of coding tool parameters and a second type of coding tool based on second type of coding tool parameters; encoding, by the processor, a first HPS and a second HPS into the bitstream, wherein the first HPS includes the first type of coding tool parameters and the second HPS includes the second type of coding tool parameters; encoding, by the processor, a first slice header into the bitstream, the first slice header describing the encoding of a first slice among the plurality of slices, the first slice header including a first reference to the first HPS and a second reference to the second HPS; and storing, by a memory coupled to the processor, the bitstream for communication to a decoder. The HPS, also known as APS, may be used to describe video data at a lower granularity than PPS and a higher granularity than a slice header. The disclosed aspect enables a single slice header to reference multiple types of HPS. By providing a mechanism that enables a single slice header to reference multiple types of HPS, various coding tool parameters can be signaled at the HPS level. This enables coding tool parameters to vary between slices within the same picture / frame without loading extra data into the slice header. Thus, the encoder has more flexibility when performing RDO as coding tool parameters that vary between slices within the same picture do not have to be loaded into any slice header. Furthermore, the average coding efficiency increases as the encoder may have access to more coding options when finding an optimal coding solution.This in turn reduces the use of memory resources and network resources in both the encoder and the decoder when storing and transmitting video data.

[0010] Optionally, in any of the above aspects, other implementations of the aspect provide that the first HPS and the second HPS include an ALF HPS, an LMCS HPS, a scaling list parameter HPS, or a combination thereof.

[0011] Optionally, in any of the above aspects, other implementations of the aspect further include encoding, by a processor, a plurality of HPSs into a bitstream, the plurality of HPSs including the first HPS and the second HPS, and each of the plurality of HPSs being restricted to refer to coding tool parameters from other HPSs within the plurality of HPSs. In some systems, the current HPS may inherit coding parameters by reference to a previous HPS. In theory, this allows the current HPS to contain only the difference between the current HPS and the previous HPS. However, in practice, this approach often creates long inheritance chains, which in turn require the decoder to hold a number of old HPSs in the buffer as long as subsequent HPSs may refer back to old HPSs. This causes both buffer memory problems and increases the likelihood of coding errors if an HPS is lost during transfer. Some aspects of the present disclosure address this issue by requiring each HPS to include all relevant coding parameters without referring to other HPSs.

[0012] Optionally, in any of the above aspects, other implementations of the aspect provide that the first HPS is included in an access unit related to a temporal ID and that the first HPS includes a temporal ID related to the access unit including the first HPS. In one example, each HPS may be required to hold the same temporal ID as the access unit that includes that HPS.

[0013] Optionally, in any of the above aspects, other implementations of the aspect provide that the first slice is partitioned from the first picture, the first picture is associated with a temporal ID, and the first HPS includes the temporal ID associated with the first picture. In some examples, each HPS may be required to hold the same temporal ID as the picture associated with the first slice that references that HPS.

[0014] Optionally, in any of the above aspects, other implementations of the aspect provide that a plurality of HPSs including a first HPS and a second HPS are encoded, each of the plurality of HPSs and each of the plurality of slices are associated with one of a plurality of temporal IDs, and each slice having a first temporal ID is restricted from referencing any HPS having a second temporal ID greater than the first temporal ID. In some examples, the bitstreams of the pictures and slices can be associated with one of a plurality of temporal IDs (e.g., one of three). Each temporal ID is associated with a corresponding frame rate. Data items having a higher frame rate may be ignored when a lower frame rate is being rendered. In this example, a slice is made unable to reference an HPS having a higher temporal ID and thus a higher frame rate. This approach ensures that a slice does not reference an HPS with a higher frame rate that would be ignored when the slice's lower frame rate is being rendered. This can ensure that the HPS is actually available to the slice and is not ignored due to a frame rate mismatch.

[0015] In an embodiment, the disclosure includes a video coding device having a processor, a receiver coupled to the processor, a memory coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform any of the methods of the above aspects.

[0016] In an embodiment, the disclosure includes a non-transitory computer-readable medium having a computer program product used by a video coding device, the computer program product having computer-executable instructions stored on the non-transitory computer-readable medium that, when executed by a processor, cause the video coding device to execute a method of any of the above aspects.

[0017] In an embodiment, the disclosure includes receiving means for receiving a bitstream having a first HPS including first type coding tool parameters, a second HPS including second type coding tool parameters, a slice header, and a slice associated with the slice header; determining means for determining that the slice header includes a first reference to the first HPS and a second reference to the second HPS; decoding means for decoding the slice using the first type coding tool parameters and the second type coding tool parameters based on the determination that the slice header includes the first reference and the second reference; and transfer means for transferring the slice for display as a portion of the decoded video sequence.

[0018] Optionally, in any of the above aspects, another implementation of the aspect provides that the decoder is further configured to execute a method of any of the above aspects.

[0019] In an embodiment, the disclosure includes a partitioning means for partitioning a plurality of pictures into a plurality of slices, an encoding means for encoding the plurality of slices into a bitstream, where the slices are encoded by at least a first type of coding tool based on first type of coding tool parameters and a second type of coding tool based on second type of coding tool parameters; encoding a first HPS and a second HPS into the bitstream, where the first HPS includes the first type of coding tool parameters and the second HPS includes the second type of coding tool parameters; encoding a first slice header into the bitstream for describing the encoding of a first slice among the plurality of slices, where the first slice header includes a first reference to the first HPS and a second reference to the second HPS, and a storage means for storing the bitstream for communication to a decoder.

[0020] Optionally, in any of the above aspects, another implementation of the aspect provides that the encoder is further configured to execute the method of any of the above aspects.

[0021] For clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to bring about new embodiments within the scope of the present disclosure.

[0022] These and other features will be more clearly understood from the following detailed description, read in conjunction with the accompanying drawings and the claims.

[0023] For a more complete understanding of the present disclosure, reference is now made to the following brief description, read in conjunction with the accompanying drawings and detailed description. Like reference numerals represent like parts.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0025] Exemplary implementations of one or more embodiments are given below, but it should first be understood that the disclosed system and / or method may be implemented using any number of techniques, regardless of whether currently known or in existence. The disclosure should in no way be limited to the exemplary implementations, drawings, and techniques described below, including the example designs and implementations illustrated and described herein, but may be varied within the full scope of the appended claims and their equivalents.

[0026] The following acronyms, Adaptive Loop Filter (ALF), Coding Tree Block (CTB), Coding Tree Unit (CTU), Coding Unit (CU), Coded Video Sequence (CVS), Dynamic Adaptive Streaming over Hypertext transfer protocol (DASH), Joint Video Experts Team (JVET), Motion-Constrained Tile Set (MCTS), Maximum Transfer Unit (MTU), Network Abstraction Layer (NAL), Picture Order Count (POC), Raw Byte Sequence Payload (RBSP), Sample Adaptive Offset (SAO), Sequence Parameter Set (SPS), Versatile Video Coding (VVC), and Working Draft (WD) are used herein.

[0027] Many video compression techniques can be used to reduce the size of video files with minimal data loss. For example, video compression techniques can include performing spatial (e.g., intra-picture) prediction and / or temporal (e.g., inter-picture) prediction to reduce or eliminate data redundancy within a video sequence. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks that may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks within an intra-coded (I) slice of a picture are coded using spatial prediction with respect to reference samples in adjacent blocks within the same picture. Video blocks within an inter-coded unidirectional prediction (P) or bidirectional prediction (B) slice of a picture may be coded by using spatial prediction with respect to reference pictures in adjacent blocks within the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may sometimes be referred to as a frame and / or an image, and a reference picture may sometimes be referred to as a reference frame and / or a reference image. Spatial or temporal prediction results in a prediction block representing the image block. Residual data represents the pixel difference between the original image block and the prediction block. Thus, an inter-coded block is encoded with a motion vector indicating a block of reference samples forming the prediction block and residual data indicating the difference between the coded block and the prediction block. An intra-coded block is encoded according to the intra-coding mode and the residual data. For further compression, the residual data may be transformed from the pixel domain to a transform domain. These result in residual transform coefficients, which may be quantized. The quantized transform coefficients may first be arranged in a two-dimensional array. The quantized transform coefficients may be scanned to generate a one-dimensional vector of the transform coefficients. Entropy coding may be applied to achieve further compression.Such video compression techniques will be described in more detail below.

[0028] To ensure that the encoded video can be accurately decoded, the video is encoded and decoded according to the corresponding video coding standard. Video coding standards include the International Telecommunication Union (ITU) Telecommunication Standardization Sector (ITU-T) H.261, the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Moving Picture Experts Group (MPEG)-1 Part 2, ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or Advanced Video Coding (AVC), also known as ISO / IEC MPEG-4 Part 10, and ITU-T H.265 or High Efficiency Video Coding (HEVC), also known as MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multi-View Video Coding (MVC) and Multi-View Video Coding Plus Depth (MVC+D), and 3D AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multi-View HEVC (MV-HEVC), and 3D HEVC (3D-HEVC). The Joint Video Experts Team (JVET) of ITU-T and ISO / IEC has started developing a video coding standard called Versatile Video Coding (VVC). VVC is included in the Working Draft (WD) including JVET-L1001-v1 and JVET-K1002-v3, and provides algorithm descriptions, encoder-side descriptions of the VVC WD, and reference software.

[0029] To code a video image, the image is first partitioned and the partitions are coded into the bitstream. Various picture partitioning schemes are available. For example, the image can be partitioned into regular slices, dependent slices, tiles, and / or according to wavefront parallel processing (WPP). For simplicity, HEVC restricts the encoder so that only regular slices, dependent slices, tiles, WPP, and combinations thereof can be used when partitioning slices into groups of CTBs for video coding. Such partitioning can be applied to support maximum transform unit (MTU) size matching, parallel processing, and reduced end-to-end delay. The MTU represents the maximum amount of data that can be transmitted in a single packet. If the packet payload exceeds the MTU, the payload is split into two packets through a process called fragmentation.

[0030] A regular slice, also simply called a slice, is a partitioned part of an image that can be reconstructed independently of other regular slices within the same picture, despite some interdependencies due to loop filtering operations. Each regular slice is encapsulated in its own Network Abstraction Layer (NAL) unit for transmission. Further, intra-picture prediction (intra-sample prediction, motion information prediction, coding mode prediction) and entropy coding dependencies across slice boundaries may be disabled to support independent reconstruction. Such independent reconstruction supports parallelization. For example, parallelization based on regular slices uses minimal inter-processor or inter-core communication. However, since each regular slice is independent, each slice is associated with a separate slice header. The use of regular slices can incur significant coding overhead due to the bit cost of the slice header for each slice and the lack of prediction across slice boundaries. Additionally, regular slices may be used to support the requirement of MTU size matching. Specifically, since regular slices are encapsulated in separate NAL units and can be coded independently, each regular slice must be smaller than the MTU in the MTU scheme to avoid splitting the slice into multiple packets. As such, the goals of parallelization and MTU size matching can impose conflicting requirements on the slice layout within a picture.

[0031] A dependent slice is similar to a regular slice but has a shortened slice header and enables partitioning at picture tree block boundaries without interrupting intra-picture prediction. Thus, a dependent slice allows a regular slice to be fragmented into multiple NAL units, which results in reduced end-to-end delay by enabling a part of the regular slice to be sent before the encoding of the entire regular slice is complete.

[0032] A tile is a partitioned part of an image created by horizontal and vertical boundaries that form columns and rows of tiles. Tiles may be coded in raster scan order (from right to left and top to bottom). The scan order of CTBs is local within a tile. Thus, CTBs within the first tile are coded in raster scan order before processing CTBs within the next tile. Similar to regular slices, tiles also break picture - in - prediction dependencies in addition to entropy - decoding dependencies. However, a tile may not be contained within an individual NAL unit and thus may not be used for MTU - size alignment. Each tile can be processed by one processor / core, and inter - processor / inter - core communication used for picture - in - prediction between processing units decoding adjacent tiles may be limited to carrying a shared slice header (when adjacent tiles are within the same slice) and performing sharing related to loop filtering of reconstructed samples and metadata. When more than one tile is included in a slice, the entry - point byte offset for each tile other than the first entry - point offset within the slice may be signaled in the slice header. For each slice and tile, at least one of the following conditions: 1) all coded tree blocks within a slice belong to the same tile, and 2) all coded tree blocks within a tile belong to the same slice, should be satisfied.

[0033] In WPP, the picture is partitioned into a single row of CTBs. The entropy decoding and prediction mechanisms may use data from CTBs in other rows. Parallel processing is enabled through parallel decoding of CTB rows. For example, the current row may be decoded in parallel with the preceding row. However, the decoding of the current row is delayed by only two CTBs from the decoding process of the preceding row. This delay ensures that data regarding the CTB above and the top-right CTB of the current CTB within the current row is available before the current CTB is coded. This approach appears as a wavefront when represented graphically. This time-offset start enables parallelization by a number of processors / cores that is less than or equal to the number of CTB rows included in the picture. Since picture-in-picture prediction between adjacent tree-block rows within the picture is permitted, the inter-processor / inter-core communication for enabling picture-in-picture prediction can be substantial. WPP partitioning does not consider the NAL unit size. Thus, WPP does not support MTU size matching. However, regular slices may be used with WPP with a specific coding overhead to implement MTU size matching as desired.

[0034] Tiles define horizontal and vertical boundaries that partition the picture into tile columns and rows. The scan order of CTBs may be changed to be local within a tile before decoding the top-left CTB of the next tile in the order of the tile raster scan of the picture. The local scan order indicates the order of the CTB raster scan of the tile. Similar to regular tiles, tiles also break picture-in-prediction dependencies in addition to entropy decoding dependencies. However, tiles may not be contained within individual NAL units. Thus, tiles may not be used for MTU size matching. Each tile may be processed by one processor / core. Inter-processor / core communication used for picture-in-prediction between processing units decoding adjacent tiles may be restricted to carrying a shared slice header when the slice spans more than one tile. Inter-processor / core communication may also be used for sharing related to loop filtering of reconstructed samples and metadata. When more than one tile or WPP segment is included in a slice, the entry point byte offset for each tile or WPP other than the first one in the slice may be signaled in the slice header. Restrictions regarding the application of four different picture partitioning schemes may be used to support simplicity. For example, a coded video sequence (CVS) may not include both tiles and waves for most of the profiles defined in HEVC. For each slice and tile, one or both of the following conditions may be satisfied. All coded tree blocks within a slice may belong to the same tile, and all coded tree blocks within a tile may belong to the same slice. Further, a wave segment may contain exactly one CTB row. Also, when WPP is in use, slices starting within a CTB row should end within the same CTB row.

[0035] VVC may include a tile and tile group picture partitioning scheme. Tiles in VVC may be the same as those in HEVC. VVC may use tile groups instead of slices. A slice is defined as including a group of CTUs, while a tile group is defined as including a group of tiles. A coded picture may be composed of one or more slices (or tile groups). Each slice / tile group has a slice header that contains syntax elements representing information used to decode the slice. Each slice header may contain information for decoding only that slice. However, the information from a slice header may be the same for other slices within the same picture. This is because coding tools may operate at the picture level and the parameters for all slices within a picture may be the same. This situation may cause redundant information in the slice header.

[0036] A header parameter set (HPS), also known as APS, may be used to solve the problem of redundant slice header information. The HPS may include slice-level information shared by multiple slices. The HPS may be generated by the encoder and may include coding tool parameters used when decoding corresponding slices in the decoder. Some systems implement the HPS scheme by using the HPS and a reference HPS. In this scheme, the first HPS in the coding order includes all of the relevant coding tool parameters for the corresponding slice. When such parameters change for subsequent slices, subsequent HPSs include only the changed parameters. The subsequent HPS then refers to the first HPS. Thus, the first HPS serves as a reference HPS for subsequent HPSs. Then, further HPSs may be used to refer to the previous HPS, etc.

[0037] Such HPS references involve several problems. For example, enabling an HPS to reference other HPSs gives rise to complex mechanisms. As a specific example, this mechanism can lead to a series of HPS references. In some cases, this approach may cause a long chain of HPSs even without an explicit limit on the number of HPS references used in a bitstream containing video data. To manage such a scheme, the decoder may need to maintain an arbitrary number of HPSs in the decoding picture buffer to prepare for possible subsequent references. To address this problem, an HPS reset mechanism may be added to break any such extended HPS reference chain, which adds further complexity. Furthermore, this approach is potentially error-prone. For example, if a previous HPS is lost due to a transmission error, subsequent reference HPSs will not contain sufficient data to code the corresponding slice. Also, this approach can lead to a large number of HPSs in the bitstream. However, the number of HPS identifiers (IDs) may be limited to avoid coding large HPS ID values. As such, HPS IDs may be reused within the bitstream. This can cause ambiguity, for example, when an HPS references an HPS ID that is used by more than one reference HPS. Additionally, HPSs may be signaled within the coded bitstream, which is called in-band signaling. HPSs may also be signaled by an external mechanism, for example, in metadata information. Such signaling is called out-of-band signaling. Such a dual signaling mechanism further increases the complexity of the HPS scheme.

[0038] Here, various mechanisms for reducing the complexity of HPS signaling are disclosed. HPS is what is referred to as HPS in the latest standard document. Thus, the following disclosure generally refers to HPS for the sake of clarity of discussion. However, the terms HPS and APS can be used synonymously in most aspects. The present disclosure removes the complexity and error-prone nature of HPS by enabling HPS not to refer to other HPSs. The fact that HPS does not need to refer to other HPSs means that the loss of a single HPS causes only local errors. Further, the decoder may not be required to hold the HPS in memory in that later HPSs replace previous HPSs. As a specific example, multiple types of HPSs may be used, where an HPS type indicates the type of coding tool parameters included in the HPS. Such HPS types may include an Adaptive Loop Filter (ALF) HPS, a Luma Mapping with Chroma Scaling (LMCS) HPS, and / or a Scaling List Parameter HPS. In such an example, when a current HPS of a first type is obtained by the decoder, the previous HPS of the first type may be discarded in that the current HPS replaces such a previous HPS. Further, to enable multiple types of HPSs, a single slice header may refer to more than one HPS so as to reference all coding tool parameters for the corresponding slice. This is in contrast to other schemes that enable a slice header to reference a single HPS that in that case references other HPSs. Thus, enabling a single slice header to reference multiple HPSs results in an implementation that avoids HPS reference chains. Also, the present disclosure describes a mechanism that enables HPS to operate with time scaling. With time scaling, the bitstream is configured to enable the decoder and / or the user to select from multiple frame rates. To implement such a scheme, pictures / frames are each assigned a temporal ID. Frames with lower temporal IDs are displayed at each frame rate.Frames having a higher temporal ID are skipped for lower frame rates and are only displayed for higher frame rates. To support such temporal scaling, a HPS is assigned a temporal ID. The HPS may receive the temporal ID of a picture that includes the first slice that references that HPS. In other examples, the HPS may receive the temporal ID of an access unit that includes that HPS. An access unit is a bitstream data grouping that contains sufficient video data to decode the corresponding picture. To further support temporal scaling, slices associated with a lower temporal ID may be restricted from referencing a HPS that includes a higher temporal ID. This ensures that a lower frame rate setting does not cause a slice to reference a HPS that is ignored by temporal scaling, and thus prevents coding tool parameters from being unavailable when decoding a particular slice at a lower frame rate.

[0039] FIG. 1 is a flowchart of an example operational method 100 for coding a video signal. Specifically, the video signal is encoded by an encoder. The encoding process compresses the video signal by using various mechanisms to reduce the video file size. The smaller file size enables the compressed video file to be transmitted to the user while reducing the associated bandwidth overhead. The decoder then decodes the compressed video file to reconstruct the original video signal for display to the end user. The decoding process generally reflects the encoding process such that the decoder can consistently reconstruct the video signal.

[0040] In step 101, a video signal is input into an encoder. For example, the video signal may be an uncompressed video file stored in a memory. As another example, the video file may be captured by a video capture device such as a video camera and encoded to support live streaming of the video. The video file may include both an audio component and a video component. The video component includes a series of image frames that, when viewed continuously, give a visual impression of movement. The frames include pixels that are represented with respect to light, here called the luma component (or luma samples), and color, here called the chroma component (or color samples). In some examples, the frames may also include depth values to support 3D displays.

[0041] In step 103, the video is partitioned into blocks. Partitioning includes subdividing the pixels within each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC), known as H.265 and MPEG-H Part 2, a frame may first be divided into Coding Tree Units (CTUs) that are blocks of a predefined size (e.g., 64×64 pixels). A CTU includes both luma and chroma samples. Coding trees may be used to divide the CTUs into blocks and then recursively subdivide the blocks until a configuration is achieved that supports further encoding. For example, the luma component of a frame may be subdivided until the individual blocks contain relatively uniform brightness values. Further, the chroma component of a frame may be subdivided until the individual blocks contain relatively uniform color values. Thus, the partitioning mechanism varies according to the content of the video frame.

[0042] In step 105, various compression mechanisms are used to compress the image blocks partitioned in step 103. For example, inter prediction and / or intra prediction may be used. Inter prediction is designed to utilize the fact that objects within a common scene tend to appear in consecutive frames. Therefore, blocks representing objects in the reference frame do not need to be repeatedly described in adjacent frames. Specifically, an object such as a table may remain in a fixed position over multiple frames. Thus, the table is described once, and adjacent frames can refer back to the reference frame. A pattern matching mechanism may be used to match objects over multiple frames. Further, a moving object may be represented over multiple frames, for example, due to the movement of the object or the movement of the camera. As a specific example, a video may show a car moving across the screen over multiple frames. Motion vectors can be used to describe such motion. A motion vector is a two-dimensional vector that provides an offset from the coordinates of an object in one frame to the coordinates of the object in the reference frame. As such, inter prediction can encode an image block in the current frame as a set of motion vectors indicating the offsets from the corresponding blocks in the reference frame.

[0043] Intra prediction encodes blocks within a common frame. Intra prediction exploits the fact that luma and chroma components tend to group in the frame. For example, a green patch in a part of a tree tends to be located adjacent to similar green patches. Intra prediction uses multiple directional prediction modes (e.g., 33 in HEVC), a planar mode, and a direct current (DC) mode. The directional modes indicate that the current block is similar to / the same as the samples of adjacent blocks in the corresponding direction. The planar mode indicates that a series of blocks along a row / column (e.g., a plane) can be interpolated based on adjacent blocks at the end of the row. The planar mode effectively indicates a smooth transition of light / color across the row / column by using a relatively constant slope of changing values. The DC mode is used for boundary smoothing and indicates that the block is similar to / the same as the average value related to the samples of all adjacent blocks related to the angular direction of the directional prediction mode. Thus, an intra prediction block can represent an image block as various related prediction mode values instead of the actual values. Further, an inter prediction block can represent an image block as a motion vector instead of the actual values. In either case, the prediction block may not accurately represent the image block in some cases. Any difference is saved in a residual block. Transformation may be applied to the residual block to further compress the file.

[0044] In step 107, various filtering techniques may be applied. In HEVC, the filters are applied according to an in-loop filtering scheme. The block-based prediction described above may cause the generation of a block-noisy image at the decoder. Further, the block-based prediction scheme may encode the blocks and then reconstruct the encoded image for later use as a reference block. The in-loop filtering scheme repeatedly applies a noise suppression filter, a deblocking filter, an adaptive loop filter, and a sample adaptive offset (SAO) filter to the blocks / frames. These filters reduce such blocking artifacts so that the encoded file can be accurately reconstructed. Further, these filters reduce the artifacts in the reconstructed reference blocks so that the artifacts are less likely to cause further artifacts in subsequent blocks encoded based on the reconstructed reference blocks.

[0045] When the video signal is partitioned, compressed, and filtered, the resulting data is encoded in a bitstream in step 109. The bitstream includes the above data in addition to any signaling data desirable to support proper video signal reconstruction at the decoder. For example, such data may include partition data, prediction data, residual blocks, and various flags that give coding instructions to the decoder. The bitstream may be stored in memory for transmission to the decoder upon request. The bitstream may also be a broadcast and / or multicast to multiple decoders. The generation of the bitstream is an iterative process. Accordingly, steps 101, 103, 105, 107, and 109 may be performed continuously and / or simultaneously over a number of frames and blocks. The order shown in FIG. 1 is presented for clarity and ease of discussion and is not intended to limit the video coding process to a particular order.

[0046] The decoder receives a bitstream and starts the decoding process at step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into the corresponding syntax and video data. The decoder uses the syntax data from the bitstream to determine the partitioning of the frame at step 111. The partitioning should match the result of the block partitioning at step 103. The entropy encoding / decoding used at step 111 is described hereinafter. The encoder makes many choices during the compression process, such as selecting a block partitioning scheme from several possible choices based on the spatial positioning of the values within the input image. Notifying the exact choice may require using a large number of bins. As used herein, a bin is a binary value treated as a variable (e.g., a bit value that can vary depending on the situation). Entropy encoding enables the encoder to discard any choices that are clearly not executable for a particular case while leaving a set of acceptable choices. Each acceptable choice is then assigned a codeword. The length of the codeword depends on the number of acceptable choices (e.g., 1 bin for 2 choices, 2 bins for 3 to 4 choices, etc.). The encoder then encodes the codeword for the selected choice. This scheme reduces the size of the codeword because the codeword is of the desired size to uniquely indicate a choice from a small subset of acceptable choices, as opposed to uniquely indicating a choice from a potentially large set of all possible choices. The decoder then decodes the choice by determining the set of acceptable choices in the same way as the encoder. By determining the set of acceptable choices, the decoder can read the codeword and determine the choice made by the encoder.

[0047] In step 113, the decoder performs block decoding. Specifically, the decoder uses inverse transformation to generate residual blocks. Then, the decoder uses the residual blocks and the corresponding prediction blocks to reconstruct the image blocks according to the partitioning. The prediction blocks may include both intra-prediction blocks and inter-prediction blocks generated by the encoder in step 105. The reconstructed image blocks are then positioned within the frame of the reconstructed video signal according to the partitioning data determined in step 111. The syntax for step 113 may also be signaled in the bitstream via entropy coding as described above.

[0048] In step 115, filtering is performed on the frame of the reconstructed video signal in a manner similar to step 107 in the encoder. For example, a noise reduction filter, a deblocking filter, an adaptive loop filter, and an SAO filter may be applied to the frame to remove blocking artifacts. Once the frame is filtered, the video signal can be output to a display in step 117 for viewing by an end user.

[0049] Figure 2 is a schematic diagram of an exemplary coding and decoding (codec) system 200 for video coding. Specifically, the codec system 200 provides functionality to support the implementation of the operation method 100. The codec system 200 is generalized to represent components used in both the encoder and the decoder. The codec system 200 receives and partitions a video signal as described with respect to steps 101 and 103 of the operation method 100, thereby obtaining a partitioned video signal 201. The codec system 200 then compresses the partitioned video signal 201 into a coded bitstream when operating as an encoder as described with respect to steps 105, 107, and 109 of method 100. When operating as a decoder, the codec system 200 generates an output video signal from the bitstream as described with respect to steps 111, 113, 115, and 117 of the operation method 100. The codec system 200 includes a general-purpose codec control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, an intra-picture prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control analysis component 227, an in-loop filter component 225, a decoding picture buffer component 223, and a header formatting and Context Adaptive Binary Arithmetic Coding (CABAC) component 231. Such components are coupled as shown. In FIG. 2, the black lines indicate the movement of data to be encoded / decoded, and the white dashed lines indicate the movement of control data that controls the operation of other components. All components of the codec system 200 may be present in the encoder. The decoder may include a subset of the components of the codec system 200.For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded picture buffer component 223. These components are described hereinafter.

[0050] The partitioned video signal 201 is a captured video sequence that is partitioned into blocks of pixels by a coding tree. The coding tree uses various splitting modes to subdivide blocks of pixels into smaller blocks of pixels. These blocks can then be further subdivided into even smaller blocks. The blocks can be referred to as nodes on the coding tree. Larger parent nodes are split into smaller child nodes. The number of times a node is subdivided is called the depth of the node / coding tree. The divided blocks can be included in coding units (CUs) in some cases. For example, a CU can be a sub-part of a CTU that includes a luminance block, a red difference chroma (Cr) block, and a blue difference chroma (Cb) block, along with the corresponding syntax instructions for the CU. The splitting modes can include a binary tree (BT), a ternary tree (TT), and a quaternary tree (QT) that are used to partition a node into two, three, or four child nodes, respectively, whose shapes change according to the splitting mode being used. The partitioned video signal 201 is transferred for compression to a general-purpose coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, a filter control analysis component 227, and a motion estimation component 221.

[0051] The general coder control component 211 is configured to make decisions regarding the coding of the images of the video sequence into the bitstream according to application constraints. For example, the general coder control component 211 manages the optimization of the bitrate / bitstream size with respect to the reconstructed quality. Such decisions may be made based on the availability of memory space / bandwidth and the requirements of the image resolution. The general coder control component 211 also manages the buffer utilization in view of the transmission speed to mitigate buffer underrun and overrun problems. To manage these problems, the general coder control component 211 manages the partitioning, prediction, and filtering by other components. For example, the general coder control component 211 may dynamically increase the compression complexity to increase the resolution and increase the bandwidth utilization, or reduce the compression complexity to reduce the resolution and bandwidth utilization. Thus, the general coder control component 211 controls the other components of the codec system 200 to balance the video signal reconstructed quality and the bitrate concerns. The general coder control component 211 generates control data that controls the operation of the other components. The control data is also transferred to the header formatting and CABAC component 231 so as to be encoded in the bitstream to notify the parameters for decoding at the decoder.

[0052] The partitioned video signal 201 is also sent to the motion estimation component 221 and the motion compensation component 219 for inter prediction. The frame or slice of the partitioned video signal 201 may be divided into a plurality of video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter prediction coding of the received video block with respect to one or more blocks in one or more reference frames to provide temporal prediction. The codec system 200 may execute a number of coding paths, for example, to select an appropriate coding mode for each block of the video data.

[0053] The motion estimation component 221 and the motion compensation component 219 may be highly integrated, but are shown separately for conceptual purposes. The motion estimation performed by the motion estimation component 221 is a process of generating motion vectors, thereby estimating the motion of video blocks. The motion vector may indicate, for example, the displacement of a coded object relative to a prediction block. The prediction block is a block that is recognized as exactly matching the block to be coded with respect to pixel differences. The prediction block may also be referred to as a reference block. Such pixel differences may be determined by the sum of absolute differences (SAD), the sum of square differences (SSD), or other difference metrics. HEVC uses several coded objects including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU may be divided into CTBs, and a CTB may then be divided into CUs for inclusion in CUs. A CU may be encoded as a prediction unit (PU) containing prediction data for the CU and / or a transform unit (TU) containing transformed residual data. The motion estimation component 221 generates motion vectors, PUs, and TUs by using rate distortion analysis as part of the rate distortion optimization process. For example, the motion estimation component 221 may determine a number of reference blocks, a number of motion vectors, etc. for the current block / frame, and may select the reference block, motion vector, etc. having the best rate distortion characteristics. The best rate distortion characteristics balance both the quality of video reconstruction (e.g., the amount of data loss due to compression) and the coding efficiency (e.g., the size of the final encoding).

[0054] In some examples, the codec system 200 may calculate values for sub-integer pixel positions of reference pictures stored in the decoding picture buffer component 223. For example, the video codec system 200 may interpolate values for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference picture. Thus, the motion estimation component 221 may perform motion search for both full pixel positions and fractional pixel positions and output motion vectors with fractional pixel accuracy. The motion estimation component 221 calculates motion vectors for PUs of video blocks in an inter-coded slice by comparing the position of the PU with the position of the predicted block of the reference picture. The motion estimation component 221 outputs the calculated motion vectors as header formatting for encoding and motion data to the CABAC component 231 and motion data to the motion compensation component 219.

[0055] Motion compensation performed by the motion compensation component 219 may include fetching or generating a predicted block based on the motion vector determined by the motion estimation component 221. As before, the motion estimation component 221 and the motion compensation component 219 may be functionally integrated in some examples. Upon receiving the motion vector for the current video block's PU, the motion compensation component 219 may find the position of the predicted block indicated by the motion vector. Then, the residual video block is formed by subtracting the pixel values of the predicted block from the pixel values of the current video block being coded to form pixel difference values. Generally, the motion estimation component 221 performs motion estimation on the luma component, and the motion compensation component 219 uses the motion vector calculated based on the luma component for both the chroma and luma components. The predicted block and the residual block are transferred to the transform scaling and quantization component 213.

[0056] The partitioned video signal 201 is also sent to the intra picture estimation component 215 and the intra picture prediction component 217. Similar to the motion estimation component 221 and the motion compensation component 219, the intra picture estimation component 215 and the intra picture prediction component 217 may be highly integrated, but are shown separately for conceptual purposes. The intra picture estimation component 215 and the intra picture prediction component 217 perform intra prediction of the current block for blocks within the current frame as an alternative to the inter prediction performed by the motion estimation component 221 and the motion compensation component 219 between frames as described above. In particular, the intra picture estimation component 215 determines the intra prediction mode to be used to encode the current block. In some examples, the intra picture estimation component 215 selects an appropriate intra prediction mode for encoding the current block from a plurality of tested intra prediction modes. The selected intra prediction mode is then transferred to the header formatting and CABAC component 231 for encoding.

[0057] For example, the intra picture estimation component 215 calculates rate distortion values using rate distortion analysis for various tested intra prediction modes and selects an intra prediction mode having the best rate distortion characteristics among the tested modes. Rate distortion analysis generally determines, in addition to the bit rate (e.g., the number of bits) used to generate the encoded block, the amount of distortion (or error) between the encoded block and the original unencoded block that was encoded to generate the encoded block. The intra picture estimation component 215 calculates a ratio from the distortion and rate for various encoded blocks to determine which intra prediction mode exhibits the best rate distortion value for that block. Additionally, the intra picture estimation component 215 may be configured to code depth blocks of a depth map using a Depth Modeling Mode (DMM) based on rate distortion optimization (RDO).

[0058] When implemented in an encoder, the intra picture prediction component 217 may generate a residual block from a prediction block based on the selected intra prediction mode determined by the intra picture estimation component 215, or when implemented in a decoder, may read a residual block from the bitstream. The residual block includes the difference in values between the prediction block and the original block, represented as a matrix. The residual block is then transferred to the transform scaling and quantization component 213. The intra picture estimation component 215 and the intra picture prediction component 217 may act on both luma and chroma components.

[0059] The transform scaling and quantization component 213 is configured to further compress the residual block. The transform scaling and quantization component 213 applies a transform such as a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), or a conceptually similar transform to the residual block to generate a video block having residual transform coefficient values. A wavelet transform, an integer transform, a subband transform, or other types of transforms may also be used. The transform may transform the residual information from the pixel value domain to a transform domain such as the frequency domain. The transform scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling includes applying a scale factor to the residual information such that different frequency information is quantized at different granularities, which can affect the final visual quality of the reconstructed video. The transform scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bitrate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be changed by adjusting the quantization parameter. In some examples, the transform scaling and quantization component 213 may then perform a scan of the matrix containing the quantized transform coefficients. The quantized transform coefficients are transferred to the header formatting and CABAC component 231 to be encoded in the bitstream.

[0060] The scaling and inverse transform component 229 applies the inverse operation of the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies inverse scaling, transform, and / or quantization to reconstruct the residual block in the pixel domain, for example, for later use as a reference block that can be a prediction block for other current blocks. The motion estimation component 221 and / or the motion compensation component 219 may compute the reference block by adding the residual block back to the corresponding prediction block for use in motion estimation of later blocks / frames. A filter is applied to the reconstructed reference block to reduce artifacts that occur during scaling, quantization, and transformation. Such artifacts could otherwise cause incorrect predictions (and cause further artifacts) when subsequent blocks are predicted.

[0061] The filter control analysis component 227 and the in-loop filter component 225 apply a filter to the residual block and / or the reconstructed image block. For example, the transformed residual block from the scaling and inverse transform component 229 may be combined with the corresponding prediction block from the intra-picture prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter may then be applied to the reconstructed image block. In some examples, the filter may instead be applied to the residual block. Similar to the other components in FIG. 2, the filter control analysis component 227 and the in-loop filter component 225 are highly integrated and may be implemented together, but are shown separately for conceptual purposes. The filter applied to the reconstructed reference block is applied to a specific spatial region and includes a plurality of parameters for adjusting how such a filter is applied. The filter control analysis component 227 analyzes the reconstructed reference block and sets the corresponding parameters to determine where such a filter should be applied. Such data is transferred as filter control data to the header formatting and CABAC component 231 for encoding. The in-loop filter component 225 applies such a filter based on the filter control data. The filter may include a deblocking filter, a noise suppression filter, a SAO filter, and an adaptive loop filter. Such a filter may be applied, for example, in the spatial / pixel domain (e.g., to the reconstructed pixel block) or in the frequency domain depending on the example.

[0062] When operating as an encoder, the filtered and reconstructed image blocks, residual blocks, and / or prediction blocks are stored in the decoding picture buffer component 223 for later use in motion estimation as described above. When operating as a decoder, the decoding picture buffer component 223 stores the reconstructed and filtered blocks and transfers them towards the display as part of the output video signal. The decoding picture buffer component 223 may be any memory device capable of storing prediction blocks, residual blocks, and / or reconstructed image blocks.

[0063] The header formatting and CABAC component 231 receives data from various components of the codec system 200 and encodes such data into a coded bitstream for transmission to a decoder. Specifically, the header formatting and CABAC component 231 generates various headers for encoding control data such as general control data and filter control data. Further, in addition to residual data in the form of quantized transform coefficient data, prediction data including intra prediction and motion data are all encoded in the bitstream. The final bitstream contains all the information desired by the decoder to reconstruct the original partitioned video signal 201. Such information may also include an intra prediction mode index table (also called a codeword mapping table), the definition of encoding contexts for various blocks, an indication of the most likely intra prediction mode, an indication of partition information, etc. Such data may be encoded by using entropy coding. For example, the information may be encoded by using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. Following entropy coding, the coded bitstream may be transmitted to other devices (e.g., a video decoder) or may be archived for later transmission or reading.

[0064] FIG. 3 is a block diagram representing an exemplary video encoder 300. The video encoder 300 may be used to implement the encoding function of the codec system 200 and / or to perform steps 101, 103, 105, 107, and / or 109 of the operation method 100. The encoder 300 partitions the input video signal to yield a partitioned video signal 301 that is substantially similar to the partitioned video signal 201. The partitioned video signal 301 is then compressed and encoded into a bitstream by components of the encoder 300.

[0065] Specifically, the partitioned video signal 301 is transferred to an intra-picture prediction component 317 for intra prediction. The intra-picture prediction component 317 may be substantially similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. The partitioned video signal 301 is also transferred to a motion compensation component 321 for inter prediction based on reference blocks in a decoding picture buffer component 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219. The prediction blocks and residual blocks from the intra-picture prediction component 317 and the motion compensation component 321 are transferred to a transform and quantization component 313 for transformation and quantization of the residual blocks. The transform and quantization component 313 may be substantially similar to the transform scaling and quantization component 213. The transformed and quantized residual blocks and the corresponding prediction blocks (along with associated control data) are transferred to an entropy encoding component 331 for coding into a bitstream. The entropy encoding component 331 may be substantially similar to the header formatting and CABAC component 231.

[0066] The transformed and quantized residual blocks and / or corresponding prediction blocks are transferred from the transform and quantization component 313 to the inverse transform and quantization component 329 and also to the reference blocks used by the motion compensation component 321 for reconstruction. The inverse transform and quantization component 329 may be substantially similar to the scaling and inverse transform component 229. The in-loop filter in the in-loop filter component 325 is also applied to the residual blocks and / or the reconstructed reference blocks, according to an example. The in-loop filter component 325 may be substantially similar to the filter control analysis component 227 and the in-loop filter component 225. The in-loop filter component 325 may include multiple filters as described for the in-loop filter component 225. The filtered blocks are then stored in the decoding picture buffer component 323 for use as reference blocks by the motion compensation component 321. The decoding picture buffer component 323 may be substantially similar to the decoding picture buffer component 223.

[0067] Figure 4 is a block diagram representing an example video decoder 400. The video decoder 400 may be used to implement the decoding functions of the codec system 200 and / or to implement steps 111, 113, 115, and / or 117 of the operation method 100. The decoder 400 receives, for example, a bitstream from the encoder 300 and generates a reconstructed output video signal for display to an end user based on the bitstream.

[0068] The bitstream is received by an entropy decoding component 433. The entropy decoding component 433 is configured to implement an entropy decoding scheme such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 may use the header information to supply a context for interpreting additional data encoded as codewords in the bitstream. The decoded information includes any desirable information for decoding a video signal such as general control data, filter control data, partition information, motion data, prediction data, and quantized transform coefficients from the residual blocks. The quantized transform coefficients are transferred to the inverse transform and quantization component 429 for reconstruction into the residual blocks. The inverse transform and quantization component 429 may be similar to the inverse transform and quantization component 329.

[0069] The reconfigured residual block and / or prediction block is transferred to the intra-picture prediction component 417 for the reconstruction of the picture block based on the intra prediction operation. The intra-picture prediction component 417 may be similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. Specifically, the intra-picture prediction component 417 uses a prediction mode to find the position of the reference block within the frame, applies the residual block to the result, and reconstructs the intra-predicted picture block. The reconstructed intra-predicted picture block and / or residual block and the corresponding inter prediction data are transferred to the decoding picture buffer component 423 via the in-loop filter component 425. These may be substantially similar to the decoding picture buffer component 223 and the in-loop filter component 225, respectively. The in-loop filter component 425 filters the reconstructed picture block, residual block and / or prediction block, and such information is stored in the decoding picture buffer component 423. The reconstructed picture block from the decoding picture buffer component 423 is transferred to the motion compensation component 421 for inter prediction. The motion compensation component 421 may be substantially similar to the motion estimation component 221 and / or the motion compensation component 219. Specifically, the motion compensation component 421 uses the motion vector from the reference block to generate a prediction block, applies the residual block to the result, and reconstructs the picture block. The resulting reconstructed block may also be transferred to the decoding picture buffer component 423 via the in-loop filter component 425. The decoding picture buffer component 423 continues to store further reconstructed picture blocks that may be reconstructed within the frame according to the partition information. Such frames may also be arranged in a sequence. The sequence is output to the display as the reconstructed output video signal.

[0070] FIG. 5 is a schematic diagram showing an exemplary bitstream 500 including an encoded video sequence along with HPS513. For example, bitstream 500 may be generated by codec system 200 and / or encoder 300 for decoding by codec system 200 and / or decoder 400. As another example, bitstream 500 may be generated by the encoder at step 109 of method 100 for use by the decoder at step 111.

[0071] Bitstream 500 includes a sequence parameter set (SPS) 510, a plurality of picture parameter sets (PPS) 512, a plurality of HPS513, a plurality of slice headers 514, and picture data 520. SPS510 includes sequence data common to all pictures within the video sequence included in bitstream 500. Such data can include picture sizing, bit depth, coding tool parameters, bitrate limits, etc. PPS512 includes parameters applicable to an entire picture. Thus, each picture within the video sequence may refer to PPS512. It should be noted that while each picture refers to PPS512, in some examples a single PPS512 can include data for a plurality of pictures. For example, a plurality of similar pictures may be coded according to similar parameters. In such a case, a single PPS512 may include data for such similar pictures. PPS512 can indicate coding tools, quantization parameters, offsets, etc. available for slices within the corresponding picture. Slice header 514 includes parameters specific to each slice within a picture. Thus, there may be one slice header 514 for each slice within the video sequence. Slice header 514 may include slice type information, picture order count (POC), reference picture list, prediction weights, tile entry points, deblocking parameters, etc.

[0072] HPS513 is a syntax structure that contains syntax elements applicable to zero or more slices determined by zero or more syntax elements found in the slice header. Thus, HPS513 contains syntax elements for the parameters of coding tools related to multiple slices. HPS513 may also be called HPS in some systems. For example, one or more slices may refer to HPS513. Thus, the decoder can obtain HPS513 based on such a reference, obtain coding tool parameters from HPS513, and use those coding tool parameters to decode the corresponding slice. HPS513 conceptually occupies a hierarchical position between PPS512 and slice header 514. For example, certain data may be related to multiple slices rather than the entire picture. Such data may not need to be stored in PPS512 because the data is not related to the entire picture. However, such data would otherwise be included in multiple slice headers 514. HPS513 can accept such data to avoid redundant signaling across multiple slice headers 514. The coding structure of HPS513 has been introduced in VVC and there is no similar structure in HEVC or previous coding standards. Various implementations of HPS513 are described below.

[0073] The picture data 520 includes video data encoded according to inter prediction and / or intra prediction, together with corresponding transformed and quantized residual data. For example, a video sequence includes a plurality of pictures 521 coded as picture data. A picture 521 is a single frame of the video sequence and thus is generally displayed as a single unit when the video sequence is displayed. However, partial pictures may be displayed to implement specific techniques such as virtual reality, picture-in-picture, etc. Each picture 521 refers to the PPS 512. Each picture 521 is divided into slices 523. A slice 523 may be defined as a horizontal cross-section of the picture 521. For example, a slice 523 may include a part of the height of the picture 521 and the entire width of the picture 521. In some systems, a slice 523 is further subdivided into tiles 525. In other systems, a slice 523 is replaced by a tile group that includes the tiles 525. The group of slices 523 and / or tiles 525 refers to the slice header 514 and / or the HPS 513. A tile 525 may include a rectangular portion of the picture 521 and / or a portion of the picture 521 defined by columns and rows. A tile 525 is further divided into coding tree units (CTUs). A CTU is further divided into coding blocks based on a coding tree. A coding block may then be encoded / decoded according to a prediction mechanism.

[0074] The bitstream 500 is coded into VCL NAL units 533 and non-VCL NAL units 531. A NAL unit is a coded data unit sized to be placed as the payload for a single packet for transmission over a network. A VCL NAL unit 533 is a NAL unit that contains coded video data. For example, each VCL NAL unit 533 may include a tile group, CTU, and / or coding block of data that includes one slice 523 and / or the corresponding tile 525. A non-VCL NAL unit 531 is a NAL unit that includes supporting syntax but does not include coded video data. For example, the non-VCL NAL unit 531 may include SPS 510, PPS 512, HPS 513, slice header 514, etc. As such, a decoder receives the bitstream 500 in separate VCL NAL units 533 and non-VCL NAL units 531. An access unit 535 is a group of VCL NAL units 533 and / or non-VCL NAL units 531 that contains sufficient data to code a single picture 521.

[0075] In some examples, HPS513 may be implemented as follows. HPS513 may be available in-band and / or out-of-band, where in-band signaling is included in bitstream 500 and out-of-band is included in supporting metadata. HPS513 may be included in a NAL unit such as non-VCL NAL unit 531, where HPS513 is identified by the NAL unit type. HPS513 may include parameters for coding tools such as, but not limited to, ALF, SAO, deblocking, quantization matrix, inter prediction parameters, parameters related to reference picture set configuration, and / or parameters related to reference picture list configuration. HPS513 may include a type. The type defines which coding tool parameters are included in HPS513. Each HPS513 may include only one type of coding tool parameters. Different types of HPS513 may be grouped together in a group of parameter sets (GPS). Instead of referring to a single HPS513, slice 523 may refer to a GPS. HPS513 may be made available in the decoder before being referred to by the corresponding slice 523 and / or tile group. Different slices 523 of coded picture 521 may refer to different HPS513. HPS513 may be placed at any slice 523 boundary in bitstream 500. This enables the reuse of parameters (e.g., ALF parameters) in HPS513 even for all slices 523 of the current picture 521 following HPS513.

[0076] The reference from the slice header 514 to the HPS 513 may be optional. For example, the slice 523 may refer to the HPS 513 if any of the following is true. First, such a reference may be indicated in the corresponding PPS 512 that the HPS 513 is available for the bitstream 500. Second, the slice 523 may refer to the HPS 513 if at least one of the coding tools whose parameters are included in the HPS 513 is enabled for the bitstream 500. Each HPS 513 may be associated with an HPS ID. The slice 523 that refers to the HPS 513 should include the HPS ID of the referenced HPS 513. The HPS ID may be coded by a syntax element coded in zero-th order Exp-Golomb of an unsigned integer, starting from the leftmost bit first (e.g., ue(v)). The value of the HPS ID may be restricted, for example, in the range from 0 to 63. For each parameter of the coding tool, a flag may exist in the HPS 513 to indicate whether the parameter exists in the HPS 513. When the coding tool is enabled for the slice 523 and the parameter for the coding tool exists in the HPS 513 referenced by the slice 523, the parameter may not be signaled in the corresponding slice header 514. The HPS 513 may be fragmented into one or more NAL units, and each fragment of the HPS 513 may be parsed and applied independently. The slice 523 may refer to a single HPS 513 or multiple HPS 513s. When referring to multiple HPS 513s is allowed, each reference to the HPS 513 may be used to determine the parameters of a specific coding tool.

[0077] The following implementation allows HPS513 to reference other HPS513s and inherit coding tool parameters through such references. HPS513 may include one or more references to other HPS513s. In such an example, HPS513 may be called an intra-HPS when it does not reference any other HPS513. When HPS513 references another HPS513, the referencing HPS513 may copy one or more parameters from the referenced HPS513. HPS513 may have a single referenced HPS513 per parameter group and may have multiple references to other HPS513s. In some examples, a linked list of HPS513s may be formed in that a series of HPS513s are connected by a reference mechanism. When it is determined that a parameter of the coding tool does not exist in HPS513 (e.g., the value of the presence flag is equal to 0), an additional flag may exist to indicate whether the parameter can be inherited from the referenced HPS513. A reference from HPS513 to another HPS513 may be implicitly specified such that when a parameter of the coding tool does not exist in HPS513, such a parameter is inherited to be the same as the parameter of the coding tool for the previous HPS513.

[0078] In some cases, the HPS513 may no longer exist when calling for random access to Instantaneous Decoder Refresh (IDR) and / or Clean Random Access (CRA) pictures. Therefore, two HPS buffers may be used to store the HPS513, and each buffer is alternatively activated at the start of each Intra Random Access Point (IRAP) picture. In such cases, the received HPS513 is stored in the active HPS buffer. To improve error resilience, the range of HPS IDs may be defined to indicate the HPS ID currently in use. If the HPS513 has an HPS ID outside the currently used HPS ID range, a further HPS513 may use that HPS ID. The HPS ID in use may then be updated according to a sliding window approach. When this technique is used, the HPS513 may only refer to other HPS513s whose HPS IDs are within the range currently in use. A limit on the number of active HPS513s may be defined to limit the storage requirements of HPS513s in the decoder memory. When the limit is reached and a new HPS513 is received, the oldest (e.g., the earliest received) HPS513 in the buffer may be deleted and the new HPS513 inserted.

[0079] In some cases, all references to HPS513 may be resolved as soon as the HPS513 is received by the decoder. For example, the coding tool parameters may be copied as soon as the decoder receives the HPS513, and the coding tool parameters are not available with the current HPS513 but are available with the reference HPS513. Another approach to improving error resilience may require the intra HPS513 to exist for a certain period of time. When the intra HPS513 is received, all available HPS513s of the same type in the buffer may be discarded. When the parameters of the coding tool do not exist in the HPS513, a flag may exist to indicate whether the coding tool for slice 523 that references the HPS513 is disabled. Further, a flag in the NAL unit header (e.g., nal_ref_flag) may determine whether the HPS513 included in the NAL unit can be referenced by slice 523 from picture 521 used as a reference. For example, the HPS513 may only be referenced by slice 523 of a non-reference picture 521 when the nal_ref_flag of the NAL unit containing the HPS513 is equal to 0. This flag may also be used to determine which HPS513s can be used as references for other HPS513s. For example, the HPS513 may not need to reference other HPS513s included in a NAL unit having a nal_ref_flag equal to 0. The reset period for the HPS buffer may be defined in the SPS510. The buffer of the HPS513 may be reset upon the occurrence of an IRAP picture.

[0080] As can be understood from reviewing the above implementation, it can be quite complex to enable HPS513 to refer to other HPS513. Therefore, in the disclosed example, HPS513 can be restricted from referring to other HPS513. Instead, multiple types of HPS513 may be used. The type of HPS513 indicates the type of coding tool parameters included in HPS513. Such types of HPS513 may include ALF HPS, LMCS HPS, and / or Scaling List Parameter HPS. ALF HPS is an HPS that includes coding tool parameters used as part of the adaptive loop filtering for the corresponding slice. LMCS HPS includes coding tool parameters used for the LMCS mechanism. LMCS is a filtering technique that reformats the luma component based on the mapping for the corresponding chroma component to reduce rate distortion. Scaling List Parameter HPS includes coding tool parameters related to the quantization matrix used by a specific filter. In such an example, when the current HPS513 of the first type is obtained by the decoder, the previous HPS513 of the first type can be discarded in that the current HPS513 replaces such a previous HPS513. Further, to enable multiple types of HPS513, a single slice header 514 may refer to more than one HPS513 to reference all coding tool parameters for the corresponding slice 523 and / or tile group. This is in contrast to other schemes that enable the slice header 514 to refer to a single HPS513 that in that case refers to other HPS513. Therefore, enabling a single slice header 514 to refer to multiple HPS513 results in an implementation that avoids the reference chain of HPS513. This approach greatly reduces complexity and thus reduces processing resource utilization in both the encoder and decoder. Further, this process reduces the number of HPS513 buffered in the decoder. For example, only one HPS513 of each type need be buffered in the decoder.This reduces memory utilization in the decoder. Further, by avoiding the reference chain of HPS513, potential errors are found and thus reduced. This is because losing HPS513 during a transmission error can only affect slice 523 that directly references HPS513. As such, the disclosed mechanism produces improvements in both the encoder and decoder when using HPS513 in bitstream 500.

[0081] FIG. 6 is a schematic diagram illustrating an exemplary mechanism 600 for time scaling. For example, mechanism 600 may be used by a decoder such as codec system 200 and / or decoder 400 when displaying a decoded bitstream such as bitstream 500. Further, mechanism 600 may be used as part of step 117 of method 100 when outputting video for display. Also, an encoder such as codec system 200 and / or encoder 300 may encode data in the bitstream to enable mechanism 600 to occur at the decoder.

[0082] Mechanism 600 acts on a plurality of decoded pictures 601, 603, and 605. Pictures 601, 603, and 605 are parts of an ordered video sequence and are decoded, for example, from a bitstream by using the mechanism described above. The bitstream is encoded to enable a decoder to display the video sequence at one of a plurality of frame rates including a first frame rate (FR0) 610, a second frame rate (FR1) 611, and a third frame rate (FR2) 612. The frame rate is an indicator of the frequency at which frames / pictures of the video sequence are displayed. The frame rate can be measured in frames over time. The differences in frame rates enable different decoders to display the same video sequence at different qualities, taking into account variations in decoder capabilities. For example, a decoder with reduced hardware capabilities and / or a decoder streaming from a low-quality network connection may display at FR0 610. As another example, a high-quality decoder with access to a high-speed network connection may display at FR2 612. As yet another example, a decoder with certain functional impairments may be able to display at FR1 611 but may not be able to display at FR2 612. Therefore, temporal scaling (e.g., mechanism 600) is used to enable each decoder to display the video at the highest possible frame rate for the best user experience based on the varying decoder-side capabilities and constraints. In most systems, each frame rate is twice the frequency of the previous frame rate. For example, FR0 610, FR1 611, and FR2 612 may be set to 15 frames per second (FPS), 30 FPS, and 60 FPS, respectively.

[0083] To implement time scaling, pictures 601, 603, and 605 are coded into the bitstream by the encoder at the highest possible frame rate, in this case FR2 612. The encoder also assigns a Temporal Identifier (TID) to each picture 601, 603, and 605. Pictures 601, 603, and 605 have received TIDs of 0, 1, and 2 respectively. When displaying the resulting decoded video, the decoder selects a frame rate, determines the corresponding frame rate TID, and displays all frames having a TID that is less than or equal to that frame rate TID. Pictures having a TID greater than the frame rate TID of the selected frame rate are ignored. For example, a decoder selecting FR2 612 displays all pictures having a TID of 2 or less, and thus displays all pictures 601, 603, and 605. As another example, a decoder selecting FR1 611 displays all pictures having a TID of 1 or less, and thus displays pictures 601 and 603 while ignoring picture 605. As another example, a decoder selecting FR0 610 displays all pictures having a TID of 0 or less, and thus displays picture 601 while ignoring pictures 603 and 605. By using this mechanism 600, a video sequence can be temporally scaled by the decoder to the selected frame rate.

[0084] The HPS513 described in FIG. 5 can be implemented to support the time scaling of mechanism 600. This can be achieved by assigning a TID, such as TID0, TID1, or TID2, to each HPS. When time scaling is performed, HPSs having a TID less than or equal to the selected frame rate TID are decoded and HPSs having a TID greater than the selected frame rate TID are discarded. The TIDs can be assigned to the HPSs according to various embodiments.

[0085] Referring to FIG. 5, in one example, HPS513 may receive a temporal ID of picture 521 that includes a first slice 523 that refers to HPS513. In other examples, HPS513 may receive a temporal ID of access unit 535 that includes HPS513. To further support time scaling, slices 523 associated with lower temporal IDs may be restricted from referring to HPS513 that includes higher temporal IDs. This ensures that lower frame rate settings do not cause slices 523 to refer to HPS513 that is ignored by the time scaling of mechanism 600, and thus prevents coding tool parameters from being unavailable when decoding a particular slice 523 at a lower frame rate such as FR0 610 and / or FR1 611.

[0086] The above mechanism may be implemented as follows. The following aspects may be applied individually and / or in combination. An Adaptive Parameter Set (APS) is another name for an HPS. The availability of an HPS for a bitstream may all be available in-band, all be available out-of-band, and / or some may be available in-band and some may be available out-of-band. When provided out-of-band, the HPS may be present in the following. In an ISO-based media file format, the HPS may be present in a sample entry (e.g., a sample description box). In an ISO-based media file format, the HPS may also be present in a time-synchronized track such as a parameter set track or a timed metadata track.

[0087] In certain implementations, when provided outside the band, the HPS may operate as follows. In a media file format based on ISO, when there is no HPS update, the HPS may only exist in sample entries (e.g., sample description boxes). The HPS update may reuse the HPS identifier (ID), while other HPS parameters may not exist if they are different from the HPS sent previously with the same HPS ID. When there is an HPS update, for example, when the HPS includes adaptive loop filter (ALF) parameters, in a media file format based on the International Organization for Standardization (ISO), the HPS may be carried in a time-synchronized track such as a parameter set track or a timed metadata track. In this way, slices each containing a group of complete tiles may be carried in their own file format tracks. Further, the HPS may be carried in a time-synchronized track. As a result, each of these tracks may be carried in a Dynamic Adaptive Streaming of Hypertext transfer protocol (DASH) representation. For decoding and rendering a subset of slice / tile tracks, DASH representations including a subset of slice / tile tracks and DASH representations including the HPS may be requested by the client for each segment.

[0088] In other examples, the HPS may be defined as always being provided within the band. The HPS may also be carried in a time-synchronized track such as a parameter set track or a timed metadata track. As a result, the HPS may be supplied as described above. Further, in the file format specification for a video codec, the bitstream reconstruction process may compose the output bitstream from a subset of slice / tile tracks and a time-synchronized track including the HPS such that the HPS is part of the output bitstream.

[0089] The HPS shall exist and / or be available to the decoder before the first slice that refers to the HPS in decoding order. For example, when the HPS is available in-band, the HPS may precede the first slice that refers to the HPS in decoder order. Otherwise, the HPS decoding time shall be less than or equal to the decoding time of the first slice that refers to the HPS.

[0090] The HPS may contain the ID of a sequence level parameter set such as the SPS. When the SPS ID does not exist in the HPS, the following constraints may apply. When the first slice that refers to the HPS is part of an Intra Random Access Point (IRAP) picture and the HPS is carried in-band, the HPS may exist in the IRAP access unit. When the first slice that refers to the HPS is part of an IRAP picture and the HPS is carried out-of-band, the HPS decoding time may be the same as the decoding time of the IRAP picture. When the first slice that refers to the HPS is not part of an IRAP picture and the HPS is carried in-band, the HPS may exist in one of the access units inclusively between the IRAP access unit that starts the coded video sequence and the access unit that contains that slice. When the first slice that refers to the HPS is not part of an IRAP picture and the HPS is carried out-of-band, the HPS decoding time may exist inclusively between the decoding time of the IRAP access unit that starts the coded video sequence and the decoding time of the access unit that contains that slice.

[0091] If the SPS ID is present in the HPS, the following may apply. When provided within the band, the HPS may be present at the start of the bitstream or in any coded sequence, as long as the HPS precedes the first slice that references the HPS in decoding order. Otherwise, the HPS may be present in the same entry or in a time-synchronized track, as long as the decoding time of the HPS is less than the decoding time of the first slice that references the HPS.

[0092] For slice references to the HPS, the following may apply. If the SPS ID is present in the HPS, each slice and the HPS it references may reference the same SPS. Otherwise, the slice may not reference an HPS that is present in an access unit that precedes the access unit to which the slice is associated in the IRAP access unit. The slice may also be restricted from referencing an HPS that has a decoding time less than the decoding time of the IRAP access unit to which the slice is associated.

[0093] As an alternative to using a flag to identify the presence of parameters of a coding tool in HPS, a 2-bit indicator may be used (e.g., coded as u(2)). The semantics of the indicator are defined as follows. One value of the indicator (e.g., value 0) defines that the parameter does not exist in HPS and no reference to another HPS exists to derive the parameter. Another value of the indicator (e.g., value 1) defines that the parameter does not exist in HPS and a reference to another HPS exists to derive the parameter. Another value of the indicator (e.g., value 2) defines that the parameter exists in HPS and no reference to another HPS exists. Other values of the indicator may be reserved. In another example, another value of the indicator (e.g., value 3) defines that the parameter exists in HPS and a reference to another HPS exists to derive the parameter. In this case, the final parameter is derived from the input of the parameter explicitly notified in HPS and the parameter existing in the reference HPS.

[0094] When the coding tool is determined to be disabled for a coded video sequence by some indication means (e.g., an enable flag within a sequence parameter set defines that the coding tool is disabled), the following constraints may be applied individually or in combination. The flag or indication of the existence of a parameter for the coding tool, and the parameter for the coding tool, may not exist in the HPS associated with the coded video sequence. The flag or indication of the existence of a parameter for the coding tool exists, but the value may be constrained such that the parameter for the coding tool does not exist and no reference HPS for deriving and / or inferring the parameter exists.

[0095] When a slice references an HPS that may include parameters of a coding tool, the following constraints may be applied individually or in combination. When the coding tool is enabled for the slice and the parameters of the coding tool are available in the HPS, the parameters of the coding tool may not be present in the slice header. This can occur when the coding tool parameters are directly notified and / or are present in the HPS or are available through a reference HPS. When the coding tool is enabled for the slice and the parameters of the coding tool are not available in the HPS, the parameters of the coding tool may be present in the slice header. This can occur when the coding tool parameters are not directly notified and / or are not present in the HPS or are not available through a reference HPS. When the coding tool is enabled for the slice and the parameters of the coding tool are available in the HPS, the parameters of the coding tool may also be present in the slice header. This can occur when the coding tool parameters are directly notified and / or are present in the HPS or are available through a reference HPS. In this case, the parameters used to call the coding tool during decoding of the slice are those present in the slice header.

[0096] When both in-band and out-of-band transport of the HPS are used, the following may apply. When the HPS is transported in-band, the HPS may not need to reference other HPSs transported out-of-band. Otherwise, the HPS may not need to reference other HPSs transported in-band.

[0097] When both in-band and out-of-band transport of the HPS are used, the following constraints may further be applied individually or in combination. The HPS transported out-of-band may only be transported on a time-synchronized track. The HPS may only be transported out-of-band if there is an HPS update.

[0098] Instead of coding the HPS ID as a syntax element (ue(v)) coded with zero-order Exp-Golomb of an unsigned integer starting from the leftmost bit first, the HPS ID may be coded as u(v). The number of bits for signaling the HPS ID may be specified in the SPS.

[0099] Two HPSs within the same coded video sequence may have the same HPS ID, in which case the following may apply. If HPS A and HPS B have the same HPS ID, HPS B follows HPS A in decoding order, and the SPS contains an ID that references the HPS ID, then in that case, HPS B replaces HPS A. If HPS A and HPS B have the same HPS ID, the decoding time of HPS B is longer than the decoding time of HPS A, and the SPS contains an ID that references the HPS ID, then in that case, HPS B replaces HPS A. Let HPS A, HPS B, HPS C, and HPS D be HPSs included in the same coded video sequence either by having the same SPS ID (e.g., if the SPS ID exists for the HPS) or by association of the access unit containing the HPS. If the HPS IDs of HPS A and HPS D are the same and the HPS IDs of HPS A, HPS B, and HPS C are unique, then the values of the HPS IDs of HPS A, HPS B, and HPS D may be constrained to increase monotonically. A flag may exist in the SPS to specify whether an HPS can refer to another reference HPS.

[0100] When references between HPSs are not allowed, a slice header may have multiple references to the same HPS or different HPSs. In this case, the following may apply. An HPS reference may exist for each coding tool enabled for the slice, and the parameters of the coding tool may be inferred from the HPS. When a slice refers to an HPS to infer the parameters of a coding tool, the parameters of the coding tool should exist in that HPS.

[0101] If the parameter of the coding tool does not exist in the HPS and there is a reference to another HPS for that parameter, the parameter should exist in the reference HPS. The HPS may not refer to other HPSs from different coded video sequences. If the SPS ID exists in the HPS, the values of the SPS IDs of both the current HPS and the corresponding reference HPS must be the same. Otherwise, the HPS may not refer to other HPSs that exist in the access unit preceding the last IRAP access unit that precedes the HPS in decoding order. The HPS may also not refer to other HPSs that have a decoding time less than the decoding time of the last IRAP access unit that precedes the HPS in decoding order, or the decoding time of the last IRAP access unit that has a decoding time closest to and less than the decoding time of the HPS.

[0102] When an HPS refers to another HPS, the HPS ID of the referred HPS must be smaller than the HPS ID of that HPS. When an SPS ID exists in any of the HPSs, HPS A, HPS B, HPS C, and HPS D may all have the same SPS ID. When HPS B follows HPS A in decoding order, HPS C follows HPS B in decoding order, and HPS D follows HPS C in decoding order, the following constraints may be applied individually or in combination. When HPS C refers to HPS A, the HPS ID of HPS B may not be the same as the HPS ID of HPS A. When HPS B refers to HPS A and HPS C has the same HPS ID as HPS A, in that case, HPS D may not refer to either HPS A or HPS B. When HPS B and HPS A have the same HPS ID, there may not be a slice following HPS B in decoding order that refers to HPS A. When HPS B refers to HPS A and HPS C has the same HPS ID as HPS A, there may not be a slice following HPS C in decoding order that refers to either HPS A or HPS B.

[0103] When temporal scalability is used, the temporal ID of an HPS may be specified as follows. The temporal ID of an HPS may be set to be the same as the temporal ID of the access unit containing that HPS. In one example, the temporal ID of an HPS may be set to be the same as the temporal ID of the picture of the first slice that refers to that HPS.

[0104] A slice within a picture having a temporal ID (Tid) A may not refer to an HPS having an ID TidB when TidB is greater than TidA. An HPS having TidA may not refer to a referred HPS having TidB when TidB is greater than TidA. An HPS having TidA may not replace another HPS having TidB when TidA is greater than TidB.

[0105] Flags within a sequence level parameter (e.g., SPS) may exist to identify whether a slice has a reference to an HPS. When the value of the flag is set equal to 1, it may be determined that the slice refers to an HPS, and that the HPS ID exists in the slice header. When the value of the flag is set equal to 0, it may be determined that the slice does not refer to an HPS and that the HPS ID does not exist in the slice header.

[0106] The above aspect may be implemented according to the following syntax. [Table 1]

[0107] hps_present_flag may be set equal to 1 to determine that hps_id exists in the slice header. hps_present_flag may be set equal to 0 to determine that hps_id does not exist in the slice header. [Table 2]

[0108] The header_parameter_set_id may identify an HPS for reference by other syntax elements. The value of hdr_parameter_set_id may be in the range from 0 to 63. The hps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of pps_seq_parameter_set_id may be in the range from 0 to 15. alf_parameters_idc[header_parameter_set_id] may be set equal to 2 to determine that alf_data( ) exists in the HPS. alf_parameters_idc[header_parameter_set_id] may be set equal to 1 to determine that although alf_data( ) does not exist in the HPS, it is presumed to be the same as the alf_data( ) that exists in the reference HPS specified by alf_ref_hps_id[header_parameter_set_id]. alf_parameters_idc[header_parameter_set_id] may be set equal to 0 to determine that neither alf_data( ) nor alf_ref_hps_id[header_parameter_set_id] exists in the HPS. A value of alf_parameters_idc[header_parameter_set_id] equal to 3 may be reserved. alf_ref_hps_id[header_parameter_set_id] may specify the header_parameter_set_id of the reference HPS for inferring the value of alf_data( ).

[0109] An example bitstream compliance check may require that the following constraints apply. When present, the value of alf_ref_hps_id[header_parameter_set_id] may be less than the value of header_parameter_set_id. The values of hps_seq_parameter_set_id within the current HPS and within the HPS specified by alf_ref_hps_id[header_parameter_set_id] may be the same. The value of alf_parameters_idc[alf_ref_hps_id[header_parameter_set_id]] may be equal to 2.

[0110] Considering HPS A with a header_parameter_set_id equal to hpsA, HPS B with a header_parameter_set_id equal to hpsB, HPS C with a header_parameter_set_id equal to hpsC, and the current HPS, the value of alf_ref_hps_id[header_parameter_set_id] may not be equal to hpsA or hpsB if the following conditions are true. Such conditions are that HPS A precedes HPS B in decoding order, HPS B precedes HPS C in decoding order, and HPS C precedes the current HPS in decoding order. Such conditions also include that the value of alf_ref_id[hpsB] is equal to hpsA and the value of hpsC is equal to hpsA. [Table 3]

[0111] The slice_hps_id specifies the header_parameter_set_id of the HPS that the slice refers to. When the slice_hps_id does not exist, alf_parameters_idc[slice_hps_id] is assumed to be equal to 0. An example bitstream compliance check may require that the following constraints apply. The HPS with a header_parameter_set_id equal to slice_hps_id must be available before parsing the slice header. When the HPS with a header_parameter_set_id equal to slice_hps_id is available within the band, the header_parameter_set_id should be present in one of the following access units. An IRAP access unit related to the picture of the current slice, or any access unit that follows the IRAP access unit but precedes the current access unit in decoding order, or the current access unit. Considering HPS A with a header_parameter_set_id equal to hpsA, HPS B with a header_parameter_set_id equal to hpsB, HPS C with a header_parameter_set_id equal to hpsC, and the current slice, when both of the following conditions are true, in that case, the value of slice_hps_id must be equal to hpsA or hpsB. The conditions include that HPS A precedes HPS B in decoding order, HPS B precedes HPS C in decoding order, and HPS C precedes the current slice in decoding order. The conditions also include that the value of alf_ref_id[hpsB] is equal to hpsA and the value of hpsC is equal to hpsA.

[0112] Other example implementations are described below.

Table 4

[0113] alf_parameters_idc[header_parameter_set_id] may be set equally to 2 to determine that alf_data( ) exists in the HPS. alf_parameters_idc[header_parameter_set_id] may be set equally to 1 to determine that although alf_data( ) does not exist in the HPS, it is presumed to be the same as alf_data( ) existing in the reference HPS specified by alf_ref_hps_id[header_parameter_set_id]. alf_parameters_idc[header_parameter_set_id] may be set equally to 0 to determine that neither alf_data( ) nor alf_ref_hps_id[header_parameter_set_id] exists in the HPS. In the case of non-existence, lf_parameters_idc[header_parameter_set_id] may be presumed to be equal to 0. A value of alf_parameters_idc[header_parameter_set_id] equal to 3 may be reserved.

[0114] Other example implementations are described below.

[0115] The slice header may refer to multiple HPSs. A slice of a picture can refer to an HPS for ALF parameters (e.g., pic_alf_HPS_id_luma[i]), an HPS for LMCS parameters (e.g., pic_lmcs_aps_id), and an HPS for scaling list parameters (e.g., pic_scaling_list_aps_id). pic_alf_aps_id_luma[i] specifies the adaptation_parameter_set_id of the i-th ALF HPS referred to by the luma component of the slice related to the PH. slice_alf_aps_id_luma[i] specifies the adaptation_parameter_set_id of the i-th ALF HPS referred to by the luma component of the slice. The temporal ID of the HPS NAL unit having aps_params_type equal to ALF_APS and adaptation_parameter_set_id equal to slice_alf_aps_id_luma[i] shall be less than or equal to the temporal ID of the coded slice NAL unit. When slice_alf_enabled_flag is equal to 1 and slice_alf_aps_id_luma[i] does not exist, the value of slice_alf_aps_id_luma[i] is assumed to be equal to the value of pic_alf_aps_id_luma[i]. pic_lmcs_aps_id specifies the adaptation_parameter_set_id of the LMCS HPS referred to by the slice related to the PH. The temporal ID of the HPS NAL unit having aps_params_type equal to LMCS_APS and adaptation_parameter_set_id equal to pic_lmcs_aps_id shall be less than or equal to the temporal ID of the picture related to the PH. pic_scaling_list_aps_id specifies the adaptation_parameter_set_id of the scaling list HPS.The temporal ID of an HPS NAL unit having an aps_params_type equal to SCALING_APS and an adaptation_parameter_set_id equal to pic_scaling_list_aps_id shall be less than or equal to the temporal ID of the picture associated with the PH.

[0116] The temporal ID of the HPS NAL unit shall be the same as that of the access unit (AU) containing the HPS. The temporal ID of the HPS NAL unit shall be less than or equal to the temporal ID of the coded slice NAL unit that refers to the HPS. In a specific example, the value of the temporal ID of the non-VCL NAL unit is restricted as follows. When nal_unit_type is equal to DPS_NUT, VPS_NUT, or SPS_NUT, the temporal ID shall be equal to 0, and the temporal ID of the AU containing the NAL unit shall be equal to 0. Otherwise, when nal_unit_type is equal to PH_NUT, the temporal ID shall be equal to the temporal ID of the PU containing the NAL unit. Otherwise, when nal_unit_type is equal to EOS_NUT or EOB_NUT, the temporal ID shall be equal to 0. Otherwise, when nal_unit_type is equal to AUD_NUT, FD_NUT, PREFIX_SEI_NUT, or SUFFIX_SEI_NUT, the temporal ID shall be equal to the temporal ID of the AU containing the NAL unit. Otherwise, when nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, the temporal ID shall be greater than or equal to the temporal ID of the PU containing the NAL unit. When the NAL unit is a non-VCL NAL unit, the value of the temporal ID shall be equal to the minimum value among the temporal ID values of all AUs to which the non-VCL NAL unit is applied. When nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, the temporal ID may be greater than or equal to the temporal ID of the containing AU in that all PPSs and HPSs can be included at the beginning of the bitstream (e.g., when they are transported out-of-band and the receiver places them at the beginning of the bitstream), and at this time, the first coded picture has a temporal ID equal to 0.

[0117] FIG. 7 is a schematic diagram of an exemplary video coding device 700. The video coding device 700 is suitable for implementing the disclosed examples / embodiments described herein. The video coding device 700 has a transceiver unit (Tx / Rx) 710 that includes a downstream port 720, an upstream port 750, and / or a transmitter and / or receiver for communicating data upstream and / or downstream over a network. The video coding device 700 also includes a processor 730 that includes a logic unit and / or a central processing unit (CPU) for processing data, and a memory 732 for storing data. The video coding device 700 may also have electrical, optical, or wireless communication components coupled to the upstream port 750 and / or the downstream port 720 for communication of data over an electrical, optical, or wireless communication network. The video coding device 700 may also include an input and / or output (I / O) device 760 for communicating data to and from a user. The I / O device 760 may include output devices such as a display for displaying video data, a speaker for outputting audio data, etc. The I / O device 760 may also include input devices such as a keyboard, a mouse, a trackball, etc., and / or corresponding interfaces that interact with such output devices.

[0118] Processor 730 is implemented by hardware and software. Processor 730 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), Field-Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), and Digital Signal Processors (DSPs). Processor 730 communicates with downstream port 720, Tx / Rx 710, upstream port 750, and memory 732. Processor 730 has a coding module 714. Coding module 714 implements the disclosed embodiments described above, such as methods 100, 800, and 900 that may use bitstream 500 and / or mechanism 600. Coding module 714 may also implement any other method / mechanism described herein. Further, coding module 714 may implement codec system 200, encoder 300, and / or decoder 400. For example, coding module 714 can encode / decode pictures in a bitstream and encode / decode parameters related to slices of a picture in multiple HPSs. Various types of HPSs may be included along with corresponding types of coding tool parameters. The slice header can then refer to various types of HPSs to obtain the coding tool parameters for the corresponding slice. Such HPSs may also be assigned temporal IDs to function according to a temporal scaling algorithm. The use of HPSs enables the coding tool parameters used by multiple slices to be gathered in a single location (e.g., using additional HPSs when the parameters change). Thus, redundant signaling is removed, which increases coding efficiency, reduces memory resource utilization when storing the bitstream, and reduces network resource utilization when communicating the bitstream.Accordingly, the coding module 714 causes the video coding device 700 to provide additional functionality and / or coding efficiency when coding video data. As such, the coding module 714 improves the functionality of the video coding device 700 in addition to addressing issues specific to video coding techniques. Further, the coding module 714 achieves variations of the video coding device 700 to different states. Alternatively, the coding module 714 may be executed as instructions stored in the memory 732 and executed by the processor 730 (e.g., as a computer program product stored on a non-transitory medium).

[0119] The memory 732 has one or more memory types such as a disk, a tape drive, a solid state drive, a read only memory (ROM), a random access memory (RAM), a flash memory, a ternary content-addressable memory (TCAM), a static random-access memory (SRAM), etc. The memory 732 may be used as an overflow data storage device to store such programs when a program is selected for execution and to store instructions and data read during program execution.

[0120] FIG. 8 is a flowchart of an exemplary method 800 for encoding a video sequence in a bitstream such as the bitstream 500 by using the HPS. The method 800 may be used by an encoder such as the codec system 200, the encoder 300, and / or the video coding device 700 when executing the method 100. The method 800 may also encode the bitstream to support time scaling according to the mechanism 600 in a decoder such as the decoder 400.

[0121] Method 800 may start when an encoder receives a video sequence including a plurality of images. For example, based on a user input, it is determined to encode the video sequence into a bitstream. The video sequence is partitioned into pictures / images / frames for further partitioning before encoding. In step 801, a plurality of pictures are partitioned into a plurality of slices including a first slice.

[0122] In step 803, a plurality of slices including the first slice are encoded into the bitstream. The slices may be encoded by a plurality of coding tool parameters. In some examples, the slices are encoded by at least a first type of coding tool and a second type of coding tool. Specifically, the slices are encoded by the first type of coding tool based on the first type of coding tool parameters. The slices are also encoded by the second type of coding tool based on the second type of coding tool parameters. For example, such coding tools may include an ALF coding tool, an LMCS coding tool, and / or a scaling list parameter coding tool.

[0123] In step 805, a plurality of HPSs are encoded into the bitstream. The plurality of HPSs may include at least a first HPS and a second HPS. The first HPS and the second HPS each include the first type of coding tool parameters and the second type of coding tool parameters used to encode the slices in step 803.

[0124] In step 807, the first slice header is encoded in the bitstream. The first slice header describes the encoding of the first slice among a plurality of slices to support decoding at the decoder. For example, the first slice header may include a first reference to the first HPS and a second reference to the second HPS. Thus, the slice header may inherit coding tool parameters from a plurality of different types of HPSs. This allows such coding tool parameters to be omitted from the slice header, improving the coding efficiency of the bitstream by reducing the signaling of redundant coding tool parameters. As a specific example, the first HPS and the second HPS may include an ALF HPS, an LMCS HPS, a scaling list parameter HPS, or a combination thereof.

[0125] To reduce the complexity of the coding method associated with HPS, the HPS may be restricted from referring to coding tool parameters stored in other HPSs. Thus, a plurality of HPSs encoded in step 805, including a first HPS and a second HPS, are restricted from referring to coding tool parameters from other HPSs among the plurality of HPSs. Also, the HPS may be encoded to support time scaling by including a temporal ID in each HPS. In one example, the first HPS is included in an access unit associated with the temporal ID. Further, the first HPS includes the same temporal ID associated with the access unit that includes the first HPS. In another example, the first slice is partitioned from the first picture, and the first picture is associated with the temporal ID. In this example, the first HPS includes the temporal ID associated with the picture. Further, to support time scaling, each of the plurality of HPSs and each of the slices are associated with one of the plurality of temporal IDs. Further, each slice having a first temporal ID is restricted from referring to any HPS having a second temporal ID greater than the first temporal ID (e.g., associated with a higher frame rate). This ensures that slices associated with a lower frame rate do not refer to HPSs associated with a higher frame rate, as such HPSs would be ignored if a lower frame rate were used according to the time scaling mechanism.

[0126] In step 809, the bitstream is stored in memory. Upon request, the bitstream can then be sent to a decoder, for example, by a transmitter.

[0127] FIG. 9 is a flowchart of an exemplary method 900 for decoding a video sequence from a bitstream such as bitstream 500 by using HPS. Method 900 may be used by a decoder such as codec system 200, decoder 400, and / or video coding device 700 when executing method 100. The results of method 900 may also be used to support temporal scaling in the decoder according to mechanism 600. Method 900 may be used in response to receiving a bitstream from an encoder such as encoder 300, and thus, method 900 may be used in response to method 800.

[0128] Method 900 may begin when the decoder begins to receive a bitstream of coded data representing a video sequence, for example, as a result of method 800. At step 910, the bitstream is received at the decoder. The bitstream has a plurality of HPSs including a first HPS and a second HPS. The first HPS is a first type of HPS and includes coding tool parameter of the first type. The second HPS is a second type of HPS and includes coding tool parameter of the second type. The bitstream also includes a slice header and a slice associated with the slice header.

[0129] At step 903, the decoder determines that the slice header includes a first reference to the first HPS and a second reference to the second HPS. Thus, the slice header may inherit coding tool parameters from multiple HPSs of different types. This allows such coding tool parameters to be omitted from the slice header, improving the coding efficiency of the bitstream by reducing the signaling of redundant coding tool parameters. As a specific example, the first HPS and the second HPS may include an ALF HPS, an LMCS HPS, a scaling list parameter HPS, or a combination thereof.

[0130] In step 905, based on the determination that the slice header includes the first reference and the second reference, the decoder can decode the slice using the first type of coding tool parameters and the second type of coding tool parameters. To reduce the complexity of the coding method related to the HPS, the HPS may be restricted from referring to the coding tool parameters stored in other HPSs. Accordingly, a plurality of HPSs received in the bitstream in step 901, including the first HPS and the second HPS, are restricted from referring to the coding tool parameters from other HPSs among the plurality of HPSs. Also, the HPS may be coded to support time scaling by including a temporal ID in each HPS. In one example, the first HPS is included in an access unit related to the temporal ID. Further, the first HPS includes the same temporal ID related to the access unit including the first HPS. In another example, the first slice is partitioned from the first picture, and the first picture is related to the temporal ID. In this example, the first HPS includes the temporal ID related to the picture. Further, to support time scaling, each of the plurality of HPSs and each of the slices are related to one of the plurality of temporal IDs. Further, each slice having the first temporal ID is restricted from referring to any HPS having a second temporal ID greater than the first temporal ID (e.g., related to a higher frame rate). This ensures that a slice related to a lower frame rate does not refer to an HPS related to a higher frame rate, as such an HPS would be ignored when a lower frame rate is used according to the time scaling mechanism.

[0131] In step 907, the decoder can transfer the slice for display as part of the decoded video sequence.

[0132] FIG. 10 is a schematic diagram of an exemplary system 1000 for coding a video sequence of an image in a bitstream such as bitstream 500 by using HPS. System 1000 may be implemented by an encoder and decoder such as codec system 200, encoder 300, decoder 400, and / or video coding device 700. Further, system 1000 may be used when implementing methods 100, 800, and / or 900. Moreover, system 1000 may be used to support time scaling as described in connection with mechanism 600.

[0133] System 1000 includes a video encoder 1002. The video encoder 1002 has a partitioning module 1001 that partitions a plurality of pictures into a plurality of slices. The video encoder 1002 further has an encoding module 1003 that encodes the plurality of slices into a bitstream, where the slices are encoded by at least a first type of coding tool based on first type of coding tool parameters and a second type of coding tool based on second type of coding tool parameters. The encoding module 1003 is further for encoding a first HPS and a second HPS into the bitstream, where the first HPS includes first type of coding tool parameters and the second HPS includes second type of coding tool parameters. The encoding module 1003 is further for encoding a first slice header that describes the encoding of a first slice among the plurality of slices into the bitstream, where the first slice header includes a first reference to the first HPS and a second reference to the second HPS. The video encoder 1002 further has a storage module 1005 that stores the bitstream for communication to a decoder. The video encoder 1002 further has a transmission module 1007 that transmits a bitstream including the first HPS and the second HPS to support decoding of slices at the decoder based on the first type of coding tool and the second type of coding tool. The video encoder 1002 may be further configured to execute any of the steps of method 800.

[0134] System 1000 also includes a video decoder 1010. The video decoder 1010 has a receiving module 1011 that receives a bitstream having a first HPS including coding tool parameters of a first type, a second HPS including coding tool parameters of a second type, a slice header, and a slice associated with the slice header. The video decoder 1010 further has a determining module 1013 that determines that the slice header includes a first reference to the first HPS and a second reference to the second HPS. Based on the determination that the slice header includes the first reference and the second reference, the video decoder 1010 further has a decoding module 1015 that decodes the slice using the coding tool parameters of the first type and the coding tool parameters of the second type. The video decoder 1010 further has a transfer module 1017 that transfers the slice for display as part of the decoded video sequence. The video decoder 1010 may be further configured to perform any of the steps of method 900.

[0135] Except for a line, trace, or other medium between a first component and a second component, if there are no intervening components, the first component is directly coupled to the second component. If there are other intervening components between the first component and the second component other than a line, trace, or other medium, the first component is indirectly coupled to the second component. The term "coupled" and its variations include both being directly coupled and being indirectly coupled. The use of the term "about" means a range that includes ± 10% of the number that follows, unless stated otherwise.

[0136] It should be understood that the steps of the exemplary methods described herein need not necessarily be performed in the order described, and that the order of such method steps is merely illustrative. Similarly, such methods may include additional steps, and certain steps may be deleted or combined in methods according to various embodiments of the present disclosure.

[0137] Although several embodiments are provided in the present disclosure, it can be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. This example should be considered as illustrative rather than restrictive, and the intention should not be limited to the details given herein. For example, various elements or components may be combined or integrated with other systems, or certain features may be omitted or not implemented.

[0138] Furthermore, the techniques, systems, subsystems, and methods described and illustrated in various embodiments as separate or discrete may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of changes, substitutions, and alternatives can be ascertained by those skilled in the art and made without departing from the spirit and scope disclosed herein.

Claims

1. A decoding method comprising the steps of: receiving a bitstream having an adaptive loop filter (ALF) adaptation parameter set (APS) including ALF coding tool parameters, a scaling list APS including scaling list parameters, a slice header, and a slice associated with the slice header; determining that the slice header includes a first reference to the ALF APS and a second reference to the scaling list APS; based on the determination that the slice header includes the first reference and the second reference, decoding the slice using the ALF coding tool parameters and the scaling list parameters to obtain a reconstructed image block; obtaining a decoded picture based on the reconstructed image blocks; and obtaining a decoded sequence including the decoded pictures; The method according to claim 1,

2. the bitstream further comprises a plurality of APSs including the ALF APS and the scaling list APS, each of the plurality of APSs being restricted from referencing coding tool parameters from other APSs within the plurality of APSs. The method of claim 1.

3. The ALF APS is included in an Access Unit (AU) associated with a temporal identifier (ID), and the ALF APS includes the temporal ID associated with the AU that includes the ALF APS. The method according to claim 1 or 2.

4. the slice is a portion of a picture, the picture being associated with a temporal identifier (ID), and the ALF APS includes the temporal ID associated with the picture. The method according to claim 1 or 2.

5. the bitstream further comprises a plurality of APSs including the ALF APS and the scaling list APS, and a slice having a first temporal ID is restricted from referencing any APS having a second temporal ID greater than the first temporal ID.

5. The method according to any one of claims 1 to 4.

6. 1. A method of encoding, comprising: obtaining a video sequence comprising a plurality of pictures; Partitioning the pictures into slices; encoding the plurality of slices into a bitstream, the slices being encoded based at least on adaptive loop filter (ALF) coding tool parameters and scaling list parameters; encoding an ALF adaptation parameter set (APS) and a scaling list APS into the bitstream, the ALF APS including the ALF coding tool parameters and the scaling list APS including the scaling list parameters; encoding a slice header into the bitstream, the slice header including a first reference to the ALF APS and a second reference to the scaling list APS; storing said bitstream; The method according to claim 1,

7. encoding a plurality of APSs into the bitstream; the plurality of APSs includes the ALF APS and the scaling list APS, and each of the plurality of APSs is restricted from referencing coding tool parameters from other APSs in the plurality of APSs. The method according to claim 6.

8. The ALF APS is included in an Access Unit (AU) associated with a temporal identifier (ID), and the ALF APS includes the temporal ID associated with the AU that includes the ALF APS. The method according to claim 6 or 7.

9. a first slice is partitioned from a first picture, the first picture being associated with a temporal identifier (ID), and the ALF APS includes the temporal ID associated with the picture; The method according to claim 6 or 7.

10. A plurality of APS including the ALF APS and the scaling list APS are encoded, each of the plurality of APS and each of the slices is associated with one of a plurality of temporal IDs, and each slice having a first temporal ID is restricted from referencing any APS having a second temporal ID greater than the first temporal ID.

10. The method according to any one of claims 6 to 9.

11. a memory storing computer executable instructions; A processor operatively coupled to said memory and configured to execute said computer-executable instructions to perform the method of any one of claims 1 to 5. A decoding device having the above configuration.

12. a memory storing computer executable instructions; A processor operatively coupled to said memory and configured to execute said computer-executable instructions to perform the method of any one of claims 6 to 10. An encoding device having:

13. receiving a bitstream having an adaptive loop filter (ALF) adaptation parameter set (APS) including ALF coding tool parameters, a scaling list APS including scaling list parameters, a slice header, and a slice associated with the slice header, the slice header including a first reference to the ALF APS and a second reference to the scaling list APS, the ALF coding tool parameters and the scaling list parameters being used to decode the slice to obtain a reconstructed image block; storing said bitstream in a memory; 2. A bitstream storage method comprising:

14. receiving a bitstream having an adaptive loop filter (ALF) adaptation parameter set (APS) including ALF coding tool parameters, a scaling list APS including scaling list parameters, a slice header, and a slice associated with the slice header, the slice header including a first reference to the ALF APS and a second reference to the scaling list APS, the ALF coding tool parameters and the scaling list parameters being used to decode the slice to obtain a reconstructed image block; storing said bitstream in a memory; transmitting said bitstream to a decoder upon receiving a request; A bitstream transmission method comprising:

Images